Cost-Effectiveness of School Support for Orphan Girls to Prevent HIV Infection in Zimbabwe

Abstract

This cost-effectiveness study analyzes the cost per quality-adjusted life year (QALY) gained in a randomized controlled trial that tested school support as a structural intervention to prevent HIV risk factors among Zimbabwe orphan girl adolescents. The intervention significantly reduced early marriage, increased years of schooling completed, and increased health-related quality of life. By reducing early marriage, the literature suggests the intervention reduced HIV infection. The intervention yielded an estimated US$1, 472 in societal benefits and an estimated gain of 0.36 QALYs per orphan supported. It cost an estimated US$6/QALY gained, about 1% of annual per capita income in Zimbabwe. That is well below the maximum price that the World Health Organization (WHO) Commission on Macroeconomics and Health recommends paying for health gains in low and middle income countries. About half the girls in the intervention condition were boarded when they reached high school. For non-boarders, the intervention’s financial benefits exceeded its costs, yielding an estimated net cost savings of $502 per pupil. Without boarding, the intervention would yield net savings even if it were 34% less effective in replication. Boarding was not cost-effective. It cost an additional $1,234 per girl boarded (over the three years of the study, discounted to present value at a 3% discount rate) but had no effect on any of the outcome measures relative to girls in the treatment group who did not board. For girls who did not board, the average cost of approximately three years of school support was US$973.

Introduction

In the last decade, growing recognition that social, economic, environmental, and political factors directly affect HIV risk and vulnerability stimulated interest in structural approaches to HIV prevention, both in the U.S. and in sub-Saharan Africa (Gupta, Parkhurst, Ogden, Aggleton, & Mahal, 2008; Hallfors, Iritani, Miller, & Bauer, 2007). Structural interventions promote health by altering the context in which people function, eliminating barriers to healthy behavior, and facilitating the ability and motivation to make better health decisions (Khumalo-Sakutukwa et al., 2008). This approach is different from more individually oriented behavior change efforts because it addresses factors affecting individual behavior, rather than targeting behavior itself (Gupta et al., 2008). Interventions to help keep sub-Saharan African adolescents in school have shown promise in reducing HIV/sexually transmitted infection (STI) risk (see e.g., Baird, Chirwa, McIntosh, & Ozler, 2010; Duflo, Dupas, Kremer, & Sinei, 2006).

HIV prevention research for orphan adolescents, and particularly girls, is especially important, since orphans are a large and growing group in sub-Saharan Africa. Adolescent orphans are more likely than non-orphans to drop out of school and engage in earlier sexual behavior (Thurman, Brown, Richter, Maharaj, & Magnani, 2006; Birdthistle et al., 2008), marry early (Gregson et al., 2005), and be infected with HIV (Birdthistle et al., 2008; Cowan et al., 2008). The prevalence of HIV among 15–17 year old orphan girls in Zimbabwe was 5.3% compared to 2.3% among non-orphan girls overall, and 4.8% versus 1.7% among girls in rural areas (p<.01) (Central Statistical Office [CSO, Zimbabwe] & Macro International Inc., 2007).

We conducted a randomized controlled trial of orphan assistance to Zimbabwe orphan girls. An earlier report on the trial showed that providing school fees, uniforms, school supplies and a school-based teacher “helper” was effective; kept orphans in school and reduced early marriage (Hallfors et al., 2011). After two years of treatment exposure, the control group was much more likely to drop out of school (OR=8.5; 95% CI=3.6–19.8) and marry early (OR=2.9; CI=1.0–8.3) compared to the intervention group.

Decisions on adopting an intervention, however, must consider not only whether it is effective but whether it is worth what it costs. Despite this, cost-effectiveness studies of structural HIV interventions in sub-Saharan Africa are rare (Galarraga, Colchero, Wamai, & Bertozzi, 2009). Cost-effectiveness analyses (CEA) of promising interventions are urgently needed to help policymakers and health providers better allocate scarce resources to the most cost-effective interventions to combat the spread of HIV/AIDS.

This article provides a CEA for the randomized trial of three years of school assistance to orphan girls in Zimbabwe. It also assesses within the intervention group whether the incremental costs of boarding students when they reached high school yielded sufficient benefit to be cost-effective. The prior evaluation article (Hallfors et al., 2011) focused on changes in the odds of school dropout and early marriage after two years of assistance. For the cost-effectiveness analysis, we analyze a longer time period and translate those odds into changes in number of years of schooling completed and number of years that marriage was delayed.

The CEA also adds an examination of quality adjusted life years (QALYs) as an outcome and a basis for comparing benefits and costs between groups. CEA studies often use this measure of health-related quality of life to measure intervention effectiveness and to compare intervention alternatives in terms of their costs and effectiveness (Gold, Siegel, Russell, & Weinstein, 1996; Drummond, Sculpher, Torrance, O’Brien, & Stoddart, 2005). A QALY is a health outcome measure that integrates gains in an individual’s health related quality of life from reduced morbidity/improved functional status (in this study, mobility, self-care, usual activities, pain, and anxiety/depression) and from reduced mortality (Gold et al., 1996; Drummond et al., 2005).

Methods

Setting and Participants

The trial focused on a rural province of Zimbabwe. Schools were randomized to a control group or a larger intervention group. All 328 eligible orphan girls in Grade 6 at 25 primary schools [randomly assigned: 13 schools (183 girls) to intervention and 12 (145) to control)] consented to be enrolled in the trial. Orphans were defined as girls with one or both parents deceased. After completing Grade 7, the girls matriculated to high school. New helpers were selected and trained in the high schools. Of the main study high schools, five were Methodist schools (including three that were primarily boarding schools); five were secular schools. Boarders went home during holidays only (i.e., month-long breaks between each of the three school terms). We informally boarded 28 additional treatment-group students at participating secular (non-Methodist) high schools because they would have had to walk 6 miles (10 kms) or more to reach their school (sometimes bypassing a closer, non-participating school). At these schools, students lived and ate during the week with female teachers who had housing at the school. Girls boarding with teachers typically went home every weekend. Sample size dictated combining formal and informal boarders into a single boarder group for analysis.

Outcome survey data were collected from all unmarried girls at randomization and annually thereafter. Orphan girls in primary schools assigned to control and treatment groups were equivalent on outcomes at baseline. Hallfors et al. (2011) details the clustered randomized design, data collection procedures, demographics, and outcomes by group at randomization.

The Intervention

The intervention used the Social Development Model (Catalano & Hawkins, 1996) as a conceptual framework. This model suggests that connections to pro-social adults, peers, and institutions result in better adolescent health and behavioral outcomes. Loss of one or both parents leaves orphans at a distinct disadvantage in navigating the risks of adolescence. The importance of consistent connections to pro-social, caring adults has been well documented in the United States as being a prime determinant of positive behavioral outcomes in adolescents (Jessor, Turbin, & Costa, 1998; Resnick et al., 1997; Kirby, 1999). For an orphan, school may be particularly important in providing a sense of order and routine and access to caring adults, as well as education. We tested school support as a structural intervention to surround orphans with a protective environment.

School support included fees, uniforms, and school supplies (e.g., pens, exercise books, soap, underpants, and sanitary napkins). In addition, female teachers at each intervention primary school were selected and trained by research personnel as helpers (approximately one helper to 10 participants) to monitor school attendance and intervene as needed with absenteeism problems, but not to provide special HIV information or life skills training. Helpers had access to a small emergency fund when addressing attendance problems. In 2010, 50% of the intervention group who were still in school were either in formal boarding at Methodist high schools (N=51) or informal boarding at a secular high school (N=28).

The study began in September 2007 when all girls were starting their final term of grade 6. Intervention commenced in Term 3 of that year after the baseline survey and randomization and continued through grade 7, Form 1, and Form 2 (the first two years of high school) for a total duration of 3.33 years. The trial followed IRB-approved and monitored protocols.

CEA Comparator

Our CEA examined the cost per unit improvement in outcome. We adopted a societal perspective. In that perspective, everyone’s costs and benefits count including those of orphans, their relatives, employers, government, and citizens. As described below, we documented the intervention’s costs, estimated its outcomes, valued those outcomes in dollars when possible, compared costs and outcomes of students in the intervention group relative to controls, and used simulation modeling to better assess the range of the CEA estimates. Since 2008, Zimbabwe’s currency has been the US dollar. This article states all costs in 2010 US dollars.

Intervention Cost

Only the intervention group had program costs. A University of Zimbabwe accountant collected receipts for fees and purchases of uniforms and other school supplies and documented costs in spreadsheets by school and year. Receipts were periodically reviewed and verified by U.S. research staff according to protocol. Helpers and headmasters received honoraria (typically $15-$20 for a 3-month term). As well as paying boarding charges for students in boarding school, we purchased required items such as towels, sheets, and storage trunks. When high school students were boarded informally, we paid a monthly fee (typically $50/child) to a teacher which covered food and other expenses. Informal boarding costs were in the midrange of costs among schools with formal boarding so we analyzed boarders as a single group.

Outcome Measurement and Analysis

We collected student survey data annually and tracked administrative data for three years plus one school term of a fourth year. The CEA values the effects on three outcomes: retention in school which increases future expected earnings, early marriage which elevates HIV risk, and health-related quality of life. We tracked the first two measures longitudinally but only assessed quality of life in 2010. To test the significance of observed differences between the intervention and control groups in years of schooling and years unmarried, we used multivariate logit regressions that controlled for orphan age at enrollment in the trial and Apostolic church membership (which tended to promote early marriage of girls). The regressions also accounted for the original clustering of the sample by school. Although our design did not allow comparison of boarding as a distinct intervention component against the control condition, we compared its cost-effectiveness within the intervention group.

We quantified QALYs using a Shona-language version of the popular EQ-5d assessment instrument with Zimbabwe-specific scoring (Jelsma, Hansen, De Weerdt, De Cock, & Kind, 2003). EQ-5d respondents report on their mobility, self-care, usual activities (e.g., work, study, housework, family or leisure activities), pain/discomfort, and anxiety/depression. On each of the five QALY dimensions, participants report whether they had no problems, some problems, or extreme problems including an inability to perform the function. The best health state is scored at .985; the worst health state is scored at .094. We used a multivariate regression to test the significance of differences in QALY scores between groups. Using multinomial logit models, we also tested the significance of differences in level of functioning on each EQ-5d dimension. In uncertainty analysis, we used the Thai scoring of the EQ-5d (Tongsiri & Cairns, 2011) which is the only other EQ-5d scoring from a developing country. The Thai scoring derived its values with a widely used regression model that includes a non-linear valuation term for poor functional status, creating a wider and lower score range from 0.798 to −0.454. The range differential suggests that the Thai mean score will be lower than the Zimbabwe mean score; it should have less effect on the differences in scores between groups or over time.

Outcome Valuation

We used net cost per QALY gained as a cost-effectiveness measure. A CEA can have only one uncosted outcome, in this case the QALY gain. In the generally accepted formula for computing cost per QALY, direct cost savings are subtracted from the program costs when computing the CEA measure (Gold et al., 1996; Drummond et al., 2005). Health-related productivity gains are not subtracted; they are captured in the QALY measure so they are not valued separately (Gold et al., 1996).

Paying school fees and expenses, however, causes orphan girls to stay in school for reasons unrelated to their health. In accordance with CEA principles (Gold et al., 1996; Haddix, Teutsch, & Corso, 2003) and prevailing practice (Cheng et al., 2000; Frick et al., 2004), we treated earnings gains due to increased education as societal benefits independent of QALY benefits. The return to education due to increased school retention offsets the intervention cost.

Return to education

No recent data were available that we could analyze to estimate return to education in Zimbabwe. To value retention in school, therefore, we relied on an evaluation of the return to education in Zimbabwe in the late 1980s (Bennell & Malaba, 1993). Zimbabwe’s British-based education system consists of 7 years of primary school and 6 years of secondary school (4 years of O level and 2 years of A level) before students can enter university. Bennell & Malaba (1993) estimated the return from completing the four years of O-level schooling rather than dropping out after seventh grade. The school costs and internal rate of return they reported indicate that the O-level increased annual per capita income (more formally, Gross Domestic Product per capita) by Z$202 pre-independence (roughly US$101, in 1987 dollars). The estimated average per capita income in 1987 in Zimbabwe was US$358 (WorldBank, 2011; deflated to 1987 dollars), so the four years of O-level schooling raised annual per capita income by an estimated 28.2% (101/358). We assumed that percentage still applies.

In 2010, estimated per capita income in Zimbabwe averaged $1,516 per person employed in the formal sector and $591 per resident (WorldBank, 2011). We computed the present value of the annual income gains (and of all other costs and benefits) at the 3% inflation-free discount rate recommended for international use by the Panel on Cost-Effectiveness in Health and Medicine (Gold et al., 1996), with sensitivity analysis at a 7% discount rate. The computations used Zimbabwe’s 30-year worklife expectancy (Bennell & Malaba, 1993). Discounting accounts for uncertainty about the future and for the fact that future bills can be paid at a lower cost today by depositing a lesser amount in the bank and letting it earn interest. The discount rate was inflation-free because we stated all costs and benefits in 2010 dollars.

We assumed a constant return per year of high school. Thus the return from increased education equals one fourth the return from completing the O level of high school times the fraction of a year of schooling gained per child in the treatment group.

Reduction in early marriage and associated HIV

To value the reduction in early marriage, we used published estimates of the risk of HIV infection associated with early marriage and of medical costs and loss of life years associated with HIV infection. In an urban Zimbabwe study of 15–19 year old females, half of whom were orphans, 18% of the married were infected with HIV compared to 6% of those never married (Birdthistle et al., 2008). Our cost-effectiveness model uses this 12 percentage point increase in HIV risk with early marriage. A recent Zimbabwe survey of secondary school students (Cowan et al., 2008) estimated excess HIV risk if orphaned as 3.4:1 (standard error 0.867). Baseline risk was 2.2 (3.4 * 50% orphans + 1 * 50% not orphans), so the risk differential was 1.55 (3.4/2.2). We multiplied this risk adjustment times the 12 percentage point prevalence differential for early marriage. We used Beck et al.’s (2008) estimate that the annual medical spending per HIV case in Africa is US $2,402 with a range from $812-$5,510 (adjusted to 2010 dollars).

We used Creese, Floyd, Alban, and Guinness’ (2002) cost-effectiveness model parameters for Africa to estimate QALY gain associated with reduced HIV. Gain per HIV case prevented comes from preventing loss of 24 years of life expectancy plus 7 years of life with an annual QALY loss of 0.495 during home treatment (a loss estimate the model took from Murray & Lopez, 1996). In analyzing uncertainty around the CEA estimate, we instead used the annual 0.3 QALY loss for HIV treated with highly active antiretroviral therapy (HAART) in South Africa (Badri et al., 2006). By convention, gain of a year of life expectancy is equated with a gain of one QALY (Murray & Lopez, 1996; Gold et al., 1996). Creese et al.’s (2002) estimate of life expectancy loss was for a case of HIV contracted at age 25. Conservatively, in part because orphan assistance sometimes merely delayed rather than prevented infection, we did not adjust the estimate upward before applying it to our 13–19 -year-old students who married early.

Cost-Effectiveness and Uncertainty Analysis

We computed and compared cost per QALY saved. We used the Oracle Crystal Ball add-in to Excel to analyze the uncertainty around the estimated net cost of the intervention, QALY gains, and cost/QALY gained and arrive at best estimates based on 1 million simulations. Table 1 and the results tables summarize the values for the outcomes and the uncertainty estimates used in these calculations, as well as their sources. We assumed a log-normal distribution of program costs since they cannot be negative. We assumed normal distributions for all other CEA model parameters where standard errors were available. For most other parameters, we assumed a simple triangular distribution because we lacked distributional data. The estimated gain from preventing an HIV infection of a 49.5% annual QALY loss for 7 years (the values used in the primary analysis) should be representative of typical HIV care in Zimbabwe. The estimated 30% loss for 12 years is more representative of losses with more costly HAART care which is the norm in neighboring South Africa (Badri et al., 2006). The uncertainty analysis used a uniform distribution that ranged from a 25% to a 54.5% QALY gain because it is unclear if and when Zimbabwe will adopt HAART widely.

Table 1.

Values Used for the Outcomes, Uncertainty Ranges Used in the Sensitivity Analysis, and Sources for the Outcome Values

Outcome	Value	Uncertainty Range	Source
Income, Formal Sector	$1,516/year	±30%	WorldDataReport (2011)
% Rise in Income with Schooling	7.05%/year	6.05%–8.05%	Bennell & Malaba (1993)
Expected Worklife at Age 16	30 years	21–39 years	Bennell & Malaba (1993)
Rise in HIV Risk If Married	0.122	SE = .037	Birdthistle et al. (2008)
Rise in HIV Risk If Orphan	1.55 (3.4/2.2)	SE = .4335	Cowan et al. (2008)
Medical Cost of HIV	$2,402/patient-year	$812-$5,510	Beck (2010)
QALY Gain from Preventing HIV	0.495/year of lifespan	0.30/year (uniform distribution)	Murray & Lopez (1996) Badri et al. (2006)
Expected Lifespan with HIV	7 years	6–12 years	Creese et al. (2002)
Life Expectancy at Age 16	31 years	23–43 years	Creese et al. (2002)

Paralleling the previous main effects analysis of the primary outcomes (Hallfors et al., 2011; Cho et al., 2011), we used one-tailed tests in the analysis of treatment versus control group because the direction of effect was hypothesized in advance. Conversely, it was unclear if boarding would lengthen or reduce the duration measures so we used two-tailed tests.

Results

Intervention Cost

Undiscounted, over 10 school terms (3.33 school years), the intervention cost an average of $1,565. The cost averaged $1,025 per pupil exclusive of boarding. Boarding a pupil added an average of $1,234 (total cost of $2,259 for a boarded student) and typically included boarding for 6 terms (2 years of high school). Table 2 shows the costs by cost category. In this table, the $12,358 in school fees for 2007–2008 is the cost required to replicate the intervention at 2009–2010 tuition levels. Before adjusting to 2010 pricing, actual fees for 2007–2008 totaled only $4,652. Fees were the largest non-boarding expense, followed by textbooks and uniforms/shoes. The expenditures were lumpy rather than level across years. Uniform expenses, for example, were incurred as students moved into a school. Implementation problems (as well as cost) led us to discontinue textbook provision after the first full school year of the trial. A teacher strike at most schools in 2008 resulted in tuition, helper, and headmaster payments for two rather than three terms at most schools. Those savings were offset by payments for extra lessons to cover the material missed during the strike. Helper training expenses at start-up included both travel and stipend. Monitoring expenses were the labor and travel expenses that project staff incurred in supervising the helpers and ensuring the girls received the services that the project funded. If implemented more broadly, monitoring could be done by government staff, such as district education officers or nurses. The costs shown would be adequate to fund that monitoring.

Table 2.

Intervention Costs by Cost Category and Year, Undiscounted, and the Standard Deviation of Total Costs at 2009–2010 Tuition Levels

Year	2007	2008	2009	2010	Total
Terms	1	3	3	3	10
# Randomized to Treatment	183	183	183	183	183
# of Pupils In School	183	183	170*	164*	183
Fees (Tuition)	$4,076	$8,282	$19,019	$24,978	$56,355
Uniforms/shoes		$4,050	$22,048	$520	$26,618
Toiletries			$8,221	$3,465	$11,686
Extra Lessons			$3,487	$1,013	$4,500
School Supplies	$1,426	$5,542	$10,920	$3,986	$21,874
Textbooks		$28,186	$2,596		$30,781
Emergency Fund		$2,745	$342	$276	$3,363
Headmaster Honoraria	$195	$405	$660	$660	$1,920
Helper Honoraria	$300	$900	$1,140	$960	$3,300
Monitoring	$2,500	$7,500	$7,500	$7,500	$25,000
Helper Training	$2,236				$2,236
Total Excluding Boarding	$10,733	$57,610	$75,932	$43,358	$187,633
Per Pupil	$59	$315	$415	$237	$1,025
Standard Deviation	6.8	31.7	104.7	99.6	144.9
# of Pupils Boarded	4	4	73	77	80
Formal Boarding	$840	$1,680	$32,412	$36,000	$70,932
Trunks for Boarders			$3,500		$3,500
Informal Boarding			$11,556	$12,750	$24,306
Total Boarding Expenses	$840	$1,680	$47,468	$48,750	$98,738
Per Pupil Boarded **	$210	$420	$650	$633	$1,234
Grand Total	$11,573	$59,290	$123,400	$92,108	$286,372
Per Pupil Randomized	$63	$324	$674	$503	$1,565
Standard Deviation	38.0	85.6	420.8	342.4	548.8

^*Excludes 4 orphans whose families chose to pay their fees in high school.

^**In this row, the divisor is the number of pupils boarded during the year and varies by cell. Hence the numbers in the row do not sum to the total.

Outcome Measurement and Analysis

Table 3 compares the percentage distribution of EQ-5d responses by dimension between girls in the treatment and control groups. Orphans receiving assistance had significantly less problems than controls on every dimension at the 90% confidence limit or better. Table 4 shows that QALY scores of assisted girls averaged 0.055 QALYs higher than for control girls. If we use the Thai EQ-5d scoring rather than the Zimbabwe scoring, the difference would be even larger, 0.066 QALYs. Girls in the treatment group also were married for significantly less time and completed significantly more grades of schooling.

Table 3.

Percentage Distribution of EQ-5d Responses by Dimension

	Mobility		Self-Care		Usual Activities		Pain		Anxiety/Depression
	Treatment	Control	Treatment	Control	Treatment	Control	Treatment	Control	Treatment	Control
None	86.6%	76.5%	78.7%	61.6%	85.3%	74.8%	73.2%	65.8%	68.3%	53.5%
Some	13.4%	23.5%	18.3%	31.3%	11.7%	21.7%	23.2%	25.4%	22.6%	36.8%
Extreme	0.0%	0.0%	3.0%	7.1%	3.1%	3.5%	3.7%	8.8%	9.1%	9.6%
N	164	115	164	112	163	115	164	114	164	114
p-value	.020		.003		.018		.061		.040

^*Regression-adjusted for design effects and age.

T = treatment, C = control

Table 4.

Mean QALY Scores, Years Married, and Grades Completed, Treatment Group vs. Control Group, and Standard Error of Difference between Groups

	Treatment	Control	Stnd Error	p-value @
QALY Score – Zimbabwe Scoring	.807	.752	.0187	.005
QALY Score – Thai Scoring	.678	.612	.0235	.007
Cases	163	112
Years Married	.114	.219	.0551	.035
Cases	178	137
Grades Completed	9.163	8.679	.1054	.004
Cases	180	138

@ = p with 1-tailed test. Marital status was unknown for 3 girls who dropped out of school and were lost to follow-up. The QALY instrument was not administered to most of the married girls who had dropped out of school or to girls lost to follow-up.

Boarding had no effect on outcomes within the intervention group. The mean QALY scores of the 79 girls who boarded versus the 80 who were still attending school but not boarding did not differ, with mean scores of 0.813 for boarders and 0.806 for non-boarders (p = .732). The boarders also had no significant or sizable difference in their probability of marrying early (5.4% per year for boarders in grades 9–10 versus 5.1% for non-boarders, p=0.899) or dropping out of school (7.6% per year in grades 9–10 for boarders versus 6.4% for non-boarders, p=0.883). Thus, the $1,234 incremental cost of boarding a student yielded no offsetting benefits. Boarding added cost but no benefit. It was not cost-effective.

Cost-Effectiveness and Uncertainty Analysis

Table 5 summarizes the net cost, QALY gain, and cost-effectiveness estimates. It shows estimates for all girls in the treatment group and estimates restricted to girls who did not board. We did not tabulate a separate cost-effectiveness estimate for boarders since boarding was not cost effective when compared to orphan assistance without boarding.

Table 5.

Intervention Costs, Estimated Cost and QALY Gains, and Estimated Cost per QALY Gained (discounted to present value using a 3% discount rate unless otherwise stated; in 2010 US Dollars; estimates for boarders are not shown because boarding cost more than tuition support without boarding but it did not improve outcomes.)

	All	Non-Boarders
Cost	1,474*	973*
Return on Added Education (RAE)
Low RAE Estimate	395	395
Medium RAE Estimate	1,028	1,028
High RAE Estimate	1,384	1,384
HIV Treatment Savings	299	299
Net Cost (Cost – RAE – HIV Savings)
Probabilistic Mean Estimate (Best)	2	(502)
Standard Deviation	682	470
Probabilistic Median Estimate	(27)	(453)
Low RAE Estimate	780	279
Medium RAE Estimate	147	(354)
High RAE Estimate	(209)	(710)
QALY Gain
QALY Gain as Student	0.05	0.05
QALY Gain from HIV Reduction	0.337	0.337
Total, Base Case	0.387	0.387
Probabilistic Best Estimate of Total	0.36	0.36
Standard Deviation	0.23	0.23
Cost per QALY Gained (Net Cost/QALY Gain)
At a 3% Discount Rate
Probabilistic Median Estimate (Best)	(120)	(1,327)
Computed from Probabilistic Means	6	(1,394)
Low RAE Estimate	2,016	721
Medium RAE Estimate	380	(915)
With Thai QALY Scoring	373	(895)
If 2010 QALY Gains Persist	324	(778)
High RAE Estimate	(540)	(1,835)
At a 7% Discount Rate
Low RAE Estimate	3,880	2,009
Medium RAE Estimate	1,805	(66)
High RAE Estimate	−912	(2,782)

^*Unlike the estimates in Table 2, these cost estimates have been discounted to present value.

The estimated net cost of the intervention is equal to its cost minus the offsetting return on added education (RAE) and HIV treatment savings. The RAE and HIV treatment cost savings were identical for all girls and girls who did not board. Using our mid-range point estimates and a 3% discount rate, over the 10 terms, the net cost was $147 per girl assisted ($1,474 - $1,028 - $299). The Crystal Ball simulation model accounted for the conservativeness of our initial assumptions, importantly our estimate of life expectancy with HIV. It suggested that a better estimate of net cost was lower, just $2 per girl. Among students who were not boarded, the intervention provided a net cost saving— estimated dollar savings exceeded intervention costs.

Our point estimate is that QALY gain averaged 0.39 QALYs per girl, but probabilistic analyses suggest 0.36 QALYs is the most likely gain. Intervention resulted in QALY gains for participants in 98.5% of the simulation estimates, so our confidence in this finding is very high.

The lower section of Table 5 shows a range of estimates of cost per QALY gained by the intervention. Almost all estimates suggest that the intervention yielded net cost savings for girls who were not boarded. Our best estimate of the intervention’s cost-effectiveness including boarders is the simulation median estimate that the savings will exceed the costs. The cost/QALY simulation yields three other findings.¹ First, dividing the mean net cost by the mean QALY gains yields an estimate of $6/QALY gained. These estimates based on the distributions of the cost, effectiveness, and benefit value estimates are well below our original point estimate of $380/QALY gained. Second, in the simulations, 52% of the net cost estimates for all participants and 87% of estimates for non-boarders yielded net cost savings. Essentially, we can say, for example, that there is an 87% probability that the intervention will offer net cost savings without boarding. Third, the WHO Commission on Macroeconomics and Health (2001) recommends that in low and middle income countries, any intervention where QALY gains cost less than three times Gross Domestic Product (GDP)/capita should be viewed as cost-effective. In the simulations, 78% of cost/QALY estimates for all participants and 97% of estimates for non-boarders were less than three times Zimbabwe’s $591 GDP/capita.

Table 5 also shows sensitivity analysis results on cost/QALY gained that rely on different assumptions than the main analysis. The intervention is far less attractive when the present value of the costs and benefits is computed with a 7% discount rate. At that discount rate, the intervention without boarding still offers net cost savings but the intervention as implemented with almost half of the girls boarded in high school exceeds the cost-effectiveness threshold.

Discussion

This study demonstrates the importance of cost effectiveness analyses as an integral part of prevention intervention research in several ways. First, it points to ways to adapt an intervention with the policy goal of maintaining or maximizing effectiveness while minimizing cost. In this study, although intuitively one would assume that boarding would add greater benefits in health outcomes for orphan girls, the assumption was not supported. Boarding had no incremental effect on years of marriage, years of schooling or QALY levels. On the other hand, it cost an additional $1,234 per pupil boarded on average.

The intervention clearly meets the WHO cost-effectiveness threshold test and merits consideration for widespread dissemination. The CEA showed that the program as implemented would cost $1,474 per pupil over 3.33 years at current prices (with costs adjusted to present value at a 3% discount rate). Our best estimate of the direct cost savings in HIV medical costs and education-related earnings was $1,472 per pupil ($1,474 - $2, from Table 5); the probability that orphan assistance yielded net cost savings was 52%.

Cost was much lower without boarding, just $973 per pupil. Our best estimate is that societal savings from intervention exceeded costs by $502 per pupil. The probability that the intervention yielded a net societal cost saving for non-boarders was 87%. Society saves more from this intervention than it spends to implement it. Furthermore, the intervention still would yield net cost savings if its effectiveness dropped by as much as 34% in dissemination (502/(502+973), from Table 5).

Lack of additional benefit with boarding is an important and somewhat surprising finding, since orphans often live in tenuous circumstances. As a structural intervention, boarding offered an environment that included food, shelter, and greater opportunities for study, such as resources for reading after dark. Boarding also eliminated the time needed to walk home and may have reduced household work responsibilities. Yet overall scores for QALY and other outcomes were essentially the same for boarders and non-boarders, providing reasonably strong evidence for policymakers and philanthropic funders to consider.

Our experience suggests further ways to reduce costs of the intervention for scale-up. For example, buying textbooks for individual students is less efficient than buying them for schools; we discontinued the practice after the first full year. We also decided that shoes could be foregone, since they are not required as part of the uniform in most rural schools where children are used to walking barefoot. At $30/pair, children quickly wear them out or outgrow them. We recommend that scale-up studies randomly assign different “doses” (e.g., school fees only versus fees plus uniforms versus fees plus uniforms plus school supplies) to determine the most effective intervention dose to help the largest number of orphans within the severe cost constraints of a sub-Saharan country like Zimbabwe.

QALYs

Another benefit of CEA analysis to prevention science is the introduction of QALYs as an outcome of interest. Although the key outcomes for this randomized controlled trial were HIV risk factors, QALYs added the dimension of an overarching public health benefit gained by an intervention. We found QALYs a very useful metric for capturing the health-related benefits of school support. We were surprised to find that measures of pain, mobility, self-care, and ability to conduct usual activities would be significantly different by condition for youth of this age. These quality of life indicators, along with the more intuitive anxiety and depression emotional states, were significantly worse for the control group, representing typical orphan teenage girls in Zimbabwe, -- many of whom were regularly chased away from school for lack of ability to pay -- compared to those who had fees, uniforms, and school supplies.

A QALY is defined as one year of life in perfect health. After three years of school-related support, the average intervention student experienced an immediate gain in quality of life of 0.055 QALYs, which equated to a 7.3% increase in quality of life above the control students’ levels. How large an effect is a one-year differential of 0.055 QALYs? It is about half of the mean annual long-term QALY loss due to the lasting disability resulting from a traumatic brain injury or upper extremity fracture treated in the emergency department or admitted to hospital in the Netherlands (Polinder et al., 2007).

The immediate 0.055 QALY gain is conservative because girls who were married were not asked to complete the EQ-5d if they were not in school. Those unpolled girls, most of them in the control group, had greatly elevated risks of HIV infection. Furthermore, one of the married girls in the control group had already died during childbirth. We did not account for either her loss of a full QALY in 2010 or her larger loss of life expectancy because our trial was not powered to detect mortality rate differences.

The immediate 0.065 QALY gain estimated with the Thai scoring of the EQ5d was slightly larger than the gain estimated with the Zimbabwe scoring. The Thai scoring uses an algorithm that is non-linear and consistently yields lower scores for bad health states than the Zimbabwe scoring. Although the score differential between treatment and control girls was similar with the Zimbabwe and Thai scoring, the absolute and perfect-health score levels for both groups were much lower on the Thai scoring, so the estimated percentage gain in health-related quality of life for treatment girls was larger, 10.6% vs. 7.3%.

In present value terms, estimated long-term gain was 0.36 QALYs per girl assisted counting only HIV-related gain and short-term gain in 2010. That is equivalent to adding 1.2 years (present value 0.36 years) to the 40 year life expectancy of the girls when the trial began.

Conservatively, our primary estimate ignored that the QALY gain in 2010 probably was preceded by a smaller gain in 2009 and probably persisted for at least one or two years. The estimated gain would be much larger if we accounted for those more speculative gains. If half as much short-term gain was achieved in 2009 and the gain persisted until 2011, each girl instead would gain 0.45 QALYs.

Outcome Measurement

The primary outcome measures in this trial were odds of marriage and odds of school dropout at specific times post-intervention. Both measures declined significantly as hypothesized (Hallfors et al., 2011), but neither was ideal for economic analysis. We therefore developed more compact secondary outcome measures that capture not only probability but timing, namely months married and grades of school completed (with half a grade arbitrarily assigned for those currently enrolled in a grade). These measures improved significantly as hypothesized. Much of the benefit from these outcomes lies in the future when reduced HIV costs and increased earnings materialize. We relied on the literature to estimate and measure these long-term impacts. The modeled long-term impacts obviously are more uncertain than the measured short-term impacts.

Limitations

Our study has several limitations that suggest future research directions. It lacked biomarker testing for HIV. This forced us to rely on the literature to estimate the STD consequences of the early marriages averted. If this intervention is further developed and tested for widespread dissemination – and the CEA suggests it should be – direct measurement of HIV status would be important. Boarding was not randomly assigned at the start of the project but instead was a function of whether one attended a primary school that was proximal to a Methodist boarding school, or the distance from one’s residence to a participating secular school. Since boarding affected almost half of treatment study participants and eligibility was likely to be largely random, we believe the study offered a reasonably good test of cost and effect.

Conservatively, we treated all school fee and uniform payments as intervention costs. In reality, relatives of some orphans would have paid these expenses absent the grant. In those instances, the outside payment would merely be a transfer payment; it shifts who pays but not the costs of schooling. We ignored the value of any additional household work that orphan girls would have done if they were not in school or were not boarding.

QALY measurement was only done after a grant supplement added a health economist to the team in 2009. It is unclear if we had baseline equivalence in QALY levels, when QALY gains began, or how they grew over time. It also is unclear how long the QALY gains will persist now that the grant is over. Similarly, it is unclear if the existing gain in school participation will lead to even larger future schooling gains. Follow-up surveys of the current treatment cohort are needed to probe these issues. The school impacts are especially important because the program’s cost-effectiveness is more sensitive to school outcomes than HIV outcomes.

Summary

CEA is rarely conducted in prevention studies, yet there are many benefits beyond the obvious policy choices. CEA provides clearer opportunities for paring down and fine-tuning effective programs so that only those elements that are most efficacious in relation to cost are included in the “best buy” package. Estimates of cost/QALY gained are the primary basis for pharmaceutical approval in most developed countries (Clement et al., 2009) and are commonly reported in clinical trials of treatment. QALY gains are more difficult to report in trials of preventive interventions because the prevention benefits and related QALY gains often do not materialize until several years after service delivery. For that reason, as in this study, QALY gains from prevention often must be modeled from short-term outcomes like delayed marriage or delayed initiation of substance use. Nevertheless, routine development of cost/QALY estimates for prevention trials would make it easier to understand the relative value of prevention and treatment and build a case that makes preventive interventions more acceptable and relevant.