Longitudinal Evaluation of the Tupange Urban Family Planning Program in Kenya

Abstract

CONTEXT

Levels of fertility and contraceptive use have long fluctuated in Kenya. The multicomponent Tupange program, part of the Urban Reproductive Health Initiative, was initiated in 2011 to increase use of modern family planning methods.

METHODS

Women aged 15–49 in the five Kenyan cities where Tupange was implemented were interviewed in 2010 and reinterviewed in 2014 to obtain information on their contraceptive use and exposure to components of the Tupange program. Fixed-effects models were estimated to identify associations between program exposure and use of modern family planning methods. Analyses were performed to determine the relative cost-effectiveness of program components.

RESULTS

During the four-year follow-up period, the proportion of women using modern contraceptives increased from 45% to 52%, and the proportion of users who were using long-acting or permanent methods rose from 6% to 19%. The fixed-effects model indicated that modern method use was associated with having heard Tupange-related local radio programming and marginally associated with having discussed family planning with a community health worker (CHW); among women who were unmarried or did not give birth during the study period, modern method use was associated with living near program facilities. Local radio programming was the most cost-effective program component, followed by proximity to Tupange facilities and discussions with CHWs.

CONCLUSIONS

Urban reproductive health programs seeking to increase use of modern family planning methods in Kenya and other Sub-Saharan African settings should consider multicomponent approaches that include CHW activities, local radio programming and improvements to the supply environment.

Kenya is a complex, challenging setting for family planning programming. In the late 1970s, the country’s total fertility rate (TFR) was eight children per woman, one of the highest national rates observed at the time.^1–5 By 1989, Kenya’s TFR had begun to decline in conjunction with increasing use of contraceptives,^4–6 eventually reaching an estimated 4.7 children per woman in the 1998 Demographic and Health Survey (DHS).⁷ However, the 2003 DHS indicated that the contraceptive prevalence rate had plateaued (at 39% in both 1998 and 2003) and that the TFR had increased slightly, to 4.9.⁸ On the basis of these findings, family planning advocates in Kenya pushed for the government to increase access to family planning and reproductive health services, particularly for poor and vulnerable populations.^9–11 By 2008, the TFR had declined to 4.6 and the contraceptive prevalence rate had increased to 46%,¹² suggesting that trends were again on track in the desired direction.

Use of long-acting and permanent methods (LAPMs) of family planning fluctuated during this period. In 1993, two-fifths of users of modern family planning methods were using a LAPM. By 2003, only one-quarter of modern method users were using a LAPM, and the 2008 DHS showed a continued decline in LAPM use.⁹ This prompted a renewed focus on LAPMs by the Kenyan government, which expanded projects to train providers, ensure commodity security, partner with private-sector organizations and create demand for LAPMs.¹³

A complicating factor for meeting family planning needs in Kenya is the high rate of population growth in urban areas, fueled by both high fertility and rural-to-urban migration. In developing countries, many urban dwellers reside in informal settlements (also called slums) where they have tenuous or no property rights and poor living conditions.^14–16 In Nairobi, about half of the population lives in informal settlements.¹⁷ Residents of slums have an elevated risk of mortality and morbidity from communicable diseases because of overcrowding, inadequate nutrition, and poor water, hygiene and sanitation. Although the health and well-being of residents of urban areas is thought to be better than that of rural residents, this urban advantage is not equally distributed; the urban poor tend to lack access to quality health and social services.^18,19

In this context, the Bill & Melinda Gates Foundation developed the Urban Reproductive Health Initiative in 2009, with the goal of increasing use of modern contraceptives in urban areas of Kenya, Nigeria, Senegal and India. Initiative programs were tasked with five objectives: integrating family planning with other maternal and child health services; improving the quality of services in high-volume facilities; increasing the role of the private sector; increasing and sustaining demand for family planning services; and working to build a supportive advocacy environment for family planning. At the same time, the foundation funded the Measurement, Learning & Evaluation (MLE) Project, based at the Carolina Population Center at the University of North Carolina at Chapel Hill, to undertake an independent impact evaluation of programs in the four countries to determine which demand-side and supply-side intervention components were effective at increasing use of modern family planning methods, particularly among the urban poor.²⁰

The purpose of this article is to present key results from the MLE evaluation of the Urban Reproductive Health Initiative in Kenya. In addition to identifying the program components associated with increases in use of modern methods, we explore whether some program components were more cost-effective than others. The findings are being used to inform future urban family planning programs in Kenya and elsewhere in the region.

METHODS

The Tupange Program

The Kenya Urban Reproductive Health Initiative program, called Tupange (“Let’s plan”), was undertaken by a consortium of partners led by Jhpiego and implemented in five cities: Nairobi, Mombasa, Kisumu, Machakos and Kakamega. These cities were strategically selected on the basis of conversations with the government of Kenya and the program mandate to work in large, urban centers.

Starting in 2011, all Tupange interventions were fully implemented in Nairobi, Mombasa and Kisumu. Program implementation was delayed in Machakos and Kakamega until interim analyses indicated which specific program activities in the other cities appeared to be the most effective; in 2013 the program was implemented in Machakos and Kakamega by the government of Kenya with technical support from Tupange.

The Tupange program tailored the five broad Urban Reproductive Health Initiative objectives to the Kenyan context. It emphasized improving access to quality of family planning services through a diverse set of supply-side activities that focused particularly on increasing use of long-acting reversible contraceptives (LARCs): implants and IUDs. Initial evidence indicated that commodity security was needed because stockouts of LARCs were common. Tupange developed a commodity management system to track supplies and redistribute them to facilities where they were needed. It also trained staff via a novel approach called “whole site training,” in which employees at all levels of a health facility received training on family planning methods so that clients could get accurate information from any staff member with whom they came in contact. Tupange also provided information and hands-on training to providers on provision of LARCs, performed outreach (carried out community activities within a facility’s catchment area) and conducted “in-reaches” (brought additional supplies and providers to facilities) on an ongoing basis at public and private facilities. Last, within facilities, the program integrated family planning services with maternal and child health services and HIV services.

The facilities where the Tupange program worked existed prior to program implementation. The improvements in facility quality and commodity availability were initiated as part of the program, and this evaluation examines the relationship between these improvements and women’s contraceptive use. During the life of the project, Tupange worked in 201 public and private health facilities across the five cities. Sixty-eight of the facilities were small, private facilities in Nairobi and Mombasa that were affiliated with the Amua Tupange social franchise network, an extension of the existing Marie Stopes International– Kenya Amua social franchise system. At the Amua Tupange facilities, Tupange carried out promotional activities, supplied equipment, supported family planning data management and linked commodity distribution to the government’s distribution program through central distribution sites.

In addition, Tupange promoted demand for contraceptives using a comprehensive outreach strategy that included organizing community activities (such as street plays and dramas); training community health workers (CHWs); airing radio and television programs; holding community meetings; and distributing posters, pamphlets and other written materials. Some of these community-based activities targeted specific groups, such as youth or out-of-school populations. As part of the program, Tupange trained a large number of CHWs, including 630 in Nairobi.²¹ Radio programs included Jongo Love, a youth-focused program that aired on seven local radio stations but achieved national coverage in Kenya, and weekly call-in talk shows that featured family planning experts and satisfied users. By the time of the endline evaluation, more than 500 radio shows that discussed family planning had aired, of which 25% covered content on topics relevant to Tupange (e.g., birthspacing, delayed first birth, male responsibility). Finally, the publications that discussed family planning included Shujaaz, an award-winning, youth-targeted comic book distributed monthly in a national newspaper. Between 2010 and 2014, nearly 26 million comic book copies were distributed, almost seven million of which contained Tupange content related to adolescent pregnancy, relationships or male responsibility.

The MLE Project

In 2010, the MLE Project collected baseline data from households and women in the five project cities in Kenya. A representative sample of women was selected from each city using a two-stage sampling design. First, for each city, primary sampling units (enumerated by the 2009 Population and Housing Census) were randomly selected with probability proportional to size; the sample size in each city was proportional to the population of the city. In Nairobi, Mombasa and Kisumu, selection was stratified so that half of selected primary sampling units were informal localities (as designated in the census frame) and half were formal. This was done to oversample the urban poor, who tend to concentrate in informal settlements. Separate sampling frames for formal and informal localities were not available for the other two cities. In all selected primary sampling units, a household listing was completed and a random sample of 30 households was selected from the listing. All women aged 15–49 in selected households were eligible for interview. The total baseline sample across the five cities consisted of 8,932 women (Figure 1).

FIGURE 1.

Flow diagram of baseline and endline samples, Kenya, 2010 and 2014

The endline household survey was carried out in 2014. The 8,850 women who had completed the baseline survey and were usual residents of the household were eligible to participate in the endline survey. Eligible women were tracked within the five study cities using detailed contact information collected at previous survey waves (the baseline survey and a midterm survey conducted in 2012). Respondents who were located were invited to take part in the endline interview; consent was obtained from both the women and the head of their household. In total, 5,217 women were successfully tracked and interviewed at end-line, for an overall response rate of 59% (range, 50% in Nairobi to 68% in Kakamega).

Standardized questionnaires were used at all survey waves to measure the same indicators over time. Survey questions covered such topics as fertility, family planning, migration, media exposure, and access to maternal and child health services. At endline, specific questions about the various program elements were added to measure women’s exposure to each of Tupange’s activities. All study procedures were approved by the Institutional Review Board at the University of North Carolina at Chapel Hill and the Kenya Medical Research Institute Ethical Review Committee. The study design and baseline and endline surveys have been described previously.^22,23

Sample and Variables

Sample

The analytic sample was limited to women who responded to all relevant questions at both baseline and endline. Twenty-four women did not provide information on their marital status, education or religion and were excluded, resulting in a final unweighted sample of 5,193 women, who contributed a total of 10,386 observations (since each woman was observed at baseline and at endline).

Dependent variable

The dependent variable in this analysis was use of modern contraceptives. Women were asked if they or their partner were currently doing something to avoid pregnancy and, if so, what method of contraception they were using. We classified women as using either a modern method (male or female sterilization, implant, IUD, injectable, pill, male or female condoms, or lactational amenorrhea method), a traditional method (rhythm, withdrawal or periodic abstinence) or no method. We also examined change in the use of long-acting and permanent methods (LAPMs): male or female sterilization, implants and IUDs.

Program exposure variables

Table 1 outlines the specific program exposure measures included in this analysis. All were binary indicators of exposure (coded as one for exposed and zero for unexposed) and were measured at endline. Women were asked about their exposure to the Jongo Love radio program; those who reported having heard the program in the past year were considered exposed. Another key Tupange activity was the training and support of CHWs who engaged in outreach activities. Respondents were asked if they had met with a CHW in the past year and, if so, whether they had discussed family planning; those who reported such a discussion were considered exposed to this program component.

TABLE 1.

Program components and measures of exposure used in evaluation

Component	Measure of exposure
Demand
Jongo Love radio program	Heard Jongo Love in the past year
Community health workers	Discussed family planning with a community health worker in the past year
Tupange meeting	Attended meeting about family planning in the past year that was led by someone wearing clothing with Tupange logo
Tupange Imarisha Maisha brochure	Saw brochure with “Tupange Imarisha Maisha” on it in the past year
Tupange ”Celebrate life!” poster	Saw poster with Tupange or “Celebrate life!” on it in the past year
Shujaaz print media	Ever saw or heard of a Shujaaz comic book that was about adolescent pregnancy, relationships or male responsibility
Community events	Heard any information about adolescent pregnancy, relationships, male responsibility, family planning or birthspacing at a community event‡ in the past year
Supply
Tupange/Amua health facility	Lives within 1.5 kilometers of a Tupange or Tupange Amua health facility

^‡Community events include caravan road shows, community drama/puppet shows, sports competitions, beauty contests, bicycle races and public entertainment events.

Tupange carried out a variety of community-based activities, including meetings and events to generate demand for family planning and provide information about where and how to obtain contraceptives. Women were asked whether in the last year they had attended a meeting about family planning that was led by someone wearing clothing with the Tupange logo. Those who responded affirmatively were coded as exposed. Women were also asked if in the past year they had heard information about adolescent pregnancy, relationships, male responsibility, family planning or birthspacing at various community events, including caravan road shows, drama or puppet shows, sports competitions, beauty contests, bicycle races and public entertainment events. Because exposure to specific types of events was low, we grouped them and considered women exposed if they had heard information at any of them.

The Tupange program also developed print media materials. Women were asked about their exposure to the Tupange Imarisha Maisha brochure and the Tupange “Celebrate life!” poster in the past year; we created a variable indicating whether women had seen the brochure and one indicating whether they had seen the poster. Another variable specified whether women had ever seen or heard of the Shujaaz comic book covering topics related to adolescent pregnancy, relationships and male responsibility.

Because no Tupange programs had been initiated at the time of the first survey, baseline exposure to all of the above demand-side program indicators was coded zero. This includes exposure to CHWs; although CHWs were active at baseline, questions about exposure were not included in the baseline survey. This is a limitation of our analysis and is discussed further in the Discussion section.

We did not have data on whether women had been directly exposed to the activities and improvements at Tupange/Amua facilities (e.g., in-reach/outreach activities, improved commodity security, whole site training). As a proxy for exposure to these program elements, we created a supply-side variable that indicated whether women lived near a Tupange program facility at endline. Specifically, we combined Tupange monitoring data on the location of facilities where activities were undertaken with geographic information about respondents’ place of residence; if a woman lived within 1.5 kilometers of a facility where Tupange/Amua had been implemented, she was considered exposed. In creating this variable, we explored several buffer distances; the 1.5-kilometer threshold was selected because it made sense from the standpoint of women’s probable behavior, given the fairly dense settlement patterns in the five program cities. All women were considered unexposed at baseline, because the Tupange program had not yet been launched at relevant facilities.

Women’s characteristics

Finally, our analyses controlled for various characteristics of the women themselves. We included measures of age (categorized as 15–29 or as 30 or older), city (Nairobi, Mombasa, Kisumu, Kakamega or Machakos); religion (Catholic, Protestant/other Christian, Muslim or none); education (none, incomplete primary, complete primary, or secondary or more); and marital status (in union or separated/divorced/widowed/never married). Another variable indicated whether the woman’s parity had increased between baseline and endline. From data on household assets, we created an index of wealth using a DHS-style principle component analysis.²⁴ The wealth index was calculated at each round of the survey; women were divided into quintiles (poorest, poor, middle, rich or richest) across all the households in the five study cities.

Evaluation Analysis

First, we offer descriptive analyses. These simply involved univariate and bivariate comparisons of variable values at baseline and endline, and bivariate comparisons of program exposure between users and nonusers of modern contraceptives at endline. Descriptive samples were weighted using baseline or endline sample weights across the five cities. These weights adjusted for uneven selection probabilities in the baseline multistage design and then for selective attrition between baseline and endline (the latter adjustment involved a regression-based weight class analysis). These weights ensured that our longitudinal sample was representative of the baseline population of our study cities.

We attempted to estimate associations between program exposure and use of modern family planning methods through multivariable analysis. Specifically, we exploited the longitudinal nature of the data to estimate fixed-effects regression models. Because our sample consisted of women for whom we had both baseline and endline measurements, the fixed-effects approach amounts to treating change in an individual’s contraceptive behavior as a function of change in the exposure (policy) variables while adjusting for other variables that changed or may have changed between the two time points (e.g., age, marital status). This differencing reduces bias by removing from the analysis unobserved fixed characteristics of the individuals (e.g., motivation) that may affect or be associated with contraceptive use and the program exposure variables. Similarly, our approach removes any bias resulting from the program’s targeting of underserved areas.

The multivariate analyses are unweighted. Our choice to weight descriptive statistics and not the multivariate results is guided by the work of Solon and colleagues.²⁵ They noted that weighting is called for in descriptive statistics to ensure that the analytic sample is representative of the target population; however, weighting may not be necessary in multivariate models, and may in fact be harmful, because it can inflate the standard errors of coefficient estimates.²⁵

Because the analysis included many program exposure variables and because we hypothesized that the program’s effects might differ by women’s marital status and changes in parity, our modeling strategy was as follows. We started by including all program variables, along with all interactions of these variables with the parity increase and marital status variables. We then simplified by dropping program variables and interactions that were not significantly associated with our outcome. The variables that were time-varying (age, education, wealth, parity change and marital status) remained in the multivariate model; the remaining variables (city and religion) dropped out of the final model.

In our model, if Y_i,t is modern contraceptive use for individual i at time t (where t=0 for baseline and t=1 for endline) and X_ij,t, and P_ik,t are, respectively, controls for a characteristic j (out of J individual characteristics included in the model) of individual i at time t and for k Tupange program exposure variables (out of four program components included in the final model) of individual i at time t, then our basic model is:

$$Y_{i,1}-Y_{i,0}={\beta }_0+{\beta }_1\cdot \left(X_{i1,1}-X_{i1,0}\right)+\cdots +{\beta }_j\cdot \left(X_{iJ,1}-X_{iJ,0}\right)+{\theta }_1\cdot \left(P_{i1,1}-P_{i1,0}\right)+\cdots +{\theta }_4\cdot \left(P_{i4,1}-P_{i4,0}\right)+\phi \cdot \left(P_{ij,1}\cdot X_{ik,1}-P_{ij,0}\cdot X_{ik,0}\right)+{\epsilon }_{i,t}$$

where φ·(P_ij,1·X_ik,1 − P_ij,0·X_ik,0) represents a potential interaction between the exposure to the j^th program component and the K^th individual characteristic. We thus estimated a classic fixed-effects or first- differences model (the two are identical for the two−time-period case considered in this analysis) whereby changes in modern contraceptive use are associated with changes in time-varying individual characteristics and program exposure. Many of the variables that were significantly associated with exposure in the bivariate analyses no longer showed such associations after we adjusted for age, marital status, parity, education and wealth.

Cost-Effectiveness Analysis

Finally, we present results from a cost-effectiveness analysis of program variables that could be costed and that remained in the final multivariate analysis. The MLE Project worked closely with the Tupange program to obtain cost estimates for the various program activities over the four years of program implementation. These costs were linked to the program variables to provide some sense of program component performance once cost was taken into consideration. Some program activities (e.g., branding of Tupange clothing, materials and advertisements) were not included in the model because of the difficulty of teasing apart the actual implementation strategy to obtain accurate cost estimates.

The analysis is relatively simple. After identifying the program activities associated with contraceptive use in the multivariable analysis, we chose one activity to serve as the reference activity against which relative cost and relative effectiveness are measured. For instance, the relative cost (RC) of exposure to Jongo Love relative to discussion of family planning with a CHW would be:

$$R{C}_{\mathit{Jongo}\mathit{Love}}=\frac{\mathit{Cos}{t}_{\mathit{Jongo}\mathit{Love}}}{\mathit{Cos}{t}_{\mathit{CHW}}}$$

Relative cost would be defined similarly for the other program components considered and would be 1 for the CHW component.

Estimating relative effectiveness is slightly more complex. Activity effectiveness (E) captures the changes in behavioral outcomes associated with an activity. We define it as the marginal effect (ME) of the program activity times the probability (Pr) that an individual is exposed to that activity. Thus, if we again use Jongo Love as an example, effectiveness would be:

$$E_{\mathit{Jongo}\mathit{Love}}=\mathrm{Pr}\left(\mathit{Exposed}to\mathit{Jongo}\mathit{Love}\right)\cdot M{E}_{\mathit{Jongo}\mathit{Love}}$$

Where ME_{Jongo Love} is the marginal effect of Jongo Love shown in the impact models below. Intuitively, effectiveness is essentially the change in the behavioral outcome associated with the program component (the marginal effect, which reflects the change in an individual’s outcome associated with exposure to the program), weighted by the probability that that individual (or, more precisely, the type of person that that individual represents, as defined by the individual characteristics X_ij,t included in our model) would be exposed to that program component.

We modeled this exposure probability using a logit regression in which the dependent variable is exposure to the program component and the regressors are the individual characteristics X_ij,t that were included in the multivariate model. The marginal effect is simply the coefficient estimate for that program component from the fixed-effects regression described earlier. We then computed effectiveness as the product of the marginal effect of a program component (from the multivariable regression) and the predicted probability of exposure to that component for each individual, and averaged the result across individuals.

If we use discussion of family planning with a CHW as the reference program component against which to measure the relative effectiveness (RE) of other components, then the relative effectiveness of Jongo Love would be:

$$R{E}_{\mathit{Jongo}\mathit{Love}}=\frac{E_{\mathit{Jongo}\mathit{Love}}}{E_{\mathit{CHW}}}$$

The relative effectiveness of discussion of family planning with a CHW would be 1.

Finally, relative cost-effectiveness is simply relative effectiveness divided by relative cost. For instance, for Jongo Love, the relative cost-effectiveness would be:

$$\frac{R{E}_{\mathit{Jongo}\mathit{Love}}}{R{C}_{\mathit{Jongo}\mathit{Love}}}$$

Because we have used discussion of family planning with a CHW as the reference activity for cost and effectiveness, its relative cost-effectiveness is 1. The relative cost-effectiveness of all other activities is measured relative to discussion of family planning with a CHW; values less than 1 imply that the activity is less cost-effective than discussion of family planning with a CHW, while values greater than 1 imply that it is more cost-effective.

RESULTS

Table 2 presents the characteristics of women surveyed at baseline and endline in the five Kenyan cities. Overall, 59% of baseline participants were reinterviewed at endline. The percentage distribution of women by their baseline characteristics was similar in the two surveys, which may be due in part to the endline weights that control for any selective attrition. As the last column of Table 2 shows, the proportion of women who were found and reinterviewed at endline was positively associated with age, and ranged from 48−50% among women aged 15−24 to more than 75% among women aged 40 or older. The proportion of women who were reinterviewed also varied by marital status, city of residence, wealth and religion, and was particularly low (<52%) among women who had never married, lived in Nairobi, belonged to the richest wealth quintile or reported no religious affiliation.

TABLE 2.

Percentage distribution of urban women aged 15–49 surveyed at baseline and endline, by selected baseline characteristics, according to survey year; and percentage of baseline respondents who were reinterviewed at endline, by baseline characteristics— Urban Reproductive Health Initiative, Kenya, 2010 and 2014

Baseline characteristics	2010 (N=8,845)	2014 (N=5,217)	% reinterviewed
All	na	na	59.0
Age
15–19	11.7	11.6	47.6
20–24	28.7	27.1	50.3
25–29	24.2	25.4	58.0
30–34	14.6	13.9	64.1
35–39	10.4	11.2	70.1
40–44	6.5	6.9	75.8
45–49	4.0	3.9	78.6***
Education
None	3.3	3.2	59.2
Incomplete primary	13.8	13.7	59.0
Complete primary	26.2	26.0	59.6
≥secondary	56.6	57.2	58.6
Missing	0.1	0.0	na
Marital status
Never married	33.7	34.7	48.7
Married/in union	55.6	54.4	64.1
Divorced	7.9	7.7	53.1
Widowed	2.5	3.0	75.1***
Missing	0.3	0.2	na
City
Nairobi	71.9	68.1	49.9
Mombasa	19.8	22.3	59.5
Kisumu	5.1	5.7	58.6
Machakos	1.5	1.8	66.0
Kakamega	1.7	2.1	67.5***
Wealth quintile
Poorest	17.3	16.8	58.7
Poor	19.2	19.9	62.0
Middle	19.5	17.5	60.6
Rich	20.7	22.1	61.5
Richest	23.3	23.7	51.3***
Religion
Catholic	22.3	22.7	57.5
Protestant/other Christian	65.1	63.4	59.2
Muslim	10.6	11.6	62.2
None	1.9	2.2	51.1*
Missing	0.1	0.1	na
Parity
0	30.5	15.3	47.5
1–2	45.6	48.0	57.0
3–4	17.5	27.1	70.1
≥5	6.5	9.6	77.8***
Total	100.0	100.0	100.0

^*p<05 for differences in reinterview rates among subcategories for this characteristic.

^***p<.001 for differences in reinterview rates among subcategories for this characteristic.

Notes: Baseline estimates use baseline weights; endline estimates use endline weights. Percentage reinterviewed is unweighted. na=not applicable.

Between baseline and endline, the prevalence of modern contraceptive use among women surveyed at both time periods increased from 45% to 52% (Table 3). At both baseline and endline, injectables, pills, condoms and implants were the methods most commonly used. The method whose share of use increased the most was the implant, which accounted for 6% of modern method use at baseline and 19% four years later. The shares of use accounted for by injectables, pills and condoms declined between the surveys.

TABLE 3.

Percentage distribution of urban women aged 15–49 who were surveyed at both baseline and endline, by selected measures of contraceptive use, according to survey year, Kenya, 2010 and 2014

Measure	2010	2014
Contraceptive use‡
Modern	44.6	51.8
Traditional	3.8	5.5
None	51.6	42.7
Modern method used§
Injectable	44.4	39.4
Pill	21.5	15.5
Male condom	15.2	11.4
Implant	6.4	19.3
IUD	5.5	6.7
Sterilization	5.3	5.6
Other††	1.8	2.2
Contraceptive use behaviors between surveys†‡
Used no/traditional method at both times	na	36.3
Switched from no/traditional method to modern method	na	23.0
Switched from modern method to no/traditional method	na	12.0
Used modern method at both times	na	28.8
Total	100.0	100.0

^‡Sample size was 4,609 at baseline and 5,185 at endline.

^§Among modern method users; sample size was 2,058 at baseline and 2,685 at endline.

^††Lactational amenorrhea, emergency contraceptives and female condom.

^†‡Sample size was 5,185.

Notes: All sample sizes and percentages are weighted using weights from the relevant survey. Percentages may not total 100.0 because of rounding. na=not applicable.

Examination of use behaviors at baseline and endline reveals that 36% of respondents used a traditional method or no method at both time points. Twenty-three percent of women changed from using a traditional method or no method at baseline to relying on a modern method at end-line, whereas 12% switched from using a modern method to using a traditional method or no method. Finally, 29% of respondents used a modern method at both survey waves.

We also examined changes between baseline and end-line in use of modern methods and use of LAPMs by age-group (Figure 2).^* The biggest increase in modern method use was among women aged 25–29; the proportion of these women who were using a modern method increased from 54% at baseline to 61% four years later. Furthermore, among women in the 25–29, 30–34 and 35–39 age-groups, the proportion of modern method users who were using a LAPM rose (e.g., from 11% at baseline to 30% at endline among women aged 25–29). Conversely, little change in the prevalence of modern method use and LAPM use was evident among the oldest women (aged 45–49).

FIGURE 2.

Percentage of urban women aged 20–49 using any modern method of contraception, and percentage of urban modern method users aged 20–49 relying on long-acting or permanent methods, by age-group, 2010 and 2014

The percentage of women who reported at endline having been exposed to Tupange program activities varied substantially (Table 4). We do not report exposure at baseline because the components of the Tupange program did not exist at the time of the baseline survey, with the exception of exposure to CHWs (whose activities were not specific to Tupange and for whom we do not have data on women’s baseline exposure). The program component for which women reported the lowest level of exposure was the Jongo Love radio program (6%), which specifically targeted young people with its themes about living in urban slums. Exposure to CHWs was also low; at endline, only 9% of women reported having discussed family planning with a CHW in the last year. Thirteen percent of women had seen or were otherwise aware of a Shujaaz comic book depicting the themes of adolescent pregnancy, male responsibility and relationships, and a similar percentage (14%) had attended a meeting at which a program leader was wearing Tupange-branded clothing. Exposure to Tupange brochures and posters was greater; 20% of women had seen Imarisha Maisha brochures and 32% had seen Tupange’s “Celebrate life!” posters. Forty-five percent of women reported that in the last year they had attended a community event at which family planning was discussed; exposure was highest for caravan road shows, community meetings and entertainment events (not shown). Finally, 81% of women lived within 1.5 kilometers of a Tupange/Amua health facility. Exposure to many of these activities varied by city (Appendix Table A).

TABLE 4.

Percentage distribution of urban women aged 15–49 surveyed at both baseline and endline who reported exposure to Tupange program activities at endline, by use of modern contraceptive methods, Kenya, 2014

Program activity	All (N=5,185)	Modern method use
		Yes (N=2,685)	No (N=2,501)
Heard Jongo Love	6.2	6.7	5.6
Discussed family planning with community	9.1	10.5	7.7†
health worker‡
Attended Tupange meeting	14.0	15.7	12.1†
Saw Tupange Imarisha Maisha brochure	20.3	21.0	19.5
Saw Tupange “Celebrate life!” poster	32.4	34.7	29.8*
Aware of Shujaaz comic book††	13.4	12.6	14.2
Attended community events‡,§	44.7	49.3	39.7**
Lives ≤1.5 km from Tupange/Amua facility	81.3	83.4	79.0

^*p<.05.

^**p<.01.

^†p<.10.

^‡Activity was not specific to the Tupange program.

^§Includes caravan road shows, community drama/puppet shows, sports competitions, beauty contests, bicycle races and entertainment events.

^††Refers to comic books with family planning information and other Tupange-relevant content.

Note: All sample sizes and percentages are weighted; weighted total sample size does not equal sum of weighted subgroup sample sizes because of rounding.

Bivariate analyses revealed that the percentage of women who were exposed to specific program elements often differed between users and nonusers of a modern method (Table 4). Compared with nonusers, users reported significantly higher levels of exposure to Tupange “Celebrate life!” posters (35% vs. 30%) and to community events (49% vs. 40%). They also reported greater exposure to CHWs and to Tupange meetings, although these differences were only marginally significant.

The results of our fixed-effects model, which adjusts for demographic characteristics, are expressed as classical marginal effects of each variable on use of modern contraceptive methods (Table 5). In the case of a program variable, the marginal effect expresses the estimated average expected change in the prevalence of use of modern family planning methods if each woman in the sample were switched from having not been exposed to the program component to having been exposed. For individual characteristics, the marginal effect indicates the estimated average expected change in use of modern family planning methods if each woman went from not having the characteristic in question to having it. Finally, the interaction effects essentially estimate the difference in the marginal effects of a program characteristic for two different subpopulations defined by the individual characteristic with which the program variable interacts.

TABLE 5.

Marginal effects from fixed-effects analysis of associations between program activities and use of modern contraceptive methods among urban women aged 15–49

Variable	% point change in method use
PROGRAM COMPONENTS
Heard Jongo Love	7.9 (3.1)*
Discussed family planning with	4.3 (2.3)†
community health worker‡
Aware of Shujaaz comic book	–1.2 (2.6)
Lives ≤1.5 km from Tupange/Amua health facility	2.2 (1.9)
WOMEN’S CHARACTERISTICS
Education level
No education	ref
Incomplete primary	6.3 (5.4)
Complete primary	10.0 (5.7)†
≥secondary	11.8 (6.1)†
Married/in union
No	ref
Yes	12.8 (1.9)***
Age
<30	ref
≥30	1.4 (2.0)
Wealth quintile
Poorest	ref
Poor	3.9 (1.9)*
Middle	0.4 (2.1)
Rich	0.4 (2.4)
Richest	–4.2 (3.1)
Birth during study period
No
Yes	32.8 (2.0)***
INTERACTIONS
Unmarried women
Lives ≤1.5 km from Tupange/Amua health facility	6.2 (2.4)*
Aware of Shujaaz comic book	6.2 (3.8)
Married women
Lives ≤1.5 km from Tupange/Amua health facility	0.2 (2.0)
Aware of Shujaaz comic book	–5.0 (3.3)
Women without birth during study period
Lives ≤1.5 km from Tupange/Amua health facility	5.2 (2.2)*
Women with birth during study period
Lives ≤1.5 km from Tupange/Amua health facility	–9.3 (2.7)***

^*p<.05.

^***p<.001.

^†p<.10.

^‡Activity was not specific to the Tupange program and therefore its impact cannot be attributed solely to Tupange.

Note: ref=reference group.

Jongo Love was the program component that was most strongly associated with contraceptive use in the fixed-effects model. Its marginal effect was 7.9, which we can interpret as meaning that the prevalence of modern method use would increase by about 7.9 percentage points if all women went from being unexposed to the Jongo Love program to being exposed. Furthermore, if all women went from not having discussed family planning with a CHW to having done so, we would expect the prevalence of contraceptive use to increase by 4.3 percentage points; however, this effect was only marginally significant and is likely overstated because we did not measure CHW exposure at baseline. We also found associations for living near a program facility, but only in some subpopulations (see the interaction effects at the bottom of the table). For example, the marginal effect of living within 1.5 kilometers of a Tupange/Amua facility was 6.2 percentage points among unmarried women and 5.2 percentage points among women who did not give birth during the study period.

In reality, the observed effectiveness of a program component would not be as high as its marginal value, since effectiveness takes into account the probability of exposure to that component. For example, because only 6% of women were exposed to Jongo Love, the effect in our study would be a 0.47 percentage point change (7.9*0.06), despite a large marginal effect.

Figure 3 shows the results of the cost-effectiveness analysis for the final model, which includes the program components for which we were able to identify corresponding costs. Overall, Jongo Love was the most cost-effective program activity, followed by the Tupange/Amua facility-level activities and CHW activities.

FIGURE 3.

Relative cost-effectiveness of selected Tupange program activities

DISCUSSION

In Kenya, where the TFR and contraceptive prevalence rate have fluctuated and urbanization is rapid, strong evidence of intervention effectiveness is needed. Our evaluation of the Tupange program can inform future programs. The study focused on a sample of women from urban Kenya who were interviewed in 2010 and again in 2014 to assess exposure to Tupange program components, most of which did not exist prior to program launch in 2011. By using fixed-effects regression, we controlled for selection bias and targeted program placement related to unobserved time-invariant measures, thus avoiding a common limitation of evaluations that do not randomly assign participants to study sites. We found that exposure to the Jongo Love radio program was associated with use of modern contraceptive use, and discussions with CHWs about family planning had a marginally significant association with method use. Furthermore, living within 1.5 kilometers of a Tupange/Amua facility was positively associated with method use among women were who were not living in a union and those who did not experience a birth during the follow-up period. The Jongo Love program was the most cost-effective program component, followed by living near a Tupange/Amua facility and discussions with CHWs.

Use of CHWs was generally a low-cost strategy for the Tupange program, especially since Tupange partnered with the government of Kenya, which was financially responsible for CHWs. Although the coverage of CHWs was generally low, the messages delivered tended to be deeper and more effective than those conveyed by the mass media, perhaps because CHWs could target messages or were able to identify individuals who were most in need. Exposure to the Jongo Love program was strongly associated with contraceptive use, perhaps because those who were exposed to it were young, often unmarried women who had a strong need for family planning; however, the program was not substantially more cost-effective than other Tupange components because of the low exposure to it.

Recent DHS surveys show that modern contraceptive use has been increasing in Kenya. It is important to understand what is driving this increase. Most trend studies from Kenya have used nationally representative data,^1,2,5 whereas evaluation studies have tended to focus on predominantly rural areas,^26–28 and often on special populations (such as adolescents) within those areas.^27,28 One exception is a study by Fotso and colleagues⁹ that used DHS data to examine trends in contraceptive prevalence in urban areas. They found that in 1993, LAPMs made up two-fifths of the method mix, but by 2008 they accounted for only one-quarter of modern method use.⁹ During the intervening period, commodity stockouts were increasingly common and the government’s focus on family planning declined as HIV became an important public health issue. In 2008, Kenya implemented a national strategy to increase use of LAPMs, including training and equipping health workers and ensuring commodity procurement;⁹ these efforts are likely related to the increases in LAPM use observed in our study, which, together with similar increases seen in national data,²⁹ suggest that latent demand existed for LAPMs.

Some limitations of our analysis should be noted. Because of the integrated nature of the Tupange program, disentangling the effect of one program component (such as a particular slogan or message) on contraceptive use from the effects of other program components is a challenge. A Tupange slogan may have been used in a television spot or a radio drama, printed on a billboard or featured on a community worker’s T-shirt at an event. It is also important to note that while our demand-side variables used information from individual respondents (whose responses to very specific questions allow us to say whether they were exposed to particular program components), our supply-side variable (living within 1.5 kilometers of a Tupange/Amua facility) does not provide direct evidence of exposure; we know whether a Tupange/Amua facility was geographically accessible to a woman, but not whether she used that particular (or any) facility. Given the dense supply-side environment in these Kenyan cities, we were not able to survey all of the facilities that respondents may have visited. Thus, our supply-side results (including those of the cost-effectiveness analysis) may be underestimates.

Furthermore, because targeted facilities generally carried out all of Tupange’s supply-side activities rather than various combinations of them, it is impossible to determine if, for example, quality improvements were more important than integration or engagement of private-sector partners in increasing contraceptive use. Another limitation of the findings is that CHW exposure was not measured at baseline, which may have biased our estimate of the marginal effect of this variable.

A final limitation is that at endline we were able to find and interview only 59% of women in the baseline sample. Women who were lost to follow-up may have had different contraceptive use experiences than the women whom we successfully reinterviewed. Future studies that collect longitudinal data in complex urban settings like Nairobi should identify additional strategies for tracking women, such as following them more frequently (e.g., every six months or every year) and obtaining follow-up data using new social media and other connection approaches that are becoming more widely available.

CONCLUSIONS

Policymakers and program managers working in urban areas of Kenya and of other East African countries can learn from the findings of this evaluation of the Tupange program. Our findings suggest that programs in urban areas should consider using CHWs and specific outreach and media activities that will be seen or heard by target women (and men and families). Media activities may include radio programs targeted to specific audiences, although the effect of such programs will be small, as we observed with Jongo Love, unless stations and messages have high coverage. In addition, multicomponent programs like Tupange are needed that address both demand-side components (e.g., creating demand for family planning by addressing desires for large families and providing information on the benefits and side effects of contraceptives) and supply-side activities (e.g., improving service quality and ensuring women’s access to a range of methods). These types of comprehensive urban programs should be promoted to policymakers who seek to increase the use of modern family planning methods with the goal of reducing the TFR, especially in rapidly growing urban populations.

APPENDIX

TABLE 1.

Percentage of urban women aged 15–49 surveyed at both baseline and endline who reported exposure to Tupange program activities at endline, by city

Program component	Nairobi	Mombasa	Kisumu	Machakos	Kakamega
Heard Jongo Love	6.7	4.6	5.2	3.2	12.2
Discussed family planning with community health worker‡	6.9	13.9	14.1	7.0	20.1
Attended Tupange meeting	11.6	21.3	13.0	13.1	15.1
Saw Tupange Imarisha Maisha brochure	20.9	21.2	11.2	14.8	19.8
Saw Tupange “Celebrate life!” poster	31.9	32.6	34.9	25.4	43.9
Aware of Shujaaz comic book	13.5	14.1	10.1	10.5	13.8
Attended community events‡§	43.6	44.1	56.5	44.0	53.1
Lives ≤1.5 km from Tupange/Amua facility	82.2	85.5	69.0	51.5	67.7

^‡Activity was not specific to the Tupange program.

^§Includes caravan road shows, community drama/puppet shows, sports competitions, beauty contests, bicycle races and public entertainment events. Note: All percentages are weighted.