Impact of The Challenge Initiative on Public-Sector Family Planning Service Provision in Pakistan: Evidence from a Difference-in-Differences Analysis

Abstract

Purpose

The study assesses the impact of The Challenge Initiative (TCI) on public sector provision of modern family planning (FP) services in 15 urban districts from two provinces of Pakistan. TCI partnered with the government’s Departments of Health (DOH) and Population Welfare Department (PWD) for implementation of high-impact interventions.

Patients and Methods

TCI began implementation in Punjab province in June 2022 and in Sindh province in September 2022. Monthly service data were obtained using the contraceptive Logistics Management Information System (cLMIS) from June 2020 to June 2025. Four districts were selected for the study: districts Rawalpindi (Intervention) and Gujrat (Control) from Punjab province and districts Malir (Intervention) and Sukkur (Control) from Sindh province. A difference-in-differences (DiD) analysis estimated the net effect of the intervention on average monthly family planning clients while interrupted time-series analysis (ITSA) assessed changes in trends before and after intervention.

Results

DiD analysis showed a statistically significant overall effect of 3,673 clients (p < 0.01) relative to control districts across provinces and government departments. In the Department of Health (DOH), an estimated statistically significant net effect of 6,100 clients (p < 0.001). ITSA trends aligned with DiD findings, showing sustained increases in client volume in intervention districts compared to declining trends in controls.

Conclusion

The study provides credible evidence on the positive impact of TCI on provision of modern family planning services using government departments’ facilities and infrastructure. Findings across provinces and departments underpin the effectiveness of public-private partnerships in strengthening family planning service delivery in urban areas. Collectively, these findings provide a strong basis for institutionalization of TCI’s high-impact interventions in provincial budgets, workplans, and policies.

Introduction

Background

Pakistan remains one of the largest contributors to unintended pregnancies and unsafe abortions with more than 6 million unintended pregnancies and 3.8 million abortions taking place annually.¹ This stems from the low and stagnant modern contraceptive prevalence rate (CPR) of 25%, with only 50% of demand satisfied, resulting in 9.5 million married women of reproductive age (MWRA) having an unmet need for family planning.^2,3

Further investigation into the CPR reveals a heavily skewed method mix with male condoms and female sterilization accounting for two-thirds of all method use while the contribution of long-acting reversible contraception (LARCs) is less than three percent.³ This skewness is inconsistent with the reported desire for fertility as 60% of women who reported a desire to limit births were using short-acting methods (SAM).³ A one-year discontinuation rate of 30% further compounds the problem with most women citing side effects, unavailability of the method, and poor counseling.³

Family Planning Service Provision in Pakistan

The failure to meet demand for family planning is striking considering the plethora of public- and private sector investment into family planning since the 1990s.⁴ On the private sector side, donor-driven voucher-based programs and social franchising initiatives led to short-term increases in contraceptive use, particularly in rural areas.^5–8 However, these interventions were not able to create sustainable impact once the donor funding ceased.⁸ Government’s Lady Health Worker (LHW) was initially successful in improving contraceptive coverage, but the program has faced chronic funding and supervision issues, reducing its effectiveness; moreover, family planning has been deprioritized in the LHW agenda as LHWs have been increasingly tasked with immunization, malaria, and other health programs.^9–11 Moreover, the Sindh government launched the People’s Primary Healthcare Initiative (PPHI) to upscale provision of primary healthcare, including family planning, in rural districts of the province.^12,13

Despite a dedicated ministry in Population Welfare Department (PWD) and favorable policies, Pakistan’s population has grown rapidly and has now become the fifth-most populous country in the world with a population of 241 million.² This gap between policy and implementation is further exacerbated by the country’s governance and administrative structure whereby the burden of implementation falls from the provincial level to the district level, and with 166 districts across the four provinces and three territories, the implementation gap widens with increased fragmentation.²

The Challenge Initiative

To bridge this gap, The Challenge Initiative (TCI) began implementation of its High-Impact Interventions (HIIs) in 15 urban districts across Pakistan covering 28% of the national population and 53% of the country’s urban population.² Under the central management of the Johns Hopkins Bloomberg School of Public Health, TCI functions as a global platform, with operations in 13 countries in Africa and Asia.^14–17 In Pakistan, Greenstar Social Marketing (GSM) has led the implementation of TCI Model since June 2022 in partnership with the government’s provincial departments of Population Welfare and Health as well as PPHI.¹⁸ TCI partners with PPHI in two implementing districts ie., Malir – Karachi and Hyderabad. PPHI is only operational in these two TCI districts as the remaining districts are predominantly urban and fall outside the purview of PPHI’s operations. These high-impact interventions focus on underserved urban areas and were piloted and tested under the Urban Reproductive Health Initiative (2010–2015) which operated in Nigeria, Kenya, and Senegal.^14–17,19 TCI’s service delivery HIIs include: (1) Family Health Days (FHDs) – Dedicated facility-based days for providing comprehensive family planning services;^20,21 (2) Integrated Outreaches through mobile outreach vans and satellite camps;²² (3) FP Integration into vaccination and disease prevention contacts;²³ (4) Post-Partum Family Planning;²⁴ (5) Post-Abortion Family Planning;²⁵ (6) On-the-Job Training for clinical staff;²⁶ (7) Whole Site Orientation - facility-wide training sessions for all staff, including janitorial and security personnel, to enhance their understanding of modern family planning methods and improve client facilitation;²⁷ and (8) Facility Makeover – Infrastructural enhancements and renovations of healthcare facilities, supported by financial contributions from TCI.²⁸ The primary HII under demand generation is Community Health Workers (CHWs) operationalized either through capacity building of existing government CHWs or recruitment of community health volunteers for uncovered areas.^29,30

TCI commenced in Punjab province (Faisalabad, Gujranwala, Lahore, Rawalpindi) in June 2022 and expanded to six Karachi districts, Hyderabad, and Islamabad Capital Territory by December 2022, with Multan and Khanewal joining under the Rapid Scale Initiative in December 2023.^18,31 The Rapid Scale Initiative (RSI) is a two-year adaptation of The Challenge Initiative (TCI) three-year model that aims to achieve rapid scale and measurable impact within two years by accelerating the adoption of high-impact family planning interventions. It combines the main elements of the TCI model ie, targeted technical assistance, catalytic funding, and government co-investment to strengthen local system capacity and swiftly transition implementation ownership to government for self-reliance and sustainability.³¹ TCI’s strategic framework centers on strengthening the government’s capacity to implement high-impact interventions, and advocating for institutionalization of these HIIs into governments’ work plan, policies, and budget.^{14,15,17,18,32}

Research Questions

While global evidence exists on TCI implementation, there is currently no robust evidence on the impact of TCI in Pakistan. Furthermore, similar interventions to TCI in Pakistan, have not generated evidence on their impact across public-sector service delivery cadres. The study poses three research questions:

1. What is the net effect of TCI on average monthly family planning client volumes in intervention districts relative to control districts across public-sector departments and provinces?

2. Did TCI produce a measurable change in the level and trend of family planning service delivery post-implementation?

3. Do programmatic effects differ across implementing departments (DOH, PWD, PPHI), and between Punjab and Sindh provinces?

Difference-in-differences (DiD) is well-suited to this evaluation since the programme was implemented in selected districts while control districts were not exposed to the intervention, enabling comparison with appropriate counterfactuals. DiD is complemented by Interrupted Time Series Analysis (ITSA) which discerns the impact of the intervention on the client volume trends at the point of intervention and in the subsequent post-intervention period. The two approaches combine to provide evidence of the intervention through trends and net causal effects.

This study thus aims to use difference-in-differences (DiD) and interrupted time-series analysis (ITSA) to ascertain the effectiveness of TCI interventions across the two implementing provinces, Sindh, and Punjab, disaggregated by the three implementing public-sector departments, DOH, PWD, and PPHI. The findings have major implications for family planning service delivery through public-private partnerships and platforms.

Material and Methods

Study Design

The study employed a quasi-experimental design using secondary data with pre- and post-intervention results with matched control districts. TCI implementation began in June 2022 cross four Punjab districts and in September 2022 across seven Sindh districts providing a consistent implementation timeframe which would allow for causal inferences at the provincial and district levels. From this pool of 11 districts, one intervention district per province was selected for the study against one matched control district.

District Selection and Matching

Since TCI Pakistan had selected the most urban districts in each province as the intervention districts, the only districts that remained were predominantly rural.^2,19 Control districts were identified through a pool of urban non-TCI district in each province then matched with an intervention district on urban population proportion and pre-intervention service delivery trends using Propensity Score Matching (PSM).^33,34 Table 1 illustrates the characteristics of the districts selected for the evaluation.

Table 1.

Comparison of Average Urban Population

District	Province	Group	Urban Population (%)	Average Monthly Clients Pre-Intervention (SD)	Pre-trend Correlation with Matched Control
Rawalpindi	Punjab	Intervention	73%	12,923 (11,715)	0.89***
Gujrat	Punjab	Control	41%	10,147 (11,841)	-
Malir	Sindh	Intervention	48%	1,353 (2,011)	0.44*
Sukkur	Sindh	Control	50%	2,443 (1,921)	-

Note: *p<0.05, ***p<0.001.

In Punjab province, Gujrat district was selected as the control, being the most urban non-TCI district at 41%. Rawalpindi was selected as the intervention district because it had the closest pre-intervention client volume trends to Gujrat among the four Punjab TCI districts (Pearson r = 0.89 across 24 pre-intervention months), indicating strong parallel pre-trends.

In Sindh province, Sukkur district was selected as the control at 50% urban. The six fully urban Karachi districts (>80% urban) were excluded as structurally incomparable to Sukkur. District Malir was selected as the intervention district because, at 48% urban, it was the closest match in urbanicity to Sukkur among the Sindh TCI districts, and its pre-intervention trend was significantly correlated with Sukkur’s (Pearson r = 0.44, across 27 pre-intervention months).

Data Source

Data were extracted from the contraceptive Logistics Management Information System (cLMIS) from June 2020 to June 2025. cLMIS is a national database recording district-level department-wise data on contraceptive dispensation to clients.³⁵ Commodity dispensation data for the four evaluation districts was extracted on each contraceptive method.

Intervention Description

Table 2 describes the frequency of TCI high-impact interventions across departments and districts. TCI supported the implementation of these interventions leveraging existing government staff and infrastructure. Prior to implementation, TCI conducted extensive capacity building of government officials at the district and provincial as well as health workers and service providers on TCI interventions. As part of TCI’s model, TCI advocated for institutionalization of these interventions within the government’s policy, workplan, and budget and was successful in doing so in the intervention districts.

Table 2.

TCI High-Impact Interventions Scale and Intensity

Indicator	District Rawalpindi		District Malir
	DOH	PWD	DOH	PWD	PPHI
Implementing Facilities	72	34	6	16	20
Family Health Days (FHDs) Conducted	1,392	1	149	76	478
Outreach Camps Conducted	1,214	365	4	114	3
Health Workers Trained	346	100	41	36	318
Community Health Workers (CHWs) Deployed	56	0	19	21	13
Facility Makeovers	5	2	5	1	0

Outcome Measures

The main outcome was the monthly number of family planning clients served per department and per district. Since cLMIS reports on the number of commodities dispensed to clients at health facilities, data were extracted on each contraceptive method separately, and were converted into estimated client numbers using conversion factors for condoms (10 condoms = 1 client) and oral contraceptive pills (3 pill cycles = 1 client). This was derived from the Couple-Years of Protection (CYP) USAID benchmark of 120 condoms per year and studies which cite that generally clients receive three pill cycles on a monthly visit.^36–38 For all other methods (injectables, IUDs, implants, and sterilization), the study treated one unit dispensed as equal to one client, since these methods were administered or inserted on a per-client basis. These conversion assumptions represent population-level averages and may not uniformly reflect individual practice, hence client volumes should therefore be interpreted as estimates. Client volume was aggregated by the type of method adopted ie., short-acting methods (SAM), long-acting reversible contraceptives (LARCs), and permanent methods (PM). Total and average client volumes alongside Couple-Years of Protections were used for primary analysis.

Data Analysis

Data were analyzed using two analytical techniques: Difference-in-Differences (DiD) and Interrupted Time-Series Analysis (ITSA). STATA/SE 14.1 (StataCorp, College Station, TX) was used for the analysis.

Difference-in-Differences (DiD)

Pre- and post-intervention periods were defined relative to intervention initiation (June 2022 in Punjab province, September 2022 in Sindh province). Selected districts were comparable as they were matched using urban proportion and parallel pre-intervention trends.^33,34 Parallel trends were verified using slope and intercept of regression lines fitted to the pre-intervention period.³⁴

DiD models were estimated separately for each province. Within Punjab province, the study Rawalpindi (intervention) with Gujrat (control).

DiD₁ = (Rawalpindi _post – Rawalpindi _pre) – (Gujrat _{post –} Gujrat _pre)

Within Sindh, Malir (intervention) was compared with Sukkur (control).

DiD₂ = (Malir _post – Malir _pre) – (Sukkur _{post –} Sukkur _pre)

A pooled analysis was conducted whereby the two TCI intervention districts were compared with the two control districts.

DiD₃ = (TCI _post – TCI _pre) – (Control _{post –} Control _pre)

Each estimator was stratified by department and contraceptive method. DOH and PPHI outcomes included short-acting methods (injectables, pills, EC, condoms), the SAM composite, LARCs (IUD, implants), the LARCs composite, and total clients. PWD outcomes additionally included male sterilisation, female sterilisation, and the PM composite. In total, 108 DiD models were estimated (33 for Punjab, 42 for Sindh, 33 for the pooled analysis).

Interrupted Time Series Analysis

ITSA complemented DiD by using the full monthly time series (June 2020 to June 2025) to identify discrete level and slope changes at the point of intervention. The model was specified as:

Y_dt =β₀ + β₁T_t + β₂D_d + β₃P_t + β₄ (T_t × P_t) + β₅ (D_d × P_t) + β₆ (D_d × T_t × P_t) + ε_dt

where:

• Y_dt = outcome (monthly family planning clients) in district d at time t

• T_t = time since start of observation

• D_d = intervention district indicator (1 = treated, 0 = control)

• P_t = post-intervention indicator (1 = post, 0 = pre)

• β₃ and β₄ capture post-intervention level and slope changes in control districts

• β₅ and β₆ capture additional post-intervention changes attributable to the intervention (the ITSA effects of interest)

Standard errors were adjusted for serial correlation using Newey-West corrections with lag selection based on Cumby-Huizinga tests. Seven ITSA models were estimated: three for Punjab province (Rawalpindi vs. Gujrat, stratified by DOH, PWD, and overall), and four for Sindh province (Malir vs. Sukkur, stratified by DOH, PWD, PPHI, and overall). ITSA models were not estimated for the national analysis due to the varying timelines of intervention implementation.

Statistical Power

Each DiD comparison uses 24–27 pre-intervention and 34–37 post-intervention monthly observations per district. For ITSA, published methodological guidance recommends a minimum of eight data points in both the pre- and post-intervention segments to obtain stable segmented regression estimates.^39,40 All seven ITSA models substantially exceed this threshold.

Ethics

The study protocol was reviewed and approved by Research Ethics Committee of Research and Development Solutions (RADS) (Ref: No. RADS/IRB-GSM/26-11-2024/067). Since aggregated and de-identified data were analyzed, no individual consent was required. Permissions were obtained from the provincial Departments of Health and Population Welfare in Punjab and Sindh.

Results

Provincial Analysis – Punjab Province

Districts Rawalpindi vs. Gujrat, Department of Health

The baseline period (June 2020–May 2022) coincides with the COVID-19 pandemic, during which both districts de-prioritized family planning. ITSA (Figure 1 and Table 3) confirmed a significant and large immediate level change (β2 = 15,707; 95% CI [7,168, 24,247]; p<0.001), indicating a sharp positive increase at TCI implementation. DiD analysis showed statistically significant net gains in injectables, pills, SAM composite, and overall clients (net effect 8,726; p<0.001; Table 4). The post-intervention slope change was non-significant (β₃ = −142/month; 95% CI [−515, 232]; p=0.457), suggesting the gain was sustained at the new level rather than continuing to steepen. Notably, both districts had a declining pre-intervention slope (β₁ = −128/month for Rawalpindi).

Figure 1.

Interrupted Time Series Analysis of Total Clients: DOH Rawalpindi (Intervention) vs. Gujrat (Control). Observed and fitted trends in client volume for Rawalpindi (intervention district, red and yellow lines) and Gujrat (comparison district, blue and green lines) among DOH facilities in Punjab. Solid lines represent observed values and dashed lines represent fitted values from interrupted time series models. The vertical dashed line indicates the timing of intervention implementation (June 2022).

Table 3.

ITSA Summary – Level and Slope Changes in Client Volume

Comparison	Pre-slope β1	Level Change β2 (95% CI)	p (β2)	Slope Change β3 (95% CI)	p (β3)
Rawalpindi vs Gujrat — DOH	−128	15,707 [7,168, 24,247]	0.000	−142 [−515, 232]	0.457
Rawalpindi vs Gujrat — PWD	−38	7,017 [−1,844, 15,878]	0.121	460 [−560, 1,480]	0.377
Rawalpindi vs Gujrat — Overall	−166	22,724 [9,754, 35,695]	0.001	318 [−786, 1,423]	0.572
Malir vs Sukkur — DOH	−3	208 [−690, 1,107]	0.650	423 [227, 618]	0.000
Malir vs Sukkur — PWD	−98	721 [−130, 1,573]	0.097	−43 [−155, 69]	0.456
Malir vs Sukkur — PPHI	−16	1,017 [640, 1,394]	0.000	68 [35, 102]	0.000
Malir vs Sukkur — Overall	−117	1,947 [747, 3,147]	0.001	448 [201, 695]	0.000

Table 4.

Difference-in-Differences (DiD) Analysis of Average Monthly Clients: Rawalpindi vs. Gujrat, Department of Health

Punjab DOH	Control		Intervention		Absolute Difference		Net Effect
	Baseline	Endline	Baseline	Endline	Control	Intervention	Estimate	95% Confidence Interval
Injectables	-	833	442	4,150	833***	3,708***	2,875***	[1,612, 4,136]
Pills	-	1,082	474	6,182	1,082*	5,708***	4,626***	[3,059, 6,195]
EC	-	-	5	-		−5	−5	[-11, 1]
Condoms	-	3,544	1,253	5,598	3,544***	4,345***	801	[-2,143, 3,742]
SAM	-	5,366	2,175	15,930	5,366***	13,754***	8,388***	[3,907, 12,685]
IUD	-	580	473	1,265	580***	792***	212	[−85, 509]
Implants	-	20	1	240	20**	239	219	[−291, 727]
LARCs	-	600	475	1,505	600***	1,030**	430	[−181, 1,042]
Overall	-	6,059	2,649	17,435	6,059***	14,785***	8,726***	[4,016, 13,437]
CYP	-	3,437	2,491	9,160	3,437***	6,669***	3,232**	[822, 5,642]

Note: *p<0.05, **p<0.01, ***p<0.001.

A visible decline in Rawalpindi DOH client volume from 2024 onward reflects disruptions to government’s staffing and facility reporting in cLMIS following Punjab’s administrative restructuring in 2024–25. The DiD net effect captures the full post-intervention average and remains substantially positive.

Districts Rawalpindi vs. Gujrat, Population Welfare Department

Both districts experienced post-intervention declines in client volume reflecting reduction in PWD’s budgetary allocation to procure commodities. The intervention produced a net positive effect of 5,100 (p<0.05; Table 5) as Gujrat’s decline exceeded Rawalpindi’s. ITSA (Figure 2) showed a non-significant level change (β₂ = 7,017; 95% CI [−1,844, 15,878]; p=0.121) and a non-significant slope change (β₃ = +460/month; 95% CI [−560, 1,480]; p=0.377), consistent with a gradual rather than abrupt improvement pattern.

Table 5.

Difference-in-Differences (DiD) Analysis of Average Monthly Clients: Rawalpindi vs. Gujrat, Population Welfare Department

Punjab PWD	Control		Intervention		Absolute Difference		Net Effect
	Baseline	Endline	Baseline	Endline	Control	Intervention	Estimate	95% Confidence Interval
Injectables	2,265	1,814	2,686	2,706	−451*	20	471	[−104, 1,048]
Pills	2,326	2,806	2,986	2,694	480	−292	−772	[−1,577, 31]
EC	343	400	368	624	57	256***	199*	[36, 361]
Condoms	14,693	7,499	15,848	13,649	−7,194***	−2,199	4,994*	[598, 9,392]
SAM	19,626	12,519	21,887	19,672	−7,107***	−2,215	4,892	[−14, 9,798]
IUD	661	971	1,042	1,479	310**	437**	127	[−201, 455]
Implants	2	4	69	23	2	−46**	−48**	[−78, −19]
LARCs	663	975	1,111	1,502	312**	391**	79	[−248, 405]
Male Sterilization	–	–	-	–
Female Sterilization	6	37	197	359	31***	162***	131***	[82, 179]
PM	6	37	197	359	31***	162***	131***	[82, 179]
Overall	20,296	13,532	23,196	21,533	−6,763***	−1,663	5,100*	[106, 10,095]
CYP	5,391	6,524	9,650	12,859	1,132	3,209**	2,077*	[74, 4,079]

Note: *p<0.05, **p<0.01, ***p<0.001.

Figure 2.

Interrupted Time Series Analysis of Total Clients: PWD Rawalpindi (Intervention) vs. Gujrat (Control). Observed and fitted trends in client volume for Rawalpindi (intervention district, red and yellow lines) and Gujrat (comparison district, blue and green lines) among PWD facilities in Punjab. Solid lines represent observed values and dashed lines represent fitted values. The vertical dashed line indicates the timing of intervention implementation (June 2022).

Due to the high demand for condoms and implants and the reduced volume of procurement, the declines reflect commodity shortages rather than programme failure.

Districts Rawalpindi vs. Gujrat, Pooled

Pooled across departments, Rawalpindi showed significant net gains for injectables, pills, SAM, and overall clients (net effect 6,913; p<0.001; Table 6). ITSA (Figure 3) confirmed a highly significant and large immediate level increase (β₂ = 22,724; 95% CI [9,754, 35,695]; p=0.001), with a non-significant post-intervention slope change (β₃ = +318/month; 95% CI [−786, 1,423]; p=0.572).

Table 6.

Difference-in-Differences (DiD) Analysis of Average Monthly Clients: Rawalpindi vs. Gujrat

Punjab Pooled	Control		Intervention		Absolute Difference		Net Effect
	Baseline	Endline	Baseline	Endline	Control	Intervention	Estimate	95% Confidence Interval
Injectables	1,133	1,323	1,564	3,428	191	1,864***	1,673***	[857, 2,489]
Pills	1,163	1,944	1,730	4,438	781*	2,708***	1,927**	[821, 3,033]
EC	171	200	187	312	29	125*	96	[−49, 242]
Condoms	7,347	5,521	8,551	9,623	−1,826	1072	2,898	[−868, 6,663]
SAM	9,813	8,942	12,030	17,800	−871	5,770**	6,641**	[1,886, 11,302]
IUD	331	776	758	1,372	445***	614***	169	[−79, 418]
Implants	1	12	35	131	11**	96	85	[−169, 339]
LARCs	332	788	793	1,503	456***	710***	254	[−108, 617]
Male Sterilization	–	–	-	-
Female Sterilization	6	37	197	359	31***	162***	131***	[4, 127]
PM	6	37	197	359	16***	81*	65*	[4, 127]
Overall	10,147	9,795	12,923	19,484	−352	6,561***	6,913**	[2,007, 11,820]
CYP	2,696	4,980	6,070	11,009	2,285***	4,939***	2,654**	[671, 4,638]

Note: *p<0.05, **p<0.01, ***p<0.001.

Figure 3.

Interrupted Time Series Analysis of Total Clients: Pooled Rawalpindi (Intervention) vs. Gujrat (Control). Observed and fitted trends in total client volume for Rawalpindi (intervention district, red and yellow lines) and Gujrat (comparison district, blue and green lines) across all facility types in Punjab. Solid lines represent observed values and dashed lines represent fitted values. The vertical dashed line indicates the timing of intervention implementation (June 2022).

Provincial Analysis – Sindh Province

Districts Malir vs. Sukkur, Department of Health

DiD showed significant net gains for injectables, pills, SAM, and overall clients (net effect 3,023; p<0.05; Table 7). ITSA (Figure 4) revealed a non-significant level change (β₂ = 208; 95% CI [−690, 1,107]; p=0.650) but a highly significant and sustained slope increase (β₃ = +423/month; 95% CI [227, 618]; p<0.001) indicating progressive rather than immediate service delivery growth.

Table 7.

Difference-in-Differences (DiD) Analysis of Average Monthly Clients: Malir vs. Sukkur, Department of Health

Sindh DOH	Control		Intervention		Absolute Difference		Net Effect
	Baseline	Endline	Baseline	Endline	Control	Intervention	Estimate	95% Confidence Interval
Injectables	191	591	647	3,005	400**	2,358**	1,958*	[282, 3,634]
Pills	109	146	146	515	37	369*	332*	[31, 634]
EC	22	13	–	141	−9	141	150	[−4, 304]
Condoms	42	203	152	408	161**	256*	95	[−131, 320]
SAM	363	952	945	4,069	589**	3,124**	2,535*	[561, 4,510]
IUD	16	61	20	515	45	495	450	[−43, 943]
Implants	–	31	–	70	31*	70	39	[−38, 116]
LARCs	16	92	20	586	76	566*	490	[−9, 987]
Overall	379	1,045	966	4,655	666**	3,689**	3,023*	[747, 5,301]
CYP	149	595	297	3,538	446*	3,240*	2,794*	[254, 5,335]

Note: *p<0.05, **p<0.01.

Figure 4.

Interrupted Time Series Analysis of Total Clients: DOH Malir (Intervention) vs. Sukkur (Control). Observed and fitted trends in client volume for Malir (intervention district, blue and green lines) and Sukkur (comparison district, red and yellow lines) among DOH facilities in Sindh. Solid lines represent observed values and dashed lines represent fitted values. The vertical dashed line indicates the timing of intervention implementation (September 2022).

Erratic trends visible in Malir DOH in 2024 are attributable to documented injectable and oral pill commodity stockouts in Karachi’s public sector supply chain in the second half of 2024. Recovery is evident in early 2025 data, confirming a supply-side disruption rather than programme failure. Sukkur’s trajectory remained flat throughout, with no significant level or slope change.

Districts Malir vs. Sukkur, Population Welfare Department

Significant net effects were limited to injectables (+430; p<0.01). Negative significant net effects were observed for IUDs, implants, and LARCs (Table 8). ITSA (Figure 5) showed no significant level or slope change for Malir PWD (β₂ = 721; 95% CI [−130, 1,573]; p=0.097; β₃ = −43/month; 95% CI [−155, 69]; p=0.456), consistent with non-significant overall DiD.

Table 8.

Difference-in-Differences (DiD) Analysis of Average Monthly Clients: Malir vs. Sukkur, Population Welfare Department

Sindh PWD	Control		Intervention		Absolute Difference		Net Effect
	Baseline	Endline	Baseline	Endline	Control	Intervention	Estimate	95% Confidence Interval
Injectables	2,000	1,808	775	1,013	−192	238	430**	[120, 741]
Pills	1,227	805	698	412	−422*	−286**	136	[−82, 353]
EC	397	274	370	170	−123*	−200***	−77	[−220, 68]
Condoms	907	767	1,102	425	−140	−677**	−537	[−1,185, 111]
SAM	4,531	3,654	2,945	2,020	−877***	−925**	−48	[−788, 693]
IUD	44	83	67	54	39***	−13	−52**	[−81, −21]
Implants	73	129	57	68	56***	11	−45**	[−73, −16]
LARCs	118	212	123	122	94***	−1	−95**	[−141, −50]
Male Sterilization	-	-	-	-	-
Female Sterilization	71	67	26	28	−4	2	6	[−13, 25]
PM	71	67	26	28	−4	2	6	[−13, 25]
Overall	4,720	3,935	3,094	2,173	−785**	−921**	−136	[−896, 626]
CYP	2,053	2,242	1,234	1,194	189	−40	−229	[−562, 105]

Note: *p<0.05, **p<0.01, ***p<0.001.

Figure 5.

Interrupted Time Series Analysis of Total Clients: PWD Malir (Intervention) vs. Sukkur (Control). Observed and fitted trends in client volume for Malir (intervention district, blue and green lines) and Sukkur (comparison district, red and yellow lines) among PWD facilities in Sindh. Solid lines represent observed values and dashed lines represent fitted values. The vertical dashed line indicates the timing of intervention implementation (September 2022).

The negative net effects are attributable to the commodity shortages experienced by PWD with greater focus being given to distribution of LARCs in rural areas.

Districts Malir vs. Sukkur, People’s Primary Healthcare Initiative

PPHI services in Malir were non-existent pre-TCI; TCI activated PPHI facilities from September 2023 onwards. DiD showed significant gains in pills, EC, IUDs, implants, and LARCs (Table 9). ITSA (Figure 6) confirmed a significant immediate level change (β₂ = 1,017; 95% CI [640, 1,394]; p<0.001) and a significant positive slope (β₃ = +68/month; 95% CI [35, 102]; p<0.001), together indicating both a structural break at programme activation and sustained subsequent growth.

Table 9.

Difference-in-Differences (DiD) Analysis of Average Monthly Clients: Malir vs. Sukkur, PPHI

Sindh PPHI	Control		Intervention		Absolute Difference		Net Effect
	Baseline	Endline	Baseline	Endline	Control	Intervention	Estimate	95% Confidence Interval
Injectables	1,058	1,608	–	541	550***	541***	−9	[−229, 211]
Pills	231	361	–	441	130***	441***	311**	[125, 496]
EC	–	–	–	24	-	24*	24*	[6, 42]
Condoms	450	1,073	-	194	623***	194***	−429**	[−624, −234]
SAM	1,739	3,042	-	1,200	1,303***	1,200***	−103	[−584, 376]
IUD	341	322	-	80	−19	80***	99*	[10, 188]
Implants	149	143	-	109	−6	109***	115**	[45, 185]
LARCs	490	465	-	189	−25	189***	214***	[113, 315]
Overall	2,229	3,508	-	1,389	1,279	1,389***	110	[−438, 657]
CYP	2,480	2,590	-	1,024	110	1,024***	914**	[417, 1,414]

Note: *p<0.05, **p<0.01, ***p<0.001.

Figure 6.

Interrupted Time Series Analysis of Total Clients: PPHI Malir (Intervention) vs. Sukkur (Control). Observed and fitted trends in client volume for Malir (intervention district, blue and green lines) and Sukkur (comparison district, red and yellow lines) among PPHI facilities in Sindh. Solid lines represent observed values and dashed lines represent fitted values. The vertical dashed line indicates the timing of intervention implementation (September 2022).

The significant negative net effect for condoms (−429; p<0.01) reflects Sukkur PPHI’s scale-up of condom distribution (from 450 to 1,073) against Malir PPHI starting from zero and reaching 194.

Districts Malir vs. Sukkur, Pooled

Malir showed significant gains for injectables, pills, and LARCs, with an overall net effect of 1,000 (p<0.05; Table 10). ITSA (Figure 7) showed a significant immediate level change (β₂ = 1,947; 95% CI [747, 3,147]; p=0.001) and the largest slope change across all models (β₃ = +448/month; 95% CI [201, 695]; p<0.001), confirming both an immediate step-up and a sustained accelerating trajectory that exceeds control-district growth.

Table 10.

Difference-in-Differences (DiD) Analysis of Average Monthly Clients: Malir vs. Sukkur, Pooled

Sindh Pooled	Control		Intervention		Absolute Difference		Net Effect
	Baseline	Endline	Baseline	Endline	Control	Intervention	Estimate	95% Confidence Interval
Injectables	1,083	1,336	474	1,520	253*	1,046**	793*	[152, 1,434]
Pills	522	437	281	456	−85	175*	260**	[72, 447]
EC	139	96	123	112	−43	−11	32	[−58, 123]
Condoms	466	681	418	342	215*	−76	−291*	[−565, −16]
SAM	2,211	2,550	1,297	2,430	339	1,133**	794	[−105, 1,694]
IUD	134	156	29	217	22	188*	166	[−9, 341]
Implants	74	101	19	82	27	63***	36	[−3, 76]
LARCs	208	256	48	299	48	251**	203*	[21, 384]
Male Sterilization	-	-	-	-
Female Sterilization	35	34	13	14	−1	1	2	[−10, 14]
PM	35	34	13	14	−1	1	2	[−10, 14]
Overall	2,443	2,829	1,353	2,739	386	1,386**	1,000*	[−4, 2,003]
CYP	1,561	1,809	511	1,919	248	1,409**	1,160*	[222, 2,099]

Note: *p<0.05, **p<0.01, ***p<0.001.

Figure 7.

Interrupted Time Series Analysis of Total Clients: Pooled Malir (Intervention) vs. Sukkur (Control). Observed and fitted trends in total client volume for Malir (intervention district, blue and green lines) and Sukkur (comparison district, red and yellow lines) across all facility types in Sindh. Solid lines represent observed values and dashed lines represent fitted values. The vertical dashed line indicates the timing of intervention implementation (September 2022).

National Analysis

For DOH nationally (Table 11), significant net effects were observed for injectables, pills, SAM, IUDs, LARCs, and overall clients (net 6,100; p<0.001). For PWD nationally (Table 12), significant net gains were limited to injectables and tubal ligation; implants showed a significant negative net effect (−43; p<0.01), prioritization of implant distribution in rural areas. In the pooled national analysis (Table 13), significant gains spanned injectables, pills, SAM, IUDs, LARCs, and permanent methods; overall net effect was 3,673 (p<0.01) with 4,121 additional clients in intervention districts (p<0.001).

Table 11.

Difference-in-Differences (DiD) Analysis of Average Monthly Clients: Intervention vs. Control, DOH

National DOH	Control		Intervention		Absolute Difference		Net Effect
	Baseline	Endline	Baseline	Endline	Control	Intervention	Estimate	95% Confidence Interval
Injectables	101	717	550	3,602	616***	3,052***	2,436***	[880, 2,695]
Pills	58	633	300	3,469	575*	3,169***	2,594***	[764, 2,714]
EC	11	6	2	68	−5	66	71	[2, 311]
Condoms	22	1,944	671	3,112	1,922***	2,441***	519	[−1,502, 1,306]
SAM	192	3,252	1,524	10,250	3,060***	8,726***	5,666***	[1,018, 6,154]
IUD	8	331	234	906	323***	672***	349*	[−111, 443]
Implants	–	25	1	159	25**	158	133	[−113, 302]
LARCs	8	356	235	1,065	349***	831***	482*	[−99, 619]
Overall	201	3,658	1,758	11,315	3,457***	9,557***	6,100***	[1,054, 6,652]
CYP	79	2,076	1,116	5,025	1,997***	3,909***	1,912*	[212, 3617]

Note: *p<0.05, **p<0.01, ***p<0.001.

Table 12.

Difference-in-Differences (DiD) Analysis of Average Monthly Clients: Intervention vs. Control, PWD

National PWD	Control		Intervention		Absolute Difference		Net Effect
	Baseline	Endline	Baseline	Endline	Control	Intervention	Estimate	95% Confidence Interval
Injectables	2,125	1,811	1,674	1,896	−314**	222	536*	[98, 867]
Pills	1,744	1,848	1,775	1,601	104	−174	−278	[−764, 332]
EC	371	340	369	407	−31	38	69	[−217, 48]
Condoms	7,395	4,275	8,041	7,316	−3,120*	−725	2,395	[−524, 6,107]
SAM	11,635	8,274	11,859	11,219	−3,361*	−640	2,721	[−973, 6,922]
IUD	335	546	526	797	211*	271*	60	[−235, 305]
Implants	40	64	63	44	24*	−19	−43**	[−63, 17]
LARCs	374	610	588	841	236**	253	17	[−244, 269]
Male Sterilization	-	-	-	-
Female Sterilization	40	52	106	201	12	95**	83**	[38, 141]
PM	40	53	106	202	13	96**	83**	[37, 141]
Overall	12,000	8,936	12,553	12,262	−3,064*	−291	2,773	[−1,090, 7,240]
CYP	3,624	4,473	4,775	6,880	849	2,105**	1,256	[−624, 3,135]

Note: *p<0.05, **p<0.01.

Table 13.

Difference-in-Differences (DiD) Analysis of Average Monthly Clients: Intervention vs. Control, Pooled

National Pooled	Control		Intervention		Absolute Difference		Net Effect
	Baseline	Endline	Baseline	Endline	Control	Intervention	Estimate	95% Confidence Interval
Injectables	1,101	1,330	879	2,322	229*	1,443***	1,214***	[428, 1,301]
Pills	761	1,071	820	2,130	310*	1,310***	1,000**	[129, 1,095]
EC	151	139	147	196	−12	49	61	[−62, 125]
Condoms	3,026	2,716	3,444	4,244	−310	800	1,110	[−561, 2,521]
SAM	5,040	5,238	5,291	8,893	198	3,602**	3,404**	[381, 4,595]
IUD	207	416	300	702	209***	402***	193*	[−83, 232]
Implants	47	64	25	103	17	78	61	[−32, 134]
LARCs	254	480	325	805	226***	480***	254*	[−53, 304]
Male Sterilization	-	-	-	-
Female Sterilization	26	35	69	136	9	67**	58*	[8, 66]
PM	26	35	69	136	9	67**	58*	[8, 66]
Overall	5,310	5,758	5,658	9,779	448	4,121***	3,673**	[412, 4,888]
CYP	1,983	3,142	2,396	4,845	1,159***	2,449***	1,290*	[183, 2,396]

Note: *p<0.05, **p<0.01, ***p<0.001.

Discussion

This quasi-experimental study used difference-in-differences (DiD) and interrupted time-series analysis (ITSA) to assess the impact of The Challenge Initiative (TCI) on public sector service provision of family planning services in two urban districts of Pakistan. The study provides robust evidence that TCI high-impact interventions strengthened public sector service delivery to fulfill greater demand for modern contraceptives in Rawalpindi (Punjab) and Malir (Sindh) districts.

Differential Performance of Departments

Both intervention and control districts exhibited declining pre-intervention trends reflecting the COVID-19 period during which family planning was deprioritized; TCI interventions were able to reverse these trends with greater effect. In DOH, district Rawalpindi grew from near-zero baseline to peak monthly volumes exceeding 17,000 clients (Figure 1), while Malir DOH surpassed 15,000 monthly clients against Sukkur’s plateau at approximately 5,000 (Figure 4). These trends reflect TCI’s role in reorienting DOH which has historically focused on pandemic response, immunization, and disease campaigns towards sustained family planning service delivery.^18,41,42

Contrastingly, in PWD facilities, commodity shortages and frequent stockouts limited the impact of TCI’s HIIs; periodic stockouts in PWD facilities are well-documented in previous studies as well.^35,43,44 These weaknesses in supply chain undercut the effect of TCI’s technical coaching, particularly in Malir district where negligible increases were observed.

As discussed earlier, the People’s Primary Healthcare Initiative (PPHI) focuses on service delivery in rural areas, and in the pre-intervention phase had not begun operations in Karachi city, specifically, Malir district.¹² TCI played a critical role in initiating service delivery through PPHI in Malir district by advocating for activation of PPHI operations in the predominantly rural district.

Provincial Context

Punjab province has been undergoing major structural changes since 2023 which affected the performance of Rawalpindi district. In late 2024, the Punjab government merged DOH with PWD to create a new unified Department of Health and Population.⁴⁵ This marked a major institutional shift with PWD staff and facilities absorbed into DOH facilities and campaigns. Moreover, the government decided to renovate and outsource the network of primary healthcare facilities to private consultants on a pay-for-performance model. This took place in 2025 severely affecting service delivery. The Punjab government as part of their reforms also introduced a new health management information system (HMIS) called electronic medical record (EMR) which is a client-level database.^45,46 Facility staff have been directed to prioritize reporting on EMR over DHIS2 and there currently is no linkage between EMR and DHIS2 or cLMIS thereby creating a parallel stream of reporting. This de-prioritization of reporting on DHIS2 and cLMIS is reflected in the decline in client volumes in 2025. In contrast, the system in Sindh is relatively stable with both departments operating independently with a strong focus on family planning from the leadership.

Global Comparisons

The study’s findings align with global literature which show that programs focusing on health system strengthening are more effective than short-term vertical projects.^47–50 The Urban Reproductive Health Initiative (URHI) pilot demonstrated considerable gains across Kenya, Nigeria, Senegal, and India through integration of demand generation and service delivery high impact intervention in public sector healthcare facilities.¹⁹ TCI as the successor to URHI has found significant gains as well in East and Francophone West African countries further emphasizing the role of local government financing and ownership, and TCI coaching in sustaining gains beyond the three-year implementation cycle.^14,17,51 TCI implementation in Nigeria found evidence that local governments with greater financial commitment and institutional support were able to effectively scale-up high-impact interventions.^15,17,51

The effectiveness of TCI in initiating effective service delivery through PPHI aligns with global evidence on the role of decentralized primary healthcare structures in expanding family planning service provision.^52–55 Similar approaches in Africa where semi-autonomous primary healthcare networks were capitalized on for delivering family planning services which resulted in measurable gains in terms of usage of long-acting contraception.^53,54 In comparison, vertical programs with donor support such as social franchising or voucher programs delivered results in the short-run, but found it difficult to institutionalize these gains beyond the funding cycle.^8,56 Evidence from neighboring India and Bangladesh also underpin the value of integrating family planning within broad primary and public healthcare systems.^57,58 A recent study conducted in rural Sindh found similar results whereby they found 11% increase in contraceptive uptake when embedded within public sector maternal and child healthcare delivery.⁵⁹

Policy Implications

The study’s findings have several implications for Pakistan’s family planning policy landscape. Firstly, TCI’s performance in DOH facilities in both Punjab and Sindh provinces highlight the benefits of embedding family planning into the broader public and primary healthcare agenda. DOH should leverage its extensive infrastructure and reach to scale-up TCI’s high-impact interventions into other districts, provinces, and territories for substantial gains. Secondly, the universal decline in contraceptive uptake across PWD facilities underscore the importance of addressing the persistent supply chain bottlenecks and stockouts, particularly for short-acting methods. PWD should prioritize strengthening their logistics systems to ensure uninterrupted commodity supply to leverage the TCI high-impact interventions. Lastly, the Punjab government should ensure the continuation of their priority on family planning while undergoing structural reforms. The government should leverage the support received from the World Bank for the Punjab Family Planning Program to continue to scale-up TCI high-impact interventions for sustained gains.^45,46 Collectively, government departments should continue to demonstrate high ownership with financing to ensure scale-up of TCI high-impact interventions for catalyzing Pakistan’s progress towards achieving FP2030 commitments.

Limitations

The study acknowledges limitations that should be taken into consideration in the interpretation of the findings. The study uses secondary data from the cLMIS which may be affected by inconsistent or delayed reporting. The study undertook validation exercises during the data cleaning process; however, there may be underreporting and misclassification that cannot be detected.

The evaluation was restricted to four districts which were selected on the basis of urban population proportion and pre-intervention trend matching. While this study was designed to improve comparability, it limited generalizability to rural districts. This was partially mitigated by the selection of district Malir which has an urban population proportion of less than 50%. While the study used robust analytical methodologies, it could not account for confounders such as local health campaigns, stockouts, etc. Moreover, any other client-level indicators which could influence outcomes were also unavailable.

Client reporting and volumes were severely affected in Punjab by the structural changes and reforms which made it difficult to separate the interventional effect from systemic reforms. Lastly, the study assessed outcomes in terms of client volumes as opposed to traditional indicators such as contraceptive prevalence, continuation, or quality of care. Future studies should conduct population-based surveys and compare differences in prevalence between TCI and non-TCI districts to provide definitive evidence on TCI in Pakistan.

The Pakistan Demographic and Health Survey (PDHS) 2017–18 reported that a majority of condom and pill users obtain contraceptives from private sources.³ The reported proportions were even higher in urban areas which were the focus of the TCI intervention as well.^3,60 Since demand-generation was a critical aspect of TCI’s high-impact interventions, the program may have resulted in an increase in private sector service delivery as well leading to an underestimation of TCI’s overall impact.

Future Research Direction

Future research should build on these findings through complementary study designs. Population-based surveys comparing contraceptive prevalence between TCI and non-TCI districts would confirm whether facility-level gains translate into household-level CPR increases. Method continuation studies using client-level tracking systems, such as Punjab’s nascent EMR once linked to cLMIS, would clarify whether clients acquired through TCI-supported outreach remain in the family planning system over 12 months, directly addressing Pakistan’s documented 30% one-year discontinuation rate. Qualitative implementation research exploring why TCI coaching generates stronger gains in DOH than in PWD facilities would inform targeted model refinements for supply-chain-constrained settings. Finally, a sustainability study comparing districts that have institutionalised TCI HIIs into government budgets and workplans against those that have not would provide direct evidence on the durability of gains as implementation cycles conclude.

Conclusion

This study aimed to assess the impact of The Challenge Initiative on public sector service delivery of modern contraceptive methods, and found significant effects particularly for Department of Health facilities where both short-acting and long-acting methods showed significant gains. At the national pooled level, TCI was associated with a net increase of 3,673 additional family planning clients per month (p<0.01), with a stronger effect of 6,100 additional clients per month in DOH facilities (p<0.001). Performance in Population Welfare Department facilities was more variable showing modest gains largely due to the dips in supply. By embedding interventions in government systems, TCI achieved results that are more likely to be sustainable than those of donor-driven vertical programs. These results also underpin the importance of institutionalizing TCI high-impact interventions in government’s policy, budget, and workplan. While this study focused on urban districts, the inclusion of Malir district with less than 50% urban population suggests that the TCI model may have relevance for peri-urban and rural settings. Furthermore, these findings suggest that strengthening Department of Health, reforming PWD, and engaging partners like PPHI is critical to accelerating progress towards Pakistan’s FP2030 commitments.

Funding Statement

This work was supported by the Gates Foundation and Bayer AG through the William H. Gates Institute for Population and Reproductive Health at the Johns Hopkins Bloomberg School of Public Health (Award # INV-032545 & 123709). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Abbreviations

CHV, community health volunteer; CHW, community health worker; cLMIS, contraceptive logistics management information system; COVID-19, coronavirus disease 2019; CPR, contraceptive prevalence rate; CYP, couple-years of protection; DiD. difference-in-differences; DOH, Department of Health; EC, emergency contraception; EMR, electronic medical record; FP, family planning; HII/HIIs, high-impact intervention(s); HMIS, health management information system; IRB, institutional review board; ITSA, interrupted time-series analysis; IUD, intrauterine device; LARC/LARCs, long-acting reversible contraceptive(s); LHW, lady health worker; MNCH, maternal, newborn, and child health; MWRA, married women of reproductive age; PM, permanent method(s); PPFP, post-partum family planning; PPHI, People’s Primary Healthcare Initiative; PSM, propensity score matching; PWD, Population Welfare Department; RADS, Research and Development Solutions; RSI, Rapid Scale Initiative; SAM, short-acting method(s); TCI, The Challenge Initiative; URHI, Urban Reproductive Health Initiative; USAID, United States Agency for International Development.

Disclosure

The authors report no conflicts of interest in this work.