Evaluating a Large-Scale Community-Based Intervention to Improve Pregnancy and Newborn Health Among the Rural Poor in India

Abstract

Objectives. We evaluated the effectiveness of the Sure Start project, which was implemented in 7 districts of Uttar Pradesh, India, to improve maternal and newborn health.

Methods. Interventions were implemented at 2 randomly assigned levels of intensity. Forty percent of the areas received a more intense intervention, including community-level meetings with expectant mothers. A baseline survey consisted of 12 000 women who completed pregnancy in 2007; a follow-up survey was conducted for women in 2010 in the same villages. Our quantitative analyses provide an account of the project’s impact.

Results. We observed significant health improvements in both intervention areas over time; in the more intensive intervention areas, we found greater improvements in care-seeking and healthy behaviors. The more intensive intervention areas did not experience a significantly greater decline in neonatal mortality.

Conclusions. This study demonstrates that community-based efforts, especially mothers’ group meetings designed to increase care-seeking and healthy behaviors, are effective and can be implemented at large scale.

Uttar Pradesh is the most populous state in India and among the poorest, with a per capita gross domestic product of about half the national average.¹ Health indicators in Uttar Pradesh reflect the poor economic conditions: at 63 deaths per 1000 live births in 2009, the state’s infant mortality rate is 25% higher than the country’s.¹

The Sure Start project was designed to improve health outcomes for mothers and newborns by mobilizing communities to practice healthy behaviors and use public health services. The launching of the project was preceded by the National Rural Health Mission (NRHM), introduced in 2005, which placed community health workers, called accredited social health activists (ASHAs), in villages at a ratio of 1 ASHA per 1000 villagers. The project’s main goals were as follows:

• To raise awareness of essential maternal and newborn health care through communication and advocacy activities to promote safe pregnancy and neonatal care, directly at the village level and through mass campaigns at a district level.

• To support households and communities to practice healthy behaviors, through mobilization techniques such as mentoring ASHAs to hold mothers’ group (MG) meetings as well as to improve their skills at promoting institutional delivery.

• To strengthen village health and sanitation committees and linkages with other structures of the Panchayati Raj Institution (a 3-tiered decentralized governance system; its lowest level of governance is the rural panchayat, our unit of intervention, which in Uttar Pradesh usually consists of a main village and surrounding smaller hamlets).

With the cooperation of the Uttar Pradesh government, Sure Start implemented interventions in tandem with planned activities of the NRHM to accelerate the improvement of health in rural areas. The program affected a population of 23 million (> 0.6 million pregnancies)² in 7 districts covering 7112 panchayats. Spanning the end of 2007 to the end of 2011, the project promoted implementation of key NRHM policies related to maternal and newborn health and sought to improve accountability related to the supply of funds, facilities, commodities, and services. Through the NRHM, the government launched several important initiatives to improve maternal and newborn health. The most important was a conditional cash transfer scheme—known as Janani Suraksha Yojana—that incentivizes both ASHAs and families to promote and use health facilities for delivery.

The motivation for this project came from growing evidence—shown in small-scale randomized control trials (< 500 000 people), many of which were in South Asia—that a significant decrease in neonatal mortality and morbidity can be achieved through community-level interventions to promote care-seeking and encourage healthy maternal and newborn health behaviors by mothers and family members.^3–7 A randomized control trial in Nepal achieved a 30% reduction in neonatal mortality (intention-to-treat impact: odds ratio = 0.70)⁴ by a participatory learning action cycle approach first developed in Bolivia. Local female facilitators assisted women’s groups (with mostly pregnant women) to discuss the problems leading to maternal and newborn deaths, develop practical interventions, and implement as well as evaluate their outcomes. Several other rigorous studies followed, and they provide evidence that effective community-based strategies can deliver a range of preventative and behavior change management messages targeting mothers and household maternal and newborn care practices.^3–8 To date, we have not found any reports from similar programs with large-scale, randomized implementation.

We examined the overall impact of the high-intensity Sure Start intervention (level 2, or L2), implemented at the village or panchayat level, on pregnancy, delivery, postpartum care, and newborn survival compared with the low-intensity intervention (level 1, or L1) that was implemented through district-level campaigns. We also specifically examined the impact of participating in MG meetings, a central feature of the L2 intervention. We examined the health and behavioral impact of the project’s direct contact with women (L2).

METHODS

Sure Start staff and local partners, in consultation with key stakeholders, jointly developed a package of core activities and messages focused on improving home care, increasing care-seeking behaviors, and strengthening community systems related to maternal and newborn health services delivered by the NRHM. Table 1 shows the outcomes that the project targeted.

TABLE 1—

Outcome Variables and Confounding Factors Used in Evaluation of Sure Start Project: Uttar Pradesh, India, 2007–2010

Variable	Measurement
Dependent variables: health outcomes
Neonatal mortality rate	Indicator of whether last live-born child in 2007 (baseline) or 2010 (follow-up) died within 28 days of birth, expressed per 1000 live births when aggregated. Used Demographic and Health Survey approach, except questions confined to last birth only.
Complications during pregnancy	Summed score of whether each of 16 complications was experienced during pregnancy: fever in third trimester, shortness of breath, vaginal bleeding, swelling of hands or feet, convulsions or loss of consciousness, pale eyelids and weakness, severe headache, high blood pressure, severe abdominal pain, dizziness, excessive vomiting, blurred vision, reduced fetal movement, foul-smelling discharge, rupture of membranes without onset of labor, and night blindness.
Complications during labor and delivery	Summed score of whether each of 8 complications was experienced during labor and delivery: prolonged labor, premature labor, breech or transverse position, vaginal bleeding, severe headache, visual disturbance, fever, and foul-smelling discharge.
Complications during postpartum	Summed score of whether each of 12 complications was experienced during postpartum period: heavy vaginal bleeding, severe abdominal pain, fever, foul-smelling discharge, severe headache, convulsion, red painful area or lump in breast, retained placenta, ruptured vagina, weak or faint, abdominal tenderness, and breast pain or tenderness.
Dependent variables: accessing care
Receipt of antenatal care	Receiving antenatal care or reporting at least 1 tetanus injection or iron–folic acid supplementation.
Receipt of 2 tetanus injections	Receiving 2 tetanus injections during pregnancy.
Iron–folic acid	Receiving or purchasing iron–folic acid tablets or syrup.
Supplementary nutrition	Eating food received from village health centers.
Receipt of JSY payment	Receiving honorarium for delivering baby in hospital.
JSY payment > 1400 rupees	Receiving 1400 rupees or more for delivering baby in hospital.
Institutional delivery	Delivery occurred in a health facility (includes women who were rushed to facility in emergency).
Institutional delivery without emergency	Delivery was planned to occur and occurred in a health facility.
Home delivery	Delivery occurred at home.
Skilled attendance at birth	Home delivery was attended by a skilled birth attendant (analysis run only on subset of home deliveries).
Visit to health clinic in first month	Woman reported visiting a health facility for a routine checkup within the first month after birth of child.
Dependent variables: behavior
Home deliveries
Washed hands	Whether the delivery assistant (whether skilled or otherwise) washed hands with soap and water prior to assisting in delivery.
New blade used	A clean new blade was used to cut the newborn’s umbilical cord.
Nothing on cord	Nothing was applied on the umbilical cord stump.
All 3 of the above
Newborn thermal care
Delayed bath	Newborn was not given a bath immediately after birth.
Keeping baby warm	Newborn was kept warm.
Drying	Newborn was wiped with a dry cloth.
Breastfeeding
First colostrum	Mother fed newborn the colostrum from breast.
Breast milk within an hour	Mother first put baby to breast within 1 hour of birth.
Breast milk first week	Mother did not give baby anything other than breast milk to drink in the first 7 days after delivery.
Exclusive breast milk for 6 mo	Mother exclusively breastfed baby for 6 months after delivery (analysis run only on subset of women with infants older than 6 months of age at time of survey).
Independent variables
Parity 1	All women who had 1 child.
Parity 2	All women who had 2 children.
Parity 3	All women who had 3 or more children.
Women’s literacy	Whether or not woman (interviewee) was literate.
Husband’s literacy	Whether or not the husband was literate.
Age	Age of woman.
Age first	Age of woman at first birth.
Religion and caste	Categorized as: Muslims and others; scheduled castes and scheduled tribes; and general castes and other backward castes.
House	All women living in nonmud dwellings versus women living in mud dwellings.
Water	Whether or not woman had tap inside house or had access to clean water.
Fuel	All women who used cow dung straw as fuel versus those who used electricity, biogas, kerosene, or coal.
Toilet	All women using flush, pit, or bucket versus those using open space or other means.
Electricity	All women who had access to electricity versus those who did not.
Agricultural labor	Husband was an agricultural laborer.
Usually goes to the marketa	Respondent went to the market with someone else.
Decision about major purchase or health of respondent (husband)a	Decisions about health care of respondent or major purchases taken by husband.
Decision about major purchase or health of respondent (respondent and husband jointly)a	Decisions about health care of women or major purchases taken by husband and respondent jointly.

Note. JSY = Janani Suraksha Yojana, a conditional cash transfer scheme.

^aNot Used in the regressions, but mentioned in Conclusions.

The intervention was implemented through the purposive selection of 7 districts on the basis of socioeconomic need and low institutional delivery, and also through taking account of geographic feasibility as judged by the project’s 5 lead implementing partners. The basic unit of activity in L2 was the panchayat, usually consisting of 1 main village and a few smaller hamlets. The entire district, including panchayats in L2 areas, received a basic intervention package (L1) that included advocacy and mass media and “mid-media” (e.g., local street theater) to promote messages on appropriate maternal and newborn home care and to increase demand for maternal and newborn services (antenatal care, institutional delivery, skilled postpartum care). Advocacy efforts included integration of Sure Start messages and information during community events to raise awareness and forms of media messaging, included posters, vehicle branding, street theater, and newsletters. We assigned 40% of panchayats within each district to receive the basic package plus the more intensive intervention (L2), which included community mobilization involving Sure Start community field workers working directly with ASHAs and strengthening village health and sanitation committees (established at the panchayat level to improve accountability related to supply of funds, facilities, commodities, and services needed to support core maternal and newborn health activities).

Sure Start community field workers trained ASHAs in interpersonal communication tools and group facilitation skills for 6 to 9 months; this training was in addition to the Uttar Pradesh government’s NRHM training program for ASHAs. In both L1 and L2, ASHAs visited pregnant women and their families in their homes; additionally, in L2, ASHAs encouraged the women to attend MG meetings (held once a month). In the MG meetings, they discussed 5 key themes: birth preparedness, cord care, thermal care, immediate or exclusive breastfeeding, and recognition of danger signs through use of oral and pictorial participatory processes. These activities were supported by community leaders at the panchayat level through the village health and sanitation committees. A total of 7450 ASHAs were mentored by Sure Start community field workers in 40% of the areas (L2) for each of the 7 implementation districts (Bahraich, Balrampur, Basti, Gorakhpur, Hardoi, Rae Bareli, and Barabanki).

Evaluation Design and Questions

Evaluation of the interventions was based on a cluster-randomized design and conducted by external evaluators. The sampling process closely resembled the implementation process. In each district, implementation commenced at the block level, with each block divided into clusters of 10 000 people; we randomly selected 40% of the clusters for L2 and the rest for L1. For the evaluation survey, we eliminated Bahraich and Barabanki so that there would be a total of 5 districts, 1 for each of the 5 lead partners. We randomly selected 5 blocks from each of the 5 districts and then randomly selected 5 clusters for each of the 2 intervention arms. Within the L1 and L2 clusters, we randomly selected panchayats for the evaluation. It was possible to have more than 1 village per cluster; with an expected estimation of 20 pregnant women per village, we carried out a census of the women. If there were shortfall or higher number of women in any village, we made adjustments to preserve the random selection of geographic areas. The process yielded 498 survey panchayats. The sample size, 6000 for each treatment arm, was powered at 80% with an intracorrelated coefficient of less than 0.07 to detect a difference of at least 10 per 1000 for the main outcome of interest, neonatal mortality rate (NMR). The rate of refusal was negligible, and fewer than 1% of interviews were unusable.⁹

The baseline, conducted in 2008, collected information from women completing pregnancy with delivery in 2007; the follow-up survey was completed in 2011 in the same panchayats, and information from women completing delivery in 2010 cohort was collected. A quality assurance process was in place for both surveys; for the 2011 survey, an independent agency conducted repeat interviews with 6.5% of the respondents and compared the filled questionnaires for completeness, accuracy, and consistency.

In addition to NMR, we examined the effect of the intervention on intermediate health outcomes and outputs related to accessing care and practicing healthy behaviors. Following previous studies,^3–7 we grouped the dependent variables (outcomes) into 3 categories: health outcomes, indicators of accessing care, and healthy behaviors. Table 1 lists each of the dependent variables examined in each category.

In evaluating the project, we aimed to answer 2 questions:

1. Intention to treat (ITT) impact: did the higher-intensity intervention improve neonatal survival and other health-related indicators for women living in L2 program areas relative to L1 areas?

2. Participation impact: did participation in one of the main components of L2, the MG meeting, help pregnant women achieve better health?

Statistical Analysis

To assess whether the randomization process was effective, we examined differences between the L1 and L2 villages for each outcome variable of interest and possible confounding factors at baseline. We used χ² and t tests to test for statistically significant differences in categorical and continuous variables, respectively. All statistical analyses were carried with Stata version 12 (StataCorp LP, College Station, TX).

For ease of exposition, we first present mean differences as the program’s effect. We then present results adjusted through regression methods. First, we calculated the ITT impact at the individual level (with each woman residing either in an L1 or an L2 cluster) through regression methods that adjusted for the standard confounding variables¹⁰ listed in Table 1 as independent variables in order to account for any residual differences remaining after randomization. We used 3 models: (1) Using only the follow-up survey data to report odds ratios, we employed logistic models for all variables of interest except those that describe morbidity. (2) Because the variables describing morbidity contained multiple numbers of dichotomous questions, we used regression for count variables. The results are expressed in terms of incidence rate ratio (IRR); a value of k means the number reported for the treated is k times more. (3) For those dichotomous outcomes with probability mass away from 0 or 1 (e.g., tetanus toxoid injection but not NMR), we used data from both the surveys to estimate linear probability difference-in-differences models, allowing the inclusion of variables that depict time trend and random assignment interacted with time. (Linear probability regression, unlike logistic regression, allows for such interaction terms.^11–13) All models used robust standard errors and the logistic regressions included a random effect at the cluster level to control for unobserved systematic differences across clusters. The logistic and difference-in-differences results can differ, as difference-in-differences report on relative changes whereas logistic regressions report on the differences from the follow-up data.

Because the L2 intervention was primarily delivered to women through MG meetings conducted by ASHAs, we examined whether the overall impact depended on MG participation. To estimate this participation impact, we used the follow-up survey data with dependent and independent variables averaged at the village level; we also estimated a model in which changes in outcomes were the dependent variables, using average covariates from baseline as the confounding factors. Because difference-in-differences estimations may have serial correlation,¹⁴ we emphasize the results from the follow up data. Various factors may have affected attendance at MG meetings, which was 42% in our L2 sample. Sure Start workers may have persuaded some women to attend MG meetings. In addition, some women may have attended the MG meetings because of confounding factors that were not shared by other women in L2 and L1. Failure to adjust for these factors could result in an overestimate of impact—biased because of self-selection; we present the results from such a model, which is meant to be only suggestive. We then estimated the effect on outcomes that can be attributed to MG participation, minus any selective effects that may also determine both MG participation and related outcomes.

We used local average treatment effect (LATE) analysis to assess the effect of MG participation minus selection bias. We did so by instrumenting MG attendance (at the village level) with random assignment of the panchayat and unobserved characteristics of the blocks in which these panchayats are located.¹⁵ The instrumentation accounted for potentially varying effects of program implementation dependent on unknown block-level characteristics such as the cohesiveness of women in the area or administrative factors that might contribute to MG effectiveness. Because the assignment to L1 and L2 areas was random, the instrumentation was uncorrelated with any features of the panchayat. Our methodology was similar to that of Bjorkman and Svensson.¹⁶ We ran robustness checks through additional exclusionary variables, such as NMR, for the panchayat in 2007. The impact of the MG meetings was considered a local impact because it was specific to the way implementation took place and would be different if recruitment to attend MG meetings had occurred through dissimilar methods.¹⁷ The LATE analysis enabled us to answer the participation question: given the effort of the program, does a higher rate of attendance at MG meetings result in better health outcomes for women and newborns, measured at the panchayat level? We report the results as linear relations between the percentage of women attending MG meetings in a panchayat or village (as estimated through the instrumental variable) and various outcomes of interest, usually expressed as a percentage within the panchayat or village.

RESULTS

At both baseline and follow-up, women receiving L1 and L2 interventions were comparable with regard to sociodemographic characteristics, with only a few exceptions (Table 2). At baseline, outcomes of interest were also comparable (Table 3). In both L1 and L2 areas, significant changes from baseline to follow-up were seen in terms of wealth and literacy (Table 2), as well as across most of the outcome variables we examined (Table 3). This indicates that substantial time trends in health improvement occurred in concert with the more intensive Sure Start intervention. Unexpectedly, caste and religious structure did not remain the same; there may have been some out-migration of the Muslim population (Table 2). Tests from the follow-up survey for mean differences indicated a favorable intervention impact (Table 3) for most of the outcomes. We report the adjusted results. (We do not report adjusted analyses where the rates of the outcome were high for both regions. For full results, see http://www.surestartdata.com/FileCategory.aspx?id=1).

TABLE 2—

Socioeconomic and Demographic Variables Used in Regression Analysis of Evaluation of Sure Start Project: Uttar Pradesh, India, 2007–2010

	Baseline				Follow-Up				Baseline vs Follow-Up, P
Variable	L1 (n = 6136), %	L2 (n = 6232), %	P (L1 vs L2)	Sure Start (n = 12 368), %	L1 (n = 5988), %	L2 (n = 5897), %	P (L1 vs L2)	Sure Start (n = 11 885), %	Sure Start	L1	L2
Parity
First birth (live)	22.39	23.57	.119	22.99	23.35	24.54	.128	23.94	.081	.21	.21
Parity 2	21.46	20.73	.319	21.09	21.16	22.88	.024	22.01	.083	.68	≤ .001
Parity 3	18.11	17.89	.756	18.00	18.47	16.86	.021	17.67	.504	.6	.13
Parity > 3	37.89	37.50	.654	37.69	36.97	35.68	.142	36.33	.028	.29	.04
Age, mean
Average age	26.76	26.61	.15	26.69	26.74	26.49	.016	26.61	.31	.83	.21
Age at first birth	20.40	20.37	.67	20.38	20.11	20.08	.6	20.09	≤ .001	≤ .001	≤ .001
Literacy
Mother literate	33.54	34.45	.285	34.00	36.59	38.09	.092	37.33	≤ .001	≤ .001	≤ .001
Husband literate	69.95	70.99	.205	70.47	71.23	74.24	≤ .001	72.72	≤ .001	.12	≤ .001
Religion and caste			≤ .001				.003		.036	.28	.06
Muslim and others	16.33	19.53		17.94	15.60	17.87		16.73
Scheduled castes and scheduled tribes	33.60	31.93		32.76	32.97	32.76		32.86
General castes and other backward castes	50.07	48.54		49.30	51.44	49.36		50.41
Housing			≤ .001				.567		≤ .001	≤ .001	≤ .001
Nonmud dwelling	62.76	65.98		64.38	76.95	77.40		77.17
Mud dwelling	37.24	34.02		35.62	23.05	22.60		22.83
Toilet facility			.201				.124		≤ .001	≤ .001	≤ .001
Flush, pit, or bucket disposal	11.73	12.48		12.11	14.18	15.18		14.67
Open space	88.27	87.52		87.89	85.82	84.82		85.33
Television	20.47	22.54	.005	21.52	20.49	22.23	.021	21.35	.761	.97	.67
Electricity	26.40	30.10	≤ .001	28.27	38.46	40.53	.021	39.49	≤ .001	≤ .001	≤ .001
Agricultural laborer	39.73	38.17	.075	38.95	51.80	49.25	.005	50.53	≤ .001	≤ .001	≤ .001
Fuel use			.509				.165		≤ .001	≤ .001	≤ .001
Electricity or biogas	8.88	9.48		9.18	8.78	9.77		9.27
Kerosene or coal	3.49	3.50		3.49	0.78	0.85		0.82
Straw, cow dung, others	87.63	87.02		87.32	90.43	89.38		89.91
Decision about major purchase or health of respondenta			.106				.587		≤ .001	≤ .001	≤ .001
Respondent	7.55	8.81		8.18	5.90	6.53		6.21
Her husband	38.72	38.00		38.36	26.62	27.10		26.86
Respondent and husband jointly	22.98	22.18		22.57	32.73	31.97		32.35
Someone else	29.97	30.15		30.06	34.03	33.71		33.87
Usually goes to the marketa			.445				.529		≤ .001	≤ .001	≤ .001
Alone	13.98	14.27		14.13	10.30	10.72		10.51
With someone else	58.16	56.96		57.56	57.00	57.81		57.40
Not at all	27.46	28.26		27.86	32.45	31.24		31.85

Note. L1 = level 1; L2 = level 2. L1 and L2 refer to low- and high-intensity intervention, respectively. At follow-up, attendance at mother's group meeting was as follows: in L1, n = 257 (4.29%); in L2, n = 2478 (42.0%; P ≤ .001); in Sure Start, n = 2735 (23.0%).

^aNot used in the regressions, but mentioned in the “Conclusions” section.

TABLE 3—

Outcome Variables Evaluated for Sure Start Intervention Effect: Uttar Pradesh, India, 2007–2010

	Baseline				Follow-Up
Variable	L1 (n = 6136), %	L2 (n = 6232), %	P (L1 vs L2)	Sure Start (n = 12 368), %	L1 (n = 5988), %	L2 (n = 5897), %	P (L1 vs L2)	Sure Start (n = 11 885), %	P for Sure Start (Baseline vs Follow-Up)
Health
Neonatal mortality rate	4.51	4.39	.751	4.45	3.70	3.58	.721	3.64	.002
Experiencing complications during labor and deliverya (SD)	0.884 (1.28)	0.872 (1.27)	.59	0.878 (1.28)	0.599 (1.00)	0.558 (0.96)	.02	0.578 (0.02)	≤ .001
Experiencing complications during postpartumb (SD)	1.46 (1.66)	1.43 (1.63)	.3	1.44 (1.65)	1.05 (1.46)	0.93 (1.39)	≤ .001	0.992 (1.43)	≤ .001
Experiencing complications during pregnancyc (SD)	2.28 (2.23)	2.25 (2.23)	.42	2.27 (2.23)	1.64 (1.91)	1.44 (1.81)	≤ .001	1.54 (1.86)	≤ .001
Access to care									≤ .001
2 tetanus injections	78.96	79.38	.565	79.17	82.75	86.47	≤ .001	84.59	≤ .001
Iron–folic acid tablet	45.97	44.74	.167	45.35	70.69	76.95	≤ .001	73.80	≤ .001
Supplementary nutrition	28.54	28.58	.959	28.56	30.03	32.54	.003	31.27	≤ .001
Antenatal care received	70.81	70.38	.601	70.59	77.00	82.70	≤ .001	79.83	≤ .001
JSY payment received	5.33	6.50	.006	5.92	42.33	46.60	≤ .001	44.45	≤ .001
JSY payment > 1400 rupees	3.90	4.90	.003	4.45	41.22	45.40	≤ .001	43.29	≤ .001
Visit to health clinic month	26.22	25.42	.306	25.82	9.74	10.58	.127	10.16	≤ .001
Institutional delivery	21.33	23.25	.01	22.30	52.30	57.06	≤ .001	54.67	≤ .001
Institutional delivery without emergency	12.29	14.75	≤ .001	13.53	43.59	49.45	≤ .001	46.50	≤ .001
Behavior
Home delivery, no. (% of sample)	4816 (78.5)	4767 (76.5)	.008	9583 (77.5)	2813 (46.9)	2484 (42.1)	≤ .001	5297 (44.6)	≤ .001
Skilled attendanced	15.03	15.52	.505	15.28	13.79	16.71	.003	15.16	.85
Washed handsd	84.93	84.35	.435	84.64	73.16	75.00	.128	74.02	≤ .001
New bladed	96.03	95.80	.57	95.92	96.59	97.14	.249	96.85	.004
Nothing on cordd	47.34	48.37	.312	47.86	43.44	52.74	≤ .001	47.80	.949
All 3 aboved	41.11	41.81	.49	41.46	32.88	41.43	≤ .001	36.89	≤ .001
All live births, no.	6075	6148		12 223	5860	5756		11 616
Delayed bathe	28.59	30.55	.018	29.58	59.91	74.17	≤ .001	66.98	≤ .001
Keeping baby warme	97.05	96.94	.718	97.00	95.99	96.79	.022	96.38	.008
Dryinge	92.51	92.36	.746	92.43	95.99	97.06	.002	96.52	≤ .001
First colostrume	63.93	64.09	.861	64.01	62.80	74.51	≤ .001	68.60	≤ .001
Breast milk within an houre	20.63	21.21	.427	20.92	29.10	42.37	≤ .001	35.67	≤ .001
Breast milk first weeke	26.73	26.11	.432	26.42	37.85	56.93	≤ .001	47.31	≤ .001
Exclusive breast milk 6 moe	14.54	14.49	.949	14.51	11.70	20.51	≤ .001	16.05	.02

Note. JSY = Janani Suraksha Yojana, a conditional cash transfer scheme; L1 = level 1; L2 = level 2.

^aHealth practices during labor and delivery (range = 0–9).

^bDanger signs experienced during postpartum phase (range = 0–12).

^cDanger signs experienced during pregnancy (range = 0–15).

^dBase all home deliveries.

^eBase all live births.

Health Impact

The difference in NMR between L1 and L2 seen at follow-up was not statistically significant (35.1/1000 and 37.0/1000 for L2 and L1 areas, respectively), although the temporal change was highly significant in both areas. All estimations indicated a lower NMR for the more intensive (L2) program, with none of them reporting a statistically significant result (Table 4).

TABLE 4—

Regression Results for Evaluation of the Sure Start Program’s Impact in Areas of High-Intensity Interventions: Uttar Pradesh, India, 2007–2010

Variables	ITT-L2 Over L1 OR (95% CI)a	Count Vvariables Estimates IRR (95% CI)b	DD Coefficientsc	MG attendance Non-IV, Endline OR (95% CI)d	IV Estimate-DD-LATEe	IV Estimate -MG Effect -LATEf
Health
Neonatal mortality rate	0.979 (0.805, 1.189)		0.001	0.813 (0.634 1.041)	–0.414	–0.004
Experiencing complications during labor and delivery		0.917 (0.832, 1.010)			0.135	–0.153*
Experiencing complications during postpartum		0.839*** (0.772, 0.911)			0.085	–0.408**
Experiencing complications during pregnancy		0.853*** (0.801, 0.909)			–0.345	–0.807***
Accessing Care
Antenatal care received	1.458** (1.149, 1.848)		0.061***	3.829*** (3.307, 4.432)	0.091	0.161***
2 tetanus injections	1.286** (1.070, 1.560)		0.033***	2.817*** (2.412, 3.228)	0.100***	0.093***
Iron-folic tablet	1.478*** (1.215, 1.797)		0.075***	3.380*** (2.980, 3.832)	0.171***	0.212***
Supplementary nutrition	1.126 (0.903, 1.404)		0.023	2.496*** (2.281, 2.730)	0.022	0.132***
JSY received	1.168 (0.952, 1.434)		0.031	1.891*** (1.729, 2.067)	0.047	0.136*
JSY received > 1400 rupees	1.169 (0.955, 1.429)		0.030	1.911*** (1.747, 2.089)	0.062	0.148*
Home delivery	0.843 (0.956, 1.429)		–0.026	0.560*** (0.510, 0.615)	–0.036	–0.142*
Skill attendance	1.324* (1.044, 1.677)		0.023	1.560*** (1.299, 1.873)	0.114**	0.143***
Visit to health clinic month	1.017 (0.830, 1.244)		0.016	1.308*** (1.135, 1.507)	0.006	0.0206
Institutional delivery	1.184 (0.957, 1.463)		0.027	1.763*** (1.606, 1.985)	0.037	0.143*
Institutional delivery (nonemergency only)	1.253* (1.012, 1.550)		0.032	1.867*** (1.704, 2.044)	0.056	0.191**
Behavior
Washed hands	1.147 (0.915, 1.437)		0.024	1.552*** (1.304, 1.847)	0.126*	0.103*
New blade	1.186 (0.824, 1.708)		0.007	1.508 (0.921, 2.467)	0.029	0.029
Nothing on cord	1.509*** (1.279, 1.778)		0.081**	2.197*** (1.901, 2.538)	0.307***	0.415***
All 3 above	1.507*** (1.248, 1.818)		0.078**	2.299*** (1.993, 2.650)	0.285***	0.390***
Delayed bath	1.956*** (1.683, 2.270)		0.120***	2.814*** (2.523, 3.136)	0.234***	0.387***
First colostrum	1.803*** (1.435, 2.265)		0.114***	3.428*** (3.040, 3.855)	0.314***	0.323***
Breast milk within an hr	1.896*** (1.602, 2.242)		0.125***	2.945*** (2.686, 3.229)	0.363***	0.471***
Breast milk first wk	2.326*** (1.967, 2.751)		0.194***	3.371*** (3.06, 3.707)	0.460***	0.560***
Exclusive breast milk 6 mo	2.024*** (1.714, 2.389)		0.086***	3.063*** (2.743, 3.420)	0.281***	0.333***

Note. CI = confidence interval; DD = difference in difference; ITT = intent to treat; JSY = Janani Suraksha Yojana, a conditional cash transfer scheme; L1 = level 1; L2 = level 2; LATE = local average treatment effect; MG = mother’s group; OR = odds ratio.

^aITT results expressed as an odds ratio through use of logistic regression and random effect model, ; where y stands for outcome variable for person i in region j at time 1; and D is treatment areas valued at 1 for L2, 0 for L1. The subscript 1 indicates follow up data.

^bIncidence rate ratio estimations for count variable through use of negative binomial using the above equation; not presented for non-IV estimations mother’s group attendance at the individual level.

^cLinear probability estimations for DD estimations: ; T stands for time, either 0 or 1 with subscript t standing for time either 0 or 1.

^dMother’s groups estimations without correction for selection, presented for heuristic reasons only: . Only odd ratios are presented; count data analyses were not carried out for pregnancy morbidity.

^eIV Linear estimation results with robust error adjustment by instrumenting percentage of women attending MG in a panchayat; these indicate village level estimations. The unit of analysis is the village/panchayat level. Denote the instrumented variable as MGIVavg; this column estimates: . Avg denotes averages over the village. We suspect serial correlation in this model.

^fEstimations of the following: .

*P < .05; **P < .01; ***P < .001.

The 3 indices of complications during pregnancy, labor and delivery, and the postpartum period (morbidity) decreased more in L2 areas than in L1 areas. The adjusted differences were statistically significant for pregnancy and postpartum complications in the ITT model (Table 4). The estimations for MG participation were not consistent for experiencing complications.

Accessing Health Care

The effects of the L2 intervention on access-to-care indicators were not uniform across the different models. As seen in both the overall ITT impact and the impact of MG participation rate at the village level, the intervention positively affected antenatal care attendance only when follow-up data were used, although the difference-in-differences estimation was almost significant. For women in the L2 areas, the likelihood of getting 2 tetanus injections and iron–folic acid was higher. The only significant result for supplementary nutrition was the follow-up village-level estimation.

Janani Suraksha Yojana payments were significantly correlated with higher village-level MG participation only when the follow-up data were used. However, the time trend for these payments (Table 3) was highly significant. The lower value for home delivery was in the expected directions, and significant for the village-level analysis using the follow-up data. The impact of MG meeting participation was strong. Except for difference-in-differences at the ITT level, we obtained positive outcomes for the presence of skilled workers at home deliveries. There was no perceptible impact in institutional births in the intervention area when we included emergency births. When using only the follow-up data, we found that there was a village-level impact on nonemergency institutional births (i.e., most births). The program did not lead to more visits to health centers in the first month after childbirth; in fact, there was a decline over time in the number of visits.

Healthy Behavior

Regarding home deliveries, the 3 safe birthing practices (washing hands, use of a new blade, and nothing on cord) taken as an aggregate were observed at a higher rate in L2 than in L1 areas (Tables 3 and 4). The impact on hand washing was greater in areas with more MG meeting participants. There was no impact on new blade use, as it was high everywhere. We observed strong results for behaviors related to care of the newborn by mothers. The more intensive intervention was successful in convincing women to feed colostrum to the newborns, provide breast milk within an hour, and exclusively breastfeed for the first week. The impact on exclusive breastfeeding for the first 6 months was statistically significant. However, this result must be understood in relation to the fact that the proportion of women breastfeeding for a full 6 months actually fell in the L1 areas, whereas it rose only about 5% in the L2 areas (Table 3). This is the only behavioral factor for which the trend over time was not a positive health outcome. Delayed bathing was practiced at a higher level in L2 areas. Drying and keeping the baby warm were practiced at a high rate in all areas (Table 3).

DISCUSSION

Time was a key factor in explaining changes in both L1 and L2. The decline in NMR may have been affected by many factors. Important contextual changes occurred with the initiation of the NRHM at the time Sure Start was implemented. The rapid rollout of the Janani Suraksha Yojana scheme to induce higher institutional birth contributed to reducing NMR (results not reported here), rising income levels, possible changes in fertility rates, and improvements in women’s education (Table 2). Across India, NMR fell to 32 per 1000 by 2010.¹⁸ These changes may have contributed to improved practices and mortality declines, making it more difficult to detect program effects as they were originally intended to be explored. We cannot attribute the temporal change in NMR to the impact of the Sure Start effort in either L2 or L1 areas since we have no randomized non–Sure Start area for comparison (having such areas would have been technically difficult because of the media campaign).

Sure Start, implemented to scale, was primarily a community-based intervention that sought to bring about changes through communication and advocacy behavioral change that would improve health; it worked alongside the government system to provide mentorship to ASHA services to complement the usual government facility-based services. Health centers still remain understaffed, and the NRHM has not fully addressed many inadequacies in basic services, which, for Uttar Pradesh, were even reported in the popular media.^18–20 This may explain the low level of visits to a clinic within 1 month of childbirth.

Limitations

Our study has considerable limitations. (1) The Sure Start evaluation design did not take account of temporal changes, but was based on an assumption that baseline NMR was greater than 50 per 1000. With changes in time as noted in Table 3, this study may have been underpowered to detect a further drop in L2 villages.^4–8,21 (2) Our data were dependent on surveys instead of birth and death registration to estimate NMR and perinatal care, although this is standard practice in developing countries. (3) Our findings for reported complications should be taken with caution. It is possible that women in L2 areas knew how to identify these problems and therefore recognized them at a higher rate. Thus, results for morbidity should be seen as inconclusive. A spillover effect most likely did not dampen either the ITT or the MG group impact; we found interactions outside the village to be small (Table 2), and within-village communication among childbearing women also seemed to be limited.

Conclusions

Sure Start, working in tandem with the NRHM, positively affected care-seeking behaviors in the antenatal period. There was significant behavioral improvement to achieve better health. In addition, the frontline health workers (ASHAs), who were mentored and supervised through Sure Start in L2 areas, successfully transmitted messages of safe motherhood practices through women’s participation in MG meetings, as reflected in the participation impact analysis that showed improvements in nearly all outcomes for MG attendees. With the notable exception of 6-month exclusive breastfeeding, we observed significant impacts for L2 areas in relation to healthy behavior, including breastfeeding at birth and acceptance of iron–folic acid tablets. These latter nutritional improvements are a key to better health as India continues to face problems with low birth weight babies and low anthropometric measures.¹⁸ On the basis of our results, we speculate that achieving a high level of attendance at MG meetings would result in improving health outcomes when accompanied by supply-side interventions. We therefore recommend continual implementation to scale of programs similar to Sure Start to improve maternal and newborn health practices and to increase demand for and use of health services. For the best outcomes, supply factors will probably also need to be improved.