Retrospective comparative evaluation of the lasting impact of a community-based primary health care programme on under-5 mortality in villages around Jamkhed, India

Abstract

Objective

To conduct the first rigorous evaluation of the long-term effect of the Comprehensive Rural Health Project on childhood mortality in rural Maharashtra.

Methods

Background information and full birth histories were collected by conducting household surveys and interviewing women. Control villages resembling project villages in terms of population size were randomly selected from an area enclosed by two ellipses centred around, but not including, the project area. An equal number of villages and approximately equal numbers of households and women were randomly sampled from both areas. Cox models with robust standard errors were used to compare the hazard of death among children under 5 years of age in project and control villages.

Findings

The hazard of death was reduced by 30% (95% confidence interval, CI: 6% to 48%) after the neonatal period in the project villages compared with control villages after adjustment for caste and religion of subjects and for availability of irrigation in the villages. During the neonatal period there was an increase of 3% in the hazard of death, but it was not statistically significant (95% CI: −18% to 29%).

Conclusion

Our methods provide useful tools for evaluating long-running community-based primary health care programmes. Our findings add to the growing debate on the long-term sustainability of community-based interventions designed to reduce child mortality.

Introduction

The Bellagio Child Survival Study Group brought the issue of child survival to the forefront of the global health care agenda in 2003 when it reported that 10 million children were dying every year.¹ Reducing child mortality by two-thirds before 2015 is one of the United Nations Millennium Development Goals.² As the Bellagio series and the Lancet neonatal survival series³ and Alma-Ata series⁴ all point out, most of these deaths are preventable. In India, as in many developing countries, providing even basic health care in rural areas is a major challenge for the government. Analyses of India’s health system have suggested that rural health care has been neglected by the government and that increasing privatization may further reduce health care in remote areas.^5,6

Community-based primary health care provided by trained community residents has been shown to improve child survival in areas with high child mortality. The Warmi project in Bolivia and the Society for Education, Action and Research in Community Health (SEARCH) in Maharashtra, India, have demonstrated significant reductions in perinatal and neonatal mortality.^7,8 Recent trials in Nepal and in Uttar Pradesh, India, have reported reductions in neonatal mortality of 30% and 52%, respectively.^9,10

Many aid agencies strive to address the lack of health care and improve child survival in developing countries, but they rarely conduct rigorous independent evaluations of their work. Such evaluations are expensive, carry the risk of showing negative results and are under-appreciated by donors. Consequently, there are few credible data on the impact of ongoing aid work.^11–15 Though randomized controlled interventions can generate such data,¹⁵ a study with prospectively assigned controls is seldom an option and retrospective approaches must therefore be used to assess the impact of aid efforts.

The Comprehensive Rural Health Project

The Comprehensive Rural Health Project (CRHP), one of the inspirations for the 1978 International Conference on Primary Health Care at Alma-Ata, has been working for the past 40 years to address the issues highlighted in the aforementioned Lancet series.^1,3,4 It has had a major influence in health policy circles, including the World Health Organization (WHO) and schools of public health,¹⁶ and is a rare example of a long-term community-based primary health care project. The project was highlighted in the Alma-Ata Rebirth and Revival series as a model for delivering primary health care to poor rural regions,⁴ and – if found to be effective upon rigorous evaluation – could prove important for achieving the Millennium Development Goals on child mortality reduction elsewhere in India and in other vulnerable areas.

CRHP was founded in 1970 by physicians Mabelle and Rajnikant Arole, who envisioned a system that delivered both curative and preventative care to India’s most vulnerable people. Based in the town of Jamkhed, CRHP also serves surrounding areas in the central part of Maharashtra state. The area is predominantly rural, poor and drought-prone. Public health care and education are notoriously poor. CRHP has gradually expanded from a single hospital in Jamkhed and currently covers approximately 300 communities with a total population of over 500 000 people.¹⁷

Activities in project villages

The CRHP model, which focuses on community-centred primary health care, is described in detail elsewhere.^17,18 Activities in the project villages are carried out at three levels: village health workers, mobile health teams and the secondary-care hospital in Jamkhed.

Village health workers, who are the cornerstone of the model, are local women selected by their communities, often from the lowest caste. They receive intensive training from CRHP trainers in primary health care and health promotion, including family planning, women’s and children’s health and home birth delivery. Training is also provided in community development, organization, communication skills and personal development. The primary role of these workers is to disseminate health knowledge in their respective communities through discussion groups and household visits, administer basic remedies and medications, perform safe deliveries and detect and refer high-risk pregnancies and deliveries to more qualified health care providers. During participatory discussions with community members, village health workers address issues such as care-seeking, family planning, adequate birth spacing, nutrition, hygiene, sanitation and safe drinking water.

Mobile health teams, comprising a nurse, a physician, a social worker and paramedics, visit project villages monthly to support and mentor village health workers and to refer complicated cases to the hospital. The secondary-care hospital in Jamkhed is a 40-bed low-cost facility operated by CRHP that provides quality emergency, medical, surgical and outpatient care to residents from the surrounding catchment area.

Research aims

Although many publications have praised CRHP’s work,^4,18–23 to date no rigorous impact study has been conducted. Overall child survival has improved in Maharashtra over the past few decades,^24–26 yet infant and child mortality rates in remote areas of Maharashtra remain much higher than for rural India overall.^27,28 Rigorously evaluating the effect of CRHP will provide insight into the impact of community health interventions in such areas. Our primary aim was to compare mortality among children under 5 in CRHP villages of a typical size and in nearby control villages of similar size. Secondary aims were to compare sanitation; health knowledge; number of children per woman; place and type of delivery and type of birth attendant; indicators of antenatal, delivery and postnatal care; and child morbidity.

Methods

Study design

For statistical efficiency, we sampled equal numbers of villages in CRHP and control sites and approximately equal numbers of households, women and children within each village.

Surveys

Two surveys were conducted after a specially-hired team identified and numbered every household in all study villages. A household survey collected information on household-level indicators such as wealth, sanitation and water supply. Women in the surveyed households were interviewed and a full birth history was obtained, along with information on the woman’s background, history of pregnancy-related care, health care expenditure and morbidity of children under 5 years of age. Information on health knowledge was also collected for each eligible woman present in the household. These surveys were modelled on questionnaires from the third National Family Health Survey (NFHS-3)²⁹ and were conducted by women hired locally from a women’s college near Jamkhed. They were trained for 10 days by a National Family Health Survey trainer and supervisor and supervised by locally-hired managers and a project manager. Training manuals, created by the authors, were based on the manuals for NFHS-3.³⁰ Data collection took place between September 2007 and January 2008.

Sampling CRHP villages

The CRHP worked in four small administrative regions known as blocks or talukas: three in the core area near the hospital (Jamkhed, Ashti and Karjat) and one (Akole) approximately 220 km from the core area. We focused only on villages in the core area because Akole is distant and consists largely of tribal villages with characteristics different from those found in the core-area villages. We restricted eligibility to villages with a population of between 400 and 3000 in which a CRHP village health worker had been working for at least 5 years.

In the core area, CRHP implemented its comprehensive health intervention in 153 villages. Of these, 69 were excluded because village health workers had not worked for at least five years in the village. A further nine were excluded because their populations were outside the specified range. A total of 75 villages, all of them non-tribal, in the core area satisfied all criteria and were included in the study.

Sampling control villages

To select control villages with characteristics similar to those of CRHP villages while minimizing the risk of contamination bias owing to proximity to the CRHP intervention area, a buffer zone was drawn around the three core blocks of the project. Since the core intervention region is approximately elliptical in shape, we drew an elliptical buffer line that was never less than 5 km from any intervention village. A second ellipse, with the same foci, was drawn 25 km outside the first (Fig. 1). Control villages were selected from the area between the two ellipses. The 25 km width ensured that the number of eligible control villages in the area between the two ellipses was greater than the total number of eligible intervention villages. From the 135 non-tribal villages in this area with a population between 400 and 3000 according to the 2001 national census, 75 were selected at random for the study. CRHP was the only nongovernmental organization delivering home-based care, health education and clinical services in the study villages. Government health services were available in both intervention and control villages.

Fig. 1.

Map of control and intervention areas in retrospective study of the impact of the Comprehensive Rural Health Project (CRHP), Maharashtra state, India

^a This area comprises the 153 CRHP villages, of which 75 satisfied all inclusion criteria.

^b This area comprises the 135 eligible control villages, of which 75 were randomly selected to be surveyed. The inner ellipse is at a distance of at least 5 km from any CRHP village.

Sampling households and women

In each village, 30 households were randomly selected from the mapping list of all households. An additional 60 households were randomly selected as back-ups. Our survey teams attempted to interview all potentially eligible women in the 30 selected households in each village. They returned twice to the village to reach as many selected households as possible. If it was impossible to identify and interview at least one eligible woman after two visits to a selected household, the team approached the next household on the back-up list, continuing until at least 30 households were interviewed. As six interviewers were simultaneously conducting interviews, in some villages the team interviewed more than 30 households.

Eligible women and births

Women were eligible if they were between 15 and 59 years of age, were currently or previously married, had lived in the village for at least one year before the survey and had at least one child born alive in the village in the past 15 years. Some women who lived in intervention or control villages at the time of the survey had given birth and raised children in other villages. To minimize recall bias, we included only women who had given birth to a liveborn child between 1 September 1992 and 31 December 2007 in the same village in which they were interviewed. To allow sufficient time for CRHP to have had an impact, analyses in intervention villages were further restricted to women who had given birth five years after the project began operating in their villages.

Sample size

The sample size was calculated to attain 80% statistical power to detect a 25% reduction in under-5 mortality at the 5% significance level using a conventional 2-sided test. On the basis of data from the 1998–1999 National Family Health Survey for rural India,²⁵ we assumed a 10% under-5 mortality rate in the control villages and hence 7.5% under-5 mortality in the CRHP villages, with an intra-cluster correlation coefficient of 0.0175, estimated from the National Family Health Survey data for the period 1985–1995 for the 81 non-tribal rural villages in Maharashtra state included in this data set.²⁵

Under these assumptions, 54 eligible births were needed from each of the 75 sampled villages in both the CRHP and the control area during the 15 years before the interviews. At an assumed average of two eligible births per household, at least 27 households per village were required.

Data analysis

All data obtained for all eligible villages, women and children were analysed. Demographic and other characteristics at the village, household and mother level were summarized by type of village (CRHP or control). These included characteristics considered as confounders and/or secondary outcomes, such as the educational level of the woman, treatment of drinking water, toilet facilities, number of women reporting miscarriage and/or stillbirth, number of Caesarean deliveries, number of children per woman, number of antenatal and postnatal care visits, mean health knowledge score and its standard deviation, and health care-seeking activities.

The overall health knowledge score was derived by summing the total number of correct responses given to 10 health knowledge questions. Questions with more than one possible correct answer were weighted by the inverse of the total number of correct answers so that no single question contributed more than one point out of 10 to the score.

The percentages of villages with irrigation in the CRHP and control groups were compared using Fisher’s exact test. Mean ages and health knowledge scores were compared using linear regression models with robust standard errors (Huber-White sandwich estimator) to allow for clustering within villages. Categorical characteristics and secondary outcomes at household, woman and child level were compared using logistic regression models. Robust standard errors were used, and Wald tests for joint significance were performed. Village-specific crude neonatal, infant and under-5 mortality rates were estimated by the Kaplan–Meier method and are presented separately for control and CRHP villages. Crude mortality rates for the entire set of surveyed villages are given in 5-yearly intervals.

Cox regression models were used to compare hazard rates between CRHP and control villages using robust standard errors (Lin-Wei estimator) to account for clustering of women in villages and for the non-independent outcomes for children born to the same woman. As CRHP attempted to work in the most disadvantaged villages, most of which lacked irrigation and had relatively large proportions of low-caste inhabitants, we decided a priori to adjust for irrigation status of the villages and for women’s caste and religion. Interaction terms between the intervention and different age bands (neonatal, post-neonatal to 5 years of age) were included in the models to test the proportionality assumption – i.e. whether the intervention had a differential effect on mortality at different ages. When a statistically significant interaction was found, age-specific effects of the intervention are also reported.

Ethical approval

Ethical approval was obtained from the Institutional Ethics Committee of the Maharashtra Association of Anthropological Sciences in India and from the Ethics Committee of the London School of Hygiene and Tropical Medicine.

Results

Data on the study population are presented in Table 1. More women in CRHP villages belonged to less-advantaged castes, their families were less likely to own land and they were less likely to have irrigation available (Table 2). Factors that may have been influenced by the CRHP intervention are summarized in Table 3. In 2007, more households in CRHP villages had toilet facilities and treated drinking water. Women in the intervention villages achieved, on average, a health knowledge score that was 0.15 standard deviations higher (0.26 points higher in absolute score) than the score obtained by women in the control villages. The number of children per woman and the percentage of women who reported having a stillbirth or miscarriage were similar in the two groups. Indicators of antenatal and postnatal care and of location and type of delivery of the last child were also similar. Among women reporting that in the three months before the survey their child under 5 years of age had experienced symptoms of one of the main diseases that cause child death, there was no significant difference in the use of medical centres when seeking treatment.

Table 1. Study population data from a retrospective evaluation of the impact of the Comprehensive Rural Health Project (CRHP), Maharashtra state, India, September 1992–December 2007.

Study population	Control villages (n = 75)					CRHP villages (n = 75)
	Total	No. per village				Total	No. per village
		Mean	SD	Range (min, max)			Mean	SD	Range (min, max)
Total households	12 466	166.2	68.3	67–366		13 647	182.0	70.7	34–443
Households visited	5041	67.2	13.9	42–90		5441	72.5	13.8	41–90
Households with no one at home	2369	31.6	14.6	10–58		2792	37.2	14.0	8–62
Households surveyed	2672	35.6	3.4	28–46		2649	35.3	3.6	26–47
Households with at least one woman intervieweda	2623	35.0	3.1	29–44		2590	34.5	3.4	24–46
Households with at least one eligible childb	2294	30.6	1.3	26–34		2253	30.0	2.6	20–37
Interviewed womena	3003	40.0	4.6	33–51		3002	40.0	5.3	27–54
Interviewed women with at least one eligible childb	2508	33.4	2.6	26–41		2432	32.4	3.2	23–40
Eligible children per villageb	5516	73.5	9.4	53–100		5380	71.7	9.1	50–93

SD, standard deviation.

^a Women were interviewed if they were currently or previously married, aged 15 to 59 and had at least one child born alive during the 15-year period preceding the survey.

^b Children were eligible if they were born between 1 September 1992 and 31 December 2007 to a currently or previously married woman who was aged 15–59 at the time of the survey and had given birth in the village being surveyed. In CRHP villages the child had to have been born at least five years after project operations began in the village.

Table 2. Characteristics of control and intervention villages, households and mothers in a retrospective evaluation of the impact of the Comprehensive Rural Health Project (CRHP), Maharashtra state, India, 2007.

Characteristic	Control villages			CRHP villages		P
	No.	%		No.	%
Villagesa
CRHP activities
< 10 yrs	–	–		28	37.3	–
10 to < 20 yrs	–	–		21	8.0	–
20+ yrs	–	–		26	34.7	–
Irrigation
Yes	19	25.3		9	12.0	0.06
No	56	74.7		66	88.0
Householdsb
Land owner	2024	88.2		1794	79.6	< 0.001
No information given	3	0.1		4	0.2
Mothersc
Aged	29.5d	7.0d		29.3d	7.0d	0.24
Religion						0.15
Hindu	2363	94.2		2249	92.5
Muslim	79	3.2		124	5.1
Other	66	2.6		59	2.4
Caste						0.0005
Most backwarde caste/tribe	293	11.7		415	17.1
Other backward caste	163	6.5		290	11.9
Nomad	457	18.2		424	17.5
Non-backward	1595	63.6		1298	53.5
No information given	0	0.0		5	0.2

^a There were 75 control and 75 CRHP villages.

^b There were 2294 households in control villages and 2253 households in CRHP villages.

^c There were 2508 mothers in control villages and 2432 mothers in CRHP villages.

^d Age is given as mean age in years and standard deviation. Age was missing for one woman in the control villages.

^e Most backward: scheduled caste. Scheduled castes and tribes, formerly known as “depressed classes”, are groups explicitly recognized and afforded certain protections under the Constitution of India.

Table 3. Secondary outcomes for intervention and control villages at the household, mother and child level in retrospective evaluation of the impact of the Comprehensive Rural Health Project (CRHP), Maharashtra state, India.

Level	Control villages			CRHP villages		P
	No.	%		No.	%
Householda
Has treated drinking water	1202	52.3		1443	64.1	< 0.001
No information	0	0.0		2	0.09
Has toilet facility	180	7.8		314	13.9	< 0.004
Motherb
Health knowledgec	4.54c	1.63c		4.80c	1.60c	0.001
Children per woman						0.6
1	303	12.1		325	13.4
2	790	31.5		769	31.6
3	730	29.1		677	27.8
4+	685	27.3		661	27.2
Mother ever had miscarriage/stillbirth						0.6
Yes	342	13.6		315	13.0
No	2153	85.9		2097	86.2
No answer	13	0.5		20	0.8
Birth attendant at last birth						0.15
Health professional	986	39.3		1029	42.3
Not a health professional	1501	59.9		1388	57.1
No one	16	0.6		11	0.5
No answer	5	0.2		4	0.2
Place of last birth						0.4
Medical centre	906	36.1		939	38.6
Home	1595	63.6		1488	61.2
Other	2	0.08		3	0.1
No answer	5	0.2		2	0.1
Type of delivery at last birth						0.5
Vaginal cephalic	2396	95.5		2316	95.2
Vaginal breech	15	0.6		10	0.4
Caesarean	90	3.6		99	4.1
No answer	7	0.3		7	0.3
Received antenatal care before last birth						0.08
No	317	12.6		244	10.0
Yesd	2186	87.2		2185	89.8
No answer	5	0.2		3	0.1
Received postnatal care after last birth						0.3
No	1500	59.8		1392	57.2
Yese	1003	39.9		1031	42.4
No answer	5	0.2		9	0.4
Childf
Medical treatment in the last 3 mo						0.6
Yes	656	46.7		667	45.6
No	696	49.5		745	50.9
No answer	54	3.8		51	3.5
Place of treatment						0.5
Medical centre	646	98.5		652	97.8
Home	5	0.8		7	1.0
Elsewhere	3	0.5		8	1.2
No answer	2	0.3		0	0.0

^a There were 2294 households in control villages and 2253 in CRHP villages.

^b There were 2508 mothers in control villages and 2432 in CRHP villages.

^c Health knowledge is given as mean scores and standard deviations. The scores were missing for five women in the control villages and two women in the intervention villages.

^d Among women with at least one antenatal care visit, the median number of visits was 3 in both the control and the CRHP villages.

^e Among women with at least one postnatal care visit, the median number of visits was 2 in both the control and the CRHP villages.

^f Represents children under 5 years of age at the survey. There were 1406 children in control villages and 1463 in CRHP villages.

In control villages, 42.3% of women had no education at all compared to 39.3% of women in the CRHP villages. The mean (median) number of years of education in CRHP villages was 4.4 (4) for women. In control villages the figure was 4.3 (4).

Out of 10 896 eligible live births to 4940 eligible women during the evaluation period, 619 under-5 child deaths were reported. Of those deaths, 37 in control villages and 48 in intervention villages were reported to have occurred on the day of delivery. For 2 of the children reported dead, age at death was missing. Those children were excluded from the survival analyses. For an additional 11 children, the caste of the mother was not reported. Five-yearly estimates of age-specific mortality rates for the entire study area (CRHP and control villages) over the evaluation period are shown in Table 4.

Table 4. Age-specific child mortality rates,^a by 5-yearly periods of birth, for the entire study area covered in retrospective evaluation of the impact of the Comprehensive Rural Health Project, Maharashtra state, India, September 1992–December 2007.

Age	Period of birth
	Sep 1992–Aug 1997			Sep 1997–Aug 2002			Sep 2002–Dec 2007
	Rateb	95% CI		Rateb	95% CI		Rateb	95% CI
Neonatal	38.7	33.3–45.0		38.4	32.5–45.2		32.1	26.4–39.0
Infant	58.5	51.9–66.2		52.9	46.1–60.8		43.0	36.3–50.9
Under 5 yr	65.4	58.3–73.2		58.2	51.0–66.4		56.4	37.2–85.1

CI, confidence interval.

^a Estimated by the Kaplan–Meier method.

^b Deaths per 1000 live births.

Fig. 2 shows that age-specific mortality rates in the CRHP and control groups run closely together in the neonatal period and then diverge, with consistently lower mortality in the CRHP villages from approximately 1 to 5 years of age. This pattern is also apparent when crude estimates of age-specific childhood mortality rates are compared (Table 5) and is confirmed by formal analysis using Cox regression models (Table 6).

Fig. 2.

Kaplan–Meier estimates of age-specific childhood mortality in control and intervention villages in retrospective evaluation of the impact of the Comprehensive Rural Health Project (CRHP), Maharashtra state, India, September 1992–December 2007^a

^a There were 5515 eligible live births in the control villages and 5379 eligible live births in the intervention villages.

Table 5. Age-specific child mortality rates^a in intervention and control villages for the 15 years preceding the survey to evaluate the impact of the Comprehensive Rural Health Project (CRHP), Maharashtra state, India, September 1992–December 2007.

Age	Control villages (n = 5 515)			CRHP villages (n = 5 379)
	Rateb	95% CI		Rateb	95% CI
Neonatal	35.7	31.1–41.0		37.7	33.0–43.2
Infant	53.8	48.1–60.1		50.9	45.3–57.2
Under 5 yr	60.8	54.7–67.5		55.5	49.6–62.0

CI, confidence interval.

^a Estimated by the Kaplan–Meier method.

^b Deaths per 1000 live births.

Table 6. Estimated effect of the Comprehensive Rural Health Project (CRPH) on under-5 child mortality, Maharashtra state, India, September 1992–December 2007^a.

Modelsb	HR	95% CI
Under 5 yrc
Crude	0.93	0.77–1.11
Controlled for casted +religion + irrigation	0.90	0.75–1.09
Controlled for casted + religion + irrigation + birth period	0.91	0.75–1.09
Neonatale
Crude	1.06	0.84–1.33
Controlled for casted + religion + irrigation	1.03	0.82–1.29
Controlled for casted + religion + irrigation + birth period	1.04	0.83–1.31
Post-natal but under 5 yre
Crude	0.72	0.54–0.96
Controlled for casted + religion + irrigation	0.70	0.52–0.94
Controlled for casted + religion + irrigation + birth period	0.70	0.52–0.95

CI, confidence interval; HR, hazard ratio.

^a There were 10 883 children. Those with any missing data were excluded.

^b Age at death was missing for 2 children.

^c Under 5 yr estimated from the model including insteraction between age bands and CRHP intervention.

^d Caste was missing for 11 children belonging to all age groups.

^e Neonatal and post-natal but under 5 yr estimated from the model including interaction between age bands and CRHP intervention.

Overall, the hazard of death in CRHP villages was reduced by 10% for the period from birth to 5 years of age when the data were controlled for irrigation status of villages and the caste and religion of mothers. This effect was not significant at the 5% level, with the 95% confidence interval (CI) extending from a 25% reduction to a 9% increase. However, there was evidence (from the result of a test of the proportionality assumption implicit in the Cox model) that the ratio of the hazard in the CRHP villages to that in the control villages varied according to children’s age. For this reason we applied an interaction model in which the hazard ratio comparing the CRHP villages with the control villages differed in the neonatal and post-neonatal periods. This interaction was statistically significant (P < 0.04 for the model, adjusted for irrigation, caste and religion). The model provided no evidence that neonatal mortality was lower in CRHP villages (hazard ratio = 1.03; 95% CI: 0.82 to 1.29), but showed a statistically significant reduction (P < 0.02) in post-neonatal (up to 5 years) mortality (hazard ratio = 0.70; 95% CI: 0.52 to 0.94). Although childhood mortality rates declined over the birth periods covered by the study (Table 4, heterogeneity test for the effect of time period: P < 0.003 in Table 6), the hazard ratio for CRHP did not materially change when birth period was included as an additional covariate in the models. Further, there was no evidence that the hazard ratio for CRHP villages varied between birth periods (interaction test P > 0.1).

Discussion

In our evaluation of the effect of CRHP on childhood mortality over the period September 1992 to December 2007 we found a 30% reduction in the hazard of child death after the neonatal period for CRHP villages in comparison with villages in the control area; the reduction was significant at the 5% level. We did not, however, find a similar reduction in the hazard of neonatal death.

There are several possible explanations for this inconsistency. Causes of neonatal deaths in rural India – primarily birth asphyxiation, preterm birth and neonatal sepsis – are different from causes of post-neonatal child deaths – primarily pneumonia, diarrhoea and malaria.^1,3,31,32 WHO-recommended first-line treatments for malaria, pneumonia and diarrhoea can be administered in the community.^3,32 CRHP may have been more effective at ensuring prevention, early recognition and treatment of these diseases than it was at delivering clinical services, as suggested by superior levels of sanitation and water treatment in CRHP villages (Table 3).

There may also have been differential reporting of live and still births. A previous study found substantial underreporting of neonatal deaths in Maharashtra.³³ In the CRHP villages, a higher number of deaths occurring on the day of delivery (48 versus 37 among controls) and a lower number of women with stillbirths (315 versus 342 among controls, Table 3) were reported. Women in control villages may have been more likely to classify early postpartum death as a stillbirth than those in intervention villages because the CRHP had sensitized women to the notion of neonatal death and its prevention.

Our childhood mortality estimates for the entire study area are in line with other survey estimates of mortality for the region, although they are somewhat higher than official estimates for Maharashtra from the National Family Health Surveys for 1994–1998²⁵ and 2001–2005.²⁶ However, these surveys sample from both urban and rural areas and, since urban regions have lower mortality, they underestimate mortality for rural areas. Estimates based on the 1994–1998 National Family Health Survey data for rural Maharashtra only show much closer agreement with our results: neonatal, infant and under-5 mortality rates of 38.4, 52.3 and 67.6 deaths per 1000 live births, respectively.³⁴

The main strength of this study is that it provides the most carefully collected evidence to date that the hazard of child death was lower in CRHP recipient villages than in nearby non-intervention villages several years after the project was implemented. Our findings suggest that a community-based programme that trains village health workers to improve community knowledge can have long-lasting impacts on child mortality. We designed the study carefully to minimize the risk of bias when controls are selected retrospectively. We believe the methods used here could contribute to future evaluations of similar long-term projects.

Our study had several limitations. Reported mortality may have been impacted by migration. Movement of women from intervention to control villages may have reduced the mortality rate in their new residence and the inverse may also have occurred when women moved from control to intervention villages. We were only able to reduce this dilution partially by restricting our analyses to children whose mothers were resident in the same village at the time of both delivery and survey.

Despite careful selection of the controls, women from CRHP villages may have answered some of the questions in the survey differently from women in control villages because of their improved knowledge. As noted above, this differential reporting could have hindered our ability to show a reduction in neonatal mortality in CRHP villages, if such a reduction did indeed exist.

CRHP may also have been more effective when the disease burden was higher. When the project began in 1970, the child mortality rate in India was 192 deaths per 1000 live births. By 2000 the rate had dropped by more than half,³⁵ partly owing to a reduced incidence of malaria, improved treatment of diarrhoea and pneumonia and increased access to clinical services, including safer birthing practices, as a result of programmes introduced by the Government of India and by Maharashtra state.³⁶ Although these programmes and the clinical and health services provided by the government affected both our intervention and control villages equally, the large improvement in child mortality might have reduced the scope for measuring CRHP-generated improvements in recent years.

Funding cuts have forced CRHP to stop its work in 44 of our 75 study villages at some point in the past 25 years. Thus, at the time of our surveys only 31 (i.e. less than half) of the study villages had CRHP-led activities. If CRHP had run intensively in all study villages from its inception to the end of our survey period, we might have seen greater impact. Nevertheless, we have found that the project area continues to have better child survival outcomes and health knowledge among mothers than surrounding areas. Our results contrast with the reported failure of voluntary village health worker programmes in other studies.³⁷ Since CRHP village workers received only minor monetary compensation, remuneration is unlikely to account for the difference. Future studies should look specifically at the effect of compensation (versus, for example, empowerment of village health workers to give them a sense of “ownership” of the programme) on the success of community-based primary health care programmes. We believe that appropriate retrospective evaluation of the lasting impact of such programmes will make an important contribution to the evidence for short-term effects from randomized control trials.