How valuable are environmental health interventions? Evaluation of water and sanitation programmes in India

Abstract

Objective

To evaluate and quantify the economic benefits attributable to improvements in water supply and sanitation in rural India.

Methods

We combined propensity-score “pre-matching” and rich pre–post panel data on 9500 households in 242 villages located in four geographically different districts to estimate the economic benefits of a large-scale community demand-driven water supply programme in Maharashtra, India. We calculated coping costs and cost of illness by adding across several elements of coping and illness and then estimated causal impacts using a difference-in-difference strategy on the pre-matched sample. The pre–post design allowed us to use a difference-in-difference estimator to measure “treatment effect” by comparing treatment and control villages during both periods. We compared average household costs with respect to out-of-pocket medical expenses, patients' lost income, caregiving costs, time spent on collecting water, time spent on sanitation, and water treatment costs due to filtration, boiling, chemical use and storage.

Findings

Three years after programme initiation, the number of households using piped water and private pit latrines had increased by 10% on average, but no changes in hygiene-related behaviour had occurred. The behavioural changes observed suggest that the average household in a programme community could save as much as 7 United States dollars per month (or 5% of monthly household cash expenditures) in coping costs, but would not reduce illness costs. Poorer, socially marginalized households benefited more, in alignment with programme objectives.

Conclusion

Given the renewed interest in water, sanitation and hygiene outcomes, evaluating the economic benefits of environmental interventions by means of causal research is important for understanding the true value of such interventions.

Introduction

Over one billion people in the world lack access to clean water and more than twice that many lack access to basic sanitation.¹ Inadequate water and sanitation services, the second most common cause underlying medical conditions that lead to child mortality, impose considerable illness and coping costs on households in developing countries. These costs fall disproportionately on the poor, women and children. One of the Millennium Development Goals (MDGs) is to halve the number of people in the world who live without access to adequate water and sanitation services.^2,3 Reaching this goal would reduce coping costs, illness costs (including those generated by malnutrition), and therefore poverty as well. However, without clear empirical economic inputs into the planning and evaluation of interventions for improving water and sanitation services, the MDGs will be difficult to attain.

While epidemiological studies of the benefits of water and sanitation interventions have shown that diarrhoea can be reduced by 30–50%,⁴ these studies have had two limitations. First, they have not estimated the economic benefits of improving water and sanitation, despite the fact that such estimates are needed to properly allocate investments among different health interventions, different programmes that promote health or non-health water uses, and different sectors of the economy. Second, a recent meta-analysis and update of previous reviews found that half of them did not meet quality standards.⁴ For example, they did not: (i) account for baseline diarrhoea rates and pre-intervention hygiene behaviour, (ii) establish control groups, or (iii) consider whether non-programme events or conditions might have caused the observed effects. There are very few studies on the benefits of water and sanitation policies in India.

The limited literature on economic benefits is dominated by “stated preference” studies, such as contingent valuation and conjoint methods studies, which directly measure households’ “willingness to pay” for contingent (hypothetical) improvements in water and sanitation infrastructure (Table 1).^23–25 “Revealed preference” studies, on the other hand, measure economic benefits by examining actual preventive behaviours, such as treating the water.^{13–19,26,27} Particular types of revealed preference studies^15,16 known as avoided or coping cost studies – thus named because they examine the prevention costs incurred to cope with poor water and sanitation – measure the savings in prevention costs resulting from improvements in water and sanitation.

Table 1. Selected valuation studies of water supply based on observational cross sectional data.

Study	Design	Summary of resultsa
Briscoe et al. (1990), Brazil5b	Contingent valuation	WTP for improved service = 15.5–18.1 if improved services available WTP for private tap = 8–8.5 in areas if improved services not yet available
Bohm et al. (1993), Philippines6	Contingent valuation	WTP for improved service = 6.2 if you have private tap connection WTP for private tap = 3.5 or 5.7 if you use a well or a stand post
Altaf et al. (1993), Pakistan7	Contingent valuation	WTP for improved water service = 2.7–13.4 WTP for private tap = 3.7 if you do and 12.4 if you do not have a tap WTP for public tap = 7.9 if you do not have a tap
Whittington et al. (1993), Ghana8	Contingent valuation	WTP for private tap = 5.4
Griffin et al. (1995), India9	Contingent valuation	WTP for improved service = 6.4 if you have private tap connection WTP for private tap = 2.5 if you do not have a private tap, but neighbourhood has taps WTP for private tap = 1.4 if neither you nor your neighbourhood has private taps
Whittington et al. (2002), Nepal10	Contingent valuation	WTP for improved water supply = 57 if you have a private tap WTP for private tap = 46.5 if you do not have a private tap WTP for shared tap = 12.7 if you do not have a private tap
Gunatilake et al. (Forthcoming), Sri Lanka11	Contingent valuation	WTP for improved water supply = 14.9 if you have a private tap WTP for private tap = 4.2 if you do not have a private tap
Rosado et al. (2006), Brazil12	Contingent valuation	WTP for improved or new water connection = 4.78
Whittington et al. (1990), Kenya13	Coping costs	Cost of buying from vendors = 30.2 Cost of collecting from kiosks = 13.7 Cost of collecting from open wells = 8.2
Whittington et al. (1990), Nigeria14	Coping costs	Cost of buying from vendors = 21.4–34.2
Pattanayak et al. (2005), Nepal15	Coping costs	Cost of collecting, treating, storing, buying = 12 (have piped connection) Cost of collecting, treating, storing, buying = 11 (lack piped connection)
Alam & Pattanayak (2009), Bangladesh16	Coping costs	Cost of collecting, treating, storing, buying = 4.9
Anselin et al. (2008), India17	Hedonic	House rent premium for improved piped water service in Bangalore = 10 House rent premium for improved piped water service in Bhopal = 4.3
Komives (2003), Panama18	Hedonic	House rent premium for in-house piped water = 202 in Panama City (formal settlement) House rent premium for in-house piped water = 41.7 in Panama City (informal settlement) House rent premium for yard tap = 14.6 in Panama City (formal settlement) House rent premium for yard tap = 22.9 in Panama City (informal settlement)
Komives (2003), Ecuador18	Hedonic	House rent premium for in-house piped water = 85 in Quito (have access to network) House rent premium for yard tap = 21.3 in Quito (have access to network) House rent premium for in-house piped water = 49.6 in Guayaquil House rent premium for yard tap = 46 in Guayaquil
North & Griffin (1993), Philippines19	Hedonic	House rent premium for in-house piped water = 11.4 (high-income) House rent premium for in-house piped water = 13.1 (middle-income) House rent premium for in-house piped water = 8.2 (low-income) House rent premium for deep well or yard tap = 5.1 (high-income) House rent premium for deep well or yard tap = 5.5 (high-income)

WTP, willingness to pay.

^a All values reported are adjusted for purchasing power parity and inflated to 2007 United States dollars.^20–22

^b These values are only adjusted for inflation because purchasing power parity was not available for the study year.

While stated preference benefit estimates are comprehensive, they are vulnerable to validity threats and may overestimate true economic benefits. Savings in coping costs, on the other hand, may underestimate the true economic benefits of a given intervention because they do not capture the economic value of a lowered risk of death or of reduced pain and suffering.^28,29 We view avoided illness costs as coping costs because medical spending can lead to the prevention of severe diarrhoea and mortality.^28,30 Although an extensive literature exists on the appropriate theoretical model for evaluating the impact of environmental interventions on economic welfare,^28,29,31 no causal empirical studies have measured the economic benefits generated by water and sanitation policies.³²

To address the knowledge gaps described above, we embedded a valuation study within a rigorous impact evaluation study and estimated the economic value of the average “treatment effect” of a community demand-driven water and sanitation programme. We employed a unique combination of propensity-score “pre-matching” and large panel data to estimate the economic impacts of a multi-dimensional environmental health programme. The results of the study are timely and relevant for new intervention planning and design efforts, including cost–benefit analyses of global and national investments in water and sanitation programmes aimed at improving child health (MDG 4) and improving sustainable access to safe drinking water and basic sanitation (MDG 7).

Methods

We estimated the economic impact of a community demand-driven programme launched by the government of the state of Maharashtra, India, with support from The World Bank to improve water, sanitation and hygiene in rural areas of the state. The main objectives of the programme were to increase access to drinking water and sanitation services in rural areas, institutionalize the decentralized delivery of such services by local governments, and improve rural living conditions. Village residents organized to improve their water supply and sanitation and hygiene conditions by choosing interventions that best matched their needs and capabilities, and they applied to the state government to participate in the programme. The latter lasted five years (2004–2008) and was implemented in approximately 2800 villages in 26 of the state’s 33 districts. Programme sustainability was promoted by requiring communities to pay 10% of capital costs and 100% of the operational and maintenance costs of the programme.

For our study of the programme’s benefits we sampled 242 villages from four geographically different districts. A published study protocol describes how restrictions, stratification and matching were combined to reduce sampling bias.³² Using propensity score matching, we matched each of 80 programme (treatment) villages in these districts to two similar non-programme (control) villages to ensure comparability between villages based on community indicators (e.g. demographics, housing, socioeconomic conditions, village infrastructure) and household variables (e.g. health outcomes, water and sanitation, personal hygiene). Table 2 illustrates the resulting similarity in key indicators. In control communities other government programmes, such as the Total Sanitation Campaign, were conducted as usual.³² Statistical power analysis indicated that a sample size of approximately 50 households with children 5 years of age or younger from each of the 242 villages would generate 80% power to detect an intervention impact of ≥ 30% in the population of children under 5 with a baseline diarrhoea prevalence of 22%. Household and community survey questionnaires were designed to collect impact indicator data on child health (as measured primarily by diarrhoea among children under 5 in the two weeks before the survey) and on child growth, personal benefits, and variables for computing cost savings (e.g. time costs), plus a range of individual variables (e.g. sex, age, social class, caste, religion) and household variables (e.g. family size and composition, education, housing conditions, assets, water and sanitation practices). Pre-intervention data were collected from a sample of approximately 9500 households before (May–June) and after (August–September) the 2005 monsoon. Post-intervention data were collected from the same households in the same months in 2007 to account for etiologic, structural and behavioural differences across seasons. To minimize confounding, we stratified our analysis by season and compared outcomes within seasons, but not across them. Thus, we can comment on seasonal differences but cannot fully explain them.

Table 2. Test of balance across treatment and control villages using pre-intervention (2005) data in study of community demand-directed water, sanitation and hygiene programme in Maharasthra, India.

Covariate of interest, mean/average value	Treatment	Control	z-valuea
Based on secondary census data used in matching model
Percentage of children < 6 years old in village	17	16	0.74
Percentage of scheduled castes in village	10	8	1.78
Percentage of scheduled tribes in village	29	31	−0.36
Percentage of female workers in village	45	45	−0.05
Percentage of cultivators in village	49	52	−1.18
Percentage of agricultural labourers in village	39	34	1.55
No. of households in village	385	387	−0.03
Household size in village (no. of dwellers)	5	5	0.03
Percentage of literate females in village	52	54	−1.05
Percentage of households with private tap in block	42	42	0.07
Percentage of households without toilets in block	83	84	−0.49
Based on baseline (2005) household survey data
Percentage of children aged < 5 years with diarrhoea	11	10	1.62
Percentage of households using private tap	18	24	−1.55
Percentage of households using private toilet	13	10	0.96
No. of critical times a caregiver washed hands	2.3	2.4	−0.51
No. of critical times a child washed hands	1.1	1.2	−0.44
No. of households treating drinking water	64	63	0.11
No. of households stating public well water quality was poor	19	24	−1.77
No. of households stating public tap water quality was poor	24	22	0.44

^a For mean differences after adjusting standard errors to account for clustering at the village level.

Data from Pattanayak et al., 2009.³²

Each programme community was expected to improve in all intervention components (water supply, sanitation and hygiene). However, most village programme plans focused on strengthening water sources and/or on developing new ones, treating sources and distributing piped water, and constructing private toilets and drains. Unfortunately, hygiene education was not demanded by the community.

We followed the methods described in the literature to compute coping costs due to inadequate water and sanitation services¹⁵ and the private costs of water-related illnesses.³⁴ For both sets of computations, time was valued at one-half of the village-level gender-specific average hourly wage.¹³ All costs are reported in terms of monthly household costs in purchasing power parity (PPP) adjusted to 2007 United States dollars (US$).

Monthly coping costs were calculated as the sum of expenses attributable to: (i) the time spent collecting water, (ii) the time spent on going to defecate, (iii) water treatment (i.e. filtering, boiling and use of chemicals), and (iv) water storage. To monetize the time spent collecting water, we multiplied the number of hours spent walking to and from water sources and waiting at the sources by the number of trips taken and the applicable wage estimate. For sanitation related costs, we assumed each household member made only one trip per day to the main sanitation site, aggregated the total sanitation-related travel time to the household level, and multiplied it by one-half of the hourly wage.

Turning to water treatment costs, for households that used filters, we converted the one-time cost of filters into monthly costs by amortizing the reported value over the lifespan of the filters at an estimated annual discount rate of 15%, typical for India. For households that boiled their water, we relied on previously reported estimates of the cost of boiling water^27,35 (i.e. the estimated cost of boiling 5 litres was 27 Indian rupees (Rs), or US$ 0.68) and assumed 5 litres of boiled water per day provided enough drinking water in these households. Finally, to estimate storage costs, we first obtained the costs and the useful life of storage containers from households and shop keepers. For each type of storage container, we first amortized the storage costs over the respective lifespan at a discount rate of 15% and then multiplied the implied monthly costs by the number of each type of container the household owned.

The private cost of illness due to the most recent episode of diarrhoea was represented by the sum of out-of-pocket expenditures (i.e. costs borne by private households rather than the public health system) and lost income. We monetized the time costs incurred by patients and caregivers as above using community average wages. Following an efficiency wage logic, we valued children’s time in proportion to their age. For example, a 4-year-old’s time costs were pre-multiplied by 0.25, and those of a 16-year-old by 1.

The resulting measures of coping costs (or defensive expenditures) and cost of illness measures were used to estimate the economic impacts caused by the programme by applying a difference-in-difference strategy on this pre-matched sample. The difference-in-difference estimate is the mean of the difference between the difference in outcomes (over time) in programme villages and the difference in outcomes in control villages.^33,36 We inflated the standard errors to account for correlated information from multiple households in the same village using robust variance estimation techniques. The pre–post data collection allowed us to use a difference-in-difference estimator to measure “treatment effect” by comparing coping cost and illness cost outcomes in the treatment and control villages during both periods.

Results

Baseline findings

At baseline only 21% of households were using private taps and only 12% were using individual household latrines. Although 50% of respondents reported that they filtered water, less than 5% reported using chemicals to purify it and only about 2% reported boiling water. On average, household members washed their hands during critical times – i.e. before preparing food or cooking, before eating, before feeding children, after changing babies or handling children's faeces, and after defecating – only half as often as required.

As shown in Fig. 1, the average monthly time costs of collecting water were: in the dry season, US$ 31.94 in 2005 and US$ 30.24 in 2007; in the rainy season, US$ 38.76 in 2005 and US$ 20.10 in 2007. The household-level mean monthly time costs related to sanitation were: in the dry season, US$ 9.31 in 2005 and US$ 8.05 in 2007; in the rainy season, US$ 9.32 in 2005 and US$ 7.73 in 2007.

Fig. 1.

Average household coping costs, by season, year and intervention, in community demand-directed water, sanitation and hygiene programme, Maharasthra, India, 2005 and 2007

The mean monthly cost of boiling water was: in the dry season, US$ 1.96 in 2005 and US$ 0.56 in 2007; in the rainy season, US$ 1.64 in 2005 and US$ 0.83 in 2007. On average, the monthly cost of filtration was about US$ 0.13. Finally, households that treated their drinking water with chemicals spent no more than US$ 0.07 per month.

For storage, household monthly expenses during the dry season were an average of US$ 10.37 in 2005 and US$ 8.36 in 2007, and during the rainy season, they averaged US$ 6.75 in both 2005 and 2007.

Cost of illness

One diarrhoea case had occurred in the two weeks before the interviews in at least 30% of the households in 2005 and about 20% in 2007. Out-of-pocket medical expenses included clinic/hospital fees, medicines, transportation, lodging and meals. In 2005, households spent an average of US$ 16.63 in the dry season and US$ 8.58 in the rainy season, whereas in 2007, they spent an average of US$ 12.52 in the dry season and US$ 5.50 in the rainy season (Fig. 2).

Fig. 2.

Average household illness costs, by season, year and intervention, in community demand-directed water, sanitation and hygiene programme, Maharasthra, India, 2005 and 2007

As shown in Fig. 2, lost implicit income per household due to diarrhoea episodes among adult patients amounted, on average, to US$ 4.24 in 2005 and US$ 3.18 in 2007 during the dry season, and to US$ 3.86 in 2005 and US$ 2.19 in 2007 during the rainy season. During the dry season, caregiving led to an average loss in implicit income of US$ 4.11 in 2005 and US$ 3.38 in 2007; in the rainy season, the loss amounted to US$ 4.39 in 2005 and US$ 2.25 in 2007.

Programme impact

Difference-in-difference estimates showed that the community demand-driven water, sanitation and hygiene programme had a moderate but significant impact on the reported adoption of taps and toilets: we found 13% more private tap use and 7% more private toilets in programme villages compared to control villages. However, we did not find a statistically significant impact on handwashing or home water treatment.

The changes in tap and toilet use reduced coping costs among households in treatment villages by an average of US$ 6.98 compared with households in control villages (Table 3). Most of these savings varied by season and were due to reductions in the time household members spent travelling to and waiting at water sources (a saving of about US$ 5) and going to defecate in the open (a saving of about US$ 2). Difference-in-difference analyses of cost of illness components and overall cost of illness showed no programme impact in either the dry or the rainy season, since both programme and control villages incurred lower illness-related costs in 2007 than in 2005. This is not surprising because the programme did not result in any differential change in diarrhoea or in handwashing or other key hygiene practices associated with diarrhoea.

Table 3. Estimated effects of a community demand-directed water, sanitation and hygiene programme on coping costs^a and cost of illness^a in the dry and rainy seasons, Maharasthra, India, 2005 and 2007.

DID analysisb	Dry	Rainy
Total monthly household coping costs	−6.98**	−0.37
Time costs for water	−4.84*	0.40
Time costs for sanitation	−1.64***	−1.42**
Filter costs	0.0023	0.0094
Costs of boiling	0.36	−0.21
Chemical costs	0.027*	0.002
Storage costs	−0.84	0.81**
Total household COI	1.25	0.11
Out-of-pocket medical expenses	1.88	−0.23
Patients' lost income	−0.52	0.15
Caregivers' lost income	−0.11	0.19

COI, cost of illness; DID, difference-in-difference. *P < 0.10; **P < 0.05; ***P < 0.01.

^a All coping cost and COI values are adjusted for purchasing power parity (PPP) and inflated to 2007 United States dollars (US$).^20–22 In 2007, the PPP exchange rate was 15.139 Indian rupees for US$ 1.

^b DID estimation includes covariates unbalanced at baseline (household knowledge of public health messages regarding handwashing and safe handling and storage of food and water; household belief that having a water supply is a public policy priority; household belief that sanitation is a public health priority; and household participation in the village water and sanitation committee). Standard errors were corrected for clustering at the village level.

Coping cost savings were larger for households below the poverty line, whereas scheduled castes and tribes saved more in both absolute and relative terms (Table 4). The programme targeted communities with a high proportion of poor and socially marginalized groups and attempted to involve them in programme planning, implementation and monitoring. Therefore, the better outcomes attained by these groups are in line with the programme’s objectives.

Table 4. Estimated effects of a community demand-directed water, sanitation and hygiene programme on coping costs^a and cost of illness^a in the dry and rainy seasons, by subgroup, in Maharasthra, India, 2005 and 2007.

DID analysisb	Overall	BPL	APL	SCST	Open
Dry season (May–June)
Total monthly household coping costs	−6.98**	−6.24*	−6.52*	−9.64***	−6.21*
Total household cost of illness	1.25	−4.85	7.02	−6.71*	4.96
Household welfare	−5.76	−11.33**	0.51	−16.45***	−1.28
Rainy season (August–September)
Total monthly household coping costs	−0.37	−0.79	−1.57	−6.52	1.96
Total household cost of illness	0.11	2.27	−0.92	−0.14	0.28
Household welfare	−0.53	1.04	−2.62	−6.51	1.83

APL, above poverty line; BPL, below poverty line; DID, difference-in-difference; SCST, scheduled caste and scheduled tribe. *P < 0.10; **P < 0.05; ***P < 0.01.

^a All coping cost and cost of illness values are adjusted for purchasing power parity (PPP) and inflated to 2007 United States dollars (US$).^20–22 In 2007, the PPP exchange rate was 15.139 Indian rupees for US$ 1.

^b DID estimation includes covariates unbalanced at baseline (household knowledge of public health messages regarding handwashing and safe handling and storage of food and water; household belief that having a water supply is a public policy priority; household belief that sanitation is a public health priority; and household participation in the village water and sanitation committee). Standard errors were corrected for clustering at village level.

Although some sanitation-related time savings occurred in the rainy season, programme benefits were noted primarily in the dry season. While the study was not designed to explain seasonal differences, we found, for example, that in the rainy season household members used more water sources and walked and waited less. These behavioural adjustments could be induced by structural factors such as the availability of water for longer periods or higher wages in the rainy season. Also, a secular trend suggests that more households can access taps and toilets during the rainy season (which follows the dry season) at baseline, so that there is less potential for improvements in the follow-up period. Our study was not designed and powered to determine which factor dominated.

Discussion

This paper reports on the average monthly household-level cost savings attributable to a community demand-driven water, sanitation and hygiene programme in rural India. Our rigorous quasi-experimental impact evaluation relied on: (i) data on household cohorts collected before and after the intervention (2005 and 2007, respectively) from programme and control villages; (ii) propensity score estimation to pre-match programme villages with observationally equivalent control villages; (iii) sufficient sample size (four repeat measures of approximately 9500 households in 242 communities); and (iv) a panel-based difference-in-difference estimation strategy to control for pre-existing differences between programme and control villages. These costly data collection and analysis strategies will reduce bias, but may not have eliminated them. However, given the political and practical considerations of studying a real world programme, we have shown that this quasi-experimental design is the best way to maintain rigor, without interfering with programme implementation.³²

This paper makes several contributions to what is known about the economic benefits of programmes that improve water and sanitation. First, we present methods that make it possible to attribute such benefits to environmental programmes and policies in the sector using a real world case study. Second, we found that the average household in programme communities could save an average of US$ 6.98 per month, or roughly 5% of monthly cash expenditures. Moreover, the benefits were higher in poorer and socially marginalized households, in line with the social targeting objectives of the programme. These coping cost reductions were largely due to improved access to better water and sanitation services, which shortened the time household members spent travelling to and waiting at the service source. Improved access leads to greater use of better services and to better public health outcomes as long as households also adopt complementary hygienic practices, such as handwashing, in response to effective hygiene promotion.

Diarrhoea incidence and the associated illness costs fell during the evaluation period in both intervention and control villages presumably because of overall socioeconomic development in rural Maharashtra and routine water and health programmes (e.g. activities in control villages). This general improvement in socioeconomic conditions, coupled with the lack of behavioural change in programme villages, may explain why, on average, we failed to find an impact on child diarrhoea and illness costs when we compared treatment villages with control villages.

Our intention-to-treat estimates of programme benefits are realistic because they are based on the actual uptake of improved services by households (e.g. use of taps) rather than on assumed or planned full coverage. Because the programme was driven by community demand, the communities and the households in them had to be willing and able to access and use the improved services. In practice this resulted in less than 100% community coverage. Consequently, in direct contrast to multivariate regressions of cross-sectional data on connections to piped sewage and water, our method avoids excessive attribution of economic benefits to the programme.

For comparison of our results with those of other studies (Table 1) on the economic benefits of obtaining a tap or a toilet per se, we used the difference-in-difference strategy to compare the cost change among adopters (those who used taps and toilets) to the cost change among non-adopters and found that adopting taps and toilets resulted in a cost reduction of US$ 15.59 in 2007 PPP terms. While these figures represent the economic benefit of adopting and using taps and toilets, they do not reflect the programme’s overall impact. Instead, the coarser intention-to-treat statistic (US$ 6.98) provides a conservative estimate of the programme’s benefits by averaging across adopters and non-adopters in programme communities.

In summary, renewed attention to the call for improving water and sanitation has spurred several global and regional assessments of the costs and benefits of delivering water and sanitation services.^37–39 These assessments rely heavily on the extrapolation of limited data or on assumptions because of a scanty and inconclusive literature on the causal economic benefits of water, sanitation and hygiene policies. Unless environmental health interventions apply the kind of rigorous causal impact evaluations described in this paper, designed to avoid excessive attribution, water and sanitation programmes will be pipe dreams.