Impact of a Randomized Controlled Trial in Arsenic Risk Communication on Household Water-Source Choices in Bangladesh

Abstract

We conducted a randomized controlled trial in rural Bangladesh to examine how household drinking-water choices were affected by two different messages about risk from naturally occurring groundwater arsenic. Households in both randomized treatment arms were informed about the arsenic level in their well and whether that level was above or below the Bangladesh standard for arsenic. Households in one group of villages were encouraged to seek water from wells below the national standard. Households in the second group of villages received additional information explaining that lower-arsenic well water is always safer and these households were encouraged to seek water from wells with lower levels of arsenic, irrespective of the national standard. A simple model of household drinking-water choice indicates that the effect of the emphasis message is theoretically ambiguous. Empirically, we find that the richer message had a negative, but insignificant, effect on well-switching rates, but the estimates are sufficiently precise that we can rule out large positive effects. The main policy implication of this finding is that a one-time oral message conveying richer information on arsenic risks, while inexpensive and easily scalable, is unlikely to be successful in reducing exposure relative to the status-quo policy.

1 Introduction

The use of labeling and information disclosure policies is on the rise in environmental and other health policy domains. The explicit goal of these policies is to promote positive changes in behavior by increasing awareness of health risks from certain activities. The provision of health information is a potentially important policy tool in developing countries because information campaigns can often be conducted at relatively low cost and with less state regulatory infrastructure compared to other policy options [13, 16, 19, 20, 24, 27, 37]. Environmental information campaigns have also been heavily used in richer countries. For instance, in the United States, labels have been used to differentiate environmentally-friendly products [8, 11, 36] and to better inform the public of environmental hazards [6, 10, 21, 26, 25, 34, 35, 38].

Despite the proliferation of such information-based policies, there is little agreement in the literature on how to present information in ways that best motivate health-improving behaviors. A key issue is how best to convey risks in ways that individuals can readily understand and use in decision making. Should this information be provided unprocessed to consumers or should it be reduced to one or more simple thresholds for evaluating risk? For example, Smith et al. [35] conducted a randomized experiment on household response to different formats of information on radon risk in homes. They found that the combination of qualitative information and specific decision guidance based on a threshold level of radon was associated with higher levels of radon mitigation activity relative to quantitative risk information which provided richer, but also more complex, decision guidance.

Our paper contributes to this literature by studying the impact of information format on house-hold decisions to mitigate health risks from arsenic in drinking water in Bangladesh. We compare household water-source choices when presented with simple binary safety guidance to decisions when a richer emphasis message is added that highlights that lower arsenic exposure is better, irrespective of a specific safety threshold. An additional contribution of this study is that it provides evidence of responses to variations in risk presentation in a developing country, among a largely illiterate population.

The natural occurrence of arsenic in groundwater is an important public health concern. Long-term exposure to arsenic in drinking water has been linked to several health risks, including skin lesions, cancers and cardiovascular diseases, with a latency period of 5 to 15 years for early health effects and 20 years or more for cancers (see [12] and references therein). In Araihazar upazila, one of about 500 similar administrative units in Bangladesh, Argos et al. [4] estimate that drinking groundwater with greater than 150 micrograms per liter (or parts per billion, ppb) arsenic almost doubled all-cause mortality compared to drinking water with up to 10 ppb.

The magnitude of the arsenic problem in Bangladesh became evident after almost five million wells were tested for arsenic by the Bangladesh Arsenic Mitigation and Water Supply Program (BAMWSP) beginning in 1998. This testing revealed that 35 million people (about 30% of the population) were habitually drinking tubewell water that exceeded the Bangladesh standard of 50 ppb, while 57 million (45%) drank water with concentrations above the lower World Health Organization (WHO) recommended limit of 10 ppb. The results of the BAMSWP arsenic tests were conveyed to local populations using a bright-line, binary format. Wells with arsenic levels in excess of the Bangladesh health standard (50 ppb) were labeled “unsafe” and painted red, while those with arsenic levels below 50 ppb were labeled “safe” and painted green [1].

A key feature of the distribution of arsenic in groundwater is that it is highly heterogeneous, to the point that users of unsafe wells usually live within 100 meters of a safe well [41]. In communities where tubewells’ arsenic contamination is known, well-sharing among households is often a viable short-term mitigation strategy [40]. Previous research in Bangladesh suggests that providing households with bright-lines (safe versus unsafe) information on arsenic status of tube wells encourages significant switching from unsafe to safe wells. In Araihazar, in locations proximate to our study areas, households informed of the unsafe concentration of arsenic in their well water have been shown to be 26–46% more likely to switch to a different source of drinking water within one year than others whose water had been tested safe [28, 31, 33].

Despite high levels of switching, there remain significant environmental health concerns. First, it is widely acknowledged that no known level of arsenic exposure is completely safe and that health benefits can result from any decrease in arsenic exposure. It is then cause for concern that the Bangladeshi standard is five times as large as the threshold adopted by the WHO and other governmental organizations such as the United States Environmental Protection Agency (EPA). Second, decreased arsenic exposure from, say, 400 to 200 ppb yields a significantly greater reduction in risk than reducing exposure from 60 to 30 ppb [29, 30]. However, the bright-lines risk presentation by BAMWSP encourages the latter switch, but not the former. Finally, millions of wells remain untested [42]. In fact, new wells are being privately installed every year, while little testing has been carried out outside of the BAMSWP project.¹ In view of the possibility of future new testing campaigns, evidence suggesting which communication modes are more effective at inducing mitigating behavior is of high value.

This paper reports the results of a cluster randomized controlled trial (RCT) conducted to evaluate alternative formats of risk presentation about arsenic concentrations in tubewell water in 43 villages of Araihazar. In our study, all wells were tagged using the same type of plate and all households were informed about the arsenic level in their well and whether that level was above or below the Bangladesh standard for arsenic. The experimental variation came from a one-time oral message that was added in a randomly determined subset of “emphasis” villages. Households in these villages received additional information explaining that lower-arsenic well water is always safer and these households were encouraged to seek water from wells with lower levels of arsenic, irrespective of the national standard. In contrast, households in the other group of villages, which we refer to as “status-quo,” were simply encouraged to seek water from wells below the national standard. The spirit of the communication intervention parallels those used in environmental information programs in the United States for both radon and lead in homes. For both of these toxic chemicals, there is an EPA action level (a level above which the homeowner is encouraged to take action to mitigate the risk), combined with information that emphasizes that no level is completely safe and benefits may results from taking action even below the level. What is different in our context is that we encouraged switches to a lower-arsenic well even if that well was still above the action level. In addition, our message is conveyed one time, orally, to largely illiterate households.

We develop a simple theoretical model of household water-source choice and evaluate how the emphasis message is likely to affect such choices. We argue that the overall predicted impact is of ambiguous sign. Empirically, receiving the emphasis message resulted in a six to seven percentage point decrease in switching, but that effect is not statistically significant. However, the estimates are sufficiently precise that we can rule out large positive treatment effects, which would imply increased switching and reduced exposure from the emphasis message. Specifically we can rule out treatment effects larger than two percentage points at the 95% confidence level. This is a useful finding for policy because many areas of Bangladesh have very-high levels of arsenic in groundwater and switching to lower arsenic wells, even wells with moderately-high levels of arsenic, would likely have substantial health benefits. Our results suggests that conveying the complexity of arsenic risk with a one-time communication, while easy to implement and scale up to the national level, is unlikely to increase mitigating behavior relative to the current safe/unsafe message used in Bangladesh.

As an extension, we investigate possible heterogeneity in responses to the emphasis message conditional on baseline arsenic levels. Our randomized experiment was not designed to estimate treatment effects by sub-group and these analysis are exploratory and serve as the foundation for future hypothesis testing. Our non-experimental results indicate that there may be heterogeneity in treatment impacts that warrant further study.

The paper proceeds as follows: Section 2 describes the design of the randomized controlled trial. Section 3 provides a conceptual framework to understand how the emphasis message may affect household water-source choices using both rational choice and behavioral economics lenses. Section 4 describes the data. Method and results can be found in Section 5. The final section concludes.

2 Randomized Control Trial (RCT) Design

This study took place in a 100-km² area of Araihazar upazila located approximately 30 km from the capital city of Dhaka. In 2003, BAMWSP conducted arsenic tests throughout Araihazar and painted wells red or green based on arsenic content. In addition, the Columbia University Superfund Basic Research Program has had an active interdisciplinary research program in some areas of Araihazar since 2000.² An earlier study analyzed responses to BAMSWP in 75 villages in neighboring areas, in order to test whether switching rates differed substantially in locations where the Superfund research team did not operate [33]. At the data collection stage of this earlier study, in 2005, field workers drew water samples from any new or previously untested wells identified in the study villages. This led to water samples from 558 wells in 48 villages. We excluded data from two villages because they were used to pre-test our survey instrument and train enumerators, and another three because they included a single well. ³ Our study thus uses data from a total of 43 villages.

In 2008, we conducted a randomized controlled trial on risk communication as part of the effort to disseminate the results of tests on the water samples collected in 2005. Unlike BAMSWP, which adopted the red-green painting to communicate safety levels, our team tagged the wells with an anodized and embossed, laser-engraved aluminum plate designed within the Columbia University project. Figure 1 shows one such plate, whose most visible component is a pictorial representation of whether the well water is suitable for drinking. The plate shows a hand holding a drinking cup that is covered by a large cross when the arsenic level is above 50 ppb (as in the case of the well in the figure). The plate also indicates the year when the water sample was collected and the specific arsenic concentration detected in the water tested from that well. The metal plate also indicates a unique well ID, which was used to match the test results to the tube-wells, together with geographical coordinates and records of the tube-well owners’ names. The ID and coordinates were used to verify whether the tube-well whose water was sampled was the same for which results were being delivered. This step was crucial because tube-well owners sometimes transfer pipes and hand-pumps from an old to a new well.

Figure 1. Tubewell and Test Result Plate.

The picture shows an example of the black-and-white plate used to communicate results: the plate indicates the well I.D., the year when the water sample was taken, the arsenic level and a hand holding a drinking cup, in this example crossed out because the arsenic level (336ppb) is above the safety threshold. The smaller, stainless steel plate visible above the black-and-white one indicates the well I.D. and was attached to the well in 2005, at the time of the water sample collection.

The intervention randomly varied how the results were orally communicated to households in a one-time discussion with survey staff. We chose to randomize at the village level in order to avoid information spillovers that could have been expected if households from the same village had received different messages. Villages were stratified by union and randomly assigned to treatment or control within strata, using a pseudo-random number generator. ⁴ Households in a first group were read a script whose English translation was as follows:

The national safety standard in Bangladesh is 50 ppb (again, that is micrograms per liter). That means the federal government says drinking water with more than 50 ppb arsenic is not safe. When possible you should seek to fetch drinking water from a well that is labeled safe.

Although the plate attached to the well also indicated the arsenic level, the script followed a bright-line format that only stressed the difference between ‘safe’ and ‘unsafe’ wells. This script was similar to that used by the Government of Bangladesh, as well as by the Columbia University team. We will thus refer to locations where results were orally communicated with this format as “status-quo” villages.

Households in the second group of locations, which we refer to as “emphasis” villages, received a message that also provided the government standard but, in addition, emphasized that if given a choice between two or more wells, the well with the lowest arsenic level should be chosen, even in situations where all wells have the same binary safety status. Specifically, the message read as follows:

The national safety standard in Bangladesh is 50 ppb (again, that is micrograms per liter). That means the federal government says drinking water with more than 50 ppb arsenic is not safe. However, we want to emphasize that whatever the level of arsenic in your drinking water now, if you have a choice of water from several wells it is better to drink water from the well with the lowest level of arsenic. For example, if you have a choice between a well with 200 ppb arsenic and a well with 100 ppb arsenic, drinking water from the well with 100 ppb arsenic is better for you. If you have a choice between a well with 40 ppb arsenic and a well with 10 ppb arsenic, drinking water from the well with 10 ppb arsenic is better for you. When possible you should seek to fetch drinking water from the well with the lowest arsenic level.

The difference between the two communication modes was thus relatively subtle: all wells were tagged with identical plates and both messages included much of the same information, but the experimental (richer) message emphasized that drinking water with lower arsenic is always safer and encouraged households to switch to lower arsenic wells, irrespective of the national safety threshold. The exclusive focus of the RCT design on variations in how results were communicated orally, rather than on the plates, was a function of logistical constraints. We could not vary the result plates, because these had already been manufactured prior to our study according to a design followed by the Columbia University Superfund research team throughout their study area in Araihazar (Figure 1). In addition, ethical concerns would have made it inappropriate to withdraw the bright line message from one subgroup, because this had been routinely included, at the time, as part of the labeling campaigns throughout Bangladesh. ⁵

3 Conceptual Framework

In this section, we develop a simple choice model to illustrate how the emphasis message may affect household water-source selection. We assume that decisions are made at the household level and that households decide to switch to an alternative source of drinking water if such a move increases utility. Let S denote a binary variable = 1 if the household decides to change the main source of drinking water, and let R (S) denote the household‘s perceived arsenic risk associated with a given water source. Utility is defined as a function of risk perceptions and switching: U (R(S), S). We make two further assumptions about the utility function:

• (Assumption 1) conditional on perceived arsenic risk, a household prefers the status-quo well because switching is costly (that is, U(R, 0) > U (R, 1)), and

• (Assumption 2) conditional on the decision whether to switch, utility is decreasing in perceived arsenic risk (that is, U (R, S) < U (R′, S) S if R > R′). ⁶

The household will switch if:

$$U(R(1),1)>U(R(0),0).$$ (1)

The impact of the emphasis message hinges critically on its impact on households’ perceived risk reduction from a particular switch in water sources. Suppose, for instance, that the emphasis message leads households to have expected risk reductions that are increasing with the actual reduction in arsenic (A) in the water, while instead, the status-quo message leads households to think that there is no difference in arsenic risk conditional on ‘safety’. This scenario is depicted in Figure 2A, where for illustrative purposes we assume that perceived risk is R(A) = A for individuals receiving the emphasis message. For illustration, we assume that baseline arsenic is A = 150 while an alternative well has a lower level of arsenic, A′ = 100. In this scenario, no household receiving the status-quo message will switch to another well in the same safety class, because such a move leads to no perceived benefits, but entails switching costs (see Assumption 1). In contrast, the emphasis message may lead to switching among wells in the same safety class as long as households’ perceived utility gain of switching from A to A′ is sufficiently large relative to the cost of switching under equation (1). Consider again the example in Figure 2A, and suppose that U = −A −S. Households in emphasis villages will switch from A to A′ if S < 50, and will remain at the baseline well otherwise. In this scenario, the predicted impact of the emphasis message is positive, that is, it increases switching to safer wells.. Furthermore, because perceived benefits of switching are increasing in baseline arsenic levels (Assumption 2), equation (1) is more likely to be satisfied among households whose baseline arsenic is higher. Under these plausible conditions, there is heterogeneity in treatment effects, with treatment effects non-decreasing in baseline arsenic, conditional on safety status.

Figure 2. Stylized Illustration of Theoretical Responses to Arsenic Information.

However, the impact of the emphasis message can be negative under alternative and equally plausible scenarios. For instance, consider a case where A = 60 and A′ = 40, so that the reduction in arsenic from switching is small. We illustrate this example in Figure 2B, where we have again assumed that perceived risk is a step function among status-quo households and is identical to the arsenic level in emphasis communities. The emphasis message may make salient the relatively small health benefits of a switch from A to A′, causing R(1) and R(0) to be close and generating low switching rates if switching costs are not negligible. How these rates compare to those of households with the same level of arsenic who receive the status-quo message hinges critically on what those households perceive the risk reductions to be from moving from an ‘unsafe’ to a ‘safe’ well. If status-quo households perceive wells with arsenic levels below 50 ppb to be sufficiently safe (as depicted in Figure 2B), the gap between R(1) and R(0) may be large enough to result in more switching among status-quo households. The two cases presented in Figure 2 highlight the theoretical ambiguity of the impact of the emphasis treatment. They also illustrate that the expected sign of the effect may vary based on baseline arsenic level and the arsenic level of the wells in the household’s choice set.

The ambiguity of the sign of the treatment effect is maintained if one introduces in the choice problem elements borrowed from behavioral economics and related fields. Recent research demonstrates that more information does not always induce people to make better choices. More choices can be demotivating and actually make people less likely to leave the status quo even when better alternatives are present [23]. Studies of contributions to 401(k) retirement plans have demonstrated that when the choice set is large, people prefer simple, easy-to-understand options [22]. Our emphasis message increases the perceived complexity of the choice set and is, perhaps, less easy-to-understand because we stress that not all safe (unsafe) wells are actually equally safe (unsafe). In addition, the emphasis message could potentially generate confusion because it contrasts the binary safe/unsafe dichotomy households are familiar with. Both of these impacts imply that the emphasis message may increase switching costs relative to the status-quo message, thereby decreasing switching rates at all levels of arsenic, ceterus paribus.

A number of behavioral explanations also suggest possible heterogeneity in responses depending on baseline arsenic levels. A first possibility is that some respondents in the emphasis group may interpret the emphasis message to mean that moderately unsafe wells are ‘almost safe’, thereby decreasing the subjective expectations of health benefits from switching rates in this group. A second possibility is that the emphasis script may lead some users of wells with very high levels of arsenic to feel discouraged, erroneously believing that, given the high past exposure, there would be no longer any health benefits from mitigating behavior.

This discussion illustrates that there are several ways in which complex information may affect mitigating behavior. The sign of the treatment effect depends on unknown parameters in the household’s utility function. Regardless of the expected sign there is the possibility of heterogeneity in response based on baseline arsenic levels. Similar results can be obtained from a behavioral framework. Because of the theoretical ambiguity, the impact of the information treatment must be determined empirically.

4 Data

The data used in this paper were collected in two separate household surveys carried out in 2008. Figure 3 provides a timeline of data collection activities. The baseline survey was completed between February and April 2008. While conducting the baseline survey, we also delivered the results of the arsenic tests. We completed interviews with the household who owned the tested well and also with another household who used the same well, if present. ⁷ Enumerators were instructed to conduct the interviews with a female adult respondent, if possible the wife of the household head. This ensured that in a large majority of cases the respondent was the individual who made decisions about where to obtain drinking water for the household. In November 2008, we completed a follow-up survey to assess household responses to the information on arsenic that was presented at baseline. Whenever possible, the follow-up survey was conducted with the same respondent as in the baseline.

Figure 3. Timeline of Data Collection.

The timeline for data collection, including numbers of wells and households located and surveyed at each time. Dark gray coloring indicates the data were collected by our research team, while hashed coloring indicates that data collection was done by other groups.

We restrict our analysis to include only households using a well that had not been moved from the original tested location, who were using the tested well as a source of drinking water at baseline, and who completed the follow-up survey. At baseline, we surveyed 746 households using the tube-wells sampled in 2005. However, for 78 of those households the well that was tested in 2005 was no longer in existence in the original, tested location and, hence, we were able to deliver results to 668 households. Of the households that received well test results, 596 were still using the tested tube well as their primary drinking-water source at the time of our baseline survey. The follow-up survey was completed for 533 of these households.

Table 1 reports selected summary statistics measured at baseline for the 533 sample households. Overall, households were relatively large (5.3 members on average), with low education and relatively low income and expenditure. ⁸ Mean total monthly household expenditure corresponded to approximately 262 USD, using purchasing power parity exchange rates. Only 39% of household heads were literate and 21% achieved at least a secondary school diploma, although enrollment rates among 6 to 14-year olds was relatively high, at 76%. Most households (80%) used sanitary latrines and 11% lived in a “pakka” (good quality) dwelling.

Table 1.

Baseline Summary Statistics and Randomization Tests

	(1)	(2)	(3)	(4)	(5)	(6)
					Difference between experimental arms (emphasis – status-quo)4
	Obs	Mean	Median	Sth Dev	Mean	p-value
Well Arsenic Level	533	116	80	149	17	0.60
As > 50 ppb	533	0.61	1	0.49	−0.06	0.62
# Household members	533	5.3	5	2.50	0.09	0.75
Household head: age	525	46	45	14	0.25	0.88
Household head: can read/write	521	0.39	0	0.49	−0.02	0.62
Household head: secondary education or above	515	0.21	0	0.41	−0.02	0.74
Household head: woman	525	0.19	0	0.39	−0.05	0.15
Number of adult females	533	1.70	1	0.89	−0.11	0.25
Number of children	533	3.10	3	1.60	−0.05	0.78
Fraction of members age > 10 who can read/write	533	0.55	0.5	0.33	−0.02	0.68
Fraction of 6–14 in a household in school	375	0.76	1	0.36	−0.02	0.61
Well Network size	533	1.30	1	1.50	−0.12	0.45
Value of food consumed: monthly 1	530	5073	4333	3339	507	0.07*
Total expenditure: monthly 1	527	5942	5000	3227	371	0.16
Typical income: monthly 1	495	7672	6000	7193	515	0.40
Expenditure on medicines: monthly 1	510	543	250	1227	−159	0.13
Uses sanitary latrine	533	0.80	1	0.40	0.00	0.98
Pakka house or pakka walls (good quality)	533	0.11	0	0.31	−0.01	0.83
Fraction of members with symptoms of As poisoning	533	0.01	0	0.07	0.00	0.69
Any member with symptoms of arsenic poisoning	533	0.04	0	0.19	0.02	0.56
Mean # days of illness last year (ages 6 to 70) 2	532	19	13	22	−0.23	0.91
Household owns well	527	0.7	1	0.46	−0.019	0.60
Distance to nearest well with As < 50 ppb (meters)	533	34	18	51	6	0.64
Number of wells closer than 100m	533	19	18	9.4	0.58	0.82
Number of wells closer than 100m with As < 50 ppb	533	6	5	6.1	1.5	0.43
Average concern about short-term arsenic risk 3	513	1.2	1	0.74	0.087	0.35
Average concern about long-term arsenic risk 3	510	7.8	8	1.6	0.098	0.64

Source: Author's calculations from baseline survey data (Spring 2008).

¹All income and expenditures are derived from simple questions about aggregates. All monetary units are in Bangladesh Takas (BDT). The PPP exchange rate was 22.64 BDT per dollar [44].

²Number of days lost to work or school in the last year.

³“Concern’ for As risk is measured on a 0 to 10 scale, by asking about the chance that someone drinking unsafe water will develop serious health conditions within 1 month or 1 year (short-term), or within 10 or 20 years (long-term). See text for details.

⁴The figures in columns 5 show the mean difference between treatment groups for the whole sample. The figures in columns 6 show the p-value for a test of equality of means for the two samples. Variances used for the tests are robust and adjusted for intra-cluster correlation of residuals. Asterisks indicate significance at the 1 (***), 5 (**) or 10 (*)% level.

The mean level of arsenic was 116 ppb, more than twice the threshold used by the Government of Bangladesh to identify “unsafe wells”. There was also considerable variation within the sample, as shown by the standard deviation of 149. While the high mean is partly driven by some outliers, the median is high as well (80 ppb) and 61% of the tested wells had unsafe levels of arsenic. Consistent with the commonly found spatially heterogeneous geographic distribution of arsenic [12, 41], we find substantial heterogeneity in arsenic contamination even within villages. The heterogeneity is evident in Figure 4, which maps the study area and shows the location of safe and unsafe wells over the territory. The larger points represent the wells used in the RCT study and the smaller points indicate wells that were tested by BAMWSP in 2003. It is evident from the map that there is a significant number of wells exceeding the Bangladesh standard of 50 ppb in the study area. Nonetheless, all unsafe wells are close to at least one ‘safe’ well (by the Bangladesh national standard). Using geographic information system (GIS) coordinates, we estimate that 94% of households are no more than 100 meters (as-the-crow-flies) from a safe well, and the average household has six safe wells within that radius. The mean distance of sample households to the nearest known safe well was 34 meters. ⁹

Figure 4. Spatial Distribution of Arsenic in Study Area.

The picture shows the spatial distribution of arsenic in the study area. Red dots indicate unsafe wells and green dots indicate safe wells according to the Bangladesh standard of 50 ppb. The larger points correspond to wells tested as part of this RCT while the smaller points correspond to wells tested by BAMWSP in 2003. The map was constructed using ArcMap 10. Well locations are based on latitudes and longitudes collected by the research team. The data were projected onto the DeLorme World Base Map using WGS_1984_UTM_Zone_46N.

Despite high levels of arsenic, very few individuals reported symptoms of arsenic poisoning. In less than 4% of households (and for 1% of individuals) do we find anyone reporting any symptoms. In almost all cases, such symptoms consisted of skin lesions. ¹⁰ We also asked respondents about other households they knew whose wells were tested in the past. The variable, that we somewhat loosely label “network size”, captures how many neighbors the respondent was able to recall whose well they knew had previously been tested. On average, households were able to recall 1.3 other households with an arsenic test.

We also measured perceptions about the health risks posed by arsenic. Beliefs about arsenic risk were elicited by posing a series of questions to the respondent about future events and asking respondents to express how likely they thought the described event would occur by counting physical objects. Each respondent was given 10 marbles and a plastic cup and asked to put more marbles into the cup if she felt the perceived event was more likely. After each question was asked, the cup was emptied so that the respondents started each question with 10 marbles and an empty cup. ¹¹ This method was used to elicit respondents’ beliefs about health risks from drinking arsenic from a generic hypothetical “unsafe well.” Each respondent was asked to think of a well that had “just the amount of arsenic that the government says is unsafe.” She was asked to think about a family that had been consuming arsenic-free water so far, but had switched to this hypothetical “unsafe” well on the day of the interview. The respondent was then asked about the chances that an adult from this family would develop “serious health problems” (defined as health complications that would impede normal daily activities) by drinking water from this well within alternative time horizons of 1 month, 1 year, 10 or 20 years. At baseline, these generic-well beliefs were elicited before the respondent was given her well test results. The beliefs data indicate that respondents were very aware of arsenic risk, and understood the cumulative nature of arsenic risk. Using the zero to 10 scale described above, the mean perceived risk was 1.2 over the short term (the average of 1 month and 1 year exposure), but it increased to 7.8 in the long-term (the mean of 10 and 20 year).

Column 5 of Table 1 displays differences in means of observed characteristics among treatment and control households, where treatment households were in villages receiving the emphasis message and control households were in villages receiving the status-quo message. Column 6 shows the p-values of the tests of equality between groups, calculated by regressing each variable on a treatment dummy, and testing the null hypothesis that the corresponding coefficient is equal to zero. Given that the randomization was conducted at the village level, all inference is conducted by estimating variances using standard Huber-White formulas generalized to allow for intra-village correlation. ¹²

Randomization was overall successful in balancing the 27 listed characteristics among experimental arms. The null of equality between arms is never rejected at the 5% level, and in only one case (mean value of monthly food consumption) at the 10% level. The differences in arsenic levels between treatment arms were small (17 ppb, or 11% of a standard deviation) and not significant (p-value= 0.6). More generally, Figure 5 shows that the distributions of arsenic for the two groups are very similar. ¹³ To test for equality of the distributions more formally, we divided arsenic levels into six bins and carried out a Pearson’s chi-squared test. ¹⁴ The p-value of the test is 0.52 indicating that we cannot reject the null hypothesis of equality of the arsenic distributions. The null is also not rejected if we test the equality of distributions only for unsafe wells (p-value=0.14).

Figure 5. Histogram of Arsenic Levels by Treatment.

Histograms exclude three observations for which the arsenic level was above 750 ppb.

Attrition was similar across experimental arms (12% in status-quo vs. 9% in emphasis communities) and in both groups it was about two percentage points higher among users of unsafe wells, although the difference is not statistically significant. There were some differences in covariates among households that were not included in the final sample as indicated in Table 2. Most of the statistically significant differences are between households that were dropped from the sample because they were not drinking water from the tested well at the time of the baseline survey. In particular, these households had higher arsenic levels in the tested well than those who were still using the tested well for drinking water. This could indicate some private learning or inference about well safety as many other wells had been tested by BAMWSP earlier and, perhaps, an untested well surrounded by wells painted red was inferred (correctly) to be unsafe. It could also be that shallower wells tend to have higher arsenic and older wells are more likely to be shallow. These wells may simply have stopped working before results were disseminated. Households that were not drinking water from the tested well at baseline were also richer than households who were still drinking from the tested well. There were also a few differences in covariates between households who were not available at follow-up and those in the sample. In particular, households that were not available at follow-up were poorer. This is consistent with the anecdotal explanations for these households’ absences that were given in the field, namely that the household had migrated to urban areas for work.

Table 2.

Means of Key Covariates by Attrition Status

	(1)	(2)	(3)	(4)
	Final Sample	Tested Well Could Not Be Found	Tested Well Not Used For Drinking At Baseline	Household Not Found at Follow-up
Well Arsenic Level	116	134.7	186.5**	109.0
As > 50 ppb	0.61	0.74	0.68	0.67
# Household members	5.3	5.6	5.4	4.9
Household head: age	46	46.4	50.2**	44.0
Household head: read/write	0.39	0.56**	0.46	0.47
Household head: secondary education or above	0.21	0.31	0.29	0.27
Household head: woman	0.19	0.16	0.17	0.25
Number of adult females	1.7	1.8*	1.9*	1.4**
Number of children	3.1	3.3	3.4*	2.7
Fraction of members > 10 can read/write	0.55	0.63*	0.62**	0.57
Fraction of 6–14 in a household in school	0.76	0.82	0.70	0.81
Well network size	1.3	1.6	1.3	1.2
Value of food consumed: monthly 1	5.1	6.1*	6.6	4.5**
Total expenditure: monthly 1	5.9	7.2*	7.9*	5.4*
Typical income: monthly 1	7.6	8.4	11.1*	6.4***
Expenditure on medicines: monthly 1	0.54	0.82	1.1	0.57
Uses sanitary latrine	0.8	0.73	0.79	0.78
Pakka house or pakka walls (good quality)	0.11	0.15	0.25**	0.16
Fraction of members with symptoms of As poisoning	0.01	0.01	0.01	0.01
Any member with symptoms of As poisoning	0.04	0.06	0.06	0.03
Mean # days of illness last year (ages 6 to 70) 2	19	14.9*	18.1	15.5
# of observations	533	78	72	63

Source: Author’s calculations from baseline survey data (Spring 2008).

¹All income and expenditures are derived from simple questions about aggregates. All monetary units are in Bangladesh Takas (BDT). The PPP exchange rate was 22.64 BDT per dollar [44].

²Number of days lost to work or school in the last year.

³Asterisks indicate significance at the 1 (***), 5 (**) or 10 (*) % level of the difference in means of the attritted sample relative to the final sample. Significance levels were determined using standard errors that are robust and adjusted for intra-cluster correlation.

5 Methods and Results

In this section, we describe the impacts of our informational interventions on the choice of primary drinking-water source. In the follow-up survey, respondents were asked whether they continued to use the well they had been given results for at baseline. If the answer was no, the survey enumerator was instructed to walk with the respondent to the new source. Whenever possible, the safety status and arsenic level of the new source were recorded. Thus, we feel confident that self-reported switching actually measured real changes in behavior. Indirect evidence supporting this view is also provided by earlier studies completed in areas proximate to our study locations, which showed significant declines in arsenic exposure (as measured through urinary arsenic concentration) in households who reported having abandoned the use of an unsafe well for drinking water (e.g. [12]).

5.1 Average Treatment Effects

We first examine switching decisions, measured by the binary dependent variable S_vh, which is = 1 if household h in village v changed the main source of drinking water between baseline and follow-up, and = 0 otherwise, conditional on safety status of the baseline well. This analysis does not use the experimental variation, but allows us to compare overall switching rates in our sample area with other non-experimental studies in neighboring areas of Bangladesh to assess how representative the decisions of these households are. At follow-up, 7–9 months after the information on arsenic levels was delivered, 35% of the households who received an ‘unsafe’ result had switched source, while only 7% of those using a safe well did (column 1, Table 3). These findings are overall consistent with switching rates documented in earlier studies conducted in the area [12, 28, 31, 33].

Table 3.

Switching Decisions, Well Arsenic-Safety Status and Communication Mode

	(1)	(2)	(3)	(4)	(5)
Intercept	0.0725*** (0.0266)	0.289*** (0.0354)	0.359*** (0.0440)	0.303*** (0.0978)	0.375*** (0.0432)
Emphasis		−0.0910* (0.0512)	−0.0737 (0.0440)	−0.0579 (0.0404)	−0.0673 (0.0432)
Unsafe	0.280*** (0.0431)
of Household Members				0.0153 (0.0119)
Household head: age				−0.000344 (0.00146)
H. head: can read/write				0.121* (0.0636)
H. head: secondary educ. or above				−0.212*** (0.0546)
H. head; woman				−0.00125 (0.0499)
Number of adult females				−0.0549 (0.0458)
Number of children				0.0184 (0.0303)
% members age < 10 read/write				0.0153 (0.0866)
Network Size				0.00531 (0.0178)
Monthly food expenditure (1,000s)				−0.000799 (0.00290)
Monthy total expenditure (1,000s)				−0.00401 (0.00635)
Uses sanitary latrine				−0.0986** (0.0404)
House/walls of ’good quality’				−0.0354 (0.0648)
Mean sick days last year				0.00135 (0.000840)
Owns well				0.0303 (0.0319)
Distance to nearest safe well (100m)				0.000349 (0.000500)
Number of safe wells 100m				−0.00671** (0.00302)
Stratum Dummies	No	No	Yes	Yes	Yes
Observations	533	533	533	502	502
R-squared	0.101	0.011	0.051	0.112	0.050
Number of Clusters	43	43	43	42	42

Source: Author’s calculations from baseline and follow-up data. All models are estimated with OLS, with the same binary dependent variable = 1 if the household switched to a different source of drinking water after the communication of the test results, and = 0 otherwise. Robust standard errors adjusted for intra-cluster correlation are provided in brackets. Asterisks indicate significance at the 1 (***), 5 (**) or 10 (*) % level. The different sample size in column 4 is due to missing data in covariates. Column 5 replicates the specification in Column 3, but with the sample that includes all covariates used in Column 4.

Next, we examine the average impact of the emphasis message on switching. Column 2 of Table 3 contains the results of a linear probability model, estimated with ordinary least squares (OLS), where we regress S_vh on an indicator E_vh, which is = 1 for households in emphasis villages, and = 0 otherwise. ¹⁵ The results show that switching rates were nine percentage points lower in emphasis villages. This translates into 31% lower switching rates relative to status-quo areas, where 29% of households moved to a different source. The difference is statistically significant, although only at the 10% level (p-value= .083). In addition, the estimate becomes smaller in magnitude (−0.07) and (barely) insignificant at standard levels (p-value= 0.102) if we include strata dummies, as suggested by Bruhn and McKenzie [9] for stratified designs such as ours (column 3). ¹⁶

Column 4 includes a full set of covariates. When covariates are included, the impact of the emphasis message decreases further to a negative six percentage points and is insignificant. However, inclusion of the covariates lowers the sample size from 533 households to 502 households (a 6% loss in sample size) due to missing values for the covariates and some of the decrease in the magnitude of the treatment effect and precision of the estimate is due to the reduced sample size. For comparison, Column 5 contains the base model with strata dummies, but limited to the sample with full data for all covariates. The decrease in magnitude of the treatment effect appears to be approximately equally explained by loss of sample size (0.074 − 0.067 = 0.007) and inclusion of covariates (0.067 − 0.058 = 0.009).

In sum, we find a treatment effect that has a negative sign, but that is statistically indistinguishable from zero in most models. Therefore, there is no evidence in our sample that the richer emphasis message increased switching to wells with lower arsenic levels. In particular, we can rule out positive treatment effects larger than two percentage points at the 95% confidence level. This is important because in areas with many unsafe wells ranging from 50 to over 300 ppb, encouraging within-safety-class switching could greatly increase health benefits. Our results suggest that a one-time oral communication about the health benefits of within-safety-class switches will not be sufficient to increase this type of switching. Further research is required to develop and test alternatives for motivating increased switching away from very unsafe wells.

5.2 Exploratory Analysis of Heterogeneous Effects

As indicated in Section 3, there is no strong a priori hypotheses about how the emphasis message would impact differentially switching choices for households with different arsenic levels, but the model does predict that a degree of heterogeneity in responses is likely to be present. Our experiment was not designed to test for differences in treatment effects among sub-groups, including sub-groups of baseline arsenic. However, given the theoretical predictions of potential heterogeneity in effects of communication across the arsenic distribution, we explore patterns of switching by communication mode across baseline arsenic levels both non-parametrically and parametrically. Because this relies on non-experimental variation in arsenic levels, one should not interpret this as causal inferences of heterogenous treatment effects, but rather as preparatory work for future hypothesis testing. Further, while the conceptual framework discussed in Section 3 applies to all initial arsenic levels, too few households with safe wells chose to switch sources, so power to ascertain treatment impacts on the “safe” part of the distribution was limited. ¹⁷ As a result, we concentrate on the switching decisions among households whose baseline well was considered unsafe.

We estimate switching rates across treatment arms conditional on arsenic using non-parametric locally linear regressions [17]. We chose a bandwidth of 50 ppb which highlights the data-driven shape of the regressions while still smoothing out some of the noise in the data. For each regression, we also estimate 90% confidence bands using 250 bootstrap replications. ¹⁸ We use block-bootstrap to reflect the clustered nature of the data, so that in each replication we re-sample clusters of village-specific observations rather than households. For each point on a grid of arsenic levels, we construct the lower and upper bounds of the 90% confidence band by selecting respectively the 5th and 95th percentile from the bootstrap distribution of 250 point-specific estimated regressions. ¹⁹ Panels A and B, in the top section of Figure 6, display the results, estimated separately for status-quo and emphasis villages.

Figure 6. Non-parametric regressions of switching rates on Arsenic levels.

Source: Author’s estimates from follow-up (post-intervention) data. We estimate all regressions using non-parametric locally linear regressions [17], with a bandwidth equal to 50 ppb. The top two panels show switching rates conditional on arsenic levels for unsafe wells in the bright line (graph A, n = 170) and the emphasis (graph B, n = 156) experimental arm respectively. The two graphs are overlapped in panel C. Panel D shows the estimated vertical distances between the two graphs, with 90% confidence bands estimated using block-bootstrap replications. We construct the bands by selecting respectively, the 5th and 95th percentile from the bootstrap distribution of 250 point-specific estimated regressions.

In areas where the status-quo message was used, switching rates are close to zero at arsenic levels close to the safe threshold of 50 ppb, and only become substantially larger than 20% for arsenic levels above 100 ppb. Switching rates then increase visibly with arsenic levels up to 160 ppb and stabilize around 40–50%. The shape of the regression is significantly different in emphasis villages. First, almost half of the households are estimated to switch for arsenic levels close to 50 ppb, which actually represents the highest switching rate for all levels of arsenic among emphasis households. Second, we find no evidence of increased switching for higher levels of arsenic. If anything, the regression suggests declining switching rates for high levels of arsenic. The stark difference between the two lines is highlighted in panel C of Figure 6, where we display both regressions. Finally, in the bottom-right panel D, we display the difference between the emphasis and status-quo regressions together with 90% confidence bands. The difference shows a marked declining pattern relative to arsenic levels. Although estimation error is large enough that we cannot reject the null of equality over most of the arsenic range, the difference is positive, large (about 40 percentage points) and statistically significant at the 10% level in a neighborhood of the safety threshold of 50 ppb and negative, large (about 35 percentage points) and significant at the 10% level for high arsenic levels around 250 ppb.

Given the non-experimental nature of the heterogeneous treatment effects, a concern is that they may be at least partly driven by differences in covariates across the arsenic distribution. ²⁰ We therefore turn to the estimation of parametric forms of the regressions shown in Figure 6. Table 4 shows the result of OLS estimates of the parametric equivalent of the non-parametric regressions, modeled as follows:

$$S_{vh}={\beta }_0+{\beta }_1{E}_{vh}+{\beta }_2{\mathit{\text{High}}}_{vh}+{\beta }_3{E}_{vh}\times {\mathit{\text{High}}}_{vh}+{\beta }_4^{\prime }{X}_{vh}+{\epsilon }_{vh,}$$ (2)

where E_vh is an indicator of emphasis village, X_vh is a vector of observed pre-intervention household characteristics, and High_vh is a binary variable = 1 if the well used by household h has ‘very high’ levels of arsenic. For the purposes of this analysis we define moderately-unsafe wells to have arsenic levels between 50 and 100 ppb and very-unsafe wells to have arsenic levels above 100 ppb. These cutoffs are not theory-driven and are based on broad patterns of responses present in the non-parametric results.

Table 4.

Communication Mode and Switching Decisions from Unsafe Wells

	(1)	(2)
Intercept (β0)	0.23 (0.064)***	0.34 (0.193)*
emphasis (β1)	0.12 (0.109)	0.14 (0.073)*
High (As > 100 ppb) (β2)	0.24 (0.070)***	0.22 (0.054)***
emphasis × High (β3)	−0.30 (0.124)***	−0.27 (0.093)***
Controls Included	No	Yes
Strata Dummies	No	Yes
Observations	326	310
Clusters	42	42

Source: Author’s calculations from baseline and follow-up data. Only households that fetched water from wells with As> 50 ppb are included. The number of clusters is reduced to 42 because one village only included safe wells. All models are estimated with OLS. The binary dependent variable is = 1 if the household switched to a different source of drinking water after the communication of the test results, and = 0 otherwise. The regressions in column 2 also includes the controls used in Table 3, column 4 (full results available upon request). Standard errors in brackets are robust and adjusted for intra-cluster correlation. Asterisks indicate significance at the 1 (***), 5 (**) or 10 (*) % level.

The results in column 1, where we do not include household-level controls, are consistent with the non-parametric estimates discussed earlier. Switching from moderately-unsafe wells was 12 percentage points higher (52%) in emphasis relative to status-quo villages, although the difference is not significant at standard levels (p-value= 0.276). However, while in status-quo areas switching is 24 percentage points higher from highly-unsafe wells relative to moderately-unsafe ones (p-value= 0.002), in emphasis villages switching is not nearly as strongly related to arsenic level and actually declines by six percentage points among high-arsenic households relative to moderate-arsenic households. The net effect, measured by the coefficient on the interaction term β̂₃, is then large, negative, and statistically significant at the 5% level (β̂3 = −0.3, p-value= 0.019). In column 2, we include the usual covariates and strata dummies and the magnitude of the effect of the emphasis message among households with very-high baseline arsenic levels decreases only marginally and remains statistically significant at the 1% level. Although these results suggest the existence of heterogeneity in treatment effects, our study was not stratified by arsenic level prior to randomization of the treatment, so further research is required to confirm these patterns and test their possible origins.

6 Conclusions and Discussion

In many different areas of environmental and health policy, there is a need to use information campaigns to inform individuals or households about risks. The hope is that this information then induces voluntary changes in behavior that reduce those risks. But little is known on how best to present information on risks in a way that leads to the greatest health benefits. One frequently used approach is to reduct the risk spectrum to two states:“safe” and “unsafe”. However, this simplification necessarily results in a loss of information–information that, in theory, could be used to make healthier choices. The cluster randomized controlled trial described in this paper was designed to test the hypothesis that providing households with more detailed information on the relationship between arsenic exposure and arsenic risk could induce more households to switch water sources, relative to the status-quo “safe/unsafe” risk presentation. We do not find empirical support for this hypothesis. The point estimate of the average treatment effect is actually negative, but the null hypotheses of zero cannot be ruled out at conventional levels.

There are three important caveats for our findings. First, the status-quo safe/unsafe message is prevalent throughout Bangladesh and certainly households in Araihazar had been accustomed to hearing information about arsenic in the safe/unsafe format. While our findings, that the emphasis message did not result in greater rates of health-improving behavior among the most exposed households, may well hold for Bangladesh, it may not hold in other settings where the status-quo message is not already dominant.

Second, our intervention was limited in scope. Presentation of the gradient in arsenic risk was limited to one-time oral information given to respondents when the test results were displayed. The test result plates themselves focus largely on the safe/unsafe message and these plates were the same in both emphasis and status-quo households. A more sophisticated presentation of the gradient in risk, perhaps gradients of colors or a bar that represents the level of arsenic in the well relative to others in the village, may have greater impacts. Similarly we might expect different results if the oral message was repeated to households at several points in time.

Third, we examined switching choices over a relatively short time horizon. The time between our baseline and follow-up surveys was less than one year. In other work, we find a significant fraction of households switch away from unsafe wells more than two years after they are first informed that they are using an unsafe well [5]. The effect of the emphasis information may take time to have an impact.

The results in this paper have direct policy implications. While the status-quo presentation (safe/unsafe) may discourage health-beneficial switches within a safety class (for example, from 200 to 100 ppb), the emphasis message, which highlighted the benefits of switching to a lower arsenic well regardless of initial arsenic level, did no better. Other alternatives including well labels with multiple safety categories (very safe, safe, unsafe, very unsafe), household-specific information on safer water sources, village-level maps of the safest water sources will need to be explored. Our exploratory analysis also suggests that future research might fruitfully be deployed in examining heterogeneous impacts of risk communication messages along different measures of baseline risk.