Consistency of Use and Effectiveness of Household Water Treatment Practices among Urban and Rural Populations Claiming to Treat Their Drinking Water at Home: A Case Study in Zambia

Abstract

Household water treatment (HWT) can improve drinking water quality and prevent disease, if used correctly and consistently. While international monitoring suggests that 1.8 billion people practice HWT, these estimates are based on household surveys that may overstate the level of consistent use and do not address microbiological effectiveness. We sought to examine how HWT is practiced among households identified as HWT users according to international monitoring standards. Case studies were conducted in urban and rural Zambia. After a baseline survey (urban: 203 households, rural: 276 households) to identify HWT users, 95 urban and 82 rural households were followed up for 6 weeks. Consistency of HWT reporting was low; only 72.6% of urban and 50.0% of rural households reported to be HWT users in the subsequent visit. Similarly, availability of treated water was low, only 23.3% and 4.2% of urban and rural households, respectively, had treated water on all visits. Drinking water was significantly worse than source water in both settings. Only 19.6% of urban and 2.4% of rural households had drinking water free of thermotolerant coliforms on all visits. Our findings raise questions about the value of the data gathered through the international monitoring of HWT practices as predictors of water quality in the home.

Introduction

Diarrheal diseases remain a leading cause of morbidity and mortality among children under 5 years of age in middle- and low-income countries. Recent estimates for the Global Burden of Disease 2013 project indicate that 1.4 million premature deaths and 83.9 million disability-adjusted life years globally were due to inadequate water, sanitation, and hygiene (WASH).¹

Household water treatment (HWT), including boiling, chlorination, filtration, and solar disinfection, can improve the quality of drinking water at the point of use and reduce the risk of diarrhea among the millions of people that rely on unimproved drinking water sources, and among those that rely on improved water sources that are nevertheless contaminated, an estimated 1.8 billion people.²

Evidence from systematic reviews has shown a protective effect of HWT against diarrhea.^3–5 Some have criticized unblinded trials of HWT interventions that rely on a subjective outcome (self-reported diarrhea), which currently constitute the bulk of the evidence, because they could be subject to differential outcome reporting bias; which if present, could overstate diarrhea reductions due to HWT.⁶ Nevertheless, based on this evidence, the World Health Organization (WHO) and United Nations Children's Fund (UNICEF) have included HWT in their seven-point plan for comprehensive diarrheal control in low- and middle-income countries.⁷

Although there is a substantial body of literature of HWT in intervention or experimental settings as well as promotional settings, comparatively little is known about HWT practices outside these contexts. Commencing in 2005, the WHO/UNICEF Joint Monitoring Program for Water and Sanitation (JMP), assigned as the official United Nations mechanism for monitoring progress on water and sanitation, started to gather data on HWT practices through their routine monitoring mechanisms. The JMP recommended the inclusion of two core questions on whether and how HWT is practiced to their nationally representative household surveys: 1) do you treat your water in any way to make it safer to drink, and, if the response is affirmative, 2) what do you usually do to the water to make it safer to drink.⁸

The objectives of these questions were to determine the baseline prevalence of this practice and by classifying HWT as adequate (if the method has been shown to be microbiologically effective, i.e., boil, bleach, filter, and solar disinfection) or inadequate (if not, i.e., strain through a cloth and stand and settle), to assess whether these questions could act as proxy indicators of water quality in home.^8,9 Analysis of these data shows that an estimated 1.1 billion people among 67 low- and middle-income countries¹⁰ and over 1.8 billion people, if data from China are included, are practitioners of HWT.¹¹

While relying on household-based surveys may be the most practical and cost-effective way of gathering these data at the national and regional scale, as is the case with survey-based data on access to drinking water as well as sanitation, limitations may exist regarding the reliability and value of these data for public health and policy purposes.^12–14

First, overreporting of “good” practices has been shown to be a problem in survey-based studies,^15–18 especially in government-sponsored surveys.¹⁹ Overreporting of HWT in experimental settings is a common phenomenon,^20–23 and so, it may also be prevalent outside this context. Second, as depicted in Figure 1 , HWT is a complex behavior, often performed as a batch process where small batches of drinking water are treated at a time, on a daily or otherwise frequent basis,²⁴ and requires the user to remain motivated and committed to integrate the practice in their daily routines.²⁵ As shown by several quantitative microbial risk assessment models, the effectiveness as well as the consistency of its use (also referred to in the literature as adherence or compliance) are the key aspects to ensure the protective effects of HWT.^26–28 It thus remains unclear if the two core JMP questions on HWT use can capture the complexity of this behavior, and are able to act as good proxy indicators of the consistency and effectiveness of the practice. Finally, it remains unclear whether practitioners of HWT remain exposed to waterborne pathogens in the home. There are multiple examples in the literature where current users of HWT continue to consume untreated water at home (i.e., supplement treated water with untreated water) and thus remain exposed to waterborne diseases.^22,29,30

Figure 1.

Individual steps necessary for correct and consistent household water treatment (HWT) use in a monitoring context.

With the support and funding of the JMP, a series of case studies were conducted in both urban and rural settings in India, Peru, and Zambia; covering a range of HWT technologies and national HWT prevalences (based on JMP figures). The main objectives of these studies were to 1) document HWT practices among populations self-reporting to be practitioners of HWT according to JMP monitoring procedures, 2) characterize the microbiological quality of drinking water among self-reported HWT users, and 3) asses to what extent the JMP questions capture these aspects of HWT.

We previously reported the results from Peru,³¹ a country with a high profile of boilers according to JMP figures (77.6% of households). Here we report results from Zambia, a lower middle-income country where, according to JMP figures, only 41.0% of households have access to improved drinking water sources and 34.9% of households report HWT, mainly by boiling (15.2%) or chlorination (25.9%).³²

Materials and Methods

Study design.

The study aimed to gain an understanding of actual practices among those self-reporting to be HWT practitioners according to current JMP monitoring procedures. A mixed-methods approach, relying on both quantitative and qualitative methods, was used to investigate and cross-check the relationship between reported and actual HWT behaviors. A baseline survey was undertaken to identify HWT users according to JMP procedures. The baseline survey closely resembled the questionnaire used in the demographic and health surveys (DHSs) and the multiple indicator cluster surveys, which the JMP relies on for monitoring purposes. The research was presented to participants as a study to improve the reporting methods of national surveys but with a special focus on child-caring practices; no mention was made of the subjects of water supply, drinking water quality, or HWT practices. A random sample of 116 households that provided an affirmative response to the JMP core question on HWT use (i.e., self-reported HWT use) was selected for follow-up. Follow-up consisted of either a second survey (hereinafter, the “HWT practices survey”) or an in-depth interview (IDI), and three rounds of observational spot check visits and water sampling (Figure 2 ). Households were followed for 6 weeks. Unlike the case studies in Peru, structured home observations were not included in this study as initial formative research showed them to be logistically difficult due to opposition to the idea of fieldworkers spending extended periods (4–6 hours) in the homes of participants.

Figure 2.

Schematic representation of the study design (U = urban, R = rural).

Study setting and participant eligibility.

The urban sub-study was conducted in Misisi Compound, a township on the outskirts of the city of Lusaka; the rural study was conducted in three neighboring villages, Njouvu, Ndango, and Shisholeka, in Chongwe District, 40 km east of Lusaka. In the urban setting, due to the informal nature of the community, the township was divided into nine sectors of similar size and households were selected by using systematic sampling. In the rural setting, given the small size of the selected villages, all households were contacted for participation. Households were eligible for participation if 1) at least one household member lived permanently in the household, 2) the head or spouse had no impediments to providing informed consent, and 3) the household reported to have no intention of moving out of the area in the following 3 months. The study was conducted from March through June in 2011.

Sample size.

Sample size calculations were based on precision estimates. Aiming for a precision of ±10% and an anticipated proportion of interest of 0.7, a minimum of 80 households self-reporting HWT were required. To account for loss to follow-up, this was increased to 115 households. On the basis of the reported prevalence of HWT use in urban and rural areas in the province of Lusaka from the Zambia Demographic and Health Survey 2007³² (54.6% and 42.2%, respectively), it was estimated that 211 and 273 households would need to be approached in the urban and rural settings, respectively, to identify the required number of households reporting to practice HWT.

HWT practices survey and IDIs.

A convenience sample of 15% of households was purposely selected for IDIs. Only households with at least one child < 5 years were eligible for the IDIs. The remainder of households completed the HWT practices survey. Both the IDIs and the HWT practices survey followed the same line of questioning, but the former allowed extra flexibility for probing and covered certain aspects such as perceptions, knowledge, and purposes of HWT in more detail. The aims of the survey/IDI were to 1) assess the reliability of the core JMP question by administering the core question on HWT use a second time and 2) gain further insight on HWT practices, with a special focus on consistent use.

Observational spot check visits.

At the end of the HWT practices survey (or IDI), and in two further, unannounced occasions, at an interval of 4–12 days, an observational spot check was completed in all participating households. The aim of these observational spot check visits was to obtain objective indicators of actual HWT use. Data were gathered on 1) availability of treated water at the time of the visit (based on self-report), 2) ability to show the materials used to perform HWT, and 3) objective proxy indicators of HWT use, such as free chlorine residuals (FCR) for those reporting chlorination, water in the filter for those reporting filtration, and time of boiling and temperature of boiled water for those reporting boiling. To reduce the potential for reporting bias in future visits, sanitation as well as hand washing facilities were inspected before water sampling and questioning. On average, the field supervisor was present on 27% of visits. All forms were reviewed at the end of the day.

Water quality.

Coinciding with the observational spot check visits, paired samples of drinking water and source water were collected for microbiological assessment from each participating household. A sample from the most frequently used drinking container was collected. If children under the age of 5 years resided in the household, the sample was collected instead from the drinking container used by the children. Samples were collected in 125 mL Whirl-Pak bags (Nasco International, Fort Atkinson, WI) and tested for the presence of thermotolerant coliforms (TTC) within 4 hours of collection using the membrane filtration method. Commercially purchased distilled water was used as negative controls for each batch of water quality analysis. Drinking and source water were tested for FCR using the Oxfam Delagua chlorine color test kit (Robens Institute, University of Surrey, Guilford, Surrey, UK).

Data analysis.

To determine the consistency of reporting of HWT practices among those identified as HWT practitioners at baseline, we assessed the concordance of reporting HWT at both questioning events (at baseline and during the HWT practices survey or IDI). To assess the level of consistent HWT use, we calculated the number of visits in which a household stated to have treated water. To further evaluate the consistency of HWT use, we cross-checked reported daily use and exclusive drinking of treated water against the availability of treated water at the time of the visit, based on self-report.

The distribution of TTC counts was zero inflated and right-skewed. Since medians, the interquartile range, and Williams mean provide a better measure of central tendency for skewed data, these are presented together with arithmetic means for source and drinking water.³³ The Williams mean is calculated by adding 1 to all the data values, then taking the geometric mean, and then subtracting 1 again.³⁴ Log₁₀ transformation of TTC counts did not ensure a normal distribution of the data. For this reason, a nonparametric test (Wilcoxon signed-ranked test), was applied to analyze each round of water quality data. To further assess if the drinking water of self-reported users of adequate methods identified at baseline was of higher quality than their source water, TTC counts were Log₁₀ transformed after imputing a value of 1 to the zero counts and the difference of the paired source-drinking water samples was calculated for each of the three rounds of follow-up.

To assess the overall difference in water quality across all three rounds of data collection, we used negative binomial regression.³⁴ We used raw mean counts of TTC, with outcomes expressed as risk ratios, estimating the change in the relative mean number of events between categories.^35,36 The analysis used robust variance estimation to adjust for clustering at the household level. All statistical analyses were conducted using STATA version 10 (Stata Corp., College Station, TX).

Ethics.

The study was reviewed by the Ethics Committee of the London School and Hygiene and Tropical Medicine (reference no. 5696 dated April 13, 2010) and the University of Zambia Research Ethics Committee (reference no. 016-10-10 dated February 25, 2011). Written informed consent was obtained from all participating household.

Results

Baseline characteristics.

Overall, 203 and 276 households completed the baseline survey in the urban and rural sub-studies, respectively (Supplemental Table 1). Just over 55% of the heads of households in both settings had completed secondary or higher education. In the urban setting, the majority of households relied on public standpipes for their drinking water needs (93.6%). Similarly, almost all households replied on improved sanitation facilities (92.6%), although in 94.1% of cases these were shared, with an average of 5.8 households sharing each facility. In the rural counterpart, households relied on boreholes and protected (65.2%) or unprotected wells (33.0%) for their drinking water. A minority of households (22.8%) had access to improved sanitation facilities. Of these, only 37.1% were shared, with an average of 3.1 households sharing each facility.

HWT was common in the urban setting, with 79.3% of households providing an affirmative response to the JMP core question on HWT use. HWT use was less prevalent in the rural setting (31.9%). The use of chlorine or bleach was the most prevalent method of HWT in both settings (U: 76.9%, R: 27.9%), followed by boiling (U: 11.3%, R: 4.0%).

HWT practices survey.

Overall, 81 and 66 households completed the HWT practices survey in the urban and rural communities, respectively (Table 1). At this second questioning event, 72.6% and 50.0% of households provided an affirmative response to the JMP core question on HWT use in the urban and rural studies, respectively. Consistent with baseline data, using chlorine or bleach was the predominant method in both settings. Seasonal HWT use was uncommon in both settings, with most households reporting the use of HWT year around (U: 74.1%, R: 78.8%). Similarly, all family members consumed treated water. In the urban setting, in 10.7% of cases, the participants reported that HWT would be performed at the public standpipe, while in the remaining 89.3% of cases, HWT would be performed at home. In the rural setting, all households performed HWT at home.

Table 1.

Summary of reported HWT practices as reported during the HWT practices survey

Characteristic	Rural		Urban
	n	%	n	%
Number of households	66	–	81	–
Water handling practices
Store drinking water at home	66	100.0	81	100.0
Percentage of households with more than one type of storage container	10	15.2	10	12.3
Percentage of households with a wide opening container*	52	78.8	70	86.4
Percentage of households with a narrow opening container*	24	36.4	9	11.1
Report covering drinking container	63	95.5	75	94.9
Access drinking water (only for wide vessels)
Dip a glass	47	90.4	58	82.9
Use a ladle	3	5.8	5	7.1
Use a tap	1	1.9	7	10.0
Other	1	1.9	0	0.0
HWT practices
Reported HWT use	33	50.0	58	71.6
Reported method
Boil only	0	0.0	1	1.7
Boil and use chlorine or bleach	1	3.0	6	10.3
Boil and let stand and settle	0	0.0	0	0.0
Use chlorine or bleach	32	97.0	51	87.9
All household members consume the treated water	33	100.0	58	100.0
Use treated water for other purposes	9	27.3	35	60.3
HWT performed year around	26	78.8	43	74.1
Reported frequency of HWT use
Daily	14	42.4	26	44.8
Every time water is collected	13	39.4	17	29.3
Regularly but not every day	5	15.2	10	17.2
Rarely	1	3.0	4	6.9
Cross-checking reported data on daily HWT use
Last treatment performed > 2 days before survey (among reported daily users)	7	50.0	7	26.9
Availability of treated water if reported daily use†	3	23.1	16	64.0
Water reported to be chlorinated showing FCR ≥ 0.2 mg/L‡	0	0.0	6	37.5

FCR = free chlorine residue; HWT = household water treatment.

^*Respondents may report multiple types of container, so the sum of containers may exceed 100%.

^†Among those with available water at the time of the visit (U [N]: 25, R [N]: 13). On the basis of self-report.

^‡Among households claiming to have chlorinated water at time of visit (U [N]: 16, R [N]: 3).

Daily use of HWT was reported by just under half of the households in both settings (U: 44.8%, R: 42.4%). However, cross-checking these data against other indicators revealed some inconsistencies. First, 26.9% (95% confidence interval [CI]: 8.7–45.2) of urban and 50.0% (95% CI: 20.0–80.0) of rural households claiming to be daily users reported that their water had been treated more than 2 days before the visit. Second, only 64.0% (95% CI: 43.8–84.2) of urban daily users had treated water at home at the time of sampling (based on self-report); the corresponding figure was only 23.1% (95% CI: 0.1–49.6) for rural households. Third, adequate chlorine levels (FCR ≥ 0.2 mg/L) among daily users that claimed to have chlorinated their current drinking water were detected in only 37.5% of urban and 0.0% of the rural cases.

In both settings, the consumption of untreated water among self-reported practitioners of HWT was common (U: 39.7%, R: 48.5%; Supplemental Table 2); however, only a minority reported doing so on a daily basis (U: 4.4%, R: 6.3%). Cross-checking against other indicators suggests that overreporting of nonexclusive drinking of treated water may be taking place, especially, in the rural setting. It was observed that 12.9% (95% CI: 0.4–25.4) of urban and 37.5% (95% CI: 10.9–64.1) of rural households that reported not to drink untreated water were consuming untreated water at the time of the visit. Furthermore, 64% and 100% of urban and rural households, respectively, that claimed not to supplement their treated water with untreated water and that reported to have chlorinated their water had no detectable FCR. Nonexclusive drinking of treated water was similarly common for children < 5 years; 41.9% (95% CI: 23.5–60.3) of urban and 65.0% (95% CI: 42.1–87.9) of rural caretakers reporting that their children would drink untreated water at home.

In-depth interviews.

Fourteen urban (14.5%) and 16 rural (19.5%) households completed the IDI. Overall, the findings were in line with the results obtained in the HWT practices survey. A substantial proportion of urban (21.4%) and rural (25%) households reported not treating their water at this second questioning event. In addition, upon probing after the core JMP question on HWT, it was noted that two and four households in the urban and rural settings, respectively, reported that they actually had their water treated at the source by a nongovernmental organization or a water vendor. It seems that in these cases respondents misinterpreted the JMP core questions on HWT use.

Of those that reported treatment at home (U [N]: 9, R [N]: 8), all reported doing so for health-related reasons. All of these households reported that HWT could prevent diarrhea and four urban and two rural households specifically reported that HWT could prevent cholera.

In both settings most participants were aware of only two HWT methods: chlorination and boiling. As observed in the HWT practices survey, chlorination was the main method reported and preferred over boiling. Further information about participant's perceptions of the cost, availability, and taste of chlorine can be found in Supplemental Information.

As observed in the HWT practices survey, nonexclusive drinking of treated water at home by respondents claiming to be HWT practitioners was common in both settings. This was observed despite the fact that all these households had previously reported in the IDI that unsafe water could lead to health problems. The main reasons given for this was either not having the products available at all times or forgetting at times to treat the drinking water. Nonexclusive drinking of treated water for children < 5 years was similarly common in both settings. All but one rural household acknowledged this was a risk to their and their children's health.

Consistency of HWT use.

Overall, 72.6% of urban and 50.0% of rural households reported to be HWT practitioners at both probing events (Table 2). Over the follow-up period, 56 urban (61.1%) and 71 rural (86.6%) households completed all three follow-up visits and had drinking water available on all three sampling events.

Table 2.

Consistency of HWT use among households that self-reported performing HWT at baseline in the urban and rural communities

Characteristic	Rural		Urban
	n	%	n	%
Consistent reporting of HWT use in the baseline and HWT practices survey/IDI*	41	50.0	69	72.6
Consistent reporting of HWT method among those reporting use in both occasions	37	90.2	61	88.4
Consistent reporting in all five HWT reporting events†	2	2.8	13	23.2
Number of home visits with available treated water (based on self-report)†
Three	3	4.2	13	23.2
Two	0	0.0	3	5.4
Two and one do not know	2	2.8	1	1.8
One	16	22.5	16	28.6
None	50	70.4	23	41.1
Subgroup analysis- Claimed to have treated water on all three collection points:
Among reported daily HWT use‡	0	0.0	7	35.0
Among reported non-supplementers§	2	13.3	10	50.0
Among reported supplememters∥	0	0.0	1	4.6
Household that claimed chlorinating at baseline with FCR ≥ 0.2 mg/L at follow-up visits¶
First visit	1	1.45	19	22.5
Second visit	1	1.39	21	28.38
Third visit	4	5.88	17	27.8

FCR = free chlorine residue; HWT = household water treatment; IDI = in-depth interview.

^*Among households that completed both visits (U [N]: 95, R [N]: 82).

^†Among households that completed all five visits and had water available at all three points (U [N]: 56, R [N]: 71).

^‡Among households that had water at all three points and reported daily HWT use (U [N]: 20, R [N]: 15).

^§Among households that had water at all three points and reported to be non-supplementers (U [N]: 20, R [N]: 15).

^∥Among households that had water at all three points and reported to be supplementers (U [N]: 22, R [N]: 18).

^¶Among households that had water at all three points and reported to be supplementers (U [N]: 22, R [N]: 18).

Consistent reporting of HWT use during follow-up among households identified as HWT users at baseline following JMP guidelines was low, especially in the rural setting. Only, 23.2% of urban households (95% CI: 11.8–34.6) reported HWT use at both questioning events and claimed to have treated water (based on self-report) on all three water sampling visits, while 41.1% of households (95% CI: 27.8–54.5) had untreated water on all three occasions. By contrast, only 2.8% of rural households reported HWT at both questioning events and reported to be consuming treated water on all water sampling visits. Moreover, 70.4% of rural households (95% CI: 59.5–81.3) were consuming untreated water on all three water sampling visits.

Among reported daily users that had water available on all collection visits, only a minority of urban (35.0%) and none of the rural households reported to have treated water on all three collection points (Table 2). Furthermore, in the rural setting, 60% of these daily users reported drinking untreated water on all three visits.

Inconsistencies in HWT use among households that reported not to supplement their treated water with untreated water were observed (Table 2). A substantial proportion of participants reporting not to supplement their treated water were consuming untreated water on all three follow-up visits (U: 20.0%, R: 40.0%). Similarly, when we compared FCR among households reporting chlorination during baseline, only a minority had FCR ≥ 0.2 mg/L in any of the collection visits in either setting (Table 2). Similarly, availability of chlorine among the households claiming to chlorinate their water was uncommon (U: 18.4%, R: 4.3%).

Water quality.

Among both the urban and rural population of self-reported practitioners of adequate HWT that were identified during baseline (U [N]: 95, R [N]: 81), most drinking water samples in the home were contaminated with TTC; in fact, they had higher levels of fecal contamination than the corresponding source water (Table 3) . Overall, in the urban setting, 39.7% (95% CI: 33.3–46.2) of drinking water samples were free of TTC compared with 25.9% (95% CI: 20.2–31.6) of rural samples (Figure 3 ). Overall, in the urban sub-study drinking water had significantly higher TTC counts compared with source water (RR: 9.6, 95% CI: 5.9–15.6, P < 0.001). In the rural counterpart, drinking water also showed increased TTC counts than source water (RR: 2.1, 95% CI: 1.6–2.8, P < 0.001). During the entire follow-up period, only 19.6% of urban and 2.4% of rural households identified as adequate HWT users at baseline had drinking water free of TTC on all completed follow-up visits.

Table 3.

Summary statistics of the microbiological quality of samples of source and drinking water collected at each follow-up visit

	Source							Drinking							Paired samples
	n	Median	IQR	AM	95% CI	WM	95% CI	n	Median	IQR	AM	95% CI	WM	95% CI	n	Mean Log10 difference	95% CI	Wilcoxon signed-rank test
Rural
Visit 1	80	0.5	0, 35	51.2	25.2–77.2	5.0	2.7–8.6	77	60.0	3, 500	263.0	181.2–344.8	38.1	20.9–68.9	76	−0.8	−1.12 to −0.48	P < 0.001
Visit 2	80	17.5	0, 254	147.0	101.1–193.0	16.5	8.8–30.3	78	50.5	0, 224	187.8	126.7–248.9	27.3	15.0–49.0	78	−0.21	−0.43 to −0.02	P = 0.180
Visit 3	73	0.0	0, 80	86.2	45.7–126.7	6.7	3.3–12.6	73	16.0	0, 142	164.3	94.8–233.7	16.8	9.0–31.0	73	−0.37	−0.61 to −0.12	P = 0.004
Urban
Visit 1	77	0.0	0, 2	58.2	23.9–92.5	3.0	1.8–4.8	78	113.0	0, 800	360.9	266.7–455.0	39.7	19.8–78.4	74	−1.13	−1.4 to −0.8	P < 0.001
Visit 2	82	0.0	0, 0	24.1	1.2–47.0	1.7	1.2–2.3	82	13.5	0, 500	279.8	190.7–368.9	20.4	10.2–39.8	77	−1.15	−1.5 to −0.9	P < 0.001
Visit 3	70	0.0	0, 0	4.9	0.4–10.2	1.4	1.1–1.8	64	2.0	0, 332	197.7	120.8–274.5	13.5	6.1–28.9	59	−0.99	−1.3 to −0.7	P < 0.001

AM = arithmetic mean; CI = confidence interval; IQR = interquartile range; WM = Williams mean.

Figure 3.

Fecal contamination in water samples of households claiming to use adequate methods of household water treatment (HWT) at baseline.

A similar picture was observed when we assessed the quality of the drinking water based on the reported HWT status at the time of sample collection as opposed to what was reported at baseline in response to the core JMP question on HWT. Overall, 49.4% of all urban drinking water samples identified as treated (N = 77) at the time of collection were free of TTC; the figure was 39.3% (N = 28) in the rural sub-study. Self-reported treated water samples were significantly more contaminated than source samples in the urban context (RR: 7.6, 95% CI: 4.1–14.0, P < 0.001). In the rural sub-study, samples of treated drinking water had 1.5 (95% CI: 0.8–2.8) higher TTC counts than source water, but the association was nonsignificant (P = 0.17). In the urban setting, of the reported samples treated with chlorine and which had adequate levels of FCR (N = 31), 83.9% were free of TTC. Because of the small number of samples with actual FCR in the rural context, we do not report this figure. Over the entire follow-up, 14.2% and 7.3% of urban and rural water samples collected yielded plates that were too numerous to count.

Discussion

We conducted case studies in urban and rural communities in Zambia to obtain further insight on HWT practices among households identified as HWT users according to current JMP monitoring standards. Our findings raise serious questions about the value of a single household survey for monitoring the consistency and microbiological effectiveness of the practice—key conditions to prevent exposure and disease.

Despite reporting HWT use during the baseline survey, a substantial proportion of households did not report the use of HWT upon administration of the JMP core question on HWT during the second survey or IDI, suggesting low reliability of this question as an indicator of HWT use and raising concerns about the reliability of the current JMP global estimates for HWT use. This uncertainty may have knock-on effects on analyses that rely on such data, as for example, the WHO global burden of diarrheal disease from inadequate WASH assessment.^1,37 In this assessment, figures of the global prevalence of HWT were not adjusted to reflect uncertainty of the HWT use data in terms of uncertainty around the prevalence estimates or the effectiveness of the practice.^1,6,38 While the assessment acknowledged that boiled water could become recontaminated, the assessment assumed safe storage for all households filtering or boiling their water, as information on recontamination was not available. The case studies conducted in Zambia, together with the replicate case studies conducted in Peru,³¹ suggest that results relying on JMP estimates of HWT use should be taken with caution.

We observed that a sizeable proportion of households that identified themselves as HWT practitioners, reported practicing HWT at a frequency that is unlikely to meet all the drinking water needs of the household. This was further confirmed by the fact that a large proportion of households, especially in the rural setting, relied on untreated water during the follow-up period. This lack of treated water at follow-up visits among households identified as HWT users has been previously observed in Zambia and other middle-income countries.^24,39 Similarly, householders readily acknowledged that they consumed untreated water, and provided the same to children < 5 years.

The lack of the necessary materials to perform HWT as well as the absence of FCR in the drinking water samples from the households identified as chlorine users at baseline corroborate the untreated status of the drinking water. This stark difference between reported HWT status in cross-sectional survey settings and longer term assessments using more objective indicators of HWT use raises concerns about the reliability and accuracy of the JMP figures on HWT use and their potential contribution in providing protection against waterborne diseases.

Data from DHS reports published after the initiation of this study provide some corroborating evidence on the variability of HWT practices based on self-report. In the Cambodia 2010 DHS⁴⁰ and the Zimbabwe 2010–2011 DHS⁴¹ surveys, the core JMP questions on HWT use were followed by “Always or sometimes?” While in Cambodia 6.1% and 19.6% of urban and rural households reported to only treat their water sometimes, over half of the households in Zimbabwe reported to treat their water only sometimes (U: 56.4%, R: 66.8%).

In addition, this study provides suggestive evidence of reporting bias of certain HWT-related practices such as nonexclusive drinking of treated water and frequency of HWT use. This might hamper efforts to evaluate and monitor the consistency of HWT use. These data also highlight the need for better indicators to assess consistent HWT use and the need to develop and implement, in a more rigorous manner, tools for monitoring the level of HWT use in HWT intervention studies and dissemination programs. Recently, the WHO published a tool kit for monitoring and evaluating HWT and safe storage programs⁴² with the aim to fill this current need. As part of the document, a set of 20 indicators were recommended, including reported and observed use; correct, consistent use and storage; knowledge and behavior; other environmental health interventions; and water quality. As suggested by this study, the use of some of these indicators might be hampered due to courtesy or reporting bias.

Data from the IDI indicate that lack of knowledge is not a reason for not practicing HWT. Most of the participants interviewed were well aware of the reasons for chlorinating their water; however, this did not stop them from consuming untreated water at times. Moreover, a large proportion of the households acknowledged that consuming untreated water at times was a risk to their children and their own health. This disconnect between knowledge and action has been shown in many other environmental health interventions.^15,43

In this study, in both the urban and rural settings, households that were identified by self-report as users of adequate methods of HWT according to JMP guidelines had drinking water of significantly worse quality than their source water. This suggests that a one-point assessment of HWT use based on self-report is a poor predictor of drinking water quality in the home. Similarly, a substantial proportion of samples that were claimed to be treated at the time of collection were contaminated with TTC. This is likely a reflection of 1) misreporting of the actual HWT status of the drinking water from the participant, in which case, there is a wealth of evidence showing that stored drinking water in the home is often of worse quality than source water⁴⁴ and 2) partly improper treatment and storage of the treated water.^45,46

This study has certain limitations. Neither of the two communities was randomly selected and so may not be representative of the country as a whole. In addition, the study was conducted over a short period during the warm and wet season and so may not be representative of HWT practices during other seasons. Similarly, we cannot rule out the potential for reactivity due to repeated follow-up visits.⁴⁷ It should also be noted that in these case studies, samples of drinking water were collected from the drinking water vessel. Research has shown that drinking water served in cups is often more contaminated than the corresponding stored drinking water,^45,48 and so our case studies may be overestimating the actual microbiological quality of drinking water in the home. Finally, a major limitation of this study is the reliance on self-reported data to determine consistent HWT use, highlighting the lack of high-quality objective indicators of exclusive HWT use.

Notwithstanding these limitations, these findings raise important questions about the value of the current approach of assessing HWT practices using the JMP core questions for public health and policy purposes. An affirmative response to an HWT use question in a single cross-sectional survey may provide little indication of drinking water quality in the home or actual use of HWT. Other strategies, including the actual assessment of drinking water quality at the household, should be considered. This may become viable with the rapid development of rapid bacterial detection tests.⁴⁹ This would not only provide an actual indicator of drinking water quality in the home, but would allow assessing the effectiveness of reported HWT use at large scale. In fact, monitoring of fecal contamination is considered by the JMP to be one of the next steps in improving global monitoring of access to safe drinking water,^50–52 and water quality testing of safely managed water sources is to be an integral part of the Sustainable Development Goals targets on drinking water, sanitation and hygiene, and waste management.⁵³ However, issues with the high temporal variability of microbial quality in drinking water, including seasonal trends, will need to be taken into account if these assessments are to provide accurate and representative data.⁵⁴

In conclusion, the lack of consistency and microbiological effectiveness of HWT practices among those identified as HWT users according to current monitoring standards raise important questions about the potential contribution of HWT in low- and middle-income countries in reducing the risk of waterborne diseases. As our findings are largely in line with similar case studies in Cambodia,²⁴ India,⁵⁵ and Zambia,³⁹ it seems fitting to reconsider the manner in which HWT is promoted, implemented, and monitored to improve its consistency and effectiveness—key aspects to achieve its full health impact.

Disclaimer: The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.