Modeling the Sustainability of a Ceramic Water Filter Intervention

Abstract

Ceramic water filters (CWFs) are a point-of-use water treatment technology that has shown promise in preventing early childhood diarrhea (ECD) in resource-limited settings. Despite this promise, some researchers have questioned their ability to reduce ECD incidences over the long term since most effectiveness trials conducted to date are less than one year in duration limiting their ability to assess long-term sustainability factors. Most trials also suffer from lack of blinding making them potentially biased. This study uses an agent-based model (ABM) to explore factors related to the long-term sustainability of CWFs in preventing ECD and was based on a three year longitudinal field study. Factors such as filter user compliance, microbial removal effectiveness, filter cleaning and compliance declines were explored. Modeled results indicate that broadly defined human behaviors like compliance and declining microbial effectiveness due to improper maintenance are primary drivers of the outcome metrics of household drinking water quality and ECD rates. The model predicts that a ceramic filter intervention can reduce ECD incidence amongst under two year old children by 41.3%. However, after three years, the average filter is almost entirely ineffective at reducing ECD incidence due to declining filter microbial removal effectiveness resulting from improper maintenance. The model predicts very low ECD rates are possible if compliance rates are 80-90%, filter log reduction efficiency is 3 or greater and there are minimal long-term compliance declines. Cleaning filters at least once every 4 months makes it more likely to achieve very low ECD rates as does the availability of replacement filters for purchase. These results help to understand the heterogeneity seen in previous intervention-control trials and reemphasize the need for researchers to accurately measure confounding variables and ensure that field trials are at least 2-3 years in duration. In summary, the CWF can be a highly effective tool in the fight against ECD, but every effort should be made by implementing agencies to ensure consistent use and maintenance.

1. Introduction

Early childhood diarrhea (ECD) is a scourge that kills an estimated 1.6 million children worldwide each year and is a problem that is largely due to poor access to water, sanitation and hygiene (WASH) infrastructure (WHO, 2006). This problem is further exacerbated by the fact that water frequently becomes contaminated after collection but before consumption in many developing world countries (Wright et al., 2004) limiting the ability of community safe water sources to prevent ECD. This is a particular problem for households who must travel long distances to collect water (Mellor et al., 2012b). This is why many have advocated for the use of point-of-use water treatment devices as a means of improving health (Clasen, 2010). Biosand filtration (Tiwari et al., 2009), solar disinfection, and chlorination (Arnold and Colford, 2007) have all shown promise as means of improving household drinking water quality. Ceramic water filters (CWFs) are one such technology that can be produced in local communities using methods and materials that do not need to be imported and are an environmentally sound technology (Ren et al., 2013). They have been shown to be a highly effective means of removing E. coli and other pathogens in controlled environments (Brown and Sobsey, 2010) and are typically impregnated with colloidal silver (Oyanedel-Craver and Smith, 2008). They have been shown to be effective at removing E. coli and total coliform in the field (Kallman et al., 2011) and at reducing ECD incidence (Fewtrell et al., 2005).

Despite this promise, most recent research has suggested that the evidence in support of point-of-use treatment technologies might be subject to recall bias since few studies have been blinded (Schmidt and Cairncross, 2009). Other researchers have found that factors including duration of follow-up and blinding were significant predictors of intervention effectiveness and that point-of-use water treatment device interventions would decline in effectiveness over time (Hunter, 2009). Blinding is important because of the courtesy bias that can be associated with the provisioning of useful products to research participants which can bias their reported disease rates. Enger et al. (2012) used a quantitative microbial risk assessment (QMRA) model to find that there are diminishing returns for improved log reduction efficiency in point-of-use water treatment devices when they are not used consistently, and that log reduction values above 2 generally prevented little additional diarrhea assuming 80% compliance. Other researchers have seen microbial effectiveness declines over time (Kallman et al., 2011) and that there can be contamination problems in the lower reservoir of CWFs due too poor maintenance (Lantagne, 2001).

Two studies have been conducted recently to understand realistic water filter compliance in the field. In the first paper Brown et al. (2009) found a linear decrease in use of the filters of approximately 2 percentage points each month. They also found that the odds ratio for using a filter was 1.7 when study participants were collecting surface water compared to 0.56 when using ground water which is of presumably better quality. A second study by Casanova et al. (2012) found that study participants were almost twice as likely to use their CWF when E. coli were present in the water. They likewise found that those with tap water were less likely to use the filters compared to those who used well water exclusively.

A final component essential for long-term sustainability is a willingness to pay (WTP) for new water filters after a filter breaks. Given that more than ~12% of filters can break over the course of a year (Brown et al., 2009), it would be desirable to have replacement filters available for purchase. However, it is unclear how much households might be willing-to-pay for new filters (especially if they were free initially) and how the availability of filters for purchase might affect ECD incidences.

A novel means of studying WASH interventions in resource limited-settings are agent-based models (ABMs) (An, 2008). ABMs are models that can be useful for understanding complex systems. They have previously been used to study schistosomiasis transmission (Hu et al., 2010), water usage in U.S. and Dutch households (Linkola et al., 2013), the impact of upstream water management on downstream agriculture in Thailand (Becu et al., 2003) and domestic water management in Spain (Galán et al., 2009). Coupled approaches have also been used to study subsistence farming, land cover and hydrology (Bithell and Brasington, 2009). The ABM technique is therefore an ideal tool to study the complex WASH system found in many developing world communities. This sort of analysis compliments intervention-control trials because such trials look at interventions in isolation, are susceptible to differences in intervention quality and are subject to large heterogeneities due to the difficulties of using self-reported ECD as a indicator of poor drinking water quality (Schmidt et al., 2011). Furthermore, it is clear that ECD is due to multiple technological, environmental and behavioral factors (Ezzati et al., 2005) leading some to suggest that systems approaches are preferable (Eisenberg et al., 2012).

Given the questions surrounding the ability of CWFs or any other point-of-use water treatment technology to effectively reduce ECD rates in the long term, there is a clear need to better understand the complexities of point-of-use water treatment technologies in a realistic setting. Therefore, the goal of this project is to investigate the role of factors affecting the imperfect use of CWFs in preventing early childhood diarrhea using an extension of an ABM described previously (Mellor et al., 2012a). This extension is based on three years of follow-up field data about filter microbial removal effectiveness, compliance and breakage. Specifically, the following factors were investigated to understand their relation to the outcome metrics of household drinking water quality and ECD rates:

• Filter prevalence

• Filter compliance

• Effects of measured declines in microbial removal effectiveness over time

• Filter breakage percent

• Filter breakage date

• Filter cleaning interval

• Filter microbial removal effectiveness

• Linear decreases in compliance over time

• WTP for new filters

• Perceptions of water quality and filter compliance

This study therefore helps implementing agencies improve CWF interventions, attempts to answer open research questions (Lantagne et al., 2006), demonstrates the adaptability of ABMs to understand intervention complexities and can both explain heterogeneity seen in previous intervention-control trials and inform future ones.

2. Methods

2.1. Study Site

The study was conducted in Venda region of the Limpopo Province of South Africa. Rural communities in South Africa suffer from the deleterious effects of decentralized access to drinking water. According to national statistics, 21% of the rural population does not have access to improved drinking water supplies (CIA, 2013). Studies have reported microbial contamination of stored drinking water and weaning food, and high prevalence of Human Immunodeficiency Virus (HIV)(Samie et al., 2006; Potgieter et al., 2011).

2.2. Study Background

As described previously, a field study was initiated in 2009 to evaluate whether CWFs can improve drinking water quality and decrease diarrhea days of HIV-positive individuals in communities in Venda (Abebe et al., 2014). The participants received CWFs coated with silver nanoparticles placed in a safe storage container with a spigot attached to prevent contamination. Water samples were collected at baseline and at the conclusion of the yearlong study. The study was closed in 2010.

In July 2012, a follow-up study was conducted on a subset of the cohort to test the effectiveness of the filters. The 20 households were randomly selected and water quality analysis was conducted on each filter to determine filter performance 3 years post-intervention.

2.3. Water Testing and Surveying Protocol

Water quality analyses were performed by collecting water samples from the common in-house water source (influent) and post-ceramic filter treatment (effluent). The influent water source was collected directly from a tap located inside the home or a water storage container. The water in storage containers was originally collected by study participants from outside sources such as rivers or community taps. The effluent sample was collected directly from the spigot of the lower reservoir of the ceramic filter. Samples were analyzed for total coliform bacteria in duplicate using standard membrane filtration techniques with MilliPore’s m-Endo medium for the 2009-2010 sampling and MilliPore’s m-ColiBlue24 medium in 2012.

User compliance was evaluated through surveys conducted during each visit. A ceramic filter satisfaction and WTP survey was conducted to assess compliance and breakage. WTP was determined through a commonly used bidding game. For the bidding game, participants were given a starting price, which they would be willing to pay for a filter. Then the price was incrementally increased until the participant was no longer willing to pay at that price, determining the maximum WTP for a ceramic filter (Wedgwood and Sansom, 2003). All surveys were conducted with the assistance of a translator. This study was approved by the Institutional Review Board of the University of Virginia.

2.4. Microbial Data Analysis

Each ceramic filter influent and effluent measurement was translated to a log reduction value for each household so that measurements for individual households could be compared across the three sampling periods for the data. 35 household measures of the enrollment log reduction values were complete enough to include in the subsequent analyses. Even so, after enrollment, the number of participating households declined. This left a number of missing data values. To rectify this, a standard multiple imputation technique (Rubin, 1978) using IBM SPSS Statistics software was used to fill in the missing log reduction values. The resulting complete data set has identical means, variability and regression parameters as the original data.

Linear piecewise fits were made between enrollment at year zero, year one and year three for each participating household. These 35 fits are shown in Figure 1. It is important to note that many filters degrade the water as it passes through resulting in negative log reduction values. This is likely due to biofilm buildup on the inside of the lower reservoir walls or biological growth in the bulk water phase and is a common occurrence in such storage containers (Mellor et al., 2013).

Figure 1.

Filter log reduction values over the three year measurement period. Each line represents a single household’s CWF effectiveness over time. Missing values were filled in using standard multiple imputation techniques.

2.5. Modeling Approach

2.5.1. Basic CWF Routine

An original ABM model described previously (Mellor et al., 2012a) was modified in Netlogo to simulate the introduction of CWFs into adjacent communities in Limpopo, South Africa. The complete code is included as part of the Supplementary Materials. The Basic CWF Routine described in this section is meant to simulate a realistic water filter campaign introduced in the communities and is based on the collected data described in previous sections. Flow diagrams of the original model are shown in Supplementary Materials Figures S1 and S4. The Basic CWF Routine modification (which was inserted into the original model) is given in Figure S2.

The model consists two types of agents: households and children. House-hold agents own the WASH attributes and behaviors described below. Child agents are all under two years of age, consume water in the household and can get sick from consuming poor quality water.

As shown in Supplementary Materials Figure S1, in the original model on every simulated day (“tick”), each household can collect from one of three community water sources if they choose to and if their preferred source is operational. If their primary source is not operational they wait for some number of days until it is operational or they can revert to a secondary source. During storage, a household’s water container can be subject to contamination from biofilm layers, hands, water transfer devices and biological regrowth which are functions of storage container type and source water type (Mellor et al., 2013). Households can also clean their storage containers which improves water quality. Households share information with their nearest neighbors at model run inception about their water sources, all of their WASH behaviors and quantified contamination sources described above. All of these aspects were quantified through previously reported field studies (Mellor et al., 2013, 2012a). The coliform bacteria concentration of the i^th household’s drinking water is defined as W Q_i and is originally a function of source water quality, but changes as described above.

Under the Basic CWF Routine modeled scenario, the filters would first be introduced for free to a percentage of the communities who would use them for the first two years of their child’s life. It is assumed for the purposes of this model that 410 children are born on the same day they receive their filters. This is an average of one child per house in the modeled communities which are comprised of 410 households. At model run inception, each household is randomly assigned one of the filter microbial-effectiveness curves shown in Figure 1.

Under the Basic CWF Routine, households can treat their water using a CWF as described in Supplementary Materials Figure S2. If a household has a CWF and if it is not broken, the household can use that filter just before consumption. The filter effectiveness for each day is computed from that household’s microbial removal effectiveness curve and the W Q_i increases or decreases as appropriate.

After the Basic CWF Routine, the household level variable W Q_i denotes the quality of water that the child in a particular household consumes on that day and the resulting potential for getting ECD is calculated daily for each child as is shown in Supplementary Materials Figure S4 and described previously (Mellor et al., 2012a). A young child’s daily propensity to drink water was also measured and incorporated. The probability of getting ECD is increased through previous cases (Moore et al., 2010) and is decreased through rotavirus vaccination (Madhi et al., 2010) and hand-washing (Curtis and Cairncross, 2003).

A household’s W Q_i value then carries over to subsequent days and is subject further biological regrowth and contamination until a house collects water again. The model is run for two years during which time children are most sensitive to poor quality water. The model outputs the mean number of ECD incidences over the first two years of a child’s life as well as mean and median daily household drinking water quality. The original ABM was carefully validated with field measurements of ECD rates and ECD-induced child growth stunting (Mellor et al., 2012a).

This Basic CWF Routine was used to understand the effects of the measured declines in filter microbial removal effectiveness, filter prevalence and compliance. This was done by keeping most values static while varying the parameter of interest with Netlogo’s behavior space analysis tool. Filter prevalence is the percent of households in the communities with a CWF. Filter compliance is the percent of days that households use a water filter if they have one. Breakage percent is the percent of filters that break during the two years. Filters have an equal chance of breaking on any model day for the Basic CWF Routine. As shown in Supplementary Materials Table S1, the baseline compliance rate is 90% and the breakage rate is 20% as was ascertained during the field measurements. The prevalence is assumed to be 100% for the baseline scenario. During the behavior space analyses each single parameter value was run 1,000 times. The resulting ECD rates and water qualities were then averaged over all runs.

2.5.2. Other Factors Tested Using Basic CWF Routine

This Basic CWF Routine described above was used to study several other important aspects of CWF use. The parameter ranges tested are given in Supplementary Materials Table S2. The baseline values of 90% compliance, 20% breakage rate and 100% prevalence were likewise used and each parameter value was run 1,000 times. The first experiment was to determine how microbial removal effectiveness impacts outcomes. This was done by replacing the microbial removal effectiveness curves with static removal percentages that are the same for all households. Effectiveness was varied from 1 to 5 log reduction. An additional analysis was conducted using the median log reduction efficiency at the 0, 1 and 3 year mark to better understand how realistic declines in effectiveness affect the outcome variables.

Researchers have suggested that the declining microbial removal effectiveness seen in Figure 1 can be reversed if the lower reservoir is cleaned throughly (Kallman et al., 2011). To simulate this, household microbial removal effectiveness declines as usual except when households clean their storage containers. At this point, their filter’s microbial removal effectiveness reverts to day-zero levels and again follows their microbial removal effectiveness curves until the next cleaning day. Cleaning interval was varied from 1 to 730 days and all households had the same cleaning interval.

Some have found that compliance of filters declines linearly over time at a rate of about 2 percentage points per month (Brown et al., 2009). To investigate the effects this might have, all household compliance percentages were decreased at a linear rate from a 90% starting value.

2.5.3. Additional CWF Routines

To further understand what parameters may play an important role in ECD incidences, the original ABM model was further modified with several Additional CWF Routines as are shown in Supplementary Materials Figure S3. These Additional CWF Routines took the place of the Basic CWF Routine described above and were input into the original ABM in the same manner. However, for the two parameter experiments summarized below, the unused portions of the code were bypassed, but included for the multi-parameter analyses described in the next section. Parameter values varied are given in Supplementary Materials Table S2. Each parameter combination was run 1,000 times.

If a filter broke early in a child’s life, it would likely not be an effective ECD reduction tool. Therefore breakage date was studied by having all filters break on the same model day. This was analyzed in conjunction with breakage rates.

There is evidence to suggest that filter users are fairly adept at knowing when their water is contaminated and are more likely to use the filter in such circumstances. To model this effect, all simulated households are 2.05 times as likely to treat their water if their W Q_i is above a varied threshold value (0 - 2000 CFU/100ml) (Casanova et al., 2012). Since this scenario is highly dependent on baseline filter compliance, the two parameters were varied concurrently for this experiment.

Given that filter breakage is a common problem with any ceramic filter intervention and that sustainability is an important component of any development project, there is a need to study how a community’s health outcomes change with WTP for a new filter if their filter breaks. To simulate this, each household was assigned a WTP in South African Rand ($1 United States Dollar ≈ 8.9 South African Rand) based on household survey data shown in Supplementary Materials Figure S5. The filters are initially distributed free of charge. Then, if a filter breaks, a household can purchase a new filter if the purchase price is equal to or less than their WTP for a new filter. An important co-variate is the breakage percentage which is varied concurrently.

2.5.4. Multiple Parameter Behavior Space Analyses

As a final assessment, two multi-parameter behavior space analyses were conducted to further elucidate system complexities and identify the parameters most responsible for large improvements in the outcome variables. In the first case, the four most salient parameters of filter compliance, microbial removal effectiveness, filter prevalence and linear usage decrease were all simultaneously varied as shown in Supplementary Materials Table S3 using the Basic CWF Routine. In the second case, the less salient parameters of filter price, cleaning interval, threshold water quality, breakage percent and breakage date were simultaneously varied as shown in Supplementary Materials Table S4. Relative salience was determined by a larger multi-parameter space analysis shown in Supplementary Materials Figure S6 which varied all parameters. The data were analyzed using and contour plots. Each parameter combination was run once for these experiments.

3. Results

Overall, results indicate that technical efficiency is just one factor in the ability of CWFs to improve water quality and reduce ECD cases.

3.1. Basic CWF Routine

The Basic CWF Routine included analyses of several relevant parameters important to the ability of CWFs to improve health including filter prevalence, compliance and microbial removal effectiveness.

As can be seen in Figure 2, the data indicate clearly that the changing microbial removal effectiveness seen in Figure 1 has a significant negative effect on the outcome variables since effectiveness decreases markedly over two years in most cases. The difference between runs with changing versus constant (i.e. day 1) microbial removal effectiveness are as follows: mean daily water quality, 259.8 vs 109.5 CFU/100mL (p < 0.001, t-test), median daily water quality, 5.37 vs 0.23 CFU/100mL (p < 0.001, t-test) and ECD cases 4.92 vs 2.70 (p < 0.001, t-test).

Figure 2.

Plots showing the mean and median daily water qualities and the mean total early childhood diarrhea (ECD) cases for changing vs constant microbial effectiveness, ceramic water filter (CWF) prevalence and CWF compliance. Results indicate that the generally declining microbial effectiveness of the filters has a highly significant impact on the three outcome metrics. CWF prevalence and compliance showed nearly identical trends with mean water quality values being nearly a linear function of those two factors. Likewise median daily water quality was generally log-linear and mean ECD cases were linear for the two factors. All analyses were run using the Basic CWF Routine.

The prevalence, or percentage of community members with a CWF had a large effect on the outcome variables. Mean water quality (p < 0.001, ANOVA), median water quality (p < 0.001, ANOVA) and ECD (p < 0.001, ANOVA) all declined significantly. Between 0 and 100% prevalence, mean daily water quality declined from 619.5 to 259.5 CFU/100mL, median daily water quality likewise went from 122.8 to 5.39 CFU/100mL and ECD cases declined from 8.43 to 4.95. The declines were linear for mean water quality and ECD incidence while they were log-linear for median daily water quality.

Even when all community members have filters, the percent of time that they use them or compliance has a large influence on the outcome variables as shown in Figure 2. Mean water quality (p < 0.001, ANOVA), median water quality (p < 0.001, ANOVA) and ECD cases (p < 0.001, ANOVA) all declined significantly. Mean water quality deteriorated from 624.3 to 242.7 CFU/100mL, median water quality went from 126.0 to 4.04 CFU/100mL while ECD cases went from 8.48 to 4.57 as compliance varied from 0 to 100%.

3.2. Other Factors Tested Using Basic CWF Routine

Theoretical changes in microbial removal efficiency had a statistically significant effect on the outcome variables with mean water quality (p < 0.001, ANOVA), median water quality (p < 0.001, ANOVA) and ECD cases (p < 0.001, ANOVA) as seen in Figure 3. There was a distinct non-linearity around a 3 LRV. There was little variation in the outcome variables for removal efficiencies greater than 3. However, the outcome variable differed a lot for inferior removal efficiencies.

Figure 3.

Plots showing the mean and median daily water qualities and the mean total early childhood diarrhea (ECD) cases as a function of log reduction value (LRV), cleaning interval, and linear compliance declines. LRV plots indicate that outcome variables are not strongly correlated with microbial effectiveness when LRVs are better than 0.001 (LOG 3). Furthermore, the median LRV at the 0, 1 and 3 year mark (2.92, 1.63 and 0.42 respectively) lead to drastically deteriorating outcome metrics. Daily cleaning significantly improved outcome variables compared to less frequent cleaning. However, even bi-monthly or cleaning every six months was highly beneficial. Linear compliance declines had large effects on all outcome metrics. CWF = Ceramic Water Filter

The model was run for the median microbial log removal effectiveness at the 0, 1 and 3 year points which were 2.92, 1.63 and 0.42 respectively. These results are shown in Figure 3 which indicate that mean water quality, median water quality and mean ECD cases changed significantly (p < 0.001, ANOVA). Mean water quality deteriorated from 72.0 to 268.5 CFU/100 mL, median water quality went from 0.1 to 32.6 CFU/100 mL and mean ECD cases increased from 1.39 to 7.48 between year zero and year three.

More frequent cleaning had a positive effect on the outcome variables as can be seen in Figure 3. There were significant variations between the outcome metrics of mean water quality (p < 0.001), median water quality (p < 0.001, ANOVA) and ECD cases (p < 0.001, ANOVA). Daily cleaning improved mean water quality compared to not cleaning by 108.9 vs 253.1 CFU/100mL, median daily water quality 0.24 vs 5.31 CFU/100mL and ECD cases by 2.70 vs 4.88.

Linear decreases in compliance led to significant variations in the outcome metrics of mean water quality (p < 0.001, ANOVA), median water quality (p < 0.001, ANOVA) and ECD cases (p < 0.001, ANOVA). Even realistic compliance declines of 20% per year resulted in a deterioration of the outcome variables. A 20% vs 0% decline changed the outcome variables as follows: mean water quality 287.1 to 259.9 CFU/100mL; median water quality from 11.89 to 5.60 CFU/100mL; and ECD cases from 5.48 to 4.93.

3.3. Additional CWF Routines

Figure 4 shows how the outcome variables of median daily water quality and ECD cases vary as a function of breakage date and the percent of CWFs that break on that day. The outcome variables are more sensitive to breakage date for the higher breakage percentages. In general, the longer filters are in use, the more effective they will be in preventing ECD, but the rate of decrease decreases markedly at the one year mark.

Figure 4.

Median water quality and average early childhood diarrhea (ECD) cases for filter breakage, threshold water quality and filter price experiments. For filter breakage, the outcome variables are more sensitive to breakage date for high breakage percents. In the threshold experiment, households are about twice as likely to treat their water when it is above a given water quality. The outcome metrics show high sensitivity of this threshold above 10 CFU/100ml. The metrics are most sensitive to the threshold when baseline compliance is 40-60%. In the filter price experiment, higher breakage percentages had improved outcome metrics for low filter prices since the microbial effectiveness of new filters is generally superior to older filters. All outcome variables showed significant improvement at around 100 South African Rand, which was the median willingness-to-pay. WQ = Water Quality.

The graphs in Figure 4 summarize a behavior space analysis of threshold water quality and baseline percent use. In general the outcome metrics are sensitive to the threshold water quality (the water quality above which a household is about twice as likely to use the CWF) for values above 10 CFU/100mL. The baseline compliances of 40-60% showed the highest sensitivity to threshold water quality although even the more realistic compliances percentages (i.e. 90%) showed a marked improvement.

An investigation of the outcome metrics as a function of filter price is shown in Figure 4. It is clear from these diagrams that improvements in water quality and ECD cases could be realized through the ability of households to have filters available for purchase for a reasonable price. This is particularly true with high breakage rates. For realistic breakage rates of around 20% for the two year period there was a deterioration of median water quality from 2.57 to 5.31 CFU/100mL and ECD cases from 4.41 to 4.91 between the lowest and highest filter prices.

3.4. Multi-Parameter Behavior Space Analyses

The variable combinations of the first analysis are given in Supplementary Materials Table S3. The same information is provided in the referenced figures below. As Figure 5 indicates, the majority of parameter combinations result in 8-9 ECD cases during the first two years of life. However, a minority of optimized parameter combinations can lead to vast reductions in ECD cases. The parameter combination thresholds leading to low ECD rates are discussed below.

Figure 5.

Multi-parameter contour plots of the four most salient parameters (compliance, prevalence, yearly compliance decreases and CWF (ceramic water filter) LRVs (log reduction values)). The x-axis of each individual plot represents the CWF prevalence while the y-axis represents CWF compliance. Optimal outcomes of 0-2 ECD cases over the first two years of life were only achievable if prevalence and compliance were high (80-90%) and compliance decreases were low (< 10% per year). Prevalence and compliance had a large effect on ECD cases for low rates of compliance decline, but ECD cases were less sensitive to prevalence and compliance for high rates of compliance decline. CWF LRVs had to be at least 3 in order to achieve optimal outcomes, but better LRVs did not necessarily reduce ECD cases. ECD incidences were relatively insensitive to prevalence, compliance and compliance declines for low LRVs.

Figure 5 is comprised of multi-parameter contour plots of the four most salient parameters (compliance, prevalence, yearly compliance decreases and CWF LRVs). Results indicated that optimal outcomes (0-2 ECD cases) are only possible if prevalence and compliance are 80-90% and compliance decreases must be below 10% per year. Furthermore, CWF LRVs had to be at least 3 in order to achieve optimal outcomes and that LRVs above 3 did not necessarily improve outcomes. It is notable that ECD incidences were rather insensitive to prevalence and compliance for low LRVs and high rates of compliance decreases.

A second multi-parameter behavior space analysis was conducted to explore several of the less salient parameters. Figure 6 indicates that most parameter combinations result in ECD rates of 2-4 with a minority resulting in lower and higher rates. These ECD rates are generally lower than those for the prior analysis because 90% compliance and 100% prevalence was assumed.

Figure 6.

Multi-parameter contour plots of the four least salient parameters (CWF price, cleaning interval, breakage percent and breakage date). All experiments assume 100% prevalence and 90% compliance. The x-axis of each individual plot represents the CWF breakage date while the y-axis represents CWF breakage percent. It is evident that optimal outcomes (ECD rates of 0-2) are only achievable if the cleaning interval is less than 120 days. However, the availability of filters for purchase for less than 100 Rand assures that ECD rates will be less than 6 irrespective of the other parameters. ECD rates are more sensitive to breakage percent and date for high purchase prices. CWF = ceramic water filter. For clarity, all plots are for threshold values of 50 CFU/100ml since ECD cases are less sensitive to that parameter.

Figure 6 indicates that optimal results (ECD rates of 0-2) can only be achieved if the cleaning intervals are less than 120-180 days. Furthermore, if filters are available for less than 100 Rand, ECD rates above 6 are impossible irrespective of the other variable values. ECD rates are more sensitive to breakage percent and date when the CWFs are expensive.

4. Discussion

By taking a systems approach to model a ceramic filter intervention in a developing world location, we have uncovered the complex coupled human/engineered/natural system dynamics that are critical for understanding the long-term sustainability of a CWF intervention. Such a modeling approach can quickly and effectively help prioritize and inform interventions, rank risk factor importance and give some insight into confounding variables and report biases in intervention-control trials.

Our model predicted that there were diminishing returns for improved log reduction efficiency when CWFs are not used consistently. This result was also found by Enger et al. (2012) for imperfect compliance levels. This has important implications for the fabrication of filters. In-country tests of filters have shown them to be 92 - 99% effective (Kallman et al., 2011; Brown and Sobsey, 2010) at removing E. coli bacteria which should be improved to achieve optimal results.

Our model has also shown that there are multiple risk factors sufficient to maintain diarrheal disease such that the elimination of single risk factors will likely have a minimal impact on ECD rates. This result is supported by a previous study (Eisenberg et al., 2007).

Our basic model predicts that ECD can be reduced by approximately 41.3% with the introduction of a ceramic filter campaign assuming a 90% compliance rate. This result is highly consistent with the 46% found by Brown et al. (2007). Other studies that investigated the use of ceramic filter candles in a similar arrangement found relative risks of 0.30 (95% CI 0.19 - 0.47) (Clasen et al., 2004), 0.40 (95% CI 0.25 - 0.64) (Clasen et al., 2005) and 0.47 (95% CI 0.24 - 0.92) for the general population (Clasen et al., 2006). A subsequent study found an odds ratio of 0.17 (95% CI 0.08 - 0.37) (Du Preez et al., 2008) for children 24-36 months of age. However all of these studies had large uncertainty and none of these studies were blinded and are therefore subject to recall bias. A meta-analysis of these studies indicates a combined relative risk of 0.34 (95% CI 0.26 - 0.43) (Hunter, 2009). This combined relative risk is less than the 0.59 calculated for this study. However, Wood et al. (2008) suggest a ratio of odds ratios of 0.75 (0.61 - 0.81) to account for the lack of blinding. This 25% correction would decrease our relative risk to 0.44, which is highly consistent with most studies cited above.

Our study can also be compared to the Hunter (2009) study which investigated the effects of study duration. Their study estimated a relative risk of 0.37 (95% CI 0.19 - 0.71) for 52 weeks of follow-up. This value is statistically equivalent to the 0.64 found by our ABM using the day 365 microbial removal effectiveness values. Correcting our value by 25% would result in a 0.48 relative risk, which is even closer to their reported value.

The most important conclusion of this work is that behavioral factors can have a huge impact on the long-term effectiveness of a CWF intervention. The presumed contamination of the lower reservoir that leads to the gradual decline in CWF effectiveness plays a significantly detrimental role in the ability of these filters to reduce ECD incidence in young children. Filters are largely ineffective at reducing ECD incidence after 3 years of realistic usage. We showed that periodic cleaning can have a positive effect on the outcome variables. Cleaning the lower reservoir at least every four months makes it possible to achieve very low ECD rates in our experiment.

Inconsistent use of the filters can have a large impact on outcome variables. This is especially important given that upwards of 95% of children under five reported drinking untreated water the previous day during a recent household water treatment trial (Boisson et al., 2010). Another interesting result from our work was the investigation into uniform linear decreases in compliance. We showed that the linear decreases of 2% per month reported previously (Brown et al., 2009) can have a negative effect on outcome variables, while larger linear decreases led to far worse outcomes. This finding quantitatively reinforces the need for implementing agencies to consider long term sustainability.

The analysis of breakage percent and breakage date illustrated how detrimental these two factors can be on the sustainability of a CWF campaign. The realistic breakage percentage of around 20% over the two year period can lead to detrimental outcomes especially when a filter breaks early in a child’s life. Complementing this experiment is our study of how the availability of a CWF for purchase can help mitigate high breakage rates. In our model, median water quality was not highly dependent on filter price below 100 Rand, but it deteriorated quickly above that amount. Based on this result, implementing agencies should strive to keep prices below this value which is equivalent to $11.23 USD.

The threshold experiment which looked at how households who treated their drinking water when it was above a given threshold also have important implications for policy makers and community health workers. According to our results, CWF users can get away without using their filters if water quality is good, but the more adept they are at recognizing poor quality water, and using their filters during high risk consumption, the more benefit they will receive. It is especially important for users to recognize when their water quality contains 10 CFU/100mL of coliform bacteria or more. Although there is no definitive way for community members to know their water quality, this finding encourages implementing agencies to educate communities about the hazards of drinking from acutely contaminated water sources and to treat their water if they must collect from those sources.

The multi-parameter investigation provided us with valuable information about which parameters are most important and therefore most critical when trying to reduce ECD incidence. The first notable fact is that the majority of parameter combinations led to ECD rates centered around 8 to 9 for the first two years of a child’s life. However, a notable minority of parameter combinations led to very good outcomes. In the case of the first multiple parameter behavior space analysis in Figure 5, it is clear that filter compliance, prevalence and linear declines were all important parameters and needed to be optimized to achieve very good outcomes. Furthermore, optimal outcomes were achievable with similar probabilities for anything from a 3 to 5 log reduction efficiency, but impossible for lower efficiencies. Also, it is important to note that the ECD rates for the optimal combinations lead to rates much lower than the rate of 4.9 cases from the baseline model which used the generally declining log reduction efficiency curves that were based on field measurements. Very low rates require users to adhere to high compliance (80-90%) and sound maintenance regimes.

The second multiple parameter analysis looked to compare several other factors. It found that container cleaning was important and that frequent container cleaning can lead to very good outcomes. In fact, adequate cleaning at least once every four months makes optimal outcomes possible. Having filters available for less than 80-100 Rand likewise excluded poor outcomes. Breakage percent and date were more critical for high purchase prices. Most importantly, this analysis again showed that very low (i.e. less than 2 ECD cases) were possible when the filters were used optimally and new filters are available for purchase.

Although our ABM is well tested (Mellor et al., 2012a), is consistent with previously reported results, is based on four years of field data collection (Mellor et al., 2013), and produces logical conclusions, there are a few notable limitations. First, our study was designed to investigate household water quality generally and the ECD incidences of children under two years of age. Children of this age range are highly sensitive to poor water quality and are likely to experience growth stunting as a result (Checkley et al., 2008). Our model therefore cannot predict the effectiveness of a filter campaign for older children. Our model would benefit from additional field research into the mechanisms of filter effectiveness declines and ways to clean the filters. Lastly, the CWF microbial and usage data as well as the WTP data was obtained from HIV positive individuals. It is possible that such a population might behave differently than the general population which could have influenced our results. However, the very high HIV rates amongst mothers in Limpopo (21.4%) (SADOH, 2011) makes it likely that our modeled communities contain a high proportion of HIV positive individuals. Furthermore, our results are consistent with previous studies looking at filter effectiveness that focused on non-HIV positive populations.

Our conclusions lead us to better understand both the results of previous randomized field trials as well as inform the development of future trials. Based on these results, the large heterogeneity seen in previous trials of CWFs and other point-of-use interventions could be due to variations in compliance, cleaning regimes, compliance declines, microbial effectiveness or breakage. It is therefore imperative that future trials accurately measure exactly how often households use their filters, how frequently participants drink from non-treated water even when “using” the filters and how compliance might decline over time or vary between household members. Researchers must also have an understanding of each household’s propensity to clean their filter and the confounding effects that might have. Future studies should also include all potential co-variates which might have an impact on outcomes. For instance, it is possible that the filters might reduce ECD incidence by a greater amount in crowded households or in households with low levels of eduction who might be less likely to maintain their filters or who might reserve the purified water only for certain household members. Lastly, these results reinforce the need for researchers to conduct trials of 2-3 years or more in order to understand the long-term effectiveness of the filters at improving water quality and reducing ECD cases. Shorter trials simply fail to capture the CWFs true effectiveness. Understanding these covariates will likely lower literature values for CWF effectiveness.

5. Conclusion

We have illustrated a novel complex systems technique for understanding the sustainability of a CWF intervention at improving household drinking water quality and reducing ECD incidences in a resource-limited setting. Our most important results are as follows:

• Broadly defined human behaviors are a primary driver of our outcome metrics.

• CWFs can reduce ECD incidence by 41% over a two year period implying that previous intervention-control trials might suffer from a ~25% bias.

• Deteriorating filter effectiveness appears to have a large impact on ECD incidence. ECD could be reduced by an additional 45% with optimal maintenance.

• CWFs are almost entirely ineffective at reducing ECD incidence after 3 years.

• Compliance should be >80-90% and linear usage declines < 10% per year to achieve optimal results.

• CWF log reduction values should be ≥ 3. Inferior CWFs largely exclude optimal results irrespective of other factors. ECD incidences are less sensitive to compliance rates for low LRVs.

• Cleaning filters at least every four months makes optimal outcomes more likely.

• Having new filters available for purchase can help to exclude poor outcomes especially if priced at less than 100 South African Rand.

• CWFs have the ability to reduce ECD incidence to very low levels if they are used optimally.

Finally we summarize the most salient points for future intervention-control trials:

• Large heterogeneity seen previously are most likely due to variations in compliance, microbial effectiveness, cleaning regimes or breakage.

• Future trials should accurately measure confounding variables such as precise compliance over time, filter cleaning, filter microbial effectiveness and socio-economic variables.

• Given the temporal reductions in effectiveness, future trials should be 2-3 years in duration.

Overall, we suggest that a CWF intervention is an effective tool in the fight against ECD, but sustainability and community engagement should be the top priorities of implementing agencies.

Highlights.

• Developed an agent-based model to study the sustainability of a ceramic water filter intervention.

• Model based on 3 years of field microbial effectiveness and compliance data.

• Filters can reduce diarrhea incidence by ~41% and human factors are a key component.

• A further 45% reduction might be possible with improved maintenance.

• Filters, as realistically used, are almost entirely ineffective after 3 years.