Proof-of-concept evaluation at Cox’s Bazar of the Safe Water Optimization Tool: water quality modelling for safe water supply in humanitarian emergencies

Abstract

Introduction

Waterborne diseases are leading concerns in emergencies. Humanitarian guidelines stipulate universal water chlorination targets, but these fail to reliably protect water as postdistribution chlorine decay can leave water vulnerable to pathogenic recontamination. The Safe Water Optimization Tool (SWOT) models chlorine decay to generate context-specific chlorination targets that ensure water remains protected up to point-of-consumption. The SWOT has not been tested in an active humanitarian response, so we conducted a proof-of-concept evaluation at a Cox’s Bazar refugee settlement to validate its modelling and assess its efficacy and effectiveness.

Methods

We trained the SWOT using data collected from July to September 2019 and evaluated using data from October to December 2019 (n=2221). We validated the SWOT’s modelling by comparing performance using training and testing data sets. We assessed efficacy using binary logistic regression comparing household free residual chlorine (FRC) when the SWOT target was delivered at tapstands versus the status quo target, and effectiveness using interrupted time series analysis of the proportion of households with protective FRC before and after SWOT implementation.

Results

The SWOT generated a context-specific FRC target of 0.85–1.05 mg/L for 15-hours protection. Validation of the SWOT’s process-based model showed R² decreased from 0.50 to 0.23 between training and testing data sets, indicating periodic retraining is required. The SWOT’s machine-learning model predicted a 1%–9% probability of household FRC<0.2 mg/L at 15 hours, close to the observed 12% and in line with the observed 7% risk during baseline and endline, respectively. Households that collected water meeting the SWOT target were more likely to have sufficient protection after 15 hours compared with the status quo target (90% vs 35%, p<0.01), demonstrating the SWOT’s efficacy. The SWOT target was not fully implemented at tapstands, so we did not observe change in household FRC during endline.

Conclusion

The SWOT can generate context-specific chlorination targets that protect water against pathogenic recontamination. Improving feedback between monitoring and treatment would help system operators unlock the SWOT’s full water safety potential.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Status quo universal water chlorination guidelines (ie, Sphere) fail to reliably protect drinking water from pathogenic recontamination in emergency settings.

• The Safe Water Optimization Tool (SWOT, www.safeh2o.app) models postdistribution chlorine decay to generate context-specific, data-driven water chlorination targets that protect water up to the household point-of-consumption—where it actually matters for public health.

• The SWOT has been demonstrated on existing water quality data sets from refugee settlements globally, but it is unknown if it can work during an active humanitarian response.

WHAT THIS STUDY ADDS

• We demonstrate that the SWOT can generate a context-specific water chlorination target using routinely collected water quality data from an active humanitarian response at the Kutupalong-Balukhali refugee settlement in Cox’s Bazar, Bangladesh.

• We show that the SWOT is highly efficacious: households collecting water at tapstands meeting the SWOT chlorination target were nearly three times more likely to have sufficient chlorine protection compared with the status quo target.

• Challenges with implementing the SWOT target consistently at tapstands can limit improvements in household water safety outcomes.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• The SWOT can help improve water safety in humanitarian emergencies, however, additional support must be provided to water system operators to improve feedback between monitoring and treatment operations to unlock its full water safety potential.

Introduction

Waterborne diseases such as hepatitis E, cholera and other diarrhoeal illnesses are among the leading causes of preventable mortality and morbidity in emergencies.¹,³ Ensuring water is free from waterborne pathogens is essential for protecting public health. Chlorination is widely used for water treatment in emergencies as it is inexpensive, simple and because it provides residual protection against pathogenic recontamination.⁴,⁸ Ensuring that treated water has at least 0.2 mg/L of free residual chlorine (FRC) is generally sufficient to keep water protected against pathogenic recontamination.^{4 5} The Sphere Handbook, which lays out minimum standards for humanitarian response, stipulates a universal FRC target of 0.2–0.5 mg/L in chlorinated water supplies at water distribution points (ie, tapstands),⁶ a target which has been adopted in many humanitarian sector guidelines.^{7 8}

This universal FRC target, however, derives from the WHO Guidelines for Drinking-Water Quality (GDWQ), which are based on conventions for piped water systems in cities.⁹ The US CDC notes that this universal FRC target is appropriate only when users drink water directly from taps of a piped system.¹⁰ It is unlikely to provide sufficient residual protection if the point-of-consumption is spatially and temporally distant from the point-of-distribution, which is commonly the case in refugee and internally displaced persons (IDP) settlements. As a consequence, this universal FRC target fails to reliably ensure that water remains protected against pathogenic recontamination up to the household point-of-consumption in these settings.^{11 12} Multiple studies in refugee and IDP settlements show that treated water is often recontaminated after distribution, contributing to the spread of waterborne diseases.¹³,¹⁶ Poor environmental hygiene conditions—arising due to limited water availability, inadequate sanitation and other factors—provide ample opportunities for pathogenic recontamination of water to occur once residual chlorine decays and the protection it offers dissipates during the postdistribution period, through collection, transport to households and multiple hours of household storage and use.^{11 12 17}

To protect public health in refugee and IDP settlements, humanitarian responders need water chlorination targets that ensure water is protected against pathogenic recontamination up to the household point-of-consumption. Chlorine decay is a context-specific process,¹²¹⁷,¹⁹ so determining the required FRC at the point-of-distribution that ensures protective residual of ≥0.2 mg/L FRC persists until the last cup is consumed in the household must be determined on a context-specific basis. The Safe Water Optimization Tool (SWOT, www.safeh2o.app) is a novel water quality modelling platform that meets this need. The SWOT harnesses routine monitoring data to model postdistribution chlorine decay and generate data-driven, context-specific water chlorination targets that optimise the probability of residual chlorine protection lasting the entire duration of household storage and use.

The SWOT implements novel process-based and machine-learning models developed using water quality data sets from refugee settlements in South Sudan, Jordan and Rwanda.^{12 17 20} These models are trained with local water quality monitoring data and outputs are integrated to generate context-specific FRC targets. The SWOT’s process-based model determines chlorine decay behaviour using an empirical power-decay model and establishes the initial tapstand FRC needed to achieve the desired residual at the household point-of-consumption.¹² The SWOT’s machine-learning model applies a probabilistic artificial neural network ensemble forecasting system (ANN-EFS) to forecast the probability of having the desired residual at the household point-of-consumption under various scenarios,^{17 20} which is then used to refine the context-specific FRC target.

The SWOT has the potential to help humanitarian responders ensure water safety and protect public health in refugee and IDP settlements during humanitarian emergencies; however, it has not yet been tested in an active humanitarian response situation. To address this gap, we carried out a proof-of-concept evaluation of the SWOT during an active humanitarian response at the Kutupalong-Balukhali refugee settlement in Cox’s Bazar, Bangladesh. The evaluation was structured around three objectives:

1. Model validation: we conducted model validation as part of training the SWOT’s process-based and machine-learning models to confirm whether the SWOT can generate a context-specific chlorination target when presented with new water quality data from an active humanitarian response.

2. Efficacy evaluation: we assessed whether the SWOT chlorination target, when implemented at water distribution points, would increase the proportion of households with protective chlorine residual in stored water after the typical duration of household storage and use at the site, compared with the status quo universal FRC target.

3. Effectiveness evaluation: we assessed whether the proportion of households with protective chlorine residual in stored water after the typical duration of household storage and use improved after SWOT implementation at Kutupalong-Balukhali, and the factors that influenced implementation and outcomes.

The findings reported in this paper can help improve best practices for safe water supply in refugee and IDP settlements and help protect public health during humanitarian emergencies.

Methods

Study design

We evaluated how the SWOT affected water quality at Camp 1 of the Kutupalong-Balukhali refugee settlement over a 6 month period (July–December) in 2019. We trained the SWOT’s models using water quality data collected at the settlement over the first 3 months (July–September) to generate a context-specific chlorination target. This target was provided to the local implementing partner, Médecins Sans Frontières (MSF), who sought to achieve this FRC target at tapstands over the next 3 months, while we continued to collect data to evaluate the efficacy of the SWOT chlorination target and the effectiveness of the overall intervention. For both efficacy and effectiveness evaluations, the primary outcome of interest was having a protective residual of ≥0.2 mg/L FRC in stored water after the typical duration of household storage at the site. This 0.2 mg/L FRC threshold has been shown to perform reasonably well as a surrogate for the absence of faecal indicators and enteric pathogens in a variety of contexts.^{4 5 13 21 22} We evaluated efficacy based on the proportion of households having ≥0.2 mg/L FRC in stored water when the SWOT chlorination target was delivered at water distribution points versus when the status quo universal FRC target (ie, 0.2–0.5 mg/L) was delivered at tapstands. We evaluated the effectiveness of the SWOT intervention using an interrupted time series (ITS) analysis to assess how the trend in household FRC ≥0.2 mg/L changed after the SWOT chlorination target was provided to the implementing partner.

Site background

At the time of the study, Kutupalong-Balukhali hosted over 600 000 Rohingya refugees who had fled state violence in Myanmar. The rapid influx of people, combined with high population densities and challenging environmental conditions, produced an acute water, sanitation and hygiene (WASH) crisis with significant burden of waterborne disease in the community.²³ Poor sanitation contributed to widespread faecal contamination of water supplies both at sources (including deep boreholes) and in households. In response, multiple water chlorination activities were initiated throughout the refugee settlement to ensure drinking water safety.²⁴ The water system at Camp 1 was built by MSF and served 83 000 people at the time of the study (online supplemental figure S1). This system comprised 10 independent subnetworks, supplied by 14 deep boreholes equipped with hybrid diesel-solar submersible pumps. Water was abstracted from a deep aquifer and chlorinated using high-test calcium hypochlorite via inline chlorinators (no other treatment). Water was distributed via pipelines from elevated reservoirs to 190 tapstands around Camp 1.

Data collection

Data collection was structured to serve two purposes. A smaller set of water quality parameters was collected at points-of-distribution (tapstands) and points-of-consumption (households) to train and validate the SWOT’s process-based and machine-learning models, and a larger set of potential covariates was collected at tapstands and households for the efficacy and effectiveness evaluations. These two sets of data were collected through a combined survey instrument that included questions relating to water quality, water handling behaviours and other factors. We systematically sampled all tapstands across Camp 1 (n=190); a few tapstands were not included due to inaccessibility or non-functionality during the study period (n=13).

Data for model training and validation

To train and validate the SWOT’s process-based and machine-learning models, paired water quality samples were collected at tapstands and households. At tapstands, we measured FRC, electrical conductivity (EC) and water temperature. We randomly approached water-users after they filled their water containers at the tapstand to request their participation in the study. If they consented, we accompanied the enrolled participant back to their household where we marked the container of collected water. We returned to the household several hours later to measure FRC from the marked container again. Each sample therefore consisted of paired FRC measurements from the tapstand and the household; EC and water temperature measurements at the tapstand and elapsed time between tapstand and household measurements. To capture the range of typical water storage durations at the site, we alternated between starting paired samples at tapstands in the morning and following up at households the same afternoon (approximately 6–8 hours elapsed time) and starting tapstand samples in the afternoon and following-up at the household the next morning (approximately 16–18 hours elapsed time). We measured FRC using PTH 7091 compact chlorometers (Palintest, Tyne & Wear, UK) and measured EC and water temperature using HI 98129 multimetres (Hanna Instruments, Woonsocket, RI, USA). Equipment was calibrated using manufacturer standards after every 1–2 days of use.

Data for efficacy and effectiveness evaluations

The primary outcome of interest for evaluating the SWOT’s efficacy and effectiveness was household FRC≥0.2 mg/L; household FRC data were included as part of the model training and validation data collection described above. However, factors other than the SWOT can also influence household FRC. To control for these factors in the efficacy and effectiveness evaluations, we mapped them out using a directed acyclic graph (DAG) and, from this, collected data on as many as were feasible in the humanitarian setting in which we were working. The DAG in figure 1 illustrates the pathway by which the SWOT intervention improves household water safety: the SWOT generates a tapstand FRC target, this target is implemented at tapstands and after postdistribution chlorine decay, households should still have sufficient FRC to prevent pathogenic recontamination. The DAG also maps out potential confounders and covariates that also influence household FRC. These can be organised into two broad groups. The first group includes factors that affect postdistribution chlorine decay, which mediates the likelihood of having FRC≥0.2 mg/L at the household point-of-consumption. Postdistribution chlorine decay is a complex phenomenon that is influenced by numerous known and unknown factors, the former of which may include water handling practices, recontamination events, air temperature and tapstand water quality. The second group includes factors that affect tapstand FRC and may lead to the SWOT target not being consistently achieved across tapstands in the community, such as location (ie, network, borehole and tapstand) and time of collection (ie, AM or PM collection from tapstands) (further explanation of these factors is provided in online supplemental appendix S1). Based on the DAG, we collected data on water handling behaviours (ie, container type, container covering and cleanliness during collection and storage, drawing method), additional tapstand water quality parameters including turbidity using PTH 092 turbidimeters (Palintest, Tyne & Wear, UK) and pH using HI 98129 multimetres, air temperature, water network ID, tapstand ID and time of collection. We were not able to gather data on source water quality, environmental hygiene or recontamination events, as these were not feasible to consistently monitor in this setting, and publicly available data on WASH service levels²⁵ did not include sufficient detail to identify changes over the 6 month period of the evaluation. Finally, while some relationships in the DAG were well defined, others were hypothetical. In these cases (shown in figure 1 with dashed lines), we used statistical tests to determine if a relationship existed in the data we collected, using the Kruskal-Wallis test for continuous outcome variables and the χ² test for binary or categorical outcome variables. Statistical tests and models were conducted in Python V.3.7,²⁶ using the SciPy package²⁷ and Statsmodel package²⁸ and considered differences significant at p<0.05.

Figure 1. Directed acyclic graph identifying factors that could potentially influence household FRC in a typical refugee settlement water system. EC, electrical conductivity; FRC, free residual chlorine; SWOT, Safe Water Optimization Tool.

Patient and public involvement

Patients and the public were not involved in the design, conduct, reporting or dissemination plans for this research.

Data analysis

Model validation

The SWOT is one of the first operations-level deployments of a machine-learning technology in the humanitarian WASH sector. Model validation—the process of testing on independent, unseen data—is a critical step in the ethical development of machine-learning tools for humanitarian response, as a major ethical concern is the potential for making faulty decisions based on an erroneous sense of accuracy.²⁹ Therefore, we conducted model validation as part of training the SWOT’s models to confirm whether the SWOT can generate a reliable and accurate context-specific chlorination target during an active humanitarian response.

To validate models, we trained the SWOT using the first 3 months of data collected at Kutupalong-Balukhali and then tested it using the next 3 months as an independent validation set. We validated the process-based model by assessing the goodness-of-fit with the training and testing data using the coefficient of determination (R²). For the machine-learning model, we compared the predicted probability of having household FRC≥0.2 mg/L with the SWOT chlorination target delivered at the tapstands to the observed probability in households that collected water at tapstands meeting the SWOT target.

Efficacy evaluation

The efficacy evaluation sought to determine whether the context-specific SWOT chlorination target, when implemented at tapstands, would increase the proportion of households with FRC≥0.2 mg/L after the typical duration of household storage and use at the site, compared with the status quo universal FRC target. To do this, we used a binary logistic regression model with a binary outcome variable set to 1 if a household had FRC≥0.2 mg/L at follow-up, or 0 if not. The treatment variable was a binary dummy variable based on whether the tapstand FRC was in line with the context-specific SWOT target or the status quo universal FRC target. To control for external factors that may also influence household FRC, we included exogenous variables from the DAG (figure 1) in the model. To ensure model parsimony, we first assessed the validity of proposed relationships in the DAG and eliminated variables for which a significant relationship was not observed. We then assessed which of the validated covariates differed significantly between comparator groups in the efficacy evaluation (ie, households receiving the context-specific SWOT target or the status quo FRC target at tapstands). To assess significance in both steps, we used the Kruskal-Wallis test for continuous variables and the χ² test for binary variables, with differences considered significant at p<0.05. The determination of which exogenous variables to include in the efficacy evaluation is detailed in online supplemental appendix S2.

Effectiveness evaluation

The effectiveness evaluation sought to assess whether the proportion of households with protective chlorine residual in stored water after the typical duration of household storage and use at the site improved after SWOT implementation. To do this, we conducted an ITS analysis in which we used a multiple linear regression model with the proportion of households with ≥0.2 mg/L FRC at follow-up as the outcome variable. We conducted a slope change analysis³⁰ by including a dummy variable of days since the start of the evaluation period (October 1) to track how household FRC changed over time following the SWOT intervention. We took this slope change approach because we assumed that improvements in household water safety would be gradual after the implementing partner received the SWOT target, as operators iteratively adjusted chlorination to achieve the SWOT FRC target at tapstands. We controlled for potential increases in household FRC over time as operators improved their ability to manage chlorination in the piped system (separate from the SWOT intervention) using a second dummy variable for days since the start of the study (July 1). This ensured that any improvements attributed to the SWOT were due to operators achieving the SWOT target, rather than overall improvements in operational control. Additionally, we controlled for external factors that could influence household FRC identified in the DAG in figure 1. As with the efficacy evaluation, we only included variables which had validated relationships and which were significantly different between the two comparator groups (ie, pre-SWOT and post-SWOT intervention). The determination of which exogenous variables to include in the effectiveness evaluation is summarised in online supplemental appendix S2.

Results

Water quality summary

We collected 2221 paired water quality and water handling observations at the study site.³¹ After data cleaning to remove erroneous measurements and/or inadmissible data prior to analysis (see online supplemental appendix S3 for details), 2094 observations remained. Many of the remaining paired samples were missing one or more non-FRC water quality measurements, including turbidity (n=90), water temperature (n=611), EC (n=1196) and/or pH (n=1382). This restricted the data available for the ANN-EFS and the ITS model (cf, effectiveness evaluation section below), which both included pH, EC and water temperature as covariates. Figure 2 shows the daily distribution of FRC concentrations, as well as the whole-data set distributions of elapsed time and other water quality parameters. Further chemical water quality data from the Camp 1 boreholes are discussed in online supplemental appendix S4.

Figure 2. Distributions of point-of-distribution (tapstand) and point-of-consumption (household) FRC concentrations (top): blue lines show the median, boxes show the IQR, and whiskers show the 90th percentile range. Histograms of household storage duration and other water quality parameters summarising the whole data set (bottom). Due to incorrect measurements and outliers, observations with water temperature above 48°C and turbidity above 5 NTU were removed from the figures. FRC, free residual chlorine; NTU, nephelometric turbidity units.

Figure 2 shows that household FRC tends to be lower than tapstand FRC, which we confirmed using a Kruskal-Wallis test of the monthly median tapstand and household FRC concentrations (online supplemental table S1). On a monthly basis, median tapstand FRC was consistently higher than median household FRC by nearly a factor of two, and this difference was statistically significant (p<0.05). The significant FRC loss observed between tapstands and households underscores the need for FRC targets that account for postdistribution chlorine decay. Online supplemental figure S2 provides additional detail on the change in FRC from tapstand to household, and the FRC loss rate (ie, change in FRC normalised by elapsed time) categorised by time of collection (AM/PM) and month of data collection.

SWOT chlorination target generation

Using data from the first 3 months (n=899), we trained the SWOT and produced a tapstand FRC target of 0.95 mg/L for 15-hours protection, representing the longest typical duration of household storage and use identified in a knowledge, attitudes and practices (KAP) survey conducted in parallel to this study (summarised in online supplemental appendix S5). Since achieving a discrete point target is challenging for operators, we provided a ±0.1 mg/L range around the target to provide water system operators with a practicable range of 0.85–1.05 mg/L. The ANN-EFS risk assessment of this target range under different times of collection and water quality scenarios indicated that the risk of household FRC being <0.2 mg/L ranged from 0.01 (1%) to 0.09 (9%) (online supplemental figure S3 and table S2). This level of risk was deemed acceptable, and we provided the SWOT tapstand FRC target recommendation of 0.85–1.05 mg/L to the water system operators at Camp 1 at the start of October 2019 to implement for the second half of the study (October–December 2019).

Model validation

The process-based model was trained using baseline data (July–September 2019), which identified optimal (best fit) parameters for the power-decay model of decay rate, k=0.052, and rate order, n=0.65, and maximum decay (worst case) parameters of k=0.059 and n=0.56. The latter model achieved an R² of 0.50 (moderately good per Moriasi et al³²) during training and 0.23 for the testing period (below satisfactory per Moriasi et al³²). While the R² remained positive during both periods, the performance of the process-based model degraded during the testing period compared with the training period.

While decay model parameters may have been overfit to the training data, the effect it had on the FRC target recommendation appeared to be limited. The machine-learning model predicted that delivering the SWOT tapstand FRC target of 0.85–1.05 mg/L would yield a risk of household FRC being <0.2 mg/L after 15 hours of between 1% and 9%. During the training period, the observed household risk of FRC being <0.2 mg/L after 15 hours was 12% (15/118) and during the evaluation period, 7% (13/182). This indicates that the probabilistic machine-learning model fairly accurately predicted the risk of household FRC being <0.2 mg/L during both the training period and the testing period, demonstrating the validity of the trained model on unseen data.

Efficacy evaluation

We evaluated the efficacy of the SWOT chlorination target by comparing the proportion of households with FRC≥0.2 mg/L at follow-up among those that received the SWOT target (0.85–1.05 mg/L) at the tapstand versus those that received the status quo universal target (0.2–0.5 mg/L). Across the full data set, 502 received the status quo universal target at the tapstand and 300 received the SWOT target, with the remaining samples falling outside of either range. We compared these two groups using a binary logistic regression model with a dummy variable for the tapstand FRC target received. We also included exogenous variables identified by the DAG that were validated and shown to be significantly different between the SWOT and status quo universal target comparator groups. Table 1 summarises the regression coefficients and p values from this binary logistic regression model. Households that collected water with tapstand FRC in line with the SWOT target had ≥0.2 mg/L FRC at the point-of-consumption much more frequently (91%, 272/300) than households that collected water with tapstand FRC in line with the status quo universal FRC target (35%, 177/502) (p<0.01), even when accounting for significant variations in household FRC linked to validated covariates of water networks and water drawing method. This indicates that the SWOT FRC target had greater efficacy for ensuring household FRC than the status quo universal FRC target.

Table 1. Regression coefficients and p values for the binary logistic regression model for the efficacy evaluation.

Variable	Coefficient	P value
Tapstand FRC in line with SWOT target (vs status quo universal FRC target)	2.0	3.5×10–21
Water drawing by scooping (vs pouring)	−0.62	0.00013
Network 2	−1.0	0.00014
Network 3	0.27	0.47
Network 4	−0.74	0.018
Network 8	−0.65	0.014
Network 9	−1.8	2.7×10–13
Network 10	0.48	0.057

Bolded variables were significant at a p value of 0.05.

FRC, free residual chlorine; SWOT, Safe Water Optimization Tool.

Effectiveness evaluation

We used an ITS analysis to assess if the SWOT intervention had an effect on household FRC between baseline and endline periods, distinct from the overall trend on how household FRC was changing over time, while controlling for validated covariates that varied significantly between preintervention and postintervention periods. The regression coefficients and p values from the ITS model are presented in table 2. There was no significant effect observed either from the start of the study (July) or from the start of the SWOT intervention (October), indicating that household FRC did not increase over time either because of water system operators improving control of the system, or as a result of the SWOT intervention. In fact, no variable was significantly associated with variations in household FRC in the ITS model.

Table 2. Regression coefficients and p values for the ITS regression model in the effectiveness evaluation.

Variable	Coefficient	P value
Time from start of study (July)	0.017	0.31
Time from start of SWOT intervention (October)	−0.0082	0.84
Tapstand water temperature	0.073	0.90
Tapstand EC	−0.0069	0.48
Tapstand air temperature	0.19	0.56
Tapstand pH	0.58	0.51
Collection container covered (vs uncovered)	0.71	0.82
Collection container clean (vs unclean)	−5.0	0.41
Household container type	0.93	0.87
Same collection and household container	0.32	0.93
Water drawing by scooping (vs pouring)	4.0	0.07
Network 3	0.97	0.69
Network 8	1.7	0.39
Network 9	1.7	0.46
Network 10	0.073	0.90
Intercept	−7.3	0.80

Bolded variables were significant at a p value of 0.05.

EC, electrical conductivity; ITS, interrupted time series; SWOT, Safe Water Optimization Tool.

Based on these findings, we conducted a second ITS analysis, this time using tapstand FRC as the outcome variable, and only including exogenous variables related to tapstand FRC (a simplified DAG for this revised analysis is presented in online supplemental appendix S6), to evaluate if the SWOT intervention had an effect on tapstand FRC, if not household FRC (table 3). From table 3, we observe that there were significant differences in tapstand FRC between some networks; however, we see no significant changes in tapstand FRC either from the start of the study (July) or from the start of the SWOT intervention (October). This explains the lack of effect on household FRC as the SWOT intervention did not have the desired effect of modifying tapstand FRC. This indicates that potential improvements to household FRC achievable through the SWOT, if the targets were implemented at the tapstand (as demonstrated in the efficacy evaluation), were not achieved at Camp 1 because chlorination system operations did not change enough to produce a significant change in tapstand FRC.

Table 3. Regression coefficients and p values for an ITS regression model with tapstand FRC as the outcome variable.

Variable	Coefficient	P value
Time from start of study (July)	−0.00041	0.73
Time from start of SWOT intervention (October)	0.0020	0.27
Network 3	0.20	0.14
Network 8	−0.35	0.045
Network 9	0.024	0.84
Network 10	−0.35	0.025

Bolded variables were significant at a p value of 0.05.

FRC, free residual chlorine; ITS, interrupted time series; SWOT, Safe Water Optimization Tool.

Discussion

The magnitude and variation in chlorine decay between point-of-distribution and point-of-consumption shown in figure 2 demonstrates the need for the SWOT’s context-specific, data-driven chlorination targets. The SWOT’s process-based and machine-learning models implicitly capture the combined effect of all factors driving chlorine decay between distribution and consumption through relationships in paired data. Model validation of the SWOT’s process-based model showed that it was moderately good at modelling postdistribution decay when presented with new data from an active humanitarian response site (R²=0.50 with training data). Performance of the trained model, however, declined during the postimplementation period (R²=0.23 with testing data), indicating that the external validity of a trained model diminishes over time as site conditions change, suggesting that periodic retraining at least every 3 months is needed to ensure that the process-based modelling is reflective of latest conditions. Despite the decline in R² of the process-based decay model, the FRC target it generated continued to ensure sufficient household FRC through the endline period, as described in the efficacy evaluation section above. ANN-EFS performance remained strong both during baseline and endline periods, with its predictions within 3% of the observed proportion of households with <0.2 mg/L FRC at follow-up during the baseline period, and precisely within range during the endline period. The varying performance of the process-based model, which takes a deterministic approach, compared with the ANN-EFS, which takes a probabilistic approach, may reflect the better suitability of a probabilistic approach for modelling high-variability data characteristic of real-world humanitarian settings. Overall, the modelling performance demonstrated at Kutupalong-Balukhali shows that the SWOT can generate context-specific, data-driven water chlorination targets using local water quality monitoring data during an active humanitarian response.

The efficacy evaluation showed that when the SWOT chlorination target is delivered at tapstands, households that collected that water had ≥0.2 mg/L FRC at the point-of-consumption >90% of the time, compared with <35% with the status quo universal FRC target, demonstrating the efficacy of the SWOT chlorination target at this site. While there are limitations with using FRC as a proxy for safe water, the main epidemic diseases in refugee and IDP settlements are caused by bacterial or viral pathogens such as Vibrio cholera and the hepatitis E virus¹,³¹¹ that are effectively disinfected by 0.2 mg/L FRC.^{21 22} Thus, when the SWOT FRC target was delivered at tapstands, it provided >90% of sampled households with protection against high-priority waterborne diseases.

While the SWOT chlorination target was efficacious, it was not fully implemented at tapstands by water system operators, so no change in either tapstand or household FRC could be assessed during endline in the effectiveness evaluation, a limitation of the present study. A randomised controlled trial design was not feasible for this study as it was not possible to subdivide the study site into intervention and control groups due to operational reasons, or to blind the delivery of the intervention. We controlled for other factors that could influence household FRC in the efficacy and effectiveness evaluations by mapping out these factors using DAGs and collecting data on as many as were feasible in the study setting, but this was still partial, another limitation of the present study.

The implementation of this study in an active humanitarian response, while presenting certain limitations described above, did however provide an opportunity to critically evaluate barriers to effective implementation of SWOT targets. Feedback from water system operators at the site indicated that lack of operational feedback during routine water quality monitoring in the community to flag that tapstand FRC was not meeting the SWOT-recommended target contributed to this outcome. Future deployments of the SWOT should therefore include support to water system operators to implement feedback mechanisms that link tapstand water quality monitoring to water treatment operations.

A perennial concern with chlorinated water supplies in humanitarian settings is taste and odour-driven rejection,¹³ and such complaints have previously been documented from Rohingya refugees at Kutupalong-Balukhali.³³ The KAP survey (online supplemental appendix S5) showed that chlorine taste and odour were not major barriers to acceptance of the chlorinated water supply at Camp 1 during this study, but the population-specific acceptability threshold is not known, a limitation of the present study. Effective chlorination practice ultimately lies between two boundaries: having sufficient chlorine residual to protect water against pathogenic recontamination, but not so much as to cause taste and odour-driven rejection. The adoption of new tools to rapidly assess population-specific chlorine taste and odour acceptability thresholds to provide an upper limit for chlorination practice will be essential for ensuring that water supplies are both protective of public health and acceptable to users.³⁴

Monitoring FRC at tapstands is standard practice in the humanitarian sector, and there is growing emphasis on household monitoring as well.⁶ Tapstand and household sampling is typically conducted in an unpaired manner with grab samples. With a minor adjustment of pairing tapstand and household measurements, the SWOT can leverage this data to improve household water safety. While a large volume of data was collected for evaluation purposes in this study, the SWOT typically requires between 100 and 150 paired samples¹² to generate an initial FRC target. This can usually be collected in a few weeks of intensive data collection as part of a household survey, or as part of routine water quality monitoring at a slower pace. Once an initial recommendation is generated, the SWOT’s models can be updated as new data come in to reflect latest field conditions. Given the observed decline in process-based modelling performance, retraining should be done periodically—at least every 3 months based on the findings of this study—to capture seasonal variations in chlorine decay conditions, but this retraining interval needs to be further evaluated.

Implications for policy and practice

The findings of this study, along with previous studies in South Sudan, Jordan and Rwanda,^{11 12} demonstrate that the universal water chlorination target presently in widespread use in emergencies fails to reliably ensure water safety at the household point-of-consumption, undermining the core public health objective of emergency safe water interventions. Context-specific chlorination targets that are appropriate to local conditions are needed to ensure that treated water stays safe up to the point-of-consumption. This study demonstrates that the SWOT can generate such targets using routinely collected water quality monitoring data in an active humanitarian response, and that these targets, when implemented at tapstands, vastly outperform the status quo universal target with respect to household water safety. Given its capacity to resolve the ‘last mile’ challenge in chlorinated water supplies, the SWOT should be considered for integration into humanitarian WASH guidelines.

Postdistribution recontamination poses public health risks not only in refugee and IDP settings, but also in communities reliant on intermittent or non-piped water systems in low-income and middle-income countries (LMICs).³⁵ Similar water safety challenges affect underserved populations in high-income countries, including in remote and Indigenous communities dependent on small water systems.³⁶ The SWOT may also offer a ‘last mile’ water safety solution for these contexts as well, supporting the realisation of the public health goals underlying the WHO GDWQ.⁹

Recent estimates by Greenwood et al, using Earth observation data, geospatial modelling and household surveys, suggest that over 4.4 billion people in LMICs lack access to safe drinking water—more than twice previous estimates for Sustainable Development Goal 6—primarily due to widespread faecal contamination of water supplies.³⁷ Scaling chlorinated water systems coupled with the SWOT to ensure water safety at the point-of-consumption offers a viable path to closing this gap. This need is especially urgent in rapidly growing refugee and IDP settlements, where displaced populations face heightened vulnerability to waterborne diseases such as cholera, which has surged globally since 2021.³⁸ The number of forcibly displaced people rose by over 50% from 79.5 million in 2019 to 122.6 million in 2024—the highest ever recorded.^{39 40} The SWOT can play an enabling role in helping safe water programmes achieve their full public health potential in these high-risk settings.

CONCLUSION

This proof-of-concept evaluation of the SWOT at the Kutapalong-Balukhali refugee settlement demonstrated that the SWOT can generate context-specific, data-driven water chlorination targets when presented with new water quality data collected in an active humanitarian response. The context-specific chlorination target generated by the SWOT for Kutapalong-Balukhali Camp 1 was associated with improved household FRC outcomes when it was delivered at water distribution points compared with the status quo universal FRC target (ie, Sphere). However, partial implementation of the SWOT target at tapstands by water system operators led to the overall intervention having no effect on tapstand or household FRC. Feedback mechanisms between water quality monitoring and water treatment operations would help water system operators fully implement SWOT chlorination targets at tapstands. Overall, the study demonstrates that the SWOT can help ensure water remains protected against pathogenic contamination up to the household point-of-consumption in refugee and IDP settlements with piped networks delivering chlorinated groundwater, but additional supports are needed to help water system operators achieve chlorination targets at tapstands.

Data availability statement

Data are available in a public, open access repository.