Skip to main content
PMC home page
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2018 Dec 11.
Published in final edited form as: J Water Sanit Hyg Dev. 2012 Jun;22(2):103–111. doi: 10.2166/washdev.2012.079

A conceptual framework to evaluate the outcomes and impacts of water safety plans

Richard J Gelting 1, Kristin Delea 1, Elizabeth Medlin 1
PMCID: PMC6289063  NIHMSID: NIHMS994033  PMID: 30546865

Abstract

A Water Safety Plan (WSP) is a preventive, risk management approach to ensure drinking water safety. The World Health Organization (WHO) guidelines place WSPs within a larger ‘framework for safe drinking-water’ that links WSPs to health, creating an implicit expectation that implementation of WSPs will safeguard health in areas with acceptable drinking water quality. However, many intervening factors can come between implementation of an individual WSP and ultimate health outcomes. Evaluating the impacts of a WSP, therefore, requires a much broader analysis than simply looking at health improvements. Until recently, little guidance for the monitoring and evaluation of WSPs existed. Drawing examples from existing WSPs in various regions, this paper outlines a conceptual framework for conducting an overall evaluation of the various outcomes and impacts of a WSP. This framework can provide a common basis for implementers to objectively monitor and evaluate the range of outcomes and impacts from WSPs, as well as a common understanding of the time frames within which those results may occur. As implementers understand the various outcomes and impacts of WSPs beyond health, a strong evidence base for the effectiveness of WSPs will develop, further enabling the scaling up of WSP implementation and provision of better quality water.

Keywords: drinking water, evaluation, impacts, logic model, outcomes, Water Safety Plans

INTRODUCTION

According to the World Health Organization (WHO) Guidelines for Drinking-water Quality, ‘The most effective means of consistently ensuring the safety of a drinking-water supply is through the use of a comprehensive risk assessment and risk management approach that encompasses all steps in water supply from catchment to consumer. In these guidelines, such approaches are called water safety plans (WSPs)’ (WHO 2011). The WSP is an approach based largely upon Hazard Analysis and Critical Control Point (HACCP) principles, a preventive risk management system developed to ensure food safety. The WHO guidelines place WSPs within a larger ‘framework for safe drinking-water’ that includes the public health context and health outcomes and also contains health-based targets and drinking water surveillance (Figure 1). As such, WSPs are specifically linked to health, with an implicit expectation that implementation of WSPs will safeguard health in areas with acceptable drinking water quality. However, the link between a WSP undertaken for an existing piped drinking water system and ultimate health outcomes may not be this direct. Many systems, prior to WSP initiation, already have multiple barriers between sources of contamination and consumers, such as source protection, water treatment with residual disinfection, safe storage, and a distribution system that prevents recontami-nation of treated water.

Figure 1 |.

Figure 1 |

Open in a new tab

WHO Framework for Safe Drinking Water.

Many intervening factors can come between implementation of an individual WSP and ultimate health outcomes. Some examples may include operational factors such as continuity of service, or institutional factors such as better training for employees that results in improved protection of water safety. Evaluating the impacts of a WSP, therefore, requires a much broader analysis than simply looking at health improvements. Just as WSPs are placed into a larger ‘framework for safe drinking-water’ in the WHO drinking water guidelines, the outcomes and impacts of WSPs must also be placed into a larger context beyond simply health. Drawing examples from existing WSPs in various regions, this paper outlines a conceptual framework for conducting an overall evaluation of the various outcomes and impacts of a WSP, and also provides a basis for the development of indicators to measure those outcomes and impacts. This framework also provides a common understanding of the time frames within which various WSP outcomes and impacts may occur, and illustrates the various benefits of implementing WSPs, even when no direct health impacts are immediately apparent at the individual project level. As WSP implementers understand the various outcomes and impacts of WSPs beyond health, a strong evidence base for the effectiveness of WSPs will develop, further enabling the scaling up of WSP implementation.

Until recently, little guidance for the monitoring and evaluation of WSPs existed. Most WSP implementers either did not evaluate their WSPs in a formal manner or developed their own methods of evaluation. Currently, more efforts are being made to standardize the monitoring and evaluation of WSPs globally. The principles laid out in this conceptual framework were developed to help WSP implementers standardize the approach they use for evaluation and consider the range of outcomes and impacts from WSPs. This framework is designed to be one of a set of tools to guide the implementation and evaluation of WSPs, along with the WHO guidelines (WHO 2011), the Water Safety Plan Manual (Bartram et al. 2009), the Water Safety Plan Quality Assurance Tool developed by WHO and the International Water Association (IWA) (WHO/IWA 2010) and other tools and resources developed for national or regional use. The framework should therefore be used as a complement to these other tools rather than as a stand-alone instrument. This paper presents a summary of a more-detailed conceptual framework that can be found at http://www.cdc.gov/nceh/ehs/GWASH/Publications/WSP_Evaluation_Framework.pdf.

METHODS: STRUCTURE OF THE CONCEPTUAL FRAMEWORK TO EVALUATE WSP IMPACTS

This conceptual framework for evaluating WSP outcomes and impacts uses a logic model structure. Logic models are ‘graphic depictions of the relationship between a program’s activities and its intended outcomes’ (U.S. Department of Health and Human Services [USDHHS] 2005). Although logic models can vary in structure, they often contain the following basic elements (W.K. Kellogg Foundation 2004; USDHHS 2005):

  • Inputs: resources available to a program (including human, financial, organizational, and community resources)

  • Activities: what a program does with the above inputs (processes, tools, actions)

  • Outputs: direct products of program activities

  • Outcomes: intermediate changes resulting from a program’s activities and outputs, sometimes divided into short and longer term outcomes (changes in behavior, knowledge, skills, status and level of functioning)

  • Impacts: ultimate change as a result of program activities

Drawing examples from existing WSPs (Table 1), this framework places WSPs into a larger context that includes these elements of a logic model. Figure 2 shows the WSP Conceptual Framework.

Table 1 |.

Water safety plan case studies

Country/reference Description of WSP case study
Australia (Mullenger et al. 2002) This water service provider was the first in Melbourne, Australia to obtain HACCPa accreditation for the supply of drinking water. The HACCP plan was developed and implemented to ensure safe, aesthetically pleasant, and ‘up to code’ water to its customers.
Bangladesh (Mahmud et al. 2007) Small rural water system WSPs in 82 communities were developed through consultation with key water sector practitioners in-country. The communities covered were spread across Bangladesh and covered all technologies, except gravity-fed piped water systems.
Guyana (Gelting 2008, unpublished This water service provider served a population of roughly 40,000 people in Linden, Guyana. The water service provider operated five water treatment plants and provided household connections for approximately 70% of residents. The WSP intended to incorporate good watershed management practices aimed at ensuring the integrity of source waters, while optimizing drinking water supply systems.
Jamaica (Environmental & Engineering Managers Ltd. 2008) The water service provider served a population of about 140,000 people in Spanishtown, Jamaica and surrounding agricultural areas. The WSP intended to enable the service provider to focus on critical areas for ensuring water of adequate quality. The WSP also aimed to build collaboration between ministries of health, environment, and water sectors, empower water operators to improve the water system, and be a model for future WSPs in the region.
South Africa (Rand Water 2007) This water service provider, the largest in South Africa, provided service to 12 million customers through two large purification plants, several booster stations and an extensive bulk distribution network (including 55 reservoirs). In keeping with modern international standards, the water service provider implemented a WSP to further improve water quality and safety for its customers.
Uganda (Howard et al. 2005) This water service provider was responsible for the provision and quality control of domestic piped water in Kampala, Uganda, while the distribution system was managed under contract by a private operator. The system served approximately 700,000 people. The WSP aimed to provide safe and high quality water to its consumers, as well as demonstrating that WSPs could be successfully implemented in lower income countries and offer cost savings.
Open in a new tab
a

HACCP =Hazard Analysis Critical Control Plan (similar to WSP – see text).

Figure 2 |.

Figure 2 |

Open in a new tab

WSP Conceptual Framework.

This conceptual framework also contains a time element, to illustrate how different WSP outcomes and impacts become apparent at different points in time. For example, longer term effects such as improvements in health will not be apparent during the output phase when the WSP is being developed, or even in the outcome phase when the first effects become apparent. The process of undertaking a WSP and any subsequent outcomes and impacts from that process is not a linear one. Figure 2 necessarily represents a simplified schematic of what can happen during this process and variations will obviously occur. In addition, feedback and interactions between the elements will occur and influence the process as well.

RESULTS AND DISCUSSION: THE CONCEPTUAL FRAMEWORK TO EVALUATE WSP IMPACTS

Inputs

Inputs are the human, financial, organizational, and community resources available to a program to implement activities. Since a WSP is designed to be a stakeholder-based process, one of the primary resources is the institutional partners involved in the WSP process. Those partners can include the obvious and expected ones such as the water supplier, regulatory and permitting agencies, and organizations with responsibility for water-shed or recharge areas. However, they should also include other less traditional partners, such as consumers or consumer groups, private industries working in water-shed or recharge areas, and local government entities. Although other stakeholders are often critical of the process and its success, the involvement and commitment of the water supplier is a sine qua non for a successful WSP.

Beyond the institutions involved in WSP implementation, individuals also often play a large role in ensuring the success of a WSP. Such ‘champions’ often help in initiating and organizing the WSP process and also motivate other partners to get or stay involved. WSP champions can come from any of the institutional partners mentioned above and are not always necessarily within the water supplier.

In order to successfully implement a WSP, the stake-holders have to provide specific resources such as time commitment of staff, facilities to hold meetings or workshops, and materials and equipment for those events, all of which represent inputs. In addition, knowledge is an input provided by the stakeholders, as the WSP process typically draws heavily on existing information and experience. Providing these resources demonstrates a level of political will and motivation to support the WSP, which is also considered an input on the part of the stakeholders and management (Summerill et al. 2010a). Although motivation is an input at the very beginning of the WSP process, it remains crucial throughout the process. This is especially true in the transition from outputs to outcomes, where recommendations from the WSP are actually implemented, resulting in positive changes. Without continuing motivation, the WSP can become an exercise in report writing that stops at the output phase when the initial WSP document is finished, never achieving the WSP’s full potential for change.

Activities and outputs

For the purposes of this conceptual framework for evaluating WSP impacts, the WSP itself is considered both the program activity and program output (Figure 2). Thus, the WSP is both what the program does (an activity) and the product of the program activity (an output). Using the inputs described above, the program undertakes the process of a WSP, an activity involving steps such as forming a WSP team, developing a water system description, assessing risks for that system, identifying control measures to manage those risks and implementing and verifying those corrective actions (Bartram et al. 2009). At the same time, the product of the WSP process is the WSP document itself, which represents an output. Although this output provides the foundation for change, it does not yet represent change; additional steps are required to actually improve drinking water safety. These changes represent outcomes and are discussed in the next section.

Outcomes

The outcomes from a WSP are the intermediate changes that result from the WSP process. Whereas products such as the WSP document represent outputs, an outcome occurs when there is a change that results from a WSP. Continued motivation and commitment is needed to carry the WSP outputs discussed above through to outcomes that can actually improve drinking water safety.

WSP outcomes can be quite diverse, and this framework classifies these outcomes into four categories: institutional, operational, financial and policy changes. Examples drawn from WSP case studies are used to illustrate each type of outcome (Table 1). As shown in Figure 2, these different outcomes typically occur at different time scales, with institutional changes often being the first ones to become apparent, followed by operational and financial changes, and, ultimately, policy changes. The case studies cited represent a spectrum of situations, from small community water systems to large urban water systems.

Outcomes: institutional change

Institutional changes are typically the first outcomes resulting from the WSP process. Many of these institutional changes occur within the water supplier, but they occur in other stakeholders involved in the WSP process as well. In the six WSP case studies, these institutional outcomes were observed to fall into three areas: increased communication and collaboration, improved knowledge and attitudes, and increased training.

(a). Increased communication and collaboration.

Increased communication and collaboration among stakeholders may be one of the most important initial outcomes from the WSP process. By catalyzing better communication and collaboration, WSPs may help to produce effective action steps toward improving drinking water safety. For example, in Guyana, the WSP process brought together various stakeholders, including the water supplier and the drinking water regulator. A representative of the water supplier stated that the WSP process had greatly improved relations and communications with the regulator, leading to better coordination of efforts to improve drinking water safety, such as monitoring of water quality in the distribution network (Gelting 2008, personal communication).

(b). Improved knowledge and attitudes.

An increase in knowledge about and understanding of the drinking water system among water supplier staff and other stakeholders often naturally occurs over the course of the WSP process. When staff are encouraged to collaborate and take an active role in the WSP development process, an increase in understanding of the water system is often observed. In South Africa and Bangladesh, due to increased understanding of all parts of the water supply system, operators were observed to have an improved ability to prevent and resolve water quality issues on their own after WSP implementation. This, in turn, encouraged staff to share their knowledge with colleagues in the utility, in order to further increase understanding among less-knowledgeable staff (Mahmud et al. 2007; Rand Water 2007). The WSP process can help create a positive environment of good communication, collaboration, and understanding, where staff feel more competent and recognized for their work. This environment can, in turn, lead to improvements in staff perceptions and attitudes toward their roles and responsibilities (Summerill et al. 2010b).

(c). Increased training.

In addition to the above institutional outcomes, more formal training can be identified as a need during the WSP process. In Australia, a noted increase in understanding and capacity due to staff involvement in the WSP process led management to implement a formal training program, leading to further improvements in knowledge, increased discipline, and increased ownership among staff for their specific roles (Mullenger et al. 2002). In South Africa, the WSP identified a need for increased training and learning opportunities for internal staff, as well as future employees. To alleviate this problem, the water supplier established professorial chairs at various universities, instituted a skills-building training program, and adopted a two-year graduate training program in which newly qualified graduates are mentored by water provider staff (Rand Water 2007).

Outcomes: operational changes

Operational changes are often the most tangible outcomes of the WSP process. In the six WSP case studies, operational outcomes were observed to fall into two areas: improved system infrastructure and implementation of improved procedures. These operational outcomes usually occur simultaneously or slightly later than the occurrence of institutional outcomes.

(a). Improved system infrastructure.

Development of the WSP itself and WSP-related documents can lead to improved system infrastructure through the use of WSP tools, such as system infrastructure assessments, water quality assessments, and monitoring plans. In the case of Bangladesh, WSPs resulted in direct action by ‘caretakers’ (community water operators) to reduce risks to drinking water, including repairing damaged water source infrastructure, moving sources of contamination, and cleaning the surroundings of the water supplies (Mahmud et al. 2007). A WSP in Uganda called for a system assessment which showed that the sanitary integrity of valve boxes was a concern, with missing inspection covers and the presence of stagnant water making valves vulnerable to contamination. As a result of these findings, the utility addressed and fixed all issues with the valve boxes. This improvement in system infrastructure was a direct result of the WSP process (Howard et al. 2005).

(b). Implementation of improved procedures.

Part of the WSP process is the creation of improved procedures for operation and monitoring. WHO’s Water Safety Plan Manual states that ‘clear management procedures documenting actions to be taken when the system is operating under normal conditions (Standard Operating Procedures or SOPs) and when the system is operating in ‘incident’ situations (corrective actions) are an integral part of the WSP’ (Bartram et al. 2009). The Australia case study provides an example of how the HACCP process, a systematic preventive approach to food safety and a precursor to the WSP methodology, aids in identifying areas for development, improvement, and/or change in already documented operating procedures in a water supplier. The HACCP approach was used in Australia for water safety management before the WSP methodology formally existed. Changes to SOPs occurred as a result of the HACCP assessment process, which led to a greater understanding of the implications and potential consequences of actions executed in the field. New SOPs became more effective than previous ones and a sense of ownership among staff developed, ensuring that new procedures were carried out fully (Mullenger et al. 2002).

Outcomes: financial changes

The WSP process can lead to financial changes for water suppliers, in terms of cost savings, cost recovery, and increased investment. These outcomes generally follow the institutional and operational changes discussed above.

(a). Cost savings.

WSPs can lead to cost savings for water suppliers, by identifying and implementing more efficient procedures. For example, in Uganda, an analysis was undertaken to estimate what the costs would be to the water provider of switching to a WSP approach to water quality monitoring compared to the costs of returning to a standard end-product testing approach. The results showed that a 30% reduction in costs of water quality control activities could be achieved, while at the same time maintaining greater assurance of drinking water safety (Howard et al. 2005).

(b). Cost recovery.

The operational changes discussed in the previous section often contribute to improvements in service, both in terms of water quality and other factors such as continuity of service, which can lead to increased customer satisfaction (Rizak et al. 2003). Because consumers are often willing to pay more for better service (Whittington 2002; Whittington et al. 2002; Constance 2004; Casey 2006; Bhandari & Grant 2007, cost recovery may be enhanced through WSPs.

(c). Increased investment.

A WSP can show donor agencies that the water supplier is willing to proactively work with other stakeholders to identify the best ways to improve its water systems. After the initial WSP document was finalized for the WSP pilot project in Jamaica, a representative of the Japan Bank for International Cooperation, which had been involved in both the WSP and other capital improvement projects, commented that ‘the WSP demonstrates that the water utility and the government are well prepared to implement and sustain donor-financed improvements’ and that ‘WSPs provide a new stage for funding assistance’ (Environmental & Engineering Managers Ltd. 2008). As WSPs become more widespread, their use as the foundation for identifying funding needs may increase.

Outcomes: policy changes

Policy level changes related to WSPs are often the last outcomes to become apparent, usually after the other types of changes discussed above have taken place. For the purposes of this paper, policy outcomes are divided into three sub-categories: informal knowledge sharing and promotion of WSPs, WSPs as norms of practice, and formal regulatory requirements for WSPs.

(a). Informal knowledge sharing and promotion of WSPs.

Initial experiences with WSPs within a country or region can lead to informal knowledge sharing and promotion of WSPs. Others interested in WSPs seek out the early adopters for information about how to get started and pros and cons of the process. This occurred in Uganda, where, once an effective WSP was established for Kampala, other water suppliers became interested in the WSP process (Howard et al. 2005).

(b). WSPs as norms of practice.

As WSPs become established and their benefits become apparent, they may become internalized into norms such as ‘best practices,’ which are often integrated into guidance documents that do not carry the mandate of regulations but nonetheless influence how water suppliers and other stakeholders operate. This occurred in the example of Bangladesh, as WSPs became well accepted by non-governmental organizations (NGOs) and other stakeholders as effective guides for consistently ensuring drinking water safety in rural areas. WSP examples and templates, which represented norms or best practices, were developed for different types of rural water supplies in Bangladesh, to facilitate the development of WSPs for these small systems (Mahmud et al. 2007).

(c). Formal regulatory requirements for WSPs.

WSPs may also be incorporated into drinking water regulations, making them mandatory. For example, in Jamaica, after the success of an initial WSP pilot project, the national drinking water regulations were being revised to include a requirement for all water suppliers to undertake WSPs (Environmental & Engineering Managers Ltd. 2008). Enacting this type of policy change can take considerable time. In Jamaica, the final regulations requiring WSPs will likely not be finalized until 8–10 years after the WSP pilot project was initially undertaken in the country.

An evolution within these policy outcomes can take place, as knowledge and promotion of WSPs initially takes place in an informal manner. Later, this type of information about WSPs may become more formalized as norms or best practices in guidance documents. Finally, WSPs may be incorporated into regulatory requirements at a national or other level, making them a formal mandate for water suppliers.

Impacts

The various outcomes discussed above subsequently lead to impacts, which are the ultimate changes desired as a result of program activities. In this conceptual framework, the initial impact of WSP outcomes (i.e., institutional, operational, financial or policy changes) is improvements in water supply (Figure 2). In the context of a WSP, these improvements are often couched primarily in terms of water quality. However, they may also involve improvements related to other WHO quantitative service factors such as quantity, continuity, coverage, and cost (WHO 2011).

It should also be noted that not all of the types of outcomes shown in Figure 2 are necessary to lead to impacts. For example, institutional and operational changes may lead to water supply improvements in some cases whereas increased investment may also be necessary in others to achieve this goal. However, any of these changes may lead to improvements in water supplies before policy changes take place.

Improvements in water supply will subsequently contribute to improvements in health, although those benefits may not be immediately apparent or easy to measure at an individual project level. In one of the few examples in the published literature where health impacts from a WSP were documented, it appeared that both hospital-acquired infections and cases of neonatal sepsis were reduced as a result of a WSP for a German hospital (Dyck et al. 2007). A water system within a hospital is typically a more controlled environment than a community drinking water system, likely making health impacts more apparent and easier to measure, but this result was still only apparent several years after the initiation of the WSP process. Therefore, considerable time may elapse before health impacts become apparent and are measureable.

Despite difficulties in measuring health impacts and the extended time frames for those impacts to become apparent, evidence at the population level makes it clear that efforts to improve drinking water safety will ultimately yield health benefits. In both the U.S. and Japan, widespread implementation of drinking water treatment in the 20th century led to dramatic declines in waterborne diseases such as typhoid fever (Japanese Ministry of Health and Welfare 1988; Cutler & Miller 2005). More recent examples show the converse: as water treatment was neglected in areas such as the former Soviet Union in the 1990s and more recently in Zimbabwe, there was a resurgence of outbreaks of waterborne diseases such as typhoid and cholera (Mermin et al. 1999; Mason 2009). Therefore, the link between improvements in water supplies and improvements in health is clear, and the expectation that those improvements in health will occur over time is consistent with prior experience and evidence.

The conceptual framework shown in Figure 2 recognizes that health improvements are influenced by multiple factors, including sanitation, hygiene, food, nutrition and other environmental exposures, and not solely dependent on drinking water safety (Pruss et al. 2002). The framework also recognizes ‘downstream’ effects of improved health such as socioeconomic benefits (Hutton & Haller 2004). These features are included to help show the larger context within which efforts to improve drinking water supplies operate, but a detailed discussion of them is not included here, as this is not the focus of this paper and they are extensively covered in other material, including the references cited above.

CONCLUSION

Implementing WSPs can lead to many positive changes, from intermediate outcomes (i.e., institutional, operational, financial, and policy changes) to ultimate impacts like improvements in health. It is important to acknowledge all of these changes, and also to recognize that not all of them will occur immediately or simultaneously. As this framework demonstrates, the impacts of WSPs must be placed into a larger context beyond simply health. Simply focusing on water quality and health improvements in the context of a WSP will overlook these important intermediate outcomes that can provide a better picture of the significance and success of the WSP. Health improvements, in particular, become apparent only after the occurrence of many of the other outcomes discussed in this paper. Ultimately, however, health and other improvements will only be sustained if water supply improvements are sustained.

As implementation of WSPs becomes more widespread, more information about the outcomes and impacts from them should become available, leading to broader recognition of the spectrum of positive changes that can result from WSPs. Increased documentation of WSP case studies detailing these results, especially in the peer-reviewed literature where broad dissemination is achieved, will also help in this process. This framework can provide a common basis for WSP implementers in objectively monitoring and evaluating the various outcomes and impacts from WSPs, which will help to establish a strong evidence base for the effectiveness of WSPs. That evidence base will, in turn, help to enable the scaling up of WSPs by providing the information necessary for developing policy environments conducive to widespread WSP implementation.

ACKNOWLEDGEMENTS

The authors wish to thank Tom Williams of the International Water Association, Bruce Gordon of the World Health Organization, Jamie Bartram of the University of North Carolina, and Jennifer de France of NSF International for comments and insights during the development of this paper. In addition, Victoria Cuellar, Brian Hubbard, and Max Zarate-Bermudez of the CDC reviewed initial drafts and made valuable comments while Wendy Worthington of CDC contributed to preliminary research for predecessor documents to this paper. Above all, thanks to the WSP participants cited throughout this paper, as their work in documenting and publishing their WSP experiences was fundamental to the development of this evaluation framework.

Footnotes

This paper is in the public domain: verbatim copying and redistribution of this paper are permitted in all media for any purpose, provided this notice is preserved along with the paper’s original DOI. Anyone using the paper is requested to properly cite and acknowledge the source as Journal of Water, Sanitation and Hygiene for Development 2(2), 103–111.

REFERENCES

  1. Bartram J, Corrales L, Davison A, Deere D, Gordon B, Howard G, Rinehold A & Stevens M 2009. Water Safety Plan Manual: Step-by-Step Risk Management for Drinking-Water Suppliers. World Health Organization, Geneva. [Google Scholar]
  2. Bhandari B & Grant M 2007. User satisfaction and sustainability of drinking water schemes in rural communities of Nepal. Sustain. Sci. Pract. Policy 3 (1), 12–20. [Google Scholar]
  3. Casey J, Kahn J & Rivas A 2006. Willingness to pay for improved water services in Manaus, Amazonas, Brazil. Ecol. Econ 58, 365–372. [Google Scholar]
  4. Constance P 2004. Service worth the price. IDBAmerica [Internet]. 2004 Sept 1; Features and Web Stories. Available from: http://www.iadb.org/features-and-web-stories/2004-09/english/service-worth-the-price-4985.html (accessed 19 September 2011)
  5. Cutler D & Miller G 2005. The role of public health improvements in health advances: the 20th century United States. Demography 42 (1), 1–22. [DOI] [PubMed] [Google Scholar]
  6. Dyck A, Exner M & Kramer A 2007. Experimental based experiences with the introduction of a water safety plan for a multi-located university clinic and its efficacy according to WHO recommendations. BMC Public Health 7 (34). Available from: http://www.biomedcentral.com/bmcpublichealth/about. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Environmental & Engineering Managers Ltd 2008. Proceedings of the Regional Water Safety Plan Workshop, 7–8 November 2007, Kingston, Jamaica, Environmental & Engineering Managers Ltd, Kingston. [Google Scholar]
  8. Howard G, Godfrey S, Tibatemwa S & Niwagaba C 2005. Water Safety Plans for piped urban supplies in developing countries: a case study from Kampala, Uganda. Urban WaterJ 2 (3), 161–170. [Google Scholar]
  9. Hutton G & Haller L 2004. Evaluation of the Costs and Benefits of Water and Sanitation Improvements at the Global Level. World Health Organization, Geneva. [DOI] [PubMed] [Google Scholar]
  10. Japanese Ministry of Health and Welfare. 1988. 50 years’ history of the Ministry of Health and Welfare [unpublished].
  11. Mahmud SG, Shamsuddin Sk. A. J., Ahmed MF, Davison A, Deere D & Howard G 2007. Development and implementation of Water Safety Plans for small water supplies in Bangladesh: benefits and lessons learned. J. Water Health 5 (4), 585–597. [DOI] [PubMed] [Google Scholar]
  12. Mason P 2009. Zimbabwe experiences the worst epidemic of cholera in Africa. J. Infect. Developing Countries 3 (2), 148–151. [DOI] [PubMed] [Google Scholar]
  13. Mermin JH, Villar R, Carpenter J, Roberts L, Samaridden A, Gasanova L, Lomakina S, Bopp C, Hutwagner L, Mead P, Ross B & Mintz ED 1999. A massive epidemic of multidrug-resistant typhoid fever in Tajikistan associated with consumption of municipal water. J. Infect. Dis 179, 1416–1422. [DOI] [PubMed] [Google Scholar]
  14. Mullenger J, Ryan G & Hearn J 2002. A water authority’s experience with HACCP. Water Sci. Technol 2 (5–6), 149–155. [Google Scholar]
  15. Pruss A, Kay D, Fewtrell L & Bartram J 2002. Estimating the burden of disease from water, sanitation, and hygiene at a global level. Environ. Health Perspect 110 (5), 537–542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Rand Water 2007. Water Safety Plan Case Study. Rand Water, South Africa. [Google Scholar]
  17. Rizak S, Cunliffe D, Sinclair M, Vulcano R, Howard J,Hrudey S & Callan P 2003. Drinking water quality management: a holistic approach. Water Sci. Technol 47 (9), 31–36. [PubMed] [Google Scholar]
  18. Summerill C, Pollard SJT, Smith JA, Breach B & Williams T 2010a. Securing Executive Buy-In FOR Preventative Risk Management – Lessons from Water Safety Plans Presented at the IWA World Water Congress (paper 2707), 19–24 September 2010, Montréal, Canada. [Google Scholar]
  19. Summerill C, Smith J, Webster J & Pollard SJT 2010b. An international review of the challenges associated with securing ‘buy-in’ for water safety plans within providers of drinking water supplies. J. Water Health 8 (2), 387–397. [DOI] [PubMed] [Google Scholar]
  20. U.S. Department of Health and Human Services – Centers for Disease Control and Prevention. Office of the Director, Office of Strategy and Innovation 2005. Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide. Centers for Disease Control and Prevention, Atlanta, GA. [Google Scholar]
  21. Whittington D 2002. Municipal Water Pricing and Tariff Design: A Reform Agenda for Cities in Developing Countries, Issue brief 02–29, Resources for the Future, Washington, DC. [Google Scholar]
  22. Whittington D, Pattanayak SK, Yang J & Bal Kumar KC 2002. Household demand for improved water services: evidence from Kathmandu, Nepal. Water Policy 4, 531–556. [Google Scholar]
  23. W.K. Kellogg Foundation 2004. W.K. Kellogg Foundation Logical Model Development Guide W.K. Kellogg Foundation, Michigan. [Google Scholar]
  24. World Health Organization 2011. Guidelines for Drinking-Water Quality, 4th edition World Health Organization, Geneva. [Google Scholar]
  25. World Health Organization/International Water Association 2010. Water Safety Plan Quality Assurance Tool. World Health Organization, Geneva. [Google Scholar]

RESOURCES