Improving quality and use of data through data-use workshops: Zanzibar, United Republic of Tanzania

Abstract

Problem

In Zanzibar, United Republic of Tanzania, as in many developing countries, health managers lack faith in the national Health Management Information System (HMIS). The establishment of parallel data collection systems generates a vicious cycle: national health data are used little because they are of poor quality, and their relative lack of use, in turn, makes their quality remain poor.

Approach

An action research approach was applied to strengthen the use of information and improve data quality in Zanzibar. The underlying premise was that encouraging use in small incremental steps could help to break the vicious cycle and improve the HMIS.

Local setting

To test the hypothesis at the national and district levels a project to strengthen the HMIS was established in Zanzibar. The project included quarterly data-use workshops during which district staff assessed their own routine data and critiqued their colleagues’ data.

Relevant changes

The data-use workshops generated inputs that were used by District Health Information Software developers to improve the tool. The HMIS, which initially covered only primary care outpatients and antenatal care, eventually grew to encompass all major health programmes and district and referral hospitals. The workshops directly contributed to improvements in data coverage, data set quality and rationalization, and local use of target indicators.

Lessons learnt

Data-use workshops with active engagement of data users themselves can improve health information systems overall and enhance staff capacity for information use, presentation and analysis for decision-making.

Introduction

Good health information systems are crucial for addressing health challenges and improving health service delivery in developing countries.¹ However, the quality of the data produced by such systems is often poor and the data are not used effectively for decision-making.² Although there has been increasing international attention to the need to develop strong health information systems, it has proved difficult to do so for several reasons, including fragmentation and lack of coordination of health programmes and insistence by international agencies on maintaining their own vertical systems³; lack of shared data standards⁴; unrealistic ambitions⁵; inability of system developers to handle complex organizational, social and cultural issues⁶; and problems of sustainability.⁷ The Health Metrics Network, established in 2005, has been instrumental in addressing the problem of fragmentation in health information systems through its technical framework,⁸ which promotes a data warehouse approach to information system integration,^3,9 and in creating global consensus on the need for all actors to join forces and work towards integrated systems. Zanzibar, in the United Republic of Tanzania, provides an early example of this shift towards integration and the use of an integrated data warehouse/repository application to achieve it.

A key hypothesis in the experience reported in this paper is that data quality and data use are interrelated: poor quality data will not be used, and because they are not used, the data will remain of poor quality; conversely, greater use of data will help to improve their quality, which will in turn lead to more data use. This hypothesis was tested through data-use workshops, which sought to encourage data use and enhance data quality by promoting systematic peer review, building teamwork and stimulating self-assessment, using indicators to measure targets.

Background and methods

Zanzibar consists of two islands, each making up a health zone; one island comprises six districts and the other four. As part of a health system strengthening project,¹⁰ in 2005 the Health Management Information System (HMIS) Unit of the Ministry of Health, with support from the Danish International Development Agency, launched a process aimed at strengthening the HMIS, improving data reporting and implementing the District Health Information Software (DHIS) (Fig. 1). DHIS is a software tool for collection, validation, analysis and presentation of aggregate statistical data managed in multiple data sets and tailored to support integrated health information management activities. It is designed to serve as a district-based country data warehouse that addresses both local and national needs. Version 2 of the program is an open-source web-based software package available free of charge (www.DHIS2.org).¹¹ It is being used by national health management information systems in numerous countries in Africa and Asia.

Fig. 1.

District Health Information Software screenshot and example of data presentation

Note: In this illustration three sets of indicators are graphed:

Bacille Calmette-Guérin (BCG) and measles immunization. Immunization trends for measles vaccine and BCG for the year are stable, since the Expanded Programme on Immunization has always used local data and data quality is therefore good.

Antenatal care versus BCG coverage. As participants began using data on women’s first visit to antenatal care, antenatal care coverage dropped from an unrealistic 125% to a more realistic 75% level.

Malaria. Since the importance of using rapid diagnostic tests was stressed during data-use workshops, malaria “incidence” dropped dramatically in children under five as clinicians moved from symptomatic diagnosis towards laboratory-confirmed diagnosis.

Quarterly data-use workshops were held in which district health management team (DHMT) members (roughly seven per district) presented their district’s routine data to their peers from other districts. The workshops lasted approximately 5 days each and were facilitated by external facilitators from the Health Information Systems Programme, supported by the Zanzibar HMIS Unit and selected health zone staff. The workshops began in 2005 and have continued since then. Most workshops are now being run by the HMIS Unit without outside help.

During the workshops, each district or programme presents and assesses its own data using standardized analysis templates based on the Millennium Development Goals and local strategic plans, after which their peers discuss and critique these presentations. The workshops encourage self-assessment and provide an opportunity to compare presentations, identify common issues relating to data quality and health services performance, promote local involvement and improve data quality. They also contribute direct feedback to HMIS planners for the revision of indicators and data sets and to software developers for the design of new functionalities, reports and other features.

As participants became more familiar and proficient with data-handling processes (e.g. indicators, analysis, display) and the tools (e.g. DHIS, Excel pivot tables and graphs), the length of their participation in workshops became shorter. At the same time, as more programmes started to use DHIS, more programme managers participated and the overall workshop duration increased.

Achievements

Improvements were noted in the following areas as a result of the data-use workshops:

Data collection

Forms were simplified on the basis of revised indicator and data sets, dramatically reducing the number of data elements collected and thus the workload of facility staff. This simplification was achieved largely because workshop participants realized that it was unnecessary to disaggregate some data (e.g. by age, sex or uncommon diseases) as the disaggregated data were not being used. Similarly, duplication of data collection by different programmes was virtually eliminated (for example, the Reproductive and Child Health Programme stopped collecting data on human immunodeficiency virus [HIV] infection, immunization and malaria) and data gaps were filled. Changes were agreed collectively through improved communication fostered by the workshops and supported by strong leadership from the Ministry of Health.

Data submission improved considerably, with districts reporting more regularly on most data forms. The process began modestly, focusing on a couple of programmes, but other programmes gradually saw the value of using the national HMIS rather than their own parallel data collection systems, and more programmes (and hospitals, including the national referral hospital) were added. Indicator set changes, including some reductions in indicators, were negotiated with programmes jointly each year through the HMIS Unit, the Health Information Systems Programme and the two health zones.

Integration

The workshops provided a stimulus for integration of the previously separate data sets and databases of primary health-care units, hospitals and programmes, and allowed district health management team members to gain a better idea of the roles played by different actors, which improved practical collaboration. Integration of programme data into a single DHIS database – with one national data set covering the Millennium Development Goals indicators, poverty reduction and national strategic plan indicators and programme-specific indicators – was a major achievement. Integration was a slow process, however, as some externally funded programmes were initially reluctant to share “their” data and did not trust the quality or timeliness of the national database.

Data quality

Data quality improved dramatically, thanks to increased use of quality checks (for timeliness, correctness, consistency and completeness of data) at the facility level, use of computer checks by districts and practical experience gained under supervision during workshops. Mistakes were identified by participants when data were presented during workshops, sometimes leading to heated discussion of quality issues, which made a strong impression on participants.

Data analysis and interpretation

At the start of the process, most district health management team staff did not think in terms of indicators, and their presentations focused on raw data. As information officers and programme managers became more competent in using the HMIS, data analysis tools (targets and indicators) became more widely used and understood, which strengthened self-assessment and “epidemiological thinking”. The link between plans, targets and indicators was emphasized, which helped to increase the use of indicators at local levels and the analysis of coverage and quality of service delivery. Fig. 1 shows a screenshot of the DHIS dashboard displaying analytical graphs. Some examples of improved local information use are summarized in Table 1.

Table 1. Examples of improved information use resulting from project for strengthening the Health Management Information System (HMIS) in Zanzibar, the United Republic of Tanzania, 2005–2008.

Health entity	Examples of improved information use
RCH Unit	– Development of indicators to monitor emergency obstetric and neonatal care availability – Monitoring of quality of antenatal care and skilled birth attendance coverage – Introduction of maternal death audits – Introduction of the “couple year protection rate” indicatora – Improved anaemia diagnosis in pregnancy
Malaria Programme	– Increased emphasis on bed net coverage – Monitoring of malaria in pregnancy – Treatment of confirmed rather than clinical cases, which in some instances resulted in data showing lower malaria incidence
Expanded Programme on Immunization	– Investigation of high dropout rates and coverage over 100% – Identification of double counting, resulting in improved quality control mechanisms – Introduction of diagnostic criteria to reduce misdiagnosis of pneumonia and malaria
HIV/AIDS and STIs	– Reduction of excessive data categories and age groupings
Hospitals	– Routine collection of basic inpatient indicators such as average length of stay and bed occupancy rate – Focus on signal functions of emergency obstetric careb and referrals, not just reporting of complications – Inclusion of laboratory data to check quality of diagnosis, particularly of malaria, tuberculosis, anaemia and syphilis – Improvement of OPD reporting to gain a more comprehensive idea of district-wide disease burden by: (i) using data elements identical to PHC units, enabling meaningful comparisons with peripheral health centres and encouraging back-referrals; and (ii) streamlining specialist OPD clinic data and adding it to the national database – Development of workload indicators to rationalize staffing needs and advocate for redistribution of staff away from central hospitals

AIDS, acquired immunodeficiency syndrome; HIV, human immunodeficiency virus; OPD, outpatient department; PHC, primary health care; RCH, reproductive and child health; STI, sexually transmitted infection.

^a The rate at which couples (specifically women) are protected against pregnancy using modern contraceptive methods.¹²

^b Key medical interventions that are used to treat the direct obstetric complications that cause most of maternal deaths around the globe. The signal functions are indicators of the level of care being provided.¹³

Problem-solving skills

District health management team members honed their ability to solve problems using HMIS data as they gained greater appreciation of the value of improved data quality and felt more competent to perform HMIS tasks and apply epidemiological approaches to daily data management issues.

Team work

Team work improved considerably as the district health management teams shared information about service delivery and used the HMIS to monitor and evaluate progress towards targets set in their district annual plans. Leaders became more confident in making evidence-based decisions to improve quality of care based on collective values developed through the data-use workshops. While a “culture of information” was not fully established, significant strides were made in that direction.

Practical computer skills

Workshop participants improved their computer skills in using DHIS for analysis, presentation and dissemination and also enhanced their knowledge of basic hardware and software maintenance, virus protection and backups. Software developers benefited from insights into new requirements and gained a better understanding of weaknesses related to local context and configuration.

Presentation skills

DHMT members’ presentation skills were initially weak, as they were unused to drawing graphs, using PowerPoint, engaging in debate or offering constructive criticism. These skills improved dramatically as a result of the workshops, especially when standardized templates for presentations were developed. Local HMIS Unit and health zone personnel acquired sufficient skills to run the workshops without outside facilitators. Box 1 summarizes the lessons learnt from this experience.

Box 1. Summary of main lessons learnt.

• The outcomes of the data-use workshops demonstrate and validate our hypothesis that the more data are used, the more data quality will improve, leading to significant innovations in the use of information and breaking the vicious cycle of non-use and poor quality of data.

• An integrated framework for HMIS, using a national data warehouse framework, provides an enabling environment in which actors, health programmes and systems can “speak to each other”, which is the foundation for improving health systems.

• Regular data-use workshops, with self-assessment and peer critique and discussion of the data presented, provide a powerful means of building a strong evidence base for HMIS improvements.

Conclusion

The data-use workshops approach developed in Zanzibar helped to strengthen the United Republic of Tanzania’s HMIS and thereby its overall health system, providing a concrete example of health information systems strengthening. The approach has been applied in other countries, such as in Kenya and Rwanda, where it has become an institutionalized part of quarterly review processes.