Resources, indicators, data management, dissemination and use in health information systems in sub-Saharan Africa: results of a questionnaire-based survey

Abstract

Objective

To describe the status of health information systems in 14 sub-Saharan African countries of the World Health Organization African Region.

Design

A questionnaire-based survey.

Setting

Fourteen sub-Saharan African countries of the African Region.

Participants

Key informants in the ministries of health, national statistics offices, health programmes, donors and technical agencies.

Main outcome measures

State of resources, indicators, data sources, data management, information products, dissemination and use of health information.

Results

The highest average score was in the identification and harmonisation of indicators (73%), reflecting successful efforts to identify priority indicators and reach international consensus on indicators for several diseases. This was followed by information products (63%), which indicated the availability of accurate and reliable data. The lowest score (41%) was in data management, the ability to collect, store, analyse and distribute data, followed by resources – policy and planning, human and financial resources, and infrastructure (53%). Data sources (e.g. censuses, surveys) were on average inadequate with a score of 56%. The average score for dissemination and use of health information was 57%, which indicated limited or inadequate use of data for advocacy, planning and decision-making.

Conclusions

National health information systems are weak in the surveyed countries and much more needs to be done to improve the quality and relevance of data, and their management, sharing and use for policy-making and decision-making.

Introduction

Data are crucial in improving health. They are the essential starting point, telling us how many people are suffering from various diseases and what health services are lacking. Statistics also tell us how successful policies and interventions are, for example whether maternal mortality has decreased, whether children are immunised and whether our health goals are achievable and strategies are on track. They alert us when we need to modify development programmes and redirect resources.

However, to be of use statistics must be both reliable and relevant. They need to be compiled correctly, following standard practices and methodology. They must also meet the needs of users. The ultimate objective of collecting data is to inform health programme planning as well as policy-making and, ultimately, global health outcomes and equity. Well-functioning health information systems empower decision-makers to manage and lead more effectively, by providing useful evidence at the lowest possible cost.

The health information systems that have been established in countries to provide such data are faced with a number of constraints in meeting these criteria. There is often a vicious circle where under-investment in health information systems results in data of poor quality, with users unwilling to use the data in policy-making. This lack of demand then leads to fewer resources being made available for data collection and quality control.

This paper describes the results of the assessment of the current status of health information systems in 14 sub-Saharan African countries of the World Health Organization (WHO) African Region and indicates what further actions countries and their development partners can undertake in order to improve the quality and availability of data.

Methods

The methods followed to assess national health information systems were drawn from a complete analysis of the knowledge landscape in the WHO African Region.¹ As regards the health information systems, all 46 WHO Member States of the Region were solicited, but only 14 were ready to undertake the assessment. Assessments were thus carried out in 14 countries, namely Benin, Cameroon, Eritrea, Ethiopia, Gambia, Ghana, Kenya, Lesotho, Mauritius, Namibia, Rwanda, Sierra Leone, South Africa, Swaziland and United Republic of Tanzania.² The procedure consisted of a systematic self-assessment by country stakeholders in ministries of health, national statistics offices, health programmes, donors and technical agencies operating in that area, under the overall coordination of WHO health information system experts. The assessment was conducted using a tool³ for rating performance of health information systems developed by the Health Metrics Network, reviewed and adapted by a team of experts from the WHO Regional Office for Africa with input from countries’ health information system managers.

A one-week workshop with all the relevant stakeholders was then organised in countries to assess the health information systems at the national level. Health information system country experts from the Regional Office and their national counterparts conducted the assessment in country regions to confirm, adjust and fix the national outcomes. During the workshops, participants were split into groups with stakeholders from each areas represented, to make sure each group was able to cover all assessed areas of the health information system. The results were categorised and analysed by the principal coordinator of the assessment.

The tool considered six dimensions with some (resources, data sources, information products, and dissemination and use) being split into subcategories. The following six dimensions of health information systems were assessed:

1. Resources: prerequisites that need to be in place for a health information system to function, including legal and policy frameworks in place supported by sufficient human and financial resources, infrastructure, etc.

2. Indicators: core health indicators identified covering determinants of health; health system inputs, outputs and outcomes; and health status (morbidity and mortality)

3. Data sources: key data available from six main sources (censuses, vital events monitoring, health facilities statistics, public health surveillance, population-based surveys and resource tracking) and standards for their use

4. Data management: optimal processes for collecting, sharing and storing data; as well as data flows and feedback loops

5. Information products: accurate and reliable data available for health status, health systems and determinants of health

6. Dissemination and use: dissemination of information and effective use of data for advocacy, planning and decision-making

Each country rated its perceived performance on all six dimensions and scores were categorised as follows:

• ‘Highly adequate’ for 80–100% perceived performance

• ‘Adequate’ for 60–79% perceived performance

• ‘Present, but not adequate’ for 40–59% perceived performance

• ‘Not adequate’ for 20–39% perceived performance

• ‘Not functional’ for 0–19% perceived performance

Results

The aggregate findings from the survey, the average scores for each of the dimensions and subdimensions, are presented in Table 1. Most of the scores were in the middle category, ‘present, but not adequate’, with no scores for the ‘highly adequate’ and ‘not functional’ options. The highest score was for the identification and harmonisation of indicators (73%), reflecting successful efforts to identify priority indicators and reach international consensus on indicators for several diseases. The lowest score (41%) was in data management (the ability to collect, store, analyse and distribute data), followed by resources (policy and planning, human and financial resources, and infrastructure).

Table 1.

National health information systems in 14 sub-Saharan African countries, 2009.

Category	Not functional (%)	Not adequate (%)	Present but not adequate (%)	Adequate (%)	Highly adequate (%)
I. Resources			53
Policy and planning			44
Human and financial resources			48
Infrastructure				66
II. Indicators				73
III. Data sources			56
Census				63
Vital statistics		36
Surveys				77
Health and disease records			54
Health service records			59
Administrative resource records			48
IV. Data management			41
V. Information products				63
Health status				72
Mortality				64
Morbidity				79
Health systems			59
Risk factors			57
VI. Dissemination and use			57
Analysis and use				64
Policy and advocacy			58
Planning and priority-setting			57
Resource allocation			50
Implementation and action				56

Scores for data sources, information products, and dissemination and use were in-between and scoring reflected uneasiness about current performance.

As part of the resources category, countries identified inadequate policy and planning resources (44%) as the most important barrier to strengthening health information systems, followed by human and financial resources (48%), and an infrastructure that is somehow adequate (66%). Only one country scored policy and planning as ‘adequate’, whereas seven countries scored it ‘not adequate’ or ‘not functional’.

Policy weaknesses included inadequate legislation, absence of a national strategic plan and non-functioning coordination mechanisms. A particular weakness identified was the absence of a regulatory framework to ensure that the private healthcare sector was part of the health information system. Indicators scored highest of all six dimensions with 11 countries in the ‘adequate’ category, reflecting that national minimum core indicators were identified that also covered the health-related Millennium Development Goals. There seemed to be no lack of indicators while data availability, quality and relevance were much more problematic.

The main sources of data for health information systems were either based on administrative or clinical information, or derived from household surveys. Eleven countries found household surveys to be ‘adequate’ and only ‘inadequate’ in exceptional cases. The survey scored censuses as ‘adequate’, but many countries have been unable to conduct a census in the past 10 years.

Data management was found to be problematic in the majority of countries and almost fell into the ‘not adequate’ category – indicating that countries do not have clear procedures for the collection, storage, analysis and distribution of data, including a centralised data depository.

Most countries (63%) scored information products as ‘adequate’, indicating that countries have accurate and reliable data for core health status, health system and determinants of health indicators. Health systems and risk factors had the lowest score, while health status information had the highest.

The average score for dissemination and use of information was 57% – ‘present, but not adequate’ – with the highest score for analysis and use being ‘adequate’. The use of data for policy and advocacy, planning and priority-setting, resource allocation, and implementation and action were all considered ‘present, but not adequate,’ with resource allocation as the lowest (50%).

Discussion

The results of the assessment show that despite the important progress made in the development of national health information systems, much more needs to be done. The middle level of scores reflects that, while efforts have been invested to improve health information systems and progress can be seen, these systems are not yet adequate. The use of average numbers and scores conceals important variations between countries and the problematic situation that many of them face.

Regarding national health information system resources, the findings show an institutional framework for health information fragmented, with responsibility for collection of health-related data being divided across different ministries. Health information systems have often evolved in an erratic, piecemeal manner due to administrative, economic and donor pressures. The system is severely fragmented by disease-specific reporting requirements, where different reporting formats and content required for tuberculosis, HIV, malaria and child health create many parallel data collection systems.⁴

The survey does not provide detailed information about the level of financial resources, but four countries scored human and financial resources as ‘not adequate’ and the rest rated them ‘present, but not adequate’. In general, countries lacked skilled human resources, including weak capacities in data analysis and use of new information technologies. Infrastructure, including the availability of computers, was considered adequate in nine countries, probably because such equipment is often provided by international donors.

The lack of adequate numbers of health information system staff with sufficient skills in the applications of computers to health information system issues emerges as a universal problem. Staff lack training, career development and motivation. Remuneration systems are poor, and donors or the private sector often lure away the best staff for their own purposes. Information systems are thus caught in a vicious circle, in which inadequate resources restrain outputs and undermine the quality of data, while the poor quality of data leads to lower demand and hence fewer resources.

The increased demand for health information from international donors has led to a proliferation of indicators and often multiple and excessive monitoring systems. Despite the high score in this dimension, there is a need to concentrate on fewer priority indicators.⁵

The high score as regards data sources is clearly a result of major international investments in household surveys. However, the limitations of surveys are the inability to disaggregate data to local levels and to provide information at short time intervals. The poor quality, availability and relevance of vital statistics are seen as major problems; these elements were considered ‘not adequate’ in eight countries and ‘adequate’ in only one country, as for administrative records with information about availability and deployment of human resources. Most low- and middle-income countries find it difficult to maintain routine data collection efforts, let alone improve them. Because of the limited development of vital registration systems, most developing countries depend on surveys and censuses to estimate population morbidity and mortality.

In many countries, there is more than one agency involved in collecting and disseminating information and most of them use different methods and approaches. The ministries of health gather data through their administrative reporting systems, while central statistical offices collect data at household level. Differences in sources and methods mean that data obtained from administrative records and data from surveys are not directly comparable. Health workers at local level are often overburdened with excessive data and reporting demands, owing to multiple and often poorly coordinated systems with a large number of uncoordinated forms to fill. Actually, a major problem is the lack of standardisation and alignment within and between reporting forms.

Other assessments⁶ of health information systems emphasise the paucity of information on leading causes of child death, data on adult mortality and availability of accurate data of morbidity for major diseases such as pneumonia, diarrhoeal diseases, malaria and diabetes. At present, stakeholders can only expect occasional incomplete snapshots of countries’ progress towards health goals, with much of the progress assessment heavily dependent on modelling rather than on empirical evidence.

Accurate data on access to services and essential drugs are lacking in most countries because of missing data on service delivery (availability and quality). Data on coverage are often quite good, thanks to population-based surveys and fairly accurate reporting systems for certain interventions (e.g. vaccinations). National Health Account exercises required to obtain comprehensive information on expenditure within a health system are only carried out in a few countries.

Several studies⁵ identify the key challenge in strengthening health information systems as being the ability to get managers at all levels to use available health information to inform decision-making. There are several reasons for the low score in this area:

• Presentation of data is often aimed at specialists

• Little effort is made to make the information understandable to policy-makers, front-line health workers or the public

• Data are presented too late

• Data production is weakly connected with data use, and more attention is given to collecting than to analysing and using information

As this area is quite specific and not easy to measure accurately, further study is required.

Although the surveys are very informative in describing the state of national health information systems, they are not well suited for enabling comparisons between countries, as they suffer from methodological limitations and are not independent. However, they have helped to identify important gaps and issues as reported by country stakeholders. Thus, the results described here should be interpreted with caution.⁷

Conclusions

There have been noticeable improvements in health information systems in Africa but there are still major gaps and much more needs to be done to improve the quality and relevance of health information. Few countries have sufficiently effective health information systems even to adequately monitor the Millennium Development Goals. There has been a chronic under-investment in systems for data collection, analysis, dissemination and use; and even when data are available, they are often out of date and unreliable.

Efforts to improve health information systems cannot fall on resource-limited countries alone, or on only a few development partners. A programme for reform and improvements must build on country ownership and efforts, and increase the demand for and supply and use of information. It should include all of the following elements:

• Building demand and country ownership

• Building on the needs of users

• Developing a comprehensive multiyear plan

• Assessing existing capacity

• Training and new tools for data collection and analysis

• Increasing financing

• Improving coordination

Declarations

Competing interests

None declared

Funding

Health Metrics Network and WHO Regional Office for Africa

Ethical approval

Not required because the survey did not touch on ethical issues requiring individual consent.

Guarantor

PEM

Contributorship

PEM wrote the paper, DK and EWS-A co-wrote and reviewed the paper, CZ reviewed the paper, WK assisted with data collection/compilation, PSLD gave overall guidance and leadership.

Acknowledgements

These surveys would not have been possible without the active participation of the ministries of health, other public agencies dealing with statistics, including statistical offices, research and academic institutions and non-governmental organisations who have given their time and effort to complete modules and questionnaires.

Provenance

Not commissioned; peer-reviewed by Claudia Pagliari