Dying to count: mortality surveillance in resource-poor settings

Abstract

Reliable cause-specific mortality data constitute a crucial resource for health monitoring, service planning and prioritisation. However, in the majority of the world's poorest settings, systematic health and vital event surveillance systems are weak or non-existent. As such, deaths are not counted and causes of death remain unregistered for more than two-thirds of the world's population.

For researchers, health workers and policy makers in resource-poor settings, therefore, attempts to measure mortality have to be implemented from first principles. As a result, there is wide variation in mortality surveillance methodologies in different settings, and lack of standardisation and rigorous validation of these methods hinder meaningful comparison of mortality data between settings and over time.

With a particular focus on Health and Demographic Surveillance Systems (HDSSs), this paper summarises recent research and conceptual development of certain methodological aspects of mortality surveillance stemming from a series of empirical investigations. The paper describes the advantages and limitations of various methods in particular contexts, and argues that there is no single methodology to satisfy all data needs. Rather, methodological decisions about mortality measurement should be a synthesis of all available knowledge relating to clearly defined concepts of why data are being collected, how they can be used and when they are of good enough quality to inform public health action.

The data we want

Mortality registration is the oldest form of disease surveillance, and its importance for epidemiology and public health planning is perhaps obvious. Two of the Millennium Development Goals and one out of three essential elements of the Human Development Index are specific mortality measures (1, 2), enhancing the need for valid and representative data on mortality and its risk factors to track progress, evaluate disease-control programmes and monitor major global health initiatives. Consistent and reliable cause-specific mortality data therefore constitute a crucial and major resource for health planning and prioritisation (3).

However, the chance of a death being registered and the cause of death documented strongly depends on the socioeconomic status of the community and nation in which it occurs (3). With little or no progress in civil registration systems in the last 50 years, between two-thirds and three-quarters of the world's population remain outside any kind of systematic health surveillance (4–8). As of 2003, 60% of United Nations member states have supplied cause-of-death data to the WHO. However, regional coverage of death registration is less than 10% in the Africa region and is considered to be complete in only one-third of the 115 reporting countries – those providing data of ‘high’ quality representing only 12% of the world's population (6, 9). This ongoing lack of knowledge on who lives and dies where and from what has been described as ‘the single most critical failure of development over the past 30 years’ and the lack of any record of the lives of billions living in poorer countries as a ‘scandal of invisibility’ (10). Tracing the imprint of a person's existence, including their birth and death, not only confirms ideals of citizenship, but also represents the first step in securing population rights to life, freedom and protection (11).

A popular application of Finagel's law to health measurement states that ‘the data we have are not the data we need; the data we need are not the data we want; the data we want are unobtainable’ (12). Underlying this saying are the methodological realities of data capture as well as the conceptual aspects of the intended users and use of data (Fig. 1). In theory, mortality measurement is based on highly interconnected relationships between the actual situation one wants to measure, the methods available for measurement and data needs, which should be informed by the intended use of the data. In practice, methodological approaches to measuring mortality and assigning causes of death do not always appear to be directly informed by the intended use of the data, but rather by an underlying presumption of a ‘one-size-fits-all’ nature and utility of mortality data.

Fig. 1. .

Conceptual framework of mortality surveillance. The methods we use determine the data we get. Appreciation of differing needs and uses of mortality data should inform the methods used so that the data we have are reconciled with the data we need.

The data we need

In practical public health terms, there are different levels at which cause-specific mortality data are needed, ranging from the local to the global. Health managers require cause-specific mortality data to effectively plan health services based on local patterns of disease. At this level, it is essential to be able to monitor major causes of death simply and cheaply. The breakdown of cause-of-death categories based on a few hundred cases is needed and very detailed causes of death would be superfluous. Epidemiologists, health service researchers and assessors of specific health interventions, such as safe motherhood interventions, need a consistent assessment of cause-specific mortality to determine trends in causes of death that enable evaluations of the effectiveness of interventions across time and regions. For such users, a reduction in rates of a specific cause of death is usually an important endpoint – for example, detailed sub-causes of maternal mortality. National and global authorities concerned with building respective pictures of health patterns require consistent and reliable cause-specific mortality data from a wide range of settings (3). For all users in low- and some middle-income settings, however, mortality registration processes and methods often have to be implemented from first principles. This has resulted in a variety of different approaches to population health surveillance, ranging from one-off cross-sectional surveys to longitudinal monitoring of population cohorts. As a result, mortality surveillance methodologies vary widely between settings and seemingly ad hoc approaches that are not always explicit about the gap in health information that they are attempting to fill or do not seem to be directly related to the intended use of the data are apparent. Furthermore, lack of standardisation and rigorous validations hinder meaningful comparisons of data between settings and over time, and may diminish the use of surveillance data for public health action.

A common, if perhaps simplistic, understanding of the purpose of population surveillance is to gain an overall impression of population composition and distribution of risk factors to inform public health action. If this is the case, then when are surrogate methods for more widespread health surveillance in resource-poor settings and the data derived from these methods ‘good enough’ for this purpose? Understanding the particular strengths and limitations of existing methods, the data that they generate and how such data can be used to meet specific needs is essential to reconcile the data we want with the data we actually need and the data that we can get. With a particular focus on Health and Demographic Surveillance Systems (HDSSs), this paper summarises recent research and conceptual development on some methodological aspects of mortality surveillance that stem from a series of empirical investigations (13).

The data we have

For the majority of the world's population, for whom vital events of births and deaths are not counted, HDSSs have emerged as a useful surrogate. Notwithstanding local and contextual variations, HDSSs maintain regular surveillance of births, deaths and migrations and, frequently, social and economic correlates of population and health dynamics, in an open cohort population within a clearly circumscribed geographic area (14). This is achieved through selecting a population, conducting an initial census and following up by periodic household surveys. Such approaches are unique in that they are able to generate data in settings with no other comparable source of information on births, deaths and causes of death, and vast amounts of high quality research, training and service provision are generated from such sites, much of which would be difficult to do without such infrastructures (15). Nevertheless, methodological variation between sites and the fact that HDSSs are localised systems that cover only a small proportion of total national populations has somewhat limited the wider utility of HDSS data by national and international researchers and practitioners.

In the context of HDSSs, it has been suggested that the resource-intensive active follow-up of individuals can only be justified if the results can be extrapolated meaningfully into the surrounding 100-fold population (5, 16). In focus here are issues of representativity and generalisability. ‘Representativity’ refers to the context of a site and the extent to which physical, cultural, religious and social characteristics approximate to other areas. ‘Generalisability’ relates to the extent to which findings from an investigation using particular methods in a particular setting (i.e. an HDSS) can plausibly be applied more widely (17). However, there remain no ‘best-practice’ guidelines for enhancing representativity or generalisability and the size and selection of HDSS populations are seemingly influenced more by economic restraints than sound sampling theory. Such determining factors have been criticised for failing to take into account the number of deaths needed to yield sufficiently robust information on cause-specific mortality (8). While mathematical formulae are available to calculate necessary sample sizes for acceptable degrees of precision (18–20), including methods for determining efficient sizes for sample-based mortality surveillance systems in situations where prior information on the cause composition of mortality is lacking (8, 21), there is no evidence that these are used in existing HDSSs. Given that budgets, geography and national contexts vary widely, there is unlikely to be a simple, one-size-fits-all solution to determining the ideal population surveillance size, but choice should be related to specific goals and intended use of the data, with appreciation for the impact on representativity and generalisability.

Thorough understanding of causal pathways and potential intervention strategies in relation to mortality requires the reliable measurement of basic population parameters such as age, gender and socioeconomic distributions, which are likely to have a wide distribution among any population. The distributions of such parameters should have important implications on the choice of sampling method, yet this is not apparent in current HDSS methods and a wide variety of sampling procedures are utilised, not least with regards to their complexity (19, 22–24). Given the reality of variation in sampling approaches between sites, it is prudent to know a priori whether and to what extent this might hinder cross-site comparisons of data. Such questions are not only important for the establishment of the HDSSs themselves, but also for one-off surveys in resource-poor settings and nested surveys within HDSS settings.

Empirical investigations into sampling approaches for population surveillance emphasises the need to consider general population distributions and uniformity of certain parameters within localities when selecting sampling methods (16, 25). While 1% samples drawn from reference datasets using different sampling approaches can represent the reference data well, distribution of parameters has been shown to be an important consideration. For example, consistent and approximately normal distribution of gender means that the proportion of males in a population can be well represented irrespective of the sampling approach. In contrast, parameters with more skewed and inconsistent distributions, such as education, are more difficult to capture. Modelling of multistage HDSS-style sampling approaches appear to perform inconsistently with regard to reliability and representativity of various demographic and health parameters, emphasising the need to consider general population distributions and uniformity of certain parameters within localities when selecting methods. As with sample size, there is unlikely to be a simple, ‘one-size-fits-all’ sampling technique that can satisfy all needs of survey design, therefore compromises, which are informed by empirical evidence, are necessary.

In terms of the generalisability of HDSS data to wider national populations, empirical comparisons with Demographic and Health Survey^† (DHS) data have shown that population composition and certain mortality risk factors identified in HDSSs are broadly applicable to regional and national populations (26–28). It appears from these investigations that HDSSs have more scope to detect the extent of local variations in population composition and health status than DHS methods, which average out local variations across regions or nations. As was the case for the sampling technique, general population distributions and uniformity of certain parameters within localities are important determinants of whether locally derived estimates can be applied nationally, and vice versa. The differing yet complementary characteristics of DHS and HDSS mean that, when combined, these two data sources have the potential to characterise national population composition and health status as well as the extent of local variation – both of which are important for health monitoring and planning. Moving on from discontinuous thinking about data sources and continually drawing comparisons, there is room for further investigations into how data from different sources, such as HDSSs and DHSs, could be combined to provide more complete pictures of population health and maximise the potential utility of existing data in supporting developing-country health systems (26–29).

Data quality

Regardless of specific methods used, a certain amount of error is to be expected in population surveillance (30, 31), and the extent to which imprecision should affect the use of mortality surveillance data is an important concept with practical implications. A significant proportion of population surveillance operations and resources are dedicated to data quality-assurance mechanisms (32). The majority of member sites of the INDEPTH^‡ network, for example, describe scheduled random re-visits of primary sampling units as a method of data quality control, with the percentage of households re-visited ranging from 2% (Agincourt HDSS, South Africa) to between 5 and 10% (Nouna HDSS, Burkina Faso) (14, 33). Recent developments in direct data capture using handheld computers or Personal Digital Assistants (PDAs) and Global Positioning Systems (GPS) present innovative approaches that may simplify data capture and enhance the quality of household and individual identification data, and several studies have demonstrated their usefulness for data capture, even in rural African settings with limited electricity supply and harsh environmental conditions (34–38). Nevertheless, error is unlikely to ever be completely eliminated from the data that we have, therefore drawing correct quantitative conclusions that can form the basis for public health intervention necessitates that the effects of measurement error in the data that we have are appreciated and accounted for (31). Recent work suggests that high levels of purely random errors may not be hugely detrimental to the utility of population surveillance data based on large samples (39). The expense and practical difficulty of detecting and correcting random errors must be considered in relation to the benefits of such efforts. Efforts will have a diminishing return as the 100% accurate dataset is approached, and so further consideration should be given to redirecting the costs of such efforts towards increasing the size or geographic spread of surveillance operations in order to increase representativity, or indeed towards analysing the data and disseminating findings.

Causes of death

Simply counting the number of deaths is not enough to develop an understanding of population-level disease profiles and important health transitions. Therefore, cause-specific mortality measurement is vital and, for the time being at least, verbal autopsy (VA) methods are the only feasible way of obtaining such data for the majority of the world's population. VA methods gather information from a close caregiver about the signs and symptoms of the deceased's terminal illness, as well as lifestyle behaviours and other characteristics. This information is then used to derive probable causes of death, most commonly through independent review of the data by local physicians who try to reach consensus on a single cause (40). Longstanding concerns over inter-observer agreement and lack of standardisation of physician review methods preclude meaningful comparisons of cause-specific mortality between regions and over time, where physicians and their methods of interpreting evidence may differ (41). This lack of standardisation has been tackled with efforts culminating in the development of various algorithmic approaches based on the concept of distilling the process of physician review into standardised rules (42). Diagnostic algorithm-based cause-of-death determination may be less accurate than physician review, but has the advantage of being transparent and repeatable. Nevertheless, algorithmic procedures make it impossible to consider parallel possibilities of causes of death along the lines of classic clinical differential diagnoses, and their consistency depends on the consistency of diagnostic criteria. Most HDSSs do not currently employ diagnostic criteria for deriving causes of death.

Limitations of physician review and traditional algorithmic approaches have led to the development of more innovative approaches to cause-of-death determination based on VAs. Application of Bayes’ theorem for VA interpretation has been developed and evaluated using VA data from Vietnam, Ethiopia and Burkina Faso (43–45). Known as InterVA (for all age mortality) and InterVA-M (for deaths in reproductive-aged women), the approach derives up to three probable causes of death from VA data and has been shown to produce comparable VA-derived cause-specific mortality fractions (CSMFs) to physician review with the advantage of being completely reliable – the same set of indicators, signs and symptoms will always lead to the same probable cause of death (43–47). An alternative method developed by King and Lu (48), directly estimates CSMFs without individual cause-of-death attribution. Their method resolves the problem of generalising VA analysis to the population based on test properties quantified in health facility validation studies. Combining King and Lu's approach with the InterVA method, Murray et al. (42) propose and have attempted to validate a new approach called the Symptom Pattern method. Such developments are welcome attempts to overcome limitations of current VA methods and the fact that these innovative methods are addressing differing data needs should be emphasised – they do not offer a ‘one-size-fits-all’ solution.

Failure to emphasise the differing data needs that VA methods are attempting to address can result in a narrow assumption that VA is a direct surrogate for Western-style cause-of-death determination. Rather than targeting specific gaps in the understanding of mortality in less-developed countries and considering whether the method is now more or less fit for purpose, therefore, VA developments tend to be discussed in terms of whether they meet medical ideals. This reinforces illusions of a ‘one-size-fits-all’ solution to long-standing information gaps, and limiting VAs to a medical model undermines their full potential as epidemiological tools, which can be adapted to any specific point along this chain of economic, social, operational, biomedical and physical events leading to death. VA may be designed to address specific public health or mortality questions in a way that Western, medical-based models cannot. Explicit targeting of a specific point along the chain of events leading to death is useful in terms of data collection and analysis and may allow more useful discussions of new methods in terms of adequacy for purpose rather than absolute validity in relation to dubious gold standards (40, 49).

Such conceptual developments, however, will need to overcome a default assumption of general medical audiences that cause-of-death determination is solely for the purposes of individual-level cause-of-death certification. Filling gaps in population-level information is arguably more important for health planning and monitoring purposes than filling gaps in individual-level data. Nevertheless, the largely individually derived and clinically oriented International Classification of Disease (ICD) coding, remains the mandatory level of coding for international reporting to the WHO mortality database (50). The purpose of such standardised disease reporting rules is to ensure comparability, however the assumption that individual deaths will be coded consistently and reliably between regions and over time and can be aggregated to identify population-level disease burdens in different regions is flawed. The use of an individual-focussed approach to address a population-level need seems inappropriate. While individually VA-determined causes may be methodologically easier to compare with individually certified causes of death from other settings, it does not necessarily imply a need for certainty at the individual level. Rather, it emphasises the need for reliable methods of interpreting CSMFs for known populations. Furthermore, determining multiple, rather than single causes of death for any particular case is more likely to accurately reflect the interaction of different diseases that lead to death and give a more complete representation of broad, population-level cause categories for which the public health response implications are essentially similar. This may be less precise in terms of ICD coding, but could be more suitable for guiding public health prioritisation on a more local level (8).

Users and uses: making deaths count

That the value of data lies in their use and not in their collection does not always seem to be appreciated by surveillance systems, often burdened with tight budgets that hinder rapid local analyses (6, 51). A major challenge facing population surveillance activities in general, and HDSSs in particular, is the accumulation of unanalysed data. All too often the period from data capture in the field to analysis, publication and use for informing public health action is very long. Even when data are processed efficiently, they are rarely made widely available or communicated effectively enough to have an immediate effect on the lives of the surveillance population.

It is debatable whether sentinel surveillance and HDSS operations in developing countries are directly responsible for practical public health action, but to justify the risks and intrusion of surveillance, the collected information must have a demonstrated utility. Within the context of humanitarian disasters, for example, important fluctuations in surveillance population mortality should be detectable and trigger action (29). Ill-defined responsibilities, complex operational procedures and long time lags between data capture and analysis are unacceptable excuses for not using the data generated from population surveillance activities for the timely detection of entirely preventable morbidity and loss of life in the surveillance population. While the standardisation, quality control and validation of surveillance methods is important, efforts are also needed to stimulate the debate and development of simple procedures for using data and clearly defined surveillance responsibilities. That population surveillance activities in developing countries typically operate in cooperation with local health authorities, universities and local and foreign government ministries means that key actors in health, development and relief are likely to be receptive to efforts to enhance communication with population surveillance organisations. In combination with data that may be collected by other parties, such as environmental and meteorological data, mortality information could enhance understanding of environmental and population inter-relationships and provide a more complete incentive for public health action. That the data must be used to justify the effort and intrusion on individual privacy is one principle that fits all surveillance activities.

Reconciling ‘want’, ‘need’ and ‘have’

Establishment of registration systems for entire populations is unlikely to occur in the short to medium term; the data we want will remain unobtainable. In the meantime, sample-based and sentinel population and mortality surveillance can yield sufficiently reliable and relevant information for programme action, and are well within the means of many developing countries. Indeed, such systems represent the only useful alternative to establish the evidence base for health policy and programme delivery for the foreseeable future in much of the developing world. That the data we have may not be exactly the data we want does not make evidence-based decision making impossible – the data and evidence that we do have should be used while efforts continue to be made to improve the evidence base (52).

Understanding the potential advantages and limitations of methods in particular contexts is important for informing appropriate population survey design within the boundaries of financial and logistical constraints. However, as this paper repeatedly emphasises, there is no single methodology that can fully satisfy all data needs. Methodological decisions about surveillance should therefore be a synthesis of all available and relevant knowledge relating to clearly defined concepts of why data are being collected, how they can be used and when they are of good enough quality. A number of mathematical principles have been developed to demarcate what is ‘significant’ statistically, but no comparable principles have been established to indicate what is significant operationally in relation to public health action. Ultimately, explicit discussion of such issues internationally as well as with surveillance communities is not only vital to improving the state of knowledge on the world's health, but also to maintaining public trust in, and understanding of, health and demographic surveillance efforts. This, in turn, may be a significant step towards more widespread, routine, vital-event surveillance and the crucial goal of not just counting deaths, but also making all deaths count.