Assessing the policy utility of routine mortality statistics: a global classification of countries

Abstract

Objective

To evaluate the utility and quality of death registration data across countries.

Methods

We compiled routine death and cause of death statistics data from 2015–2019 from national authorities. We estimated completeness of death registration using the Adair-Lopez empirical method. The quality of cause of death data was assessed by evaluating the assignment of usable causes of death among people younger than 80 years. We grouped data into nine policy utility categories based on data availability, registration completeness and diagnostic precision.

Findings

Of an estimated 55 million global deaths in 2019, 70% of deaths were registered across 156 countries, but only 52% had medically certified causes and 42% of deaths were assigned a usable cause. In 54 countries, which are mostly high-income, there is complete and high-quality mortality data. In a further 29 countries, located across different regions, death registration is complete, but cause of death data quality remains suboptimal. Additionally, 37 countries possess functional death registration systems with cause of death data of poor to moderate quality. In 30 countries, death registration ranges from limited to nascent completeness, accompanied by poor or unavailable cause of death data. Furthermore, 38 countries lack accessible data altogether.

Conclusion

By implementing more proactive death notification processes, expanding the use of digitized data collection platforms, streamlining data compilation procedures and improving data quality assessment, governments could enhance the policy utility of mortality data. Encouraging the routine application of automated verbal autopsy methods is crucial for accurately determining the causes of deaths occurring at home.

Introduction

Health planning and policy evaluation requires reliable, detailed and timely mortality statistics and cause of death data. Functional civil registration and vital statistics systems ensure the continuous registration and certification of all deaths in a population.¹ However, in many countries, the statistics on deaths and causes of death produced by civil registration and vital statistics systems are inaccurate and otherwise of limited use for public policy.² Even when proper registration and certification of deaths occur, the information is not always compiled into routine statistics to guide government policy. Gaps in mortality data are often filled by surveys and censuses despite the fact that they are not a timely and routine source of mortality statistics.³ In many low- and middle-income countries cause of death data are only collected for the small proportion of deaths that occur in hospitals. Moreover, the diagnostic accuracy and policy utility of such statistics are limited by insufficient training in correct medical certification procedures and/or incorrect coding practices.⁴ Approximately 60% of deaths occur outside of hospitals in low- and middle-income countries.⁵ Data on these deaths can be unreliable because the cause is often ascertained by non-medical, untrained personnel, with or without the use of validated verbal autopsy methods.

Civil registration and vital statistics are designed to generate continuous, disaggregated, timely and accurate data on deaths and causes of death that can be used as a source of health intelligence for governments. Mortality data that is categorized by both age and gender can be used to compute essential population health metrics like life expectancy, and to monitor a nation's progress through the epidemiological transition. Reliable measurements of excess mortality during pandemics or from mortality shocks such as natural disasters or civil conflict are critical for decision-making. In addition, civil registration and vital statistics systems facilitate calculation of mortality indicators for subnational populations; in turn enabling a more granular understanding of differentials in mortality risk. Yet, despite these multiple uses, death registration data are often underused largely because of concerns about their accuracy and potential biases.⁶

Similarly, reliable and timely statistics on causes of death are fundamental for tracking trends in the leading causes of premature mortality; for underpinning assessments of the impact of health intervention strategies and policies; and are a prerequisite for monitoring progress for seven out of the 17 sustainable development goals.^7,8

Previous global assessments of the availability and quality of mortality statistics generated by civil registration and vital statistics systems are limited to a single source, such as the World Health Organization (WHO) Mortality Database, and rely on complex transformations of the underlying data; or are now several years out of date.^1,2,9,10 A recent assessment of mortality data produced by the Global Burden of Disease (GBD) Study focused on the utility of vital registration for measuring national and subnational burden of disease; however they failed to identify common challenges in countries at different levels of development of statistical systems, and often censored data points judged to be outliers, introducing bias into the results.¹¹

Here we present a systematic evaluation of the completeness and quality of death and cause of death statistics for all publicly available sources of data where deaths were either registered or otherwise known to a government. We then use these results to classify country vital registration systems into categories that represent their utility for guiding public policy in countries. Based on these scores, we then propose a prioritized set of feasible, tested system interventions that can assist countries to derive maximum policy benefit from their administrative data systems.

Methods

We used death statistics generated from routine death registration or reporting systems designed to count all events within a specific jurisdiction, including hospital death reporting systems. We define these statistics as registered or reported deaths known to some level of government authority; hereafter we refer to them as registered deaths. Data sources were identified from an exhaustive search, and were only considered if they were compiled by national authorities and publicly available either through national reports (e.g. vital statistics reports); international databases which include only statistics notified by national authorities; national civil registration and vital statistics system assessments endorsed by governments; or data otherwise made available to the authors by relevant government agencies.^10,12,13 These sources exclude mortality and cause of death statistics collected from Sample Registration Systems; Health and Demographic Surveillance System sites; and other surveys and censuses where the primary intent was not formal death registration. We used the WHO Mortality Database for cause of death data for the majority of countries, and country-provided data in some instances.¹⁰ A full list of data sources is provided in the online repository.¹⁴ We only considered data from 2015–2019 as indicative of an operational vital registration system, and we did not consider timeliness any further.

We evaluated the policy utility of vital registration data, based on the completeness of death registration and quality of cause of death data. Completeness was calculated for registered or reported deaths as the percentage of the estimated total number of deaths, measured where possible by year of occurrence using the Adair-Lopez empirical completeness method.¹⁵ This method estimates completeness of death registration using a statistical model with covariates of the registered crude death rate, and incorporates variables representing the drivers of the true crude death rate such as: (i) the under-five mortality rate to represent mortality level; (ii) the percentage of the population aged 65 years or older to represent population age structure; (iii) the completeness of under-five death registration or reporting; (iv) calendar year; and (v) country random effects.¹⁵ Such a method also ensures comparability by applying a common approach to calculating completeness for all data.

For countries where there is either high human immunodeficiency virus mortality or conflict-driven mortality, we used the United Nations (UN) World Population Prospects¹⁶ estimated deaths to calculate completeness. The empirical completeness method is less reliable in such settings given the assumptions of the model; these countries are indicated in the online repository.¹⁴ UN estimates are predominantly derived from adult mortality calculated from registered deaths adjusted for incompleteness, and child and adult mortality estimates from censuses and surveys, which are entered into model life tables.¹⁶ We present completeness estimates in bands of five percentage points to account for uncertainty in estimation. We estimated global completeness of death registration by using the most recent completeness estimate for each country based on available data for the period 2015–2019; as applied to UN Population Division estimates of deaths in 2019.¹⁶

We included all cause of death data where the underlying cause was ascertained from the International certificate of medical cause of death (medically certified cause of death), subsequently coded according to the rules and procedures of the International statistical classification of diseases and related health problems, 10th revision (ICD-10), 2016 version.¹⁷ Garbage codes were classified into four different levels of severity (very high, high, medium and low) according to their potential effect on policy deliberations (online repository).^14,18 Using this classification, we calculated the percentage of deaths with a useable cause; that is, the percentage of estimated total deaths that have a medically certified cause of death and for which the assigned cause was not classified as a garbage code of very high, high or medium severity.

Countries with a high proportion of deaths at older ages (older than 80 years) tend to have a higher proportion of garbage codes in their cause of death data because accurately diagnosing underlying cause of death at older ages is increasingly difficult due to higher likelihood of multiple morbidities. We therefore only considered cause of death data for those individuals younger than 80 years, to ascertain the extent of garbage codes in the data.

For 46 countries the distribution of garbage codes by severity could not be calculated because ICD-10 data were not publicly available. For these, we predicted the useable fraction of causes of death for individuals younger than 80 years based on a linear regression model using death registration completeness; GBD super-region; sociodemographic index; and universal health coverage as covariates (online repository).¹⁴ Usability calculations and estimates are also presented in bands of five percentage points to account for uncertainty in the completeness estimation method. Estimating the fraction of useable causes for these 46 countries enabled their inclusion in further analyses of civil registration and vital statistics system performance. We did not include country-level estimated usability percentages in global or regional calculations of usability.

To better target interventions and identify challenges in mortality statistics, the usability of countries’ data was classified by the completeness of death registration and the quality of cause of death data, determined by the percentage of deaths in individuals younger than 80 years with a usable code. The criteria for categorizing mortality data from national vital registration systems are displayed in Table 1. Countries with 95%–100% death registration are labelled as complete since the small percentage of unrecorded deaths would unlikely influence policy decisions. Those with 75%–94% registration are termed functional; they record enough deaths to demonstrate that the system is operational, but many deaths go unrecorded. Countries with 25%–74% registration have limited systems. Despite being operational, a significant portion of deaths remain unrecorded, reducing the policy value of the data. Nascent registration indicates very few recorded deaths, highlighting a need to enhance basic system components. High-quality cause of death data means 80% or more deaths in people younger than 80 years have a usable cause, marking them as fit-for-purpose. Data with 60%–79% usable causes are moderate quality, suitable for policy discussions but requiring caution. Data with less than 60% usable causes are of poor quality. In many such countries, non-medical staff often determine causes, sometimes using verbal autopsy. While these classifications and thresholds might seem arbitrary, they offer a foundation to prioritize interventions for enhancing the usefulness of regular mortality data.

Table 1. Criteria for the classification of mortality data from national vital registration systems.

Categories of mortality data	%
	Registration completeness	Deaths assigned a useable cause of deatha
Complete registration and high quality cause of death	95–100	80–100
Complete registration and moderate quality cause of death	95–100	60–79
Complete registration and poor quality cause of death	95–100	< 60
Functional registration and high quality cause of death	75–94	80–100
Functional registration and moderate quality cause of death	75–94	60–79
Functional registration and poor quality cause of death	75–94	< 60
Limited registration and poor quality cause of death	25–74	< 60
Nascent death registration and cause of death data	> 0 but < 25	> 0 but ≤ 10
Data not available	N/A	N/A

^a Of deaths reported for those younger than 80 years.

Note: We classified countries where the most recent data were collected before 2015 as having no data available.

Results

Of the more than 55 million deaths estimated to have occurred worldwide in 2019, 70% were captured and reported by civil registration and vital statistics systems in 156 countries. Just over half (52%) had a medically certified cause of death, while only 42% of deaths in people younger than 80 years were assigned a useable cause of death; the remainder either being attributed to a garbage cause of high, medium or low severity; or where the medically certified cause of death contained insufficient detail to assess quality (Table 2). The African Region only registers a small minority of deaths (15% of estimated total deaths in that region) and even less with a medically certified cause of death (8%). Eastern Mediterranean (50%) and South-East Asia Regions (69%) each have moderate levels of death registration, but far lower medically certified cause of death completeness (28% and 17%, respectively) and cause of death usability (15% and 3%, respectively). Other regions have higher death registration completeness, with cause of death usability being highest in the European Region at 82%.

Table 2. Completeness and quality of death registration and cause of death statistics by region; 2019.

WHO Region	%
	Deaths registered	Deaths with medically certified cause of death	Deaths with useable causea
African Region	15	8	4
Region of the Americas	95	90	77
South-East Asia Region	69	17	4
European Region	99	98	82
Eastern Mediterranean Region	50	28	17
Western Pacific Region	82	72	70
Global	70	52	42

WHO: World Health Organization.

^a Includes deaths reported for those younger than 80 years.

The distribution of all 194 WHO Member States according to the two primary measures of data usability is summarized in Table 3 and presented by country in Fig. 1 and in the online repository.¹⁴ More than half (83) of the 156 reporting countries had death registration levels of at least 95%; and of these, about 54 had systems that provided a useable cause of death for at least 80% of deaths among people younger than 80 years. Death registration systems in these 54 countries can be considered as complete, and of high quality, accounting for about one-quarter of global deaths, with most of them located in the Region of the Americas, the European Region and the five-high income countries of the Western Pacific Region (Fig. 1). A further 29 geographically diverse countries have systems that successfully capture all or most deaths; but only 60%–80% of deaths that occur are assigned a useable cause in 20 of these countries, falling to less than 60% in a further nine countries. Of the remainder, more than half (43 countries) had systems that were at least 75% complete, and in some cases close to 95% complete; but where cause of death data quality was assessed as being predominantly moderate (20 countries) or poor (17 countries).¹⁸

Table 3. Distribution of WHO Member States according to completeness, availability and quality of cause of death, 2019.

Categories of mortality data	No. of countries	Estimated % of global deaths
Complete registration and high quality cause of death	54	23
Complete registration and moderate quality cause of death	20	7
Complete registration and poor quality cause of death	9	2
Functional registration and high quality cause of death	6	< 1
Functional registration and moderate quality cause of death	20	21
Functional registration and poor quality cause of death	17	20
Limited registration and poor quality cause of death	21	11
Nascent death registration and cause of death data	9	4
Data not available	38	11
Total	194	100

WHO: World Health Organization.

Note: The definition of each category is presented in Table 1.

Fig. 1.

Estimated completeness and quality of mortality and cause of death data for WHO Member States, 2019

WHO: World Health Organization.

A geographically diverse group of 21 countries account for about one tenth of global deaths and have vital registration systems that provide limited policy benefit; typically capturing between 25%–74% of deaths but with useable fractions well below 60%. We identified data for nine countries where data completeness and quality are so low that we have classified the data systems as nascent. For 38 countries, we could find no vital registration data on mortality, or the most recent data was from before 2015. Likely, these 38 countries have some form of mortality reporting system established and operated by a government; however, the data were unidentifiable using our search strategy.

Discussion

In 2019, 70% of global deaths were reported to a government agency, suggesting a potential improvement from the 59% global death registration rate indicated by the GBD Study in 2015.¹⁹ However, less than half of deaths among individuals younger than 80 years received a cause of death usable for public health planning. While nearly half of WHO Member States provided comprehensive mortality data, in 38 countries no data could be identified from 2015 to 2019.

We found that for approximately half of deaths, the cause of death is not determined from a medically certified cause of death, and when they are, a significant portion either gets attributed to a cause of no public health utility or lacks specific details. Addressing this problem requires proper physician training to ensure accurate death certification. Proper cause of death determination would also have a positive impact on reducing misdiagnoses among deaths younger than 80 years, where the cause is useable but not necessarily correct. Several studies of diagnostic accuracy in low- and middle-income countries have reported misdiagnosis levels that are typically around 30% of useable causes; further reducing the policy value of routine vital statistics.^4,20–22

We classified countries into nine categories of data usability based on a joint assessment of completeness and quality, quantified by the fraction of garbage codes found in the data. Countries that have complete registration also tend to have higher quality cause of death data. These were mostly high-income countries, and civil registration and vital statistics improvement strategies in these populations should focus on improving diagnostic practices among physicians to reduce garbage coding, especially for deaths in individuals younger than 80 years. Most countries with functional, but still incomplete registration also have substantial challenges with cause of death data quality. Countries in this category tended to be more geographically diverse. In countries where mortality data is nascent or exhibits low completeness and poor certification quality, enhancing vital statistics systems should be a priority. These efforts should include capacity-building and training in data analysis and dissemination as key strategies for reliably monitoring progress of forthcoming global health initiatives.

The coronavirus disease 2019 (COVID-19) pandemic highlighted the poor state of death registration systems in many countries, with most low- and middle-income countries being unable to generate timely and reliable data on excess mortality during the pandemic. Globally, only 73 of 194 countries produced full national data in 2020 and 2021 to enable calculation of excess mortality; most of them high-income countries.²³ Measuring excess mortality reliably becomes challenging when registration is incomplete; distinguishing actual mortality increases from the pandemic's effects on the death registration system is a complex task.^24,25 In many countries during the pandemic, death registration systems were unable to function as normal due to the impact of movement restrictions, a reliance on paper-based registration, and as a consequence of COVID-induced mortality.²⁶ In Peru, for example, health services were overwhelmed due to a surge in deaths which meant that normal registration processes were severely disrupted; while in Ghana the operations of some civil registration offices were adversely affected by movement restrictions.^26–28 The pandemic highlighted the need for governments to have reliable mortality statistics, which resulted in increased use of online registration, which has improved efficiency with attendant longer-term benefits for improved global death registration.^27–29

Improving civil registration and vital statistics systems is a significant, organizational, technical and financial challenge for many countries. Countries seeking to improve their civil registration and vital statistics systems can benefit from the WHO Civil registration and vital statistics strategic implementation plan 2021–2025. This comprehensive platform enables countries to prioritize interventions tailored to their needs, emphasizing strengthened intersectoral collaboration and governance for vital statistics.³⁰ This plan offers many feasible options for improving death registration and diagnosis based on recent methodological research and experience with practical implementation in countries. Specifically, this cumulated experience suggests that governments can work to improve the completeness of registered deaths by focusing on more active death notification processes for deaths occurring outside hospitals; on innovative application of digitized data collection platforms; on improving the national compilation of data; and on implementing processes to assess data quality.⁶ Concurrently, there should be accelerated efforts to increase use of the International medical certificate of cause of death (with appropriate training of physicians to reduce deaths with a garbage cause), and to fill the cause of death data gap for non-hospital deaths with routine, validated automated verbal autopsy methods.^21,31 Training of physicians, including in medical school curricula, should be a priority in countries with complete registration but more than 20% of garbage codes, using the effective training methods and tools available.^21,22 The introduction of ICD-11 will present additional challenges for its implementation into death registration systems; reinforcing the need for accelerated efforts to train doctors in correct medical certification procedures for deaths; and coders in the application of ICD-11 coding rules.

Monitoring the impact of civil registration and vital systems interventions through periodic assessment of data completeness and quality can now easily be performed for subnational populations using established data quality assessment software, such as ANACONDA (co-developed by the University of Melbourne, Melbourne, Australia and the Swiss Tropical and Public Health Institute, University of Basel, Basel, Switzerland).³² The composition of the package of interventions that should be prioritized under the WHO Civil registration and vital statistics strategic implementation plan 2021–2025 will vary according to the needs and capacity of countries, but is likely to be similar for all countries in each of the categories that we have identified.³⁰

The persistent application of these strategies is likely to bring about significant improvements in the availability and policy utility of mortality statistics. However, significant improvements in completeness of death registration may take some time, possibly a decade or longer in some countries. In the interim, established demographic methods applied to child survival, sibling survival and household death data available from surveys or censuses offer an alternative means for generating essential health intelligence on levels and patterns of mortality required to inform health planning.³³ The policy utility of this information will be substantially enhanced by the application of automated verbal autopsy methods to a representative sample of deaths collected in surveys or censuses to provide cause of death data, as has been demonstrated elsewhere.³⁴

Our study has some limitations, most notably the omission of recent vital statistics that are undoubtedly being produced in several low- and middle-income countries but are not readily accessible; as well as more disaggregated analysis of data quality at a subnational level where data are likely to be most relevant for planning. Estimates of death registration completeness are also subject to accuracy in the method used, either the empirical completeness method, or GBD or world population prospects estimated deaths. We used a standardized approach to estimate completeness rather than relying on country-derived estimates that would vary by country and lack suitability, which would reduce their intercountry comparability. We also did not assess completeness of death registration during the COVID-19 pandemic, in part because data for the pandemic years were not yet widely available, but also because of the abnormal registration environment that prevailed in most countries due to national responses to the pandemic, including movement restrictions. We were also unable to assess the role of public versus private facilities in reporting deaths. There is little available information on the relative completeness or accuracy of death data from different types of facilities, or even whether they were recorded in private or public institutions.

Vital statistics on deaths and causes of death are the cornerstone of any country’s health information system, and the findings of this study should guide national and global action to improve them. Better data will allow countries to make better decisions leading to enhanced population health.