Assessment of the Quality of Cause-of-Death Data in Serbia for 2005–2019 Vital Statistics Performance Index Estimation

Abstract

Background:

We aimed to evaluate the quality of the cause of death (COD) concerning mortality patterns and completeness of death registration to identify areas for improvement in Serbia.

Methods:

COD data collected from the mortality register in Serbia from 2005 to 2019 (1540615 deaths) were analyzed with the software Analysis of National Causes of Death for Action. The Vital Statistics Performance Index for Quality (VSPI(Q)) is estimated for the overall COD data quality.

Results:

The completeness of death certification was higher than 98%. Usable underlying COD was registered in 57%, 24.1% with an unusable and 18.6% with insufficiently specified COD. The VSPI(Q) was 67.2%, denoting medium quality. The typical error was using intermediate COD (24.7% of all deaths), while 13.2% and 8.5% of all garbage codes (GC) belonged to the Very High and High Severity classes. The leading underlying COD is unspecified cardiomyopathy. The analysis revealed that 39.1% of GC has been redistributed to non-communicable diseases, 2.5% to external causes and 1.1% to communicable diseases.

Conclusion:

In the 15 years’ worth of data analyzed, the true underlying COD, in many cases, was ill-defined, indicating that COD data at the national level could be distorted. The additional and continuous professional education of medical students as well as physicians is needed. It should focus on the most common GC among the leading COD and acquiring skills in certifying external causes of death.

Introduction

Analysis of the cause of death data (COD) is essential for monitoring the health situation at the national level to select priorities for better policy decisions about health interventions (1, 2). The civil register and vital data systems (CRVS) have a crucial role in generating data for responsible healthcare system governance, the health of the population, and the monitoring of sustainable development goals. The accuracy of COD reports is crucial for disease prevention, suppression of epidemics, and adequate response to communicable diseases (1, 3). However, the quality of COD data is far from expected and accurate (1, 4–6).

COD databases are usually characterized by full completeness but insufficient quality (5). The completeness of death reporting represents how well the data indicate the population they are supposed to describe, which refers to the percentage of deaths with a medically certified COD. The low quality of COD leads to considerably misrepresented mortality data and decreases their usefulness for public health. Murray and Lopez announced garbage code (GC) terms in the literature for codes and diagnoses that are not useful for public health analysis of mortality data as part of the assessment of the Global Burden of Disease (GBD) studies (7). These codes are not specific enough, are an immediate or intermediate COD, or impossible COD, and therefore reduce the utility of mortality statistics, hindering their significance as an initial source of information for planning and assessing health actions and interventions (8).

Serbia was ranked among 46 countries with adequate death registration systems capable of producing data of sufficient quality for public health, research, and planning purposes (5). In this analysis, the overall data quality rating was three out of five stars for Serbia, with well-certified deaths at 43.7% from 1980–2016 and well-certified deaths at 76% from 1995–2016. From 2005 onwards, the death registration system was upgraded by including coders from Public Health institutes in Serbia. The GBD Study ranked Serbia as having a well-performing vital statistics system assessed based on data from 2012 (5, 9). The Vital Statistics Performance Index for Quality (VSPI(Q)), as an improved assessment concept, measures completeness, the proportion of GC deaths, the rate of deceased without age or sex detail, and the timeliness of data reporting (5, 10), indicating which elements of the mortality registration system most need improvement. Additionally, more detailed and updated research on mortality data is required to profile COD errors and create precise national patterns.

Therefore, the aim of this study was to evaluate the quality of COD data in the Republic of Serbia, analyzing the completeness and quality of the Serbian mortality registry with a subsequent estimation of the VSPI(Q) from 2005 to 2019.

Methods

We conducted a cross-sectional study using mortality databases collected by the Statistical Office of the Republic of Serbia from 2005 to 2019, with 1540615 registered deaths. The period from 2005 is included since coders have been involved in the death registration system using the ICD-10 rules (11). The analysis did not include the last three years (2020 to 2022), primarily to prevent mortality fluctuations due to COVID-19 infection.

The Ethics Committee of the Medical Faculty, University of Niš (No 12-6647-2/2 from 21.06.2022.) approved the study.

The analysis presented in the study uses Analysis of Causes of National Deaths for Action (ANACONDA) version 5.0.0 (10). ANACONDA compares vital statistics data at the national level with the GBD study estimates for a specific country or region (Eastern Europe for Serbia) (4). Input data must be sorted by ICD-10 codes, divided into age groups of 5 years, except for deaths under the age of 5, divided into those younger than one year and 1–4 years (12).

Completeness represents the percentage of deaths with a medically certified cause of death. ANACONDA practices two methods to assess completeness: first, using Generalized Growth Balance, Synthetic Extinct Generations, or a combination of both (13), and the second based on an empirical model of the association between the observed crude death rate and that which would have been predicted given the population ageing, completeness of child death registration, and the level of child mortality (14).

An indicator of the quality of COD is the rate of GC. The distribution of GC was estimated based on the typology proposed by Murray and Lopez (13) in the following five categories: “Symptoms, signs, and ill-defined conditions,” “Impossible as an underlying cause of death”, “Intermediate causes of death”, “Immediate causes of death,”, “Insufficiently specified causes within ICD Chapters,”.

GC were analyzed based on levels of severity (15) into the following four groups: Very High (Level 1) - severe impact on the structure of mortality patterns in the population; High (Level 2) - a substantial impact; Medium (Level 3) - significant impact; Low (Level 4) - limited impact. GC reported among the top 20 leading COD are highlighted in red (Very High, High, and Medium severity group) and orange (Low severity).

Redistribution in ANACONDA is the process of reallocating GC to plausible underlying COD, and it is identical to the GBD study, based on clinical observations and opinions about probable misdiagnoses, statistical algorithms and proportional redistribution according to reported COD patterns (16).

The VSPI(Q) represents a summary index of overall CRVS system performance and considers the essential components of quality: Completeness of death registration, Quality of age and sex reporting, biologically plausible underlying COD, Quality of cause of death reporting (amount of GC), The level of cause-specific detail included in the input data (amount of detail in the cause of death list used for tabulation) (5, 9). The VSPI(Q) (in the range 0–100%) is categorised as follows: “very low (<25%)”, “low (25%–49%)”, “medium (50%–69%)”, “high (70%–84%)”, and “very high (85%–100%)”.

Results

Table 1 shows that almost half of all deaths in Serbia between 2005 and 2019 represent GC. The most frequent are ‘Category 3: “Intermediate causes of death” and Category 5: “Insufficiently specified causes within ICD-10 chapters”.

Table 1:

Garbage codes by typology, severity, and leading packages, Serbia, 2005–2019

GC typology	Number of deaths with GC	% of total causes	Total GC (%)
1: “Symptoms, signs, and ill-defined conditions.”	66 344	4.3	10.1
2: “Impossible as underlying causes of death.”	15 817	1.0	2.4
3: “Intermediate causes of death.”	380 057	24.7	57.7
4: “Immediate causes of death.”	39 578	2.6	6.0
5: “Insufficiently specified causes within ICD-10 chapters.”	156 949	10.2	23.8
Total	658 745	42.8	100.0
GC severity Leading packages	Number of deaths with GC	% of total causes	Total GC (%)
Very High	202 973	13.2	30.8
Left heart failure
Impossible cause of death
Cardiac Arrest and Shock
High	131 614	8.5	20.0
Hypertension
External causes UDI, type unspecified
Atherosclerosis
Medium	37 091	2.4	5.6
Unspecified site cancers
Flutter and fibrillation
Abdomen and pelvis cancer
Low	278 067	18.6	43.6
Unspecified cardiomyopathy
Unspecified stroke
Unspecified lower respiratory infectious
Diabetes, unspecified type
Total	658 745	42.8	100.0

The Serbian COD registers during 2005–2019 were almost complete for males (98.8%) and females (98.3%). The crude death rates for males and females in Serbia were 14.8 and 13.1 per 1000, respectively.

Useable underlying COD for public health policy was coded for 57.2% of imputed data. According to the three GBD broad cause-of-death groups, 55.7% were due to non-communicable diseases, 0.7% of deaths were due to communicable diseases, and 0.8% were due to external causes. GC and insufficiently specified causes with limited impact were represented for 24.2% and 18.5%, respectively (Table 2).

Table 2:

Top 20 causes of death, including unusable and insufficiently specified codes, females and males, Serbia, 2005–2019

	Males			Females
Rank	ICD code	Name of category	Deaths (%)	ICD code	Name of category	Deaths (%)
1	I42.9	Cardiomyopathy, unspecified	6.5	I42.9	Cardiomyopathy, unspecified	9.6
2	C34.9	Bronchus or lung, unspecified	5.4	I63.9	Cerebral infarction, unspecified	5.7
3	I21.9	Acute myocardial infarction, unspecified	4.7	I42.0	Dilated cardiomyopathy	5.5
4	I63.9	Cerebral infarction, unspecified	4.3	I21.9	Acute myocardial infarction, unspecified	3.0
5	I42.0	Dilated cardiomyopathy	4.0	I25.5	Ischemic cardiomyopathy	2.9
6	I25.5	Ischemic cardiomyopathy	2.4	C50.9	Breast, unspecified	2.8
7	I50.9	Hearth failure, unspecified	1.8	I50.9	Hearth failure, unspecified	2.3
8	J44.9	Chronic obstructive pulmonary disease, unspecified	1.7	I10	Essential hypertension	2.2
9	I10	Essential hypertension	1.4	C34.9	Bronchus or lung, unspecified	2.0
10	I46.9	Cardiac arrest, unspecified	1.4	I70.9	Generalized and unspecified atherosclerosis	2.0
11	I50.0	Congestive heart failure	1.4	I50.0	Congestive heart failure	1.7
12	I46.1	Sudden cardiac death, so described	1.2	I11.0	Hypertensive heart disease	1.5
13	R96.1	Death occurring less than 24 hours from onset of symptoms, not otherwise explained	1.2	I46.9	Cardiac arrest, unspecified	1.4
14	I70.9	Generalized and unspecified atherosclerosis	1.1	I63.3	Cerebral infarction due to thrombosis of cerebral arteries	1.1
15	C34.0	Main bronchus	1.0	J44.9	Chronic obstructive pulmonary disease, unspecified	1.0
16	I11.0	Hypertensive heart disease	1.0	R96.1	Death occurring less than 24 hours from onset of symptoms, not otherwise explained	1.0
17	C16.9	Stomach, unspecified	0.9	I67.8	Another specified cerebrovascular disease	1.0
18	I21.0	Acute transmural myocardial infarction of the anterior wall	0.9	I46.1	Sudden cardiac death, so described	0.9
19	N18.9	Chronic renal failure, unspecified	0.9	N18.9	Chronic renal failure, unspecified	0.8
20	I63.3	Cerebral infarction due to thrombosis of cerebral arteries	0.8	I61.9	Intracerebral haemorrhage, unspecified	0.8
sum			44.0			49.2

The proportion of deaths according to the GBD groups and deaths coded to a GC and insufficiently specified cause-by-age categories appears in Fig. 1. Notably, GCs are present in all age groups. The proportion of unusable codes is more than a third (range 34.3–44.6%) in the age groups from one to fourteen years and more than 50% (range 49.1–62.6%) in the age groups from 15 to 34 years.

Fig. 1:

Distribution of deaths by age groups and broad causes of death groups, including unusable and insufficiently specified causes, Serbia, 2005–2019

Figure 2 shows distribution and redistribution of input data by broad GBD groups. The ratio of the non-communicable diseases group to the communicable disease group has changed from 75.6% to 52.3% after redistribution.

Fig. 2:

Distribution and redistribution of input data by broad GBD groups

The VSPI(Q) score for Serbia’s 2005–2019 input data denoting medium quality. ‘Quality of age and sex reporting’ and ‘Biologically plausible COD’ have the highest scores (100%), ‘Level of cause of specific details’ was 96%, while ‘Quality of death reporting’ was 70.6%. The performed analysis revealed that priority intervention areas are the ‘Quality of cause of death reporting’ and the ‘Level of cause of specific details’.

Discussion

Our research is the first extensive evaluation of the COD data quality in Serbia during 2005–2019. The detailed assessment in this study revealed that Serbia’s mortality data ranked as medium quality between 2005 and 2019, with a VSPI(Q) score of 67.2%, very close to the lower limit of the high category of 70% (5), which indicates that Serbia can achieve a high level of quality, very soon. The completeness of death reporting was almost 99%. Over the last fifteen years, the coverage and quality of age and sex for certified deaths have constantly been completed, representing mortality statistics generally timely and publicly available, which is the merit of regulated legislation.

However, in our country, the most significant issue regarding data quality characterizes the high distribution of GCs. The distribution of deaths in the three major GBD groups will be biased if the amount of GC is more than 10%, and the data will not illustrate the actual health status of the population (17, 18). In Serbia, nearly half of the deaths were either GC (24.2%) or insufficiently specified codes (18.5%), indicating a significant field for data quality improvement. According to typology, over half a million deaths (32.6%) were unusable coded (group 1–4) in 15 years. Additionally, analysis of the five-category classification revealed that using intermediate causes (24.7% deaths) was the most common error. Compared to other studies, intermediate COD was the second most frequent error in five high-income countries (1), Greenland (17) and Tanzania (19).

Serbia has an unfavorably higher distribution of GC compared to high-income countries in recent studies (1, 2) but a favorable higher rate of Low severity class. A substantial percentage of deaths in Serbia are attributed to High-impact GC (13.2%), significantly influencing the quality of mortality statistics and health policy. A considerable proportion of these diagnoses directly indicate certification errors, insufficient medical resources, poor death coding, and possible deficit of medical care, especially among the economically under-privileged population, as established in a Brazilian study (20). The insignificant decrease in high-impact GC reported across West Europe (8) demonstrates the difficulty in dealing with these codes and the lack of organized effort to reduce their occurrence, making them the most difficult GCs to redistribute. Almost half of the unusable codes (43.6%) belong to Level 4 and are considered to have a low impact because they are not incorrect, just lacking specificity. Although this category does not have the same influence on quality as the other three, its lack of specificity limits the COD data’s value for planning and evaluating public health activities.

Certification problems could be culture-specific (2, 20, 21). The analysis of leading COD revealed that both genders were saturated with useless or low informative conditions in mortality analysis. When GC frequently occurs among the top 10 to 20 causes of death with higher ranked; they notably decrease the management value of the data by minimizing the real impact of other dominant death causes. In Serbia, five top-ten leading COD in both genders are assigned as GC, with 6.5% and 9.6% of all deaths in males and females certified as due to “Cardiomyopathy, unspecified”. This code is assigned to the fourth level, which gives adequate information to guide public health interventions, yet not for research and technology progress. Our results show similarly, as in other medium-income (Argentina, Iran, South Africa, Tunisia) and high-income countries (Japan, France) (2), that 3 out of 10 leading deaths are high-impact GC, both in males and females, offering no valuable data for health policy direction. Many deaths from these causes suggest poor COD data quality and might indicate insufficient medical resources and physicians’ attention. Conveniently, a few ICD codes are responsible for most of the GC in our case, especially from Chapter Diseases of the circulatory system. Hence, training physicians to avoid a small chunk of ICD codes, such as unspecified heart failure and unspecified cardiomyopathy, Primary hypertension, and Cardiac arrest, could rapidly reduce the frequency of GC by 10–15%.

More than half of the deaths aged 15 to 30 belonged to GC and insufficiently specified causes. This unexpected, very high distribution of ill-defined COD among the young population is inconsistent with other data (1, 17, 21). Considering that deaths in these age categories are often entirely prevented, the strategy must be based on accurate and precise COD data. Expecting that deaths at young ages are primarily the result of external causes such as traffic accidents or victims, we conclude that deaths due to external causes have systematically been underreported in the last 15 years in Serbia. Globally, the expected frequency of external causes accounts for approximately 5 to 10%, regardless of whether the country is low-, middle-, or high-income (22). Our findings reveal that less than 1% of all deaths were coded as external causes. The data analysis reported that 4.2% of GC codes used from Chapter XIX (Injury and other consequences of external causes) were GC since physicians coded traumas as the intermediate cause instead of identifying underlying COD (Chapter XX-External causes) (data are not shown in Results). Consecutively, the structure of the external cause of death is distorted.

Lack of knowledge and skills in adequately classifying the order of events leading to death and their awareness of the underlying COD affects the accuracy of the death certification. Therefore, additional research is necessary to assess why more GC are assigned to Serbia’s younger population. Previous studies (2, 17, 23) and our research imply that the main reason for the considerable prevalence of GC is the inadequacy of awareness and acknowledgement of the significance among certifying physicians of the public health worth of correctly certified COD. This gap can be addressed by strategies such as hospital committees regularly monitoring the quality assessment of causes of death (23). The second reason is the lack of standardized medical certifiers’ instructions on accurately completing the death certificate. Even high-income countries have declared a lack of standardized instructions for medical certifiers at the national level (1, 17). The efficacy of CRVS systems could be improved by simple organizational and technical decisions to improve clarity and COD details reported for deaths already captured by the system. Also, applying an automated coding system, like Iris software in Western European countries (24), could significantly improve coding practice. A limitation of our study is that we analyzed mortality data from the registration system but not primary data (medical records) with accompanying analyses and procedures that antedated death. The second limitation could be that a sub-national assessment has not been undertaken that would possibly point to differences in quality by region.

Conclusion

The completeness of the data is satisfactory, but the data quality could be improved, as concluded based on the medium-category VSPI(Q) score in Serbia. The finding of the high prevalence of unusable codes should raise awareness of the error pattern and its influence on the mortality data. Our research implies the following country error pattern: frequently using GC, commonly of intermediate COD, and codes with high severity among leading COD. Therefore, the additional and continuous professional education of medical students and physicians is needed. It should focus on the most common GC among the leading COD and acquiring skills in certifying external causes of death. Reduction of GC can lead Serbia to derive maximum health policy benefits from mortality data.

Journalism Ethics considerations

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.