Flaws and uncertainties in pandemic global excess death calculations

Abstract

Several teams have been publishing global estimates of excess deaths during the COVID-19 pandemic. Here, we examine potential flaws and underappreciated sources of uncertainty in global excess death calculations. Adjusting for changing population age structure is essential. Otherwise, excess deaths are markedly overestimated in countries with increasingly aging populations. Adjusting for changes in other high-risk indicators, such as residence in long-term facilities, may also make a difference. Death registration is highly incomplete in most countries; completeness corrections should allow for substantial uncertainty and consider that completeness may have changed during pandemic years. Excess death estimates have high sensitivity to modeling choice. Therefore different options should be considered and the full range of results should be shown for different choices of pre-pandemic reference periods and imposed models. Any post-modeling corrections in specific countries should be guided by pre-specified rules. Modeling of all-cause mortality (ACM) in countries that have ACM data and extrapolating these models to other countries is precarious; models may lack transportability. Existing global excess death estimates underestimate the overall uncertainty that is multiplicative across diverse sources of uncertainty. Informative excess death estimates require risk stratification, including age groups and ethnic/racial strata. Data to-date suggest a death deficit among children during the pandemic and marked socioeconomic differences in deaths, widening inequalities. Finally, causal explanations require great caution in disentangling SARS-CoV-2 deaths, indirect pandemic effects, and effects from measures taken. We conclude that excess deaths have many uncertainties, but globally deaths from SARS-CoV-2 may be the minority of calculated excess deaths.

Excess deaths are a hot and debated measure. They can be calculated for groups of patients, populations, entire countries, and globally for any factors speculated to change overall mortality. Global excess death estimates have become very visible during the COVID-19 pandemic. They reflect deaths directly caused by COVID-19 (with potentially large variability across demographic factors, comorbidities, and institutionalized versus community populations) and a diverse set of causes directly or indirectly affected by the pandemic and the response to it.

Several sophisticated efforts have attempted to estimate pandemic global excess deaths. The World Health Organization (WHO), the Institute of Health Metrics and Evaluation (IHME), and the Economist have generated global estimates (1-3) with substantial differences among them (4). Here, we examine several deficiencies that threaten global excess death calculations and inferences thereof (Table 1). We also make suggestions for corrections on already published estimates and for future improvements.

Table 1.

Key issues and potential for correction or improvement in global estimates of excess deaths

Issues	Potential for correction or improvement
Adjusting for changing population structure	Detailed adjustment of all excess death calculations for narrow age bins in the countries where these are available, so as to account for fine change in population structure over time.
Adjusting for changes in other high-risk indicators	Capture and adjustment for other variables that affect mortality risk, in particular residence in long-term facilities (rates may have changed over time in various countries, with different patterns for elderly and for young residents).
Completeness corrections	Allowance for uncertainty in completeness corrections; consideration that completeness may have changed during pandemic years.
Sensitivity to modeling choice	Consideration of different options regarding choice of pre-pandemic reference period and regarding imposed models; showing full range of results rather than single spuriously accurate average or weighted average.
Post-hoc corrections in specific countries	Avoidance of post-hoc corrections that are not based on pre-specified rules; pre-specification of objective, unambiguous criteria for any required post-modeling corrections.
All-cause mortality modeling	Ensuring full transparency of model and model performance, including variance explained; exploration of transportability; acknowledgement of measurement errors and biases in included variables, consideration of alternative variables and models.
Underestimation of uncertainty	Incorporation of uncertainty from each step in the modeling and from each of the variables considered; cautious interpretation since uncertainty may still be underestimated.
Excess death estimates per risk strata	Routine provision of excess death estimates per age group and according to other major risk strata (e.g., separately for community versus long-term care resident population and per ethnic/racial strata).
Causal (mis)interpretation	Avoidance of causal statements of excess deaths attributed directly to SARS-CoV-2; consideration of direct and indirect effects of the pandemic and of the measures taken; in-depth assessment of causes and attribution will require other types of studies.

Adjusting for changing population age-structure

Expected fatalities depend crucially on changes in population age-structure over time. Many high-income countries have detailed data according to age bins that capture their changing populations over many years (5,6). Several countries exhibit large shifts in population age-structure, especially with increasing proportions of elderly people. Non-consideration of this population aging process underestimates expected deaths and overestimates excess deaths. E.g. in the USA, age-adjustment decreased excess deaths estimates for March-August 2020 by 28% (7). In 33 high-income countries with age-structure information, we previously (4) estimated 2.319 million excess deaths without age-adjustment during 2020-2021, i.e. close to WHO, IHME and Economist estimates. However, estimates shrank by 31% with age-adjustment (1.610 million). Eight of 33 countries had lower death rates during 2020-2021 versus the three pre-pandemic years (4). Another age-standardized analysis (8) on excess deaths from 1/2020 to mid-2022 in 33 European jurisdictions versus 2015-2019 also found lower death rates during the pandemic versus pre-pandemic period in 8 countries (Malta −0.7%, Switzerland −0.7%, Finland −1.7%, Denmark −2.8%, Luxembourg −3.4%, Iceland −3.9%, Sweden −4.0%, Norway −4.1%). Only 3 countries had >10% excess (Bulgaria, Poland, Romania). Median excess across all 33 locations was only 3%.

Conversely, WHO, IHME and Economist (1-3) generated much higher estimates of excess deaths in high-income countries. These inflated estimates were published in prominent venues and/or received wide media attention. Setting an erroneous precedent may influence also other studies to use similar, suboptimal methods. These evaluations (1-3) did not use proper age standardization. Instead, they employed linear or spline trends (or an ensemble of multiple such methods) to model evolving mortality patterns over several years. However, modeling trend patterns alone may be too crude to sufficiently capture the impact of granular age-structure changes. Therefore, one should age-adjust excess death calculations in countries with available data. Sex-adjustment is also readily feasible, but may have a lesser impact on the excess death calculations (relative changes in male-versus-female population over time are small).

Regularly updated census enumeration of the population even in countries with the best death registration data is essential, since there can be shifts over time (e.g. due to migration), affecting some age groups more than others.

Adjusting for changes in other high-risk indicators

Other factors, besides demographics, may help capture better the changing nature of populations over time. Adjustment for indicators of frailty, e.g. residence in long-term care facilities (LTCFs), may have substantial impact on expected deaths (9) and thus also excess death estimates. Such indicators are ignored in excess death calculations to-date. Even without changes in age structure, with more frail residents, expected deaths should increase. Information on location of death is available for many countries (10) but needs better standardization. In the USA, the proportion of deaths that happen in LTCFs had been increasing even before the pandemic and was already 26.8% in 2018 (11). It would be useful to capture also additional deaths at home or hospitals of residents of LTCFs. The extent to which pandemic circumstances may lead some people to pull family members from facilities and/or more long-term reluctance for LTCFs should be carefully examined. Ideally, one would like to adjust for population-level data on frailty (regardless of residence location) and their change over time. These data are available for some populations, e.g. frailty has been documented to have increased during the pandemic in Japan (12).

Changes in frailty indicators may affect calculations not only for elderly strata, but also younger ages. In some countries a large share of COVID-19 non-elderly deaths happened in young residents of LTCFs. E.g., in South Korea (Gyeongbuk Province), LTCF residents accounted for 31% of COVID-19 deaths in the elderly, but also 20% of COVID-19 deaths in people 30-60 years old (13). Even under routine circumstances, a substantial share of deaths in non-elderly people in high-income countries happens among frail, institutionalized patients (14). One in 6 nursing home residents in the USA (210,000 of 1,315,000) is younger than 65 and the proportion increases over time (15). While rates of nursing home placement declined sharply in 2013-2019 among the elderly, they stagnated for younger people (15). Young people with frailty are a special group (16) deserving careful consideration in fatality calculations. Their numbers and trends differ across countries. E.g. Australia has had a relatively steady population of ~6,000 young nursing home residents between 2008 and 2018, i.e. <0.03% of its non-elderly population reside in LTCFs (17), 3 times lower than the USA.

Changes in other institutionalized populations may also be considered. In particular, in the USA the incarcerated population was 2.1 million in 2018 (decreasing to 1.7 million in 2020); incarcerated people had 3-fold higher COVID-19 standardized mortality than the general population (18).

Completeness corrections

Death registration is very incomplete in most countries worldwide. E.g., in a country with 80% completeness of registration, observed deaths should be multiplied by 1/0.8=1.25-fold. Even in high-income countries, very recent data should be seen as incomplete, given that some deaths are entered late in the registration systems. Moreover, causes of death may be revised. Some missing (or late arriving) data affect even high-income countries’ statistics, while elsewhere usually 10-90% of deaths remain unregistered (19-21). Fixed-value completeness corrections have been used in the WHO excess death estimates (1). However, completeness corrections carry large uncertainties requiring plausible ranges rather than fixed values. Popular sources of data (e.g. United Nations and Global Burden of Disease) give different estimates for many countries and are often highly uncertain.

Critically, it is unknown how completeness changed during the pandemic. Perhaps completeness decreased under multifarious pandemic disruptions. Conversely, completeness may have increased: most countries improve over time, plus the pandemic intensified efforts, sensitizing authorities for better completeness and encouraging the use of online resources. E.g., in-depth analysis of death registrations in Loreto, Peru showed that during the first wave peak, registered deaths increased 7-fold, but deaths may still have been 20-30% higher than those registered (22). That first wave had both positive and negative impacts on the completeness of the death registration systems (22). Importantly, after COVID-19 waves ended, numbers of registered deaths remained at much higher levels than pre-pandemic standards; this may reflect excess deaths from non-COVID-19 causes or a persisting impact of improved death registration, e.g. wider use of online systems (22).

It is therefore essential to disentangle to what extent any persisting excess deaths in 2022-2023 and beyond worldwide are genuine or reflect spurious artifacts of improved death registration. This task is very difficult, if not impossible, for countries with poor death registration practices. One option is to simply acknowledge the data inadequacy and avoid publishing and disseminating spurious excess death calculations from countries without rigorous death registration. Alternatively, one may examine whether in these countries there are any sub-populations where data collection on deaths is far more reliable. For example, in some middle income countries, large cohort studies and biobanks have been launched and these projects may afford data-collection mechanisms that can ascertain deaths with much higher accuracy than the unreliable public, country-level systems. At a minimum, if excess death calculations are undertaken, they should show how results may vary under different corrections and different patterns of pre-pandemic versus pandemic completeness.

Sensitivity to modeling choices

Excess death calculations are highly sensitive to the choice of pre-pandemic reference period and specific model imposed on pre-pandemic data to make expected death predictions for pandemic years (23-27). Typical modeling choices include specific-average, specific-average with trend, harmonic (to consider seasonality patterns) with trend, and specific trend (23-27). Trends can also be modeled with linear, multinomial, spline functions, or other approaches. Time trend models may also differ depending on whether death data are available on weekly, monthly, or only annual level, e.g. annual-level data cannot accommodate seasonality. Calculation of trends will also vary depending on how many observations are used, i.e. how lengthy is the reference period. In theory, the reference period should be lengthy enough (covering many years) so to identify any stable and clear mortality trends. However, trends seen in lengthy reference periods reflect largely what happened many years ago. This may no longer be relevant for current and future patterns.

E.g. let us suppose that, on average, death rates in a country decreased by 2% per year in the decade 2010-2019. If we believe that the same trend should have applied to 2020-2023, then a country where the pandemic years and beyond were not any different than the last pre-pandemic year will appear to have had major excess deaths. To offer an analogy, a company that remains equally wealthy in 2023 as in 2019 would lament financial devastation, because it did not markedly increase its wealth. However, the tenant that past improvements in death rates over time would/should continue also in the future is spurious. In fact, long-term evolution in death rates shows very complex patterns (28), thus extrapolations from the past to the future are precarious.

Some modeling choices may still seem more sensible, but no unequivocal gold standard exists. Therefore, excess death analyses should present results according to different modeling choices and consider the range of results as lower uncertainty boundaries. Some choices are more sensitive to minor data anomalies than others, e.g. some splines may be more sensitive than linear interpolations (29).

A multiverse approach has been also proposed (27) where all possible modeling choices are considered for parameters of interest (e.g. reference period). Ensemble approaches where different models are combined with relative weights according to their perceived performance have been used by IHME (2). However, relative weighting is hampered by the lack of a gold standard. It is thus preferable to present the wide range of results obtained with different models, rather than force a spuriously precise single weighted value.

Post-hoc corrections in specific countries

Sometimes, after all the modeling has been completed, results for specific countries may seem weird or improbable and modelers become tempted to correct them. Post-hoc corrections not based on explicit pre-specified rules should be avoided, as they are subject to overt or covert cognitive biases. Corrections may remedy obvious data and/or modeling failures even at the last minute, but their governing rules (what will instigate a last-minute correction and how it will be done) should be pre-registered before analysis. For example, the WHO team corrected post-hoc (1) the estimates for two countries, changing the completeness corrections and modeling choices (linear versus spline). Consequently, excess death estimates shrank by 37% for Germany and increased by 19% for Sweden. Paradoxically, this manipulation was deemed necessary only for two countries that probably have some of the most reliable data worldwide.

To remedy this problem, pre-registration is both feasible and desirable (30). Pre-specified rules for what would need to be corrected at the end (and how) should be objective and unambiguous. This will diminish selective reporting and spin which have been major challenges for pandemic modeling (31,32).

All-cause mortality (ACM) modeling

Most countries around the world do not collect data even on all-cause mortality (ACM) with any level of accuracy. An approximate estimate of ACM in these countries needs to be assumed before any excess death inferences. Typically ACM is modeled in countries with data first and then the model is extrapolated to countries without data. This process is precarious. Illustratively, the modeling variables for ACM in the WHO excess death calculations (1) appear in Table 2. All of them are ecological variables, subject to ecological fallacies (relationships seen with group averages may not represent what happens to individuals). Most have substantial measurement errors not accounted for in the modeling. Most importantly, these variables have very different values and distributions in countries with ACM data than in other countries without data and, typically, errors are much larger in the latter group. For several variables such as stringency, economic measures, and containment, any relationship with ACM may be entirely different in high-income countries versus in low- or middle-income countries that lack ACM data. Therefore, transportability of the ACM model across countries is low. Model fit upon cross-validation, including the proportion of variance explained, should be reported. Estimates of measurement errors should be in-built in modeling and in extrapolation and would translate to much higher uncertainty in excess death estimates in countries without ACM data.

Table 2.

Problems with variables selected for modeling all-cause mortality

Variables considered in the WHO model for all-cause mortality (see ref. 1 for details/definitions)	Ecological variable	Substantial measurement error	Different in those with versus without data	Different error in those with versus without data
High income country	Yes	No	Yes	No
COVID-19 test positivity rate	Yes	No	Yes	Yes
COVID-19 death rate	Yes	No	Yes	Yes
Temperature	Yes	No	Yes	No
Population density	Yes	No	Possibly	No
Sociodemographic index	Yes	Yes	Yes	Yes
Human development index	Yes	Yes	Yes	Yes
Stringency	Yes	Yes	Possibly	Yes
Economic measures	Yes	Yes	Yes	Yes
Containment	Yes	Yes	Possibly	Yes
2019 non-communicable disease rate	Yes	Yes	Yes	Yes
2019 cardiovascular disease rate	Yes	Yes	Yes	Yes
2019 HIV rate	Yes	Yes	Yes	Yes
Diabetes prevalence	Yes	Yes	Yes	Yes
Life expectancy	Yes	No	Yes	No
Proportion population <15 y	Yes	No	Yes	No
Proportion population >65 y	Yes	No	Yes	No

Different efforts at global excess death calculations have used different models to impute ACM countries without ACM data. WHO used 16 variables (Table 2), Economist used 144 and IHME used 16, with limited overlap among them, different modeling, and almost ubiquitous problems of ecological bias, measurement error, limited overlap between modeling set of countries and projected set of countries and high potential for differential measurement error. None of these excess death evaluations pre-registered their model before analyses were done or explained convincingly why the chosen variables and overall models should be trusted. The task of effective modeling under such circumstances is so precarious that any agreement or disagreement between different evaluations may mostly reflect agreement or disagreement of investigator expectations.

Underestimation of uncertainty

All factors mentioned above add uncertainty to estimates of global excess deaths. The uncertainty introduced in each step of assumptions and calculations is multiplicative across steps. The ratio of upper to lower 95% bound was 1.15, 1.25, and 1.63, in the global excess death estimates for 2020-2021 according to IHME (17.1-19.6 million) (2), WHO (13.3-16.6 million) (1), and Economist (12.9-21.0 million) (3) calculations, respectively. However, the true ratios are probably much larger. None of these calculations fully accounts for all sources of uncertainty. Therefore, extra caution is needed in interpreting these estimates overall, and even more so, when subsets are considered, e.g. regional estimates, single year or seasonal fluctuations, and single country performance where uncertainty can be even larger.

Excess death estimates per risk strata

The existing reported excess death estimates are difficult to put into perspective unless broken down per age and other informative risk strata. The pandemic impact on these strata may have been very different.

High-income countries have shown death deficit (fewer deaths during the pandemic than before the pandemic) among children and adolescents, wide diversity across countries in excess deaths of non-elderly adults, and substantial diversity in excess deaths among the elderly in general and also under long-term care (4,33). Implications are major for understanding the magnitude of the impact in lost life-years without disability and why different countries may have different excess deaths profiles across risk groups. E.g. excess deaths in non-elderly in mid- and low-income countries, may reflect harms from measures taken, e.g. exacerbation of poverty, starvation and disruption of basic welfare. However, data to-date from African countries suggest a large death deficit among children during 2020-2021 (34,35) E.g. in coastal Kenya mortality decreased 20% during the pandemic versus pre-pandemic levels for children 1-14 years old and even decreased 5% for 15-44 years old people (34). If this pattern is widespread, global lost life-years are far more limited than what crude pandemic excess death counts suggest. According to the WHO estimate, of the 14.9 million excess deaths in 2020-2021, the 0-39 years old stratum accounted for only 5,393 (36), which is a miniscule 0.036 percent. Other authors have also pointed out that estimates of life years lost from COVID-19 are intrinsically grossly exaggerated, because age and comorbidities information has not been fully accounted (37).

Ethnic and racial group stratification of estimates is also essential. The pandemic effected disproportionately minorities and underprivileged populations, widening inequalities (38,39). In the USA in 2020, COVID-19 death rate was 27.4-times higher for low-socioeconomic profile Hispanic men vs. high-socioeconomic profile white women (40).

Causal (mis)interpretation of excess deaths

Causal interpretation of excess death estimates is precarious. Published global assessments (1-3) acknowledge typically that excess deaths include both direct and indirect pandemic effects. However, excess deaths also include direct and indirect effects of measures taken to deal with the pandemic (41). Disentangling causes requires far more granular data than overall excess deaths. Causal language claiming that SARS-CoV-2 infection was responsible for most excess deaths should be avoided. Even for influenza, attempts to estimate its death toll through seasonal excess death estimates yields results that differ 2- to 35-fold, depending on whether the excess is estimated for all-cause mortality, respiratory and cardiovascular mortality, or pneumonia/influenza mortality (42). Excess death calculations are too crude and unreliable a tool to estimate deaths caused directly by a respiratory infection. Confounding between various respiratory diseases, including diverse viral pathogens, pneumonia, and acute exacerbations of chronic obstructive pulmonary disease is common and leads to difficulties in exact cause attribution.

Both under- and over-counting of SARS-CoV-2 deaths have occurred in different locations and periods (43). In some high-income countries, under-counting may have occurred during the first wave due to limited testing (43). However, in later phases over-counting was prominent, e.g. eventually over-counting accounted for 40% of recorded COVID-19 deaths in Finland (44). During 2022, 65-75% of recorded COVID-19 deaths in Denmark were not caused by SARS-CoV-2 infection (45). Chronic and acute dysfunctions in health systems (46) can be exacerbated by pandemic responses resulting in extra fatalities. Unemployment, bankruptcies, poverty, and lack of health insurance and health access may promote fatal outcomes from diverse causes in countries as diverse as USA and Peru (47,48).

When excess deaths far outnumber recorded COVID-19 deaths, the excess is probably mostly not by missed lethal SARS-CoV-2 infections. Illustratively, India has the largest absolute difference between reported COVID-19 deaths in 2020-2021 (n=481,080 per JHU) and excess death calculations (e.g. n=4,735,940 per WHO). Per population base, when compared (Table 3) with USA and Germany, excess deaths in India were substantially higher. The difference becomes even more prominent when only community-dwelling populations are considered (excluding deaths of LTCFs’ residents) and when excess deaths from drug overdose (a major escalating problem in the USA) (49) are also excluded. The residual mortality impact (as population percentage) appears 2-3 times higher in India (0.34%) than in USA (0.15%) or Germany (0.12%). Considering population age-structures and age-stratified infection fatality rate (IFR) estimates (50,51), the overall IFR in India should be 2.5-4 times lower than in USA/Germany – probably 3-8 times lower after accounting also for differences in obesity and immunosuppression prevalence (Table 3, Appendix). Hence, one would expect 3-8-fold lower community population mortality impact in India, not 2-3-fold higher. Correction for vaccination coverage among the elderly in the Delta wave (~35% in India versus ~70% in USA/Germany) or relative lack of effective treatments in India cannot explain most of the paradox. Therefore, if excess deaths for India were indeed that high, most deaths were caused by indirect pandemic effects and measures taken rather than by SARS-CoV-2 infections.

Table 3.

Comparison of USA, Germany, and India

	USA	Germany	India
Excess death estimates
WHO estimate (absolute count)	932,458	122,432	4,735,940
Per million population	0.28%	0.15%	0.34%
Per million non-institutionalized population (excluding COVID-19 deaths in LTCFs)*	0.18%	0.12%	0.34%
Excluding also drug overdose**	0.15%	0.12%	0.34%
Population structure and risk factors ***
Population >65 years	17%	22%	7%
Population >80 years among males	3%	6%	1%
Population >80 years among females	5%	9%	1%
Obesity in adults	42.7%	19%	5.5%
Immunocompromised among 18-64 year old	6.2%	Similar percentage to USA	Much lower percentage

^*The proportion of deaths in residents of long-term facilities among reported COVID-19 deaths in 2020-2021 is estimated at 34% for the USA and 22% for Germany (32), while it is likely to be negligible in India where long-term care facilities are very rare.

^**With an evolving an escalating drug overdose epidemic in the USA, the number of overdose deaths was preliminarily estimated at 201,277 in 2020-2021 (46) approximately 80,000 more than the average of 2015-2019 (ranging from 52,804 in 2015 to 70,630 in 2019) and this may be an underestimate. The impact of drug overdose excess in the pandemic years is considered here as negligible for Germany and India.

^***Percentage population in age and sex groups are derived from World Bank, percentage of obesity per country is from the Global Obesity Observatory (https://data.worldobesity.org/tables/prevalence-of-adult-overweight-obesity-2/) and number of immunocompromised in the USA is estimated at 15 million and 6.2% among people 18-64 years old (discussion in https://coronavirus.jhu.edu/vaccines/blog/immunocompromised-people-are-vulnerable-to-covid-19-we-owe-them-some-answers).

Focused efforts should estimate excess deaths from various non-COVID-19 causes during 2020-2023 and beyond, e.g. from disruption of healthcare services regarding acute cardiovascular disease (52), chronic diseases such as cancer (53), reversal in progress on infectious diseases such as malaria (54) or tuberculosis (55), hunger/starvation (56), mental health deterioration (57), drug overdose epidemics (49), and diverse worsening circumstances (58). The quality of relevant data varies across countries and extrapolations require great caution. For most countries, including the two most populous (India and China (9,59)), residual uncertainty is likely to be large.

Overall picture in countries with and without reliable death registration

Figures 1 and 2 illustrate the composite effects of diverse issues on the comparative performance of countries with and without reliable death registration. Even in countries with reliable overall death registration (Figure 1), information on death causes, in particular COVID-19 deaths, varies substantially in accuracy and comparisons may not be reliable. E.g., Netherlands recorded fewer COVID-19 deaths in 2020-2022 than Germany and Sweden, but many elderly deaths in the Netherlands 2020 were not recorded as COVID-19 (60), while similar deaths were registered as COVID-19 in Germany and Sweden. Excess death calculations by the Economist (3), show higher excess deaths in Germany and Netherlands than in Sweden, while Sweden recorded more COVID-19 deaths. The Economist excess death estimates are probably inflated due to modeling choices (27) and non-consideration of changes in age structure over time and of age-adjustments. With age-adjustment using the methodology described in (4), excess deaths estimates for Germany and Netherlands are less than half of the Economist’s (and lower than recorded COVID-19 deaths), while for Sweden it is estimated there were fewer deaths in 2020-2022 versus 2017-2019 (“death deficit”). In western European countries, COVID-19 deaths were typically over-counted (43-45,61). Moreover, Sweden did have many deaths in nursing homes in early 2020, but life expectancy in Swedish nursing home residents is very short; therefore, these COVID-19 deaths would not contribute excess deaths when 3 years (2020-2022) are considered (62). Conversely, USA excess death estimates are consistently the highest with the two methodologies, and outnumber recorded COVID-19 deaths. As discussed above, USA probably also over-counted COVID-19 deaths (63), but had extremely high non-COVID-19 deaths.

Figure 1.

Cumulative recorded COVID-19 deaths, excess death estimates according to the Economist, and excess death estimates with age-adjustment (using 2017-1019 as reference) for the 3 years’ period 2020-2022 in 4 countries with reliable death registration systems. See references (3) and (4) for the methods underlying the excess death calculations. Data are presented as deaths per 100 people in the general population. For Sweden, the age-adjusted excess death calculation shows a death deficit during 2020-2022 (fewer deaths than 2017-2019 after age-adjustment). The total deaths (from all causes) in 2020-2022 were as follows: 3.1% of the population died in the USA, 3.7% in Germany, 3.0% in Netherlands, 2.7% in Sweden.

Figure 2.

Cumulative recorded COVID-19 deaths, excess death estimates according to the Economist, and expected COVID-19 deaths for the period from January 2020 to end of March 2023 in 4 countries without reliable death registration systems. See reference (3) for the methods underlying the excess death calculations. The expected COVID-19 deaths are a rough, speculative estimate; their calculation assumes that almost everyone has been infected and the overall population infection fatality rate was 0.1% for India and Indonesia, 0.05% for Kenya, and 0.02% for China. See Appendix for illustrative calculations of infection fatality rate and other considerations in India. The population of Indonesia has a similar age structure to India. Median age (proportion of population over 65 years) is 28 years (6.8%) in India versus 29 years (6.9%) in Indonesia. Median age is 19 years (2.8% above 65 years of age) in Kenya thus population-level infection fatality rate was probably half (or less) compared with India/Indonesia). In these 3 countries, probably most people were infected before any vaccination, thus age-stratified infection fatality rates for pre-vaccination era (50,51) may apply to most people (IFR may be lower for those vaccinated before being infected). For China, see reference 9 for plausible estimates of COVID-19 deaths during the massive Omicron wave following removal of zero COVID policy measures All mortality estimates in these 4 countries carry large uncertainty.

In countries without reliable death registration (Figure 2) like India, China, Indonesia, and Kenya, recorded COVID-19 deaths are low and probably substantially undercounted. Excess death calculations in such countries are precarious. In the Economist calculations (3), excess deaths outnumbered COVID-19 recorded deaths by 10-20-fold in these countries. However, given their population age structures, expected COVID-19 deaths (even with everyone infected) should probably outnumber recorded COVID-19 deaths by only 2-5-fold. Under-ascertainment of infections was extreme (64), but under-ascertainment of COVID-19 deaths is more limited (43). If so, then the vast majority of excess deaths in these countries reflects non-COVID-19 deaths. Excess non-COVID-19 deaths may even exceed total excess deaths, especially in younger age strata, if COVID-19 deaths were fewer compared with typical influenza deaths in these countries (65-67).

Conclusions

Table 1 summarizes possibilities to correct existing global excess death calculations and to improve future calculation efforts. While excess deaths are an interesting, all-encompassing concept, their calculation and interpretation is fraught with difficulties.

Uncertainty is large even in high-income countries with best quality data and is notorious elsewhere. Presented estimates from different countries should come with explicit caveats. Only few high-income countries with rigorous data have uncertainty <6%, e.g. a 5% excess death estimate suggests a true value within 2% and 8%.

One may even argue whether excess death calculations should even be attempted at all in countries with unreliable data. Instead efforts may need to focus on settings with reliable data (e.g. cohorts with intensive data collection and meticulous death ascertainment), whenever they exist. National-level priorities should concentrate on improving death registration first.

For both countries with and without reliable overall death registration, causes of death information has deficiencies. Difficulties with death certificate accuracy have been long-standing (68,69) and require special commitment for improved fidelity during a pandemic crisis (70). Not only COVID-19 deaths, but also deaths from several other causes (e.g. drug overdose, suicide, etc.) may involve sensitive information; different circumstances may affect their documentation. Use of varying definitions for COVID-19 deaths and variable levels of stringency for documentation can lead to large under- or over-counting of COVID-19 deaths and make comparisons between countries very difficult. Political choices in some countries may make the situation worse if pressure or incentives are applied to generate data that support specific narratives. Reported COVID-19 deaths can be weaponized to prove success or failure of pandemic response measures. In some cases, different stakeholders may endorse extremely different estimates of COVID-19 deaths (9,59) and different interpretations of what happened during the crisis. It is essential to safeguard information on key vital statistics from political, media and other non-scientific interference.

None of the widely publicized global estimations of pandemic excess deaths to-date (1-3) has seriously addressed these issues and efforts to disentangle SARS-CoV-2 deaths from other causes of excess deaths are still at an early stage. Therefore, there is large room for improvement in calculating and interpreting global excess death estimates. Misleading estimates have the potential for unintended adverse consequences by misguiding clinical practice, funding, health policy, and public health.

APPENDIX

The table below shows the percentage of the population of the USA, Germany and India in different age brackets (data from www.populationpyramid.net for 2020), the pre-vaccination infection fatality rate (IFR) for each age bracket (based on data from mostly western country populations for community-dwelling individuals (50)) and the calculation (sum of columns c) of the overall population IFR for populations with the age structure of the USA (0.26%), Germany (0.41%) and India (0.11%) without accounting for differences in risk factors of COVID-19 death such as prevalence of obesity and immunosuppression. Given that obesity and immunosuppression are far less frequent in India than in USA and Germany, population-wide IFR in India may be much lower than 0.11%. Therefore, in unadjusted analyses India has 2.5-4-fold lower IFR than the prototypical age structure of USA or Germany, but with adjustment for risk factors, IFR may be 3-8-fold lower. In the percentages of the populations of each country, long-term care residents have been subtracted. IFR estimates in each age stratum are derived from reference (50) up to age 69, while for each decade in older ages the IFR is assumed to increase 2.2-fold which is lower than the ~3-4-fold increase per decade in the non-elderly age strata, but probably reasonable given that institutionalized populations (that carry much higher IFR) are excluded. Estimates should be seen with caution, given the need to make several assumptions.

	Population pyramids in 2020
Age group	USA (%)	Germany (%)	India (%)	IFR	USA c	Germany c	India c
0-19	25	18.5	35.4	0.0003	0.000075	0.0000555	0.000106
20-29	13.4	11.5	17.6	0.002	0.000268	0.00023	0.000352
30-39	13.6	12.8	15.4	0.011	0.001496	0.001408	0.001694
40-49	12.4	12.2	12.2	0.035	0.00434	0.00427	0.00427
50-59	13	16.1	9.2	0.123	0.01599	0.019803	0.011316
60-69	11.8	12.8	6.4	0.506	0.059708	0.064768	0.032384
70-79	6.7	8.6	2.9	1.113	0.074571	0.095718	0.032277
80-89	3	7	1	2.449	0.07347	0.17143	0.02449
90 and over	0.5	1	0.1	5.388	0.02694	0.05388	0.005388
					0.256858	0.4115625	0.112277

Data statement:

All data are in the manuscript