Excess death estimates from multiverse analysis in 2009–2021

Abstract

Excess death estimates have great value in public health, but they can be sensitive to analytical choices. Here we propose a multiverse analysis approach that considers all possible different time periods for defining the reference baseline and a range of 1 to 4 years for the projected time period for which excess deaths are calculated. We used data from the Human Mortality Database on 33 countries with detailed age-stratified death information on an annual basis during the period 2009–2021. The use of different time periods for reference baseline led to large variability in the absolute magnitude of the exact excess death estimates. However, the relative ranking of different countries compared to others for specific years remained largely unaltered. Averaging across all possible analyses, distinct time patterns were discerned across different countries. Countries had declines between 2009 and 2019, but the steepness of the decline varied markedly. There were also large differences across countries on whether the COVID-19 pandemic years 2020–2021 resulted in an increase of excess deaths and by how much. Consideration of longer projected time windows resulted in substantial shrinking of the excess deaths in many, but not all countries. Multiverse analysis of excess deaths over long periods of interest can offer a more unbiased approach to understand comparative mortality trends across different countries, the range of uncertainty around estimates, and the nature of observed mortality peaks.

INTRODUCTION

Calculation of excess deaths is considered to be a very useful tool for estimating patterns of mortality changes over time in different countries and the impact of major events, such as pandemics (1–3). Excess deaths are meant to capture the composite sum of perturbations in disease incidence and other factors, including social, health care, lifestyle and natural catastrophes that may shape population fatalities in a given year. However, excess death calculations can lead to controversy with different teams of researchers generating markedly different estimates for the same country and year(s) (4–6). The reason is that the calculation of excess deaths requires making analytical choices for which there is no consensus. Specifically, one needs to select a reference baseline period (a time window in the past that will be used for extrapolating how many deaths would be expected in subsequent years) and a projected period (the time window for which an excess death estimate is made by comparing the observed versus expected number of deaths based on the past (reference) experience). Moreover, one should decide whether there are any time patterns and what is the form of these time patterns (e.g. whether overall mortality should be declining or increasing over time and, if so, in what form, e.g. linear or spline fit). Empirical work and simulations (4–10) have shown that these choices can make a substantial difference in the obtained excess death estimates.

When results depend on analytical choices, one methodological strategy is to explore the full range of results that can be obtained when a wide range of possible analytical choices and combinations thereof are considered (11–20). Analyses may range from a few dozen to several million different options (e.g. in selecting covariate sets in regressions) (15,17). Different terminology has been used for such approaches that generalize the concept of sensitivity analysis. Commonly used terms are “multiverse analysis” (11–14), “vibration of effects” (16–18) and “multi-analyst analysis” (19,20) (when multiple researchers are each asked to select independently their preferred analysis). Here, we propose a multiverse approach for excess deaths. Instead of making unavoidably arbitrary choices in selecting reference baseline and projected periods, we consider all possible reference baseline periods and projected periods in consecutive year time windows during a lengthy period of interest. Instead of prespecifying time patterns, this multiverse approach allows the data to demonstrate what might be the time patterns and how sensitive the results are to different analytical choices. All possible choices are considered for reference baseline periods. The multiverse approach also allows us to understand to what extent excess death estimates may shrink when longer projected periods are considered, in the range of 1–4 years. If perturbations lead to excess deaths increases due to the demise of individuals with limited life expectancy (21), then excess death peaks that are seen with short projected periods (e.g. 1 year) will diminish or even disappear when longer projected periods are considered. People who died at some point due to the perturbation would have died very soon anyhow. Conversely, if perturbations result in mortality peaks due to deaths of people who had long life expectancy, extending the projected period window will not have the same impact.

We applied this approach to 33 high-income countries that have the most reliable data for mortality according to age-stratified groups for the extended period 2009–2021. This allowed us to examine whether there are consistent time patterns in different countries, how sensitive the results are to analytical choices, and whether larger projected period windows, especially in the COVID-19 pandemic and the years preceding it, shrink substantially any observed mortality peaks.

RESULTS

Variability of excess death estimates according to reference baselines

The absolute value of excess death estimates can vary a lot depending on the selection of reference years used for baseline. We considered all 66 possible time windows of whole consecutive calendar years (1 to 11 years long) in the years 2009–2019 as representing baseline values. Table 1 shows the average, standard deviation, minimum, maximum and range for estimates of relative excess deaths (expressed as percentage of expected deaths) for the two-year pandemic period 2020–2021 for each of the 33 countries. The average value is highly correlated with either the maximum or minimum value but not with standard deviation or range (correlation coefficients of 0.96, 0.95, −0.22 and −0.15, respectively). This happens as changing the reference time window shifts all years by approximately the same amount along the y-axis.

Table 1:

Average, standard deviation, minimum, maximum and range for estimates of relative excess deaths (expressed as percentage of expected deaths, p%) for the two-year pandemic period 2020–2021 for each of the 33 countries.

Country	Abbreviation p%	Average p%	SD of p%	Minimum p%	Maximum p%	Range of p%
Australia	AUS	−9.7	3.2	−16.2	−2.4	13.9
Austria	AUT	3.2	3.0	−3.4	9.2	12.6
Belgium	BEL	1.4	2.9	−5.0	8.8	13.8
Canada	CAN	2.2	2.0	−4.9	6.9	11.7
Switzerland	CHE	−1.3	3.1	−8.2	5.7	13.9
Chile	CHL	6.4	3.8	−1.7	15.1	16.8
Czechia	CZE	8.7	3.9	−0.5	16.7	17.2
Germany	DEU	1.0	1.9	−4.4	4.5	8.9
Denmark	DNK	−7.6	4.0	−18.6	−0.3	18.3
Spain	ESP	3.6	2.2	−2.6	10.9	13.5
Estonia	EST	1.7	4.8	−10.8	11.0	21.9
Europe	EUM	2.3	2.2	−3.7	7.4	11.1
Finland	FIN	−5.3	3.1	−11.9	1.6	13.4
France	FRA	2.4	2.0	−3.8	6.1	10.0
United Kingdom	GBR	4.2	1.9	−1.2	10.0	11.3
Greece	GRC	5.6	2.8	−1.3	10.6	12.0
Croatia	HRV	7.0	3.1	−1.2	14.8	16.1
Hungary	HUN	6.8	2.7	0.5	13.1	12.6
Iceland	ISL	−7.3	2.0	−12.2	−2.1	10.1
Israel	ISR	−1.5	2.9	−7.1	4.6	11.6
Italy	ITA	5.4	2.4	−0.5	10.8	11.2
South Korea	KOR	−13.5	5.2	−24.5	−1.1	23.5
Lithuania	LTU	8.6	3.2	2.0	18.8	16.8
Luxembourg	LUX	−2.6	3.9	−10.6	4.4	15.0
Latvia	LVA	7.0	3.1	−1.0	14.0	15.0
Netherlands	NLD	2.5	2.0	−2.5	7.8	10.4
Norway	NOR	−9.4	3.6	−16.1	−1.4	14.7
New Zealand	NZL	−9.1	2.5	−15.5	−4.2	11.3
Poland	POL	14.2	3.5	3.9	19.9	15.9
Portugal	PRT	3.6	2.4	−2.7	8.6	11.3
Slovakia	SVK	10.2	4.4	0.7	20.7	20.0
Slovenia	SVN	4.7	3.4	−4.0	11.8	15.7
Sweden	SWE	−6.7	3.4	−12.5	4.2	16.7
United States	USA	16.6	0.8	14.3	18.7	4.3

Stability of relative ranking for the pandemic years’ excess deaths across 33 countries

The estimates of relative excess deaths (as percentage of expected deaths) can be used to compare different countries in a given time period. Despite large variabilities in the absolute estimates, the relative ranking of the 33 countries for a given period of interest was largely unperturbed, regardless of what reference baseline years were chosen. Table 2 shows the ranking of relative excess death estimates (as percentage of expected deaths) for the pandemic years 2020–2021 in all 66 analyses with different reference baseline windows. The USA had the highest estimates of relative excess deaths among all 33 countries in 49 of 66 analyses, the second highest in 16 analyses and the fourth highest in 1 analysis. Conversely, South Korea had the lowest estimates in 58 of 66 analyses, the second to lowest in 6, the third to lowest in 1, and the sixth to lowest in 1. Eastern European and Balkan countries closely followed the USA in the top excess death ranks consistently. Scandinavian countries, Australia, and New Zealand consistently were placed among the lowest excess death ranks next to South Korea. Other Western European countries typically occupied middle ranks. Tables S1A and S1B show that the distribution of country ranks for projected periods of 1 year and 3 years are similar to that shown in Figure 1 for 2020–2021; summing over more years does blur the ranking of middle-ranked countries.

Table 2:

Distribution of the country rank of the excess death estimates in the pandemic 2-year period 2020–2021 expressed as a percentage of the expected deaths p% for the 33 countries as calculated for each of the 66 different reference baseline year sets..

Figure 1:

Variation with year from 2009 to 2021 of the excess death estimate expressed as a percentage of the expected deaths. The expected deaths are estimated from the average mortality values of each of the 66 different reference year-sets, which are all combinations of one or more consecutive years from 2009 to 2019. The average over all the reference years is shown at the black solid line, the maximum and minimum values are shown by the shaded area. The y-axis of every panel extends from −30% to 30%. The plots for different reference year sets are almost identical but shifted along the y-axis by different amounts. The sum of two years, which is particularly significant as the complete pandemic years are 2020 and 2021, is shown here.

Diversity in time patterns across 33 countries

Figure 1 maps the emerging time patterns for mortality in each of the analyzed countries for the average of the 66 analyses using different reference baseline periods and the range of maximum and minimum estimates. Although the range of estimates of relative excess deaths for each given year is large, the rank of different years for a particular country is generally the same for the 66 different sets of reference years (Figure S1). Time patterns across different countries show large variability as well. Differences exist both in the presence and magnitude/steepness of time trends; and on the presence or not of peaks of mortality impact during the COVID-19 pandemic (2020, 2021, both, or neither). All countries had some decline in mortality over the period 2009–2019, but for the USA in particular the change was minimal (change from average of 1.34% in 2009–2010 to −1.25% in 2018–2019 for an overall decline of only 2.59%, using data in Table S2B). The other 4 countries with the smallest changes for the averages between 2009–2010 and 2018–2019 were Germany, The Netherlands, Canada and the United Kingdom, (changes of −6.65%, −8.34%, −8.44% and −8.49%, respectively). Conversely, the 5 countries with the largest declines for the averages between 2009–2010 and 2018–2019 were South Korea, Estonia, Denmark, Slovakia and Norway (changes of −23.4%, −19.7%, −17.1%, −16.0% and −14.1%, respectively). For the pandemic period 2020–2021, the USA had the steepest increase (change in average 17.94% between 2018–2019 and 2020–2021). Steep increases were seen also in Eastern European and Balkan countries (changes in average from 10.18% to 17.42% between 2018–2019 and 2020–2021 for Slovenia, Hungary, Latvia, Croatia, Lithuania, Czechia, Slovakia and Poland). Most western European countries had more modest disruptions of the declining trend (changes for the averages from 2.03% to 9.96% between 2018–2019 and 2020–2021 for Luxembourg, Germany, Switzerland, France, The Netherlands, Belgium, Portugal, Austria, the United Kingdom, Spain and Italy). Some Scandinavian countries, Australia, New Zealand, and South Korea continued to have declining mortality trends during the pandemic (changes for the averages from −5.05% to −2.44% between 2018–2019 and 2020–2021 for New Zealand, South Korea, Iceland, Norway, Denmark and Australia). Figures S2A, S2B, and S2C map the time patterns shown in Figure 1 for periods of 1, 3 and 4 years, respectively.

The worst single year with the highest mortality was 2021 for 10 countries (Slovakia, Poland, United States, Latvia, Lithuania, Hungary, Croatia, Czechia, Chile and Greece), 2020 was the highest for 4 countries (United Kingdom, Italy, Spain and Belgium), 2010 was worst for Luxembourg and 2009 was worst for all other 18 countries (data from Table S2B). When considering 2-year periods, in 25 of the 33 countries, 2009+2010 were the worst pair of years. In 9 of the 33 countries (Chile, Czechia, Greece, Hungary, Italy, Lithuania, Poland, Slovakia and United States) the pandemic years 2020+2021 were the worst, and in all of them the years 2009+2010 were the second worst. (Table S2A & Table S3). In 16 countries, the pandemic years were not among the three worst years, which were always years between 2009 and 2016. When considering 3 or 4 year periods, in 31 of the 33 countries 2009–2011 and 2009–2012, were the worst, respectively. Only in Poland or the United States were period 2019–2021 and 2018–2021, which include the pandemic years, the worst, respectively, (data from Tables S2C & S2D)

Excess death estimates in recent years using different projected period time windows

Table 3 shows the effect of changing the width of the projected period of interest from 1 to 4 years for the most recent years (2021 alone, 2020 alone, 2020–2021, 2019–2021, 2018–2021). As shown, there is substantial attenuation of the relative excess mortality between the single worse pandemic year and increasingly wider periods of interest. The attenuation was most prominent for Slovakia, Latvia, Lithuania, Poland, Estonia and Croatia, with relative drops of 18.73, 14.86, 12.65, 11.91, 11.87 and 10.91 percentage points, respectively. The attenuation was least prominent for Australia, Norway, Denmark, Iceland, New Zealand and South Korea, with relative drops of 0.19, −0.47, −0.80, −0.90, −1.02 and −1.39, percentage points, respectively. The USA maintained the most prominent peak even with a 4-year window.

Table 3:

Effect of changing the width of the projected period of interest from 1 to 4 years for the most recent years (2021 alone, 2020 alone, 2020–2021, 2019–2021, 2018–2021).

		1 Year	1 Year	<2 Years>	<3 Years>	<4 Years>
Location	LOC	2020	2021	2020+2021	2019+2020+2021	2018+2019+2020+2021	max (2020,2021) - <2 Years>	max (2020,2021) - <3 Years>	max (2020,2021) - <4 Years>
Australia	AUS	−10.7	−8.2	−9.5	−8.7	−8.4	1.3	0.5	0.2
Austria	AUT	3.7	2.8	3.3	0.5	−0.6	0.4	3.3	4.3
Belgium	BEL	7.6	−4.7	1.4	−1.3	−1.9	6.2	8.9	9.5
Canada	CAN	3.5	1.0	2.3	0.2	−0.3	1.2	3.3	3.7
Switzerland	CHE	3.0	−5.3	−1.2	−3.0	−3.6	4.2	6.0	6.6
Chile	CHL	3.6	10.2	6.7	2.2	−0.1	3.4	8.0	10.2
Czechia	CZE	5.5	11.6	8.6	3.6	1.7	3.0	8.0	9.9
Germany	DEU	−0.2	2.1	1.0	−0.3	−0.2	1.1	2.3	2.3
Denmark	DNK	−9.0	−6.9	−7.8	−7.4	−6.1	0.9	0.5	−0.8
Spain	ESP	8.8	−1.4	3.6	0.2	−0.4	5.2	8.5	9.2
Estonia	EST	−6.8	9.4	1.5	−1.7	−2.5	7.9	11.1	11.9
Finland	FIN	−6.1	−4.6	−5.4	−5.8	−5.3	0.8	1.2	0.6
France	FRA	3.8	0.6	2.3	0.5	−0.1	1.5	3.3	3.9
United Kingdom	GBR	6.2	2.3	4.2	1.1	0.4	2.0	5.0	5.8
Greece	GRC	1.0	9.9	5.6	3.1	1.4	4.3	6.8	8.6
Croatia	HRV	1.9	11.8	6.9	2.3	0.9	4.9	9.5	10.9
Hungary	HUN	1.6	11.8	6.7	2.7	1.4	5.1	9.1	10.4
Iceland	ISL	−6.9	−7.5	−7.2	−6.5	−6.0	0.3	−0.4	−0.9
Israel	ISR	−1.9	−0.8	−1.3	−2.5	−3.3	0.5	1.6	2.4
Italy	ITA	8.9	2.1	5.5	2.0	0.5	3.4	6.9	8.3
South Korea	KOR	−13.1	−13.0	−13.1	−12.8	−11.6	0.2	−0.2	−1.4
Lithuania	LTU	3.9	13.3	8.5	2.7	0.7	4.9	10.6	12.7
Luxembourg	LUX	−0.6	−4.8	−2.7	−3.8	−3.4	2.2	3.2	2.8
Latvia	LVA	−2.8	16.3	6.8	2.6	1.5	9.5	13.7	14.9
Netherlands	NLD	3.3	1.8	2.5	0.1	−0.4	0.8	3.2	3.6
Norway	NOR	−10.1	−8.6	−9.4	−8.9	−8.2	0.8	0.3	−0.5
New Zealand	NZL	−10.7	−7.5	−9.0	−7.3	−6.5	1.5	−0.2	−1.0
Poland	POL	10.2	17.8	14.2	8.2	5.9	3.6	9.5	11.9
Portugal	PRT	3.6	3.3	3.5	0.9	0.2	0.1	2.7	3.4
Slovakia	SVK	−0.3	20.7	10.2	4.1	2.0	10.5	16.6	18.7
Slovenia	SVN	7.5	1.8	4.7	1.1	−0.2	2.8	6.4	7.8
Sweden	SWE	−2.2	−10.6	−6.6	−7.9	−7.2	4.5	5.7	5.1
United States	USA	15.8	17.6	16.7	10.6	7.8	0.9	7.0	9.8

With increasing projected periods, both the mean and standard deviation of the relative excess mortality declined substantially. For 2021, 2020–2021, 2019–2021, 2018–2021, the mean was 2.55%, 1.62%, −0.88%, and −1.57%, respectively. The standard deviation was 9.20%, 7.10%, 5.10%, and 4.11%, respectively.

DISCUSSION

Our application of a multiverse approach to excess death data shows that consideration of different periods for reference baseline resulted in major variability in the absolute magnitude of the exact excess death estimates, but it did not affect substantially the relative ranking of different countries compared to others for specific years. Moreover, there have been distinct time patterns across different countries during 2009–2021. Countries differed markedly on whether they had a substantial decrease over time or not during 2009–2021, on whether they had a peak during the 2020–2021 pandemic years, and, if so, how high, and in the relative contribution of 2020 and of 2021 to this peak. With longer time windows for the projected period of interest (1 to 4 years), the range of excess deaths across different countries in the pandemic years and the 2 years preceding the pandemic shrank substantially and excess death estimates became less variable across countries. However, peaks did not disappear and for the USA in particular, excess deaths remained prominent even with long projected periods of interest.

In the multiverse literature from other fields, some analytical choices may be considered more meaningful or relevant than others. When researchers are asked to select independently what analysis mode they feel is most sensible, not all analytical choices are selected (19,20) and some types of choices may seem to make more sense. This may apply also for excess death calculations. E.g. it may seem not so appropriate to use a reference window of 2009 alone for projecting mortality in 2021. Indeed, one should definitely not use the excess deaths estimate of 2021 based on the 2009 reference in isolation as a reliable measure of excess deaths in 2021. However, previous studies on excess death calculations for the COVID-19 pandemic have used extremely different reference periods, ranging from a single year to over a decade (6). While in our own work (6) we used a 3-year reference period (2017–2019), it is difficult, if not impossible, to defend that shorter or longer reference periods would be less appropriate. The proposed multiverse approach removes the hurdle of this subjective choice and allows for consistency by considering all possible reference years and time windows. Thus, it allows one to get an objective comparative picture for the relative performance of each calendar year and an unbiased comparative picture for the presence and pattern of time trends.

The obvious heterogeneity of time patterns across different countries suggests that selection of specific time trends in modeling excess deaths may be a situation where one size does not fit all. Selection of specific anticipated time trend patterns may markedly affect the results in ways that are not verifiable for their appropriateness. E.g. selecting a model that anticipates a marked decrease in mortality over time makes it difficult for a country not to have excess deaths even if it does very well in a given year – but still falls short of an anticipated stellar improvement over time. There is no guarantee that mortality rates should continue declining, let alone markedly decline, over time with medical and other progress, even in the absence of major negative perturbation events such as pandemics or natural catastrophes (6). For advanced economies with aging populations, accumulating frailty and disease burden and restrictions or ceilings to progress and available resources, trends for decreased mortality may not be sustainable. The multiverse approach, when applied to multiple countries, allows a comparative assessment of the trajectory of different countries. This may be preferable and it may offer some genuine insights about which countries do well (short-term and long-term) and which do poorly – in comparison.

In this regard, some stark differences stand out for both long-term trends and for the pandemic years. The USA consistently performed very poorly with both stagnation in mortality during the pre-pandemic years and a sharp increase during the pandemic. Eastern European and Balkan countries showed sharp decreases during the pre-pandemic years and a sharp increase during the pandemic. Most western European countries had sharp decreasing trends with modest disruption during the pandemic. All Scandinavian countries, Australia, New Zealand, and South Korea have had largely unperturbed declining mortality patterns. The markedly different patterns may reflect a combination of social, health care, and pandemic factors. The USA has an ailing health system with approximately 30 million uninsured people (22), large inequalities (23), many people with poor access to care (24), and major ongoing non-infectious epidemics, including obesity (25), opioid abuse and overdose (26), and violent deaths (27). Eastern European and Balkan countries have limited resources for their healthcare systems and lower social welfare than other European countries (28) and some countries like Greece have long suffered from austerity (29). The best performers are excelling in social welfare and health system functionality and resources, even if there are differences across countries. Exceptions may occur within circumscribed populations and adverse settings even in countries with overall excellent trajectories. For example, the dysfunctional consequences of privatization in nursing homes in countries like Sweden or Canada (30,31) translated to peaks of excess deaths during circumscribed periods in the long-term care settings (32).

Consideration of longer projected period time windows diminished substantially the range of excess deaths in some countries, but not in others. Overall, when longer periods are considered, differences between most countries become less pronounced. However, larger windows had minimal effect in the USA, and this may reflect that the problems that lead to unfavorable mortality patterns in the USA reflect chronic dysfunctions that might have been accentuated by the pandemic but pre-existed and which affect also people with long life expectancy. Poverty, marginalization, homelessness, inequalities, drug overdoses, and violence affect indeed young and middle-aged populations. We have shown previously that the USA has had 40% of excess deaths contributed by the <65 age stratum, a higher percentage than all other highly developed countries (6). Conversely, in many other countries, large time windows for the projected period shrank substantially the excess death fluctuations. This suggests that in these countries excess deaths temporarily affect mostly people with relatively limited life expectancy (21).

Europe, while not a country, has historically aggregated excess death data in the EuroMOMO data base (https://www.euromomo.eu/) to include data from 21 countries in Europe plus Israel (33). If one were to aggregate data for the 19 of these 21 countries for which we have data (excluding Cyprus & Ireland), the fictional country composite that includes Austria, Belgium, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Israel, Italy, Luxembourg, Netherlands, Norway, Portugal, Slovenia, Sweden, Switzerland, and United Kingdom has a similar population to the USA (410 million versus 330 million) and the relative excess death of this European composite is only 2.46% for the pandemic years (2020+2021), which is in stark difference to the USA figures. It is also less than two year totals for 2009+2010 and 2010+2011, with values of 5.57% and 3.30% caused by elevated Influenza pandemics (34).

Some limitations should be acknowledged. First, there are some additional sources of analytical flexibility that can be considered in excess death calculations. These include the choice of age bins for age adjustment, and the use of additional adjustments for modeling the population profile over time. These would add additional variation with more multiverse options, but probably would not invalidate the major patterns that we observed. Second, we only modeled data from 33 countries that are the ones with the most reliable data. Extrapolations to other countries would be precarious, given the unreliability of the mortality information. Time patterns observed in the 33 countries may not necessarily apply to the remaining countries around the globe and local circumstances may make a difference. Third, we considered yearly interval increments so as to capture all 4 seasons in the unit of time, but in theory, the multiverse process can be applied for smaller units of time as well. Fourth, data on population and population structure in each country on a yearly basis are typically inferred from census data collected on more sparse timing, therefore they carry some uncertainty. Fifth, the pandemic impact and its consequences as well as the consequences of aggressive measures that were taken has continued more prominently in 2022 in some countries than others (35). It would be interesting to see whether differences across countries get further attenuated and/or some countries continue to stand out prominently when longer pandemic and post-pandemic periods (e.g. 2020–2022 and 2020–2023) are considered. Preliminarily results based on the first 8 months of 2022, it seems that several countries with death deficit in 2020–2021 (e.g., Australia, New Zealand and South Korea), had considerable excess deaths in 2022, while some others continued to have limited deaths (e.g. Sweden) and some hard-hit countries like USA and Greece continued to do very poorly (6,35).

A multiverse approach to excess death calculations may offer bird’s eye views on mortality patterns in comparative assessments of a large number of countries. These patterns may be more reliably informative than efforts to obtain isolated single-country estimates of excess deaths, which are subject to substantial uncertainty even in countries with the best-collected data. It may be best to avoid pre-specifying time patterns and to allow the data to show what time patterns may be emerging. Finally, observed time patterns may not necessarily continue into the future and multiverse analyses can be updated accordingly for additional years moving forward.

MATERIALS AND METHODS

Data

All data comes from the Human Mortality Database (HMD) (36–38). The data for the most recent years comes from the Short-Term Mortality Fluctuation file stmf.csv downloaded from https://www.mortality.org/File/GetDocument/Public/STMF/Outputs/stmf.csv (last updated 20 July 2022). The data for earlier years extending back to 2009 was downloaded as the HMD archive file (see Supplementary Links to Data). By combining data from these two sources, full data for 13 years from 2009 to 2021, inclusive, is available for 34 countries. We focused on the 33 high-income countries with highly reliable death registration systems, excluding Bulgaria as done in previous work (6). The most recent data in the file stmf.csv is per week and uses five standard age-bands: 0–14, 15–64, 65–74, 75–84 and Over 85. The older data in the HMD archive uses 1-year age bands for annual all-cause deaths and annual populations are available for all 33 countries. We sum these 1-year bands to give the same five standard age bands used in stmf.csv. Because the older data is over-written by the newer data in stmf.csv, it does not matter much which older version we use (ours was downloaded on 25 May 2022). The annual death and population data taken from the hmd_statistics_20220812 archive is used to make a file in the same format as stmf.csv, which included weekly data for deaths and weekly data for mortality (deaths per population). Because the older data is for entire years, it is converted to weekly data by dividing the annual value into 52 equal weeks.

Excess death calculations

In order to be able to compare different countries and different time periods we focus on relative excess deaths expressed as the number of excess deaths divided by the number of expected deaths. Specifically, the relative excess death p% is the actual all-cause death count D minus the estimated death count E expressed as a percentage of the estimated death count or p%=(D-E)/E.

Systematic Variation of Assumptions for Multiverse Analyses

We consider all possible reference baseline spans of consecutive years in the period 2009–2019. This gives 11+10+..+1=66 different spans of length 1 to 11 years for the 66 reference baselines. For each reference period, we average the mortality of each of the five age bands. These averaged mortalities are then used to get the expected deaths in any year by multiplying the mortality of a particular age band by the population of that age band and then summing the estimated values over the five age bands to give the total estimated death count, E.

Projected time periods are also considered in all possible options of length 1 to 4 years, again considering consecutive calendar years.

Data availability

All data are in the manuscript, tables, and supplementary tables and in the publicly available databases listed in Supplementary Links to Data and deposited online at https://zenodo.org/record/7095753.

SIGNIFICANCE STATEMENT.

Excess death estimates are the ultimate assessment of the impact of multiple diseases and forces on the mortality of a population. However, their calculation can be notoriously unstable because it depends on a multitude of analytical choices. Other scientific fields have started using multiverse analysis approaches where all possible analytical choices are considered. We developed a multiverse analysis approach for excess death estimation. The approach is demonstrated with data from 33 countries for the period 2009–2021. Multiverse analysis offers a standardized way to demonstrate the sensitivity of estimates to analytic assumptions, to understand the presence of time patterns (rather than arbitrarily prespecify them), and to reveal consistent patterns that characterize excess deaths in a comparative fashion between different countries.