Estimating excess mortality during the Covid-19 pandemic in Aotearoa New Zealand

Abstract

Background

The excess mortality rate in Aotearoa New Zealand during the Covid-19 pandemic is frequently estimated to be among the lowest in the world. However, to facilitate international comparisons, many of the methods that have been used to estimate excess mortality do not use age-stratified data on deaths and population size, which may compromise their accuracy.

Methods

We used a quasi-Poisson regression model for monthly all-cause deaths among New Zealand residents, controlling for age, sex, and seasonality. We fitted the model to deaths data for 2014–19. We estimated monthly excess mortality for 2020–23 as the difference between actual deaths and projected deaths according to the model. We conducted sensitivity analysis on the length of the pre-pandemic period used to fit the model. We benchmarked our results against a simple linear regression on the standardized annual mortality rate.

Results

We estimated cumulative excess mortality in New Zealand in 2020–23 was 1040 [95% confidence interval (−1134, 2927)], equivalent to 0.7% (−0.8%, 2.0%) of expected mortality. Excess mortality was negative in 2020–21. The magnitude, timing, and age-distribution of the positive excess mortality in 2022–23 were closely matched with confirmed Covid-19 deaths.

Conclusions

Negative excess mortality in 2020–21 reflects very low levels of Covid-19 and major reductions in seasonal respiratory diseases during this period. In 2022–23, Covid-19 deaths were the main contributor to excess mortality, and there was little or no net non-Covid-19 excess. Overall, New Zealand experienced one of the lowest rates of pandemic excess mortality in the world.

Key Messages.

• We investigated the number of additional deaths that occurred in New Zealand in 2020–23 relative to what would have been expected in the absence of a pandemic.

• Excess mortality in New Zealand in 2020–23 was estimated to be 1040, with a 95% confidence interval of (−1134, 2927).

• New Zealand experienced one of the lowest rates of pandemic excess mortality in the world.

Introduction

The Covid-19 pandemic caused an estimated 14.8 million deaths globally in 2020 and 2021 [1]. Aotearoa New Zealand used a combination of border and community control measures to minimize transmission of SARS-CoV-2 until high vaccine coverage was achieved in late 2021 [2, 3]. Following the establishment of the B.1.1.529 (Omicron) variant in January 2022 [4], New Zealand experienced a series of Covid-19 waves [5]. By 31 December 2023, there had been a total of 750 Covid-19-attributed deaths per million people [6], one of the lowest rates in the world.

The number of Covid-19-attributed deaths is one measure of the pandemic’s impact on mortality, but it has limitations. It could be influenced by factors such as test availability, how Covid-19-attributed deaths are distinguished from incidental deaths in Covid-19-positive individuals, and processes for recording multiple causes of death. Furthermore, Covid-19-attributed deaths only measure the direct impact of the disease on mortality. This ignores indirect effects, such as deaths that occurred or were prevented as a result of pandemic control measures and deaths that would have occurred due to another cause within a given time period.

Another measure of the pandemic’s impact is excess mortality, i.e. the number of additional deaths relative to the expected number if no pandemic had occurred [1]. Various methods have been used to estimate the expected number of deaths, known as the baseline. The method of Karlinsky and Kobak [7], which has been widely used to make international comparisons of excess mortality via the Our World in Data dashboard [8, 9], estimated cumulative excess mortality in New Zealand in 2020–23 as 99 per million. Another widely used model developed by the Economist estimated 198 per million [10], whilst Kung et al. [11] estimated excess mortality for 2020–22 as 215 per million.

The Karlinsky and Kobak [7] baseline relies on extrapolating the pre-pandemic trend in raw death counts. This may, in some cases, mainly reflect the trend in total population size if the crude mortality rate (i.e. deaths per capita) does not vary greatly. Gibson [12] correctly pointed out that, in such cases, if the trend in total population size abruptly changes, then the Karlinsky and Kobak baseline will be systematically biased. There was a sharp drop in the rate of population growth in New Zealand in 2020 due to reduced international travel. Gibson [12] argued that controlling for population size by fitting to the trend in crude mortality rate, rather than raw deaths, led to a lower baseline than that of Karlinsky and Kobak [7] and, therefore, a higher estimate of excess mortality.

However, the trend in raw deaths may also be influenced by population ageing, which would generally lead to an upward trend in crude mortality rate. For this reason, where age-stratified data are available, it is preferable to control for age when estimating mortality baselines [13]. This will limit international comparisons to jurisdictions where age-stratified data are available but will provide more accurate estimates. Neither Gibson [12] nor Kung et al. [11] controlled for age. Reliable estimates of pandemic excess mortality in New Zealand that control for the combined effects of population growth and ageing are lacking.

Here, we use age- and sex-stratified data on all-cause mortality in New Zealand to estimate excess mortality in 2020–23 relative to the pre-pandemic baseline. We compare the results of two methods: a quasi-Poisson regression model fitted to monthly age-specific death counts and a linear regression on the age-standardized annual mortality rate.

Both methods account for changes in population size and age structure, which means they provide more reliable estimates of excess mortality than methods that do not. We use the quasi-Poisson regression model as our primary method, as this allows us to disaggregate results by monthly time periods and by age and sex. We compare aggregated yearly results from this method with the standardized mortality rate linear regression to benchmark our results against a simpler model. We then compare the estimated excess to the number of Covid-19-attributed deaths over time and in different subgroups. This enables us to estimate whether there was any net non-Covid-19 excess mortality during this period.

Methods

We obtained data on monthly all-cause deaths among New Zealand residents and the Stats NZ estimated resident population [14], stratified by sex and age, up to the end of 2023. We also obtained data on the number of deaths attributed to Covid-19 [6]. See online supplementary material Section S1.1 for further details.

We used two methods for constructing a mortality baseline from pre-pandemic data: (i) a generalized linear quasi-Poisson regression model for monthly death counts including age, sex, and month of the year as predictors, based closely on the model of the UK’s Office for National Statistics [15]; (ii) a linear regression fitted to the age- and sex-standardized annual mortality rate. Quasi-Poisson regression is a natural model for count data whilst allowing for the possibility that the variance may be higher than in a Poisson regression. See online supplementary material Section S1.2 for further details.

We fitted each model to all-cause mortality data from January 2014 to December 2019. Using a 6-year baseline is a compromise between shorter baselines, which are more sensitive to annual fluctuations in death rates, and longer baselines, which are biased by the nonlinear shape of annual mortality rates over time. We conducted a sensitivity analysis with different lengths of baseline ranging from 4 to 10 years.

We used the fitted baseline to estimate expected deaths from January 2020 to December 2023 if pre-pandemic trends had continued. We calculated excess mortality as the difference between actual deaths and expected deaths.

Results

In the pre-pandemic period, all-cause monthly deaths in the New Zealand resident population had a pronounced seasonal pattern, with relatively high deaths during the winter (i.e. June–August) respiratory disease season (Fig. 1). The quasi-Poisson regression model captured the seasonality and the gradually increasing trend in deaths. The age- and sex-standardized yearly all-cause mortality rate varied between 7.0 and 7.9 per 1000 between 2010 and 2019, with a decreasing trend (Fig. 2). The model standardized yearly mortality rate was very similar under the two methods.

Figure 1.

Total monthly all-cause deaths in the New Zealand resident population from January 2010 to December 2023 (points) along with the quasi-Poisson regression model fitted to data on monthly deaths from January 2014 to December 2019. Vertical dashed lines show the fitting period. Solid curve shows the mean monthly deaths according to the fitted model; shaded band shows the 95% confidence interval (CI). Excess mortality is estimated as the difference between actual deaths for 2020–23 and expected deaths for 2020–23 according to the model (mean and 95% CI).

Figure 2.

Age- and sex-standardized yearly all-cause mortality rate per 1000 people in the New Zealand resident population (open circles), with the results of the quasi-Poisson regression (QPR) model (solid blue curve = mean, shaded blue band = 95% confidence interval) and the standardized mortality rate linear regression (SMR-LR) model (solid red). Both models were fitted to data from 1 January 2014 to 31 December 2019 (indicated by the vertical dashed lines). All calculations use the first quarter of 2021 as the standard population.

The number of excess deaths according to the quasi-Poisson regression model was −2276 [95% confidence interval (CI) (−2663, −1941)] in 2020; −654 (−1144, −228) in 2021; 2735 (2141, 3252) in 2022; and 1235 (531, 1858) in 2023 (Fig. 3a, red). The cumulative excess for 2020–23 was 1040 (−1134, 2927), which equates to 204 (−222, 573) per million and to 0.7% (−0.8%, 2.0%) of expected mortality. The standardized mortality rate linear regression model gave very similar results (Fig. 3a, yellow).

Figure 3.

(a) Yearly Covid-19-attributed deaths (blue) along with excess deaths according to the quasi-Poisson regression (QPR) model (red) and the standardized mortality rate linear regression (SMR-LR) model (yellow). (b) Monthly Covid-19-attributed deaths (blue points) along with excess deaths according to the QPR model (red open circles). Error bars show the 95% confidence interval for the QPR model.

Covid-19-attributed deaths were extremely low in 2020 and 2021 (26 in each year). In 2022–23, the number of Covid-19-attributed deaths was close to the central estimate for excess deaths (Fig. 3a, blue), accounting for 102% of the estimated excess in 2022 and 82% in 2023, and fell within the 95% CI in both years.

Looking at monthly data (Fig. 3b), Covid-19 deaths and excess deaths, according to the quasi-Poisson regression model, followed similar trends in 2022–23. The number of Covid-19 deaths was below the 95% CI for excess deaths in February and March 2022, and above it from July to September 2022. The unexplained excess in early 2022 could represent unattributed Covid-19 deaths that occurred when the country’s laboratory testing capacity became overwhelmed [16]. The monthly discrepancies could also partly reflect temporal displacement of mortality due to disruption of seasonal respiratory disease patterns [17].

The number of Covid-19 deaths fell within the 95% CI for the number of excess deaths in each age band (0–59 years, 60–69 years, 70–79 years, and 80+ years) and for both sexes in 2022 and 2023 (Fig. 4). Note we used these four age bands because appropriate data on Covid-19 deaths were available for comparison (see online supplementary material, Fig. S1 for results disaggregated into finer age groups). The results in Fig. 4 mirror similar patterns seen in Australia [18], where there was some positive non-Covid excess in the 70–79-year age band and, despite there being more Covid-19 deaths among men than women, the central estimate for excess deaths was higher for women.

Figure 4.

Yearly Covid-19-attributed deaths (blue bars) and excess deaths (mean and 95% confidence interval) according to the quasi-Poisson model (open red circles and error bars) disaggregated by age (a, b) or sex (c, d) in 2022 and 2023.

When different pre-pandemic fitting periods were used, the estimated number of excess deaths for 2020–23 according to the quasi-Poisson regression model varied between a low of −1770 (−4973, 924) for the 4-year baseline, and a high of 1995 (500, 3368) for the 9-year baseline (Table 1). Expressed as a percentage of expected deaths, these correspond to a low of −1.2% (−3.4%, 0.6%) and a high of 1.4% (0.4%, 2.4%). The estimate from our default baseline of 6 years (2014–19) ranked third out of the seven baselines investigated. The 95% CI for the quasi-Poisson model contained zero in all cases except for the 9-year baseline. The results of the standardized mortality rate linear regression model showed a similar pattern with respect to baseline length. In both models, the variations in estimated excess seen in Table 1 were mainly driven by whether or not the first year in the fitting period had a relatively high all-cause death rate.

Table 1.

Sensitivity analysis showing the cumulative estimated number of excess deaths for the 4-year period 2020–23 with different length baselines

Baseline period	Excess (QPR)	Excess (SMR-LR)
2016–2019	−1770 [−4973, 924]	−1489
2015–2019	456 [−1999, 2743]	641
2014–2019	1040 [−1134, 2927]	1777
2013–2019	−784 [−2687, 963]	−948
2012–2019	510 [−1152, 2018]	611
2011–2019	1995 [500, 3368]	2271
2010–2019	1148 [−246, 2402]	1258

In each row of the table, the models were fitted to deaths data in a period of between 4 and 10 years ending in 2019. Results are shown for the quasi-Poisson regression (QPR) model (mean and 95% confidence interval) and the standardized mortality rate linear regression (SMR-LR) model.

The Karlinksy and Kobak estimate for 2020–23 of 517 excess deaths [7, 9] falls within the 95% CI for all seven baselines. The Economist estimate of 1026 excess deaths [10] is very close to our central estimate of 1040 for the 6-year baseline, is within the 95% CI for five of the seven baselines, and slightly above it for other two. These estimates are therefore largely consistent with our results. In contrast, Gibson estimated +4% excess mortality for the 3-year period of 2020–22 using a 5-year baseline, which translates to ∼4000 excess deaths [12]. This estimate, which failed to control for changes in population age structure, is 4500 deaths higher than our central estimate for this period using a 5-year baseline, and 2500 deaths higher than the highest upper limit of the 95% CI for ‘any’ baseline length between 4 and 10 years (see online supplementary material, Table S1).

Discussion

We have estimated the number of excess deaths in Aotearoa New Zealand between January 2020 and December 2023 using two different methods. Importantly, both methods controlled for age, sex, and population size (one by including these variables as predictors in a regression model, and one by using an age- and sex-standardized mortality rate). This avoids the potential for systematic biases caused by changes in population size or age structure.

To comprehensively measure the impact of the pandemic on deaths, estimates need to include the period after most non-pharmaceutical interventions were removed and the virus became endemic. However, the longer pre-pandemic baselines are extrapolated forward in time, the greater the uncertainty and potential for bias due to nonlinear trends over time. In New Zealand, most remaining interventions ended in September 2022 [19], except for mask requirements in healthcare and aged residential care settings and isolation requirements for confirmed cases, which continued until August 2023 [20]. Choosing the period from January 2020 to December 2023 is a reasonable trade-off that allows enough time for the transmission dynamics to settle into a relatively steady pattern without extrapolating pre-pandemic trends too far. Organizations estimating excess deaths in the UK [15] and Australia [18] have adopted a similar approach, moving to a new baseline from 2024 that partly includes the pandemic period.

Consistent with other methods [7, 10], both our models estimated negative excess mortality in 2020 and, to a lesser extent, 2021. This was likely due to the effective suppression of Covid-19 (there were only 52 Covid-19 deaths in 2020–21), the elimination of influenza [21], and the reduction in other respiratory pathogens during this period [17, 22].

Our models estimated positive excess mortality in 2022 and 2023, after New Zealand ended its elimination strategy and the Omicron variant became established. The magnitude, timing, and age-distribution of the excess closely matched those of Covid-19-attributed deaths. This suggests that the bulk of the excess in 2022–23 was directly attributable to Covid-19, either as an underlying or a contributory cause of death. This could be because there were relatively few Covid-19 deaths that were not recorded as such. Alternatively, it could be that any undocumented Covid-19 deaths were offset by reduced mortality from other causes, such as non-Covid respiratory disease.

The timing of the excess does not coincide with the timing of the Covid-19 vaccine rollout, the bulk of which occurred between February 2021 and February 2022 (more than 10 million doses were administered in this period). Excess deaths were either negative or close to zero for most of this period. In contrast, the period during which excess deaths were relatively high (March 2022 to April 2023) coincided with a period where far fewer vaccine doses (fewer than 1.7 million) were administered. This suggests that vaccines were not responsible for any substantial portion of the excess mortality, consistent with New Zealand safety data [23] and international research showing very low risk of severe adverse events following Covid-19 vaccination [21].

Mortality rates fluctuate from year to year depending on the severity of the winter respiratory illness season and other factors. This means that the fitted baseline for expected mortality can be sensitive to the choice of fitting period. To mitigate this, we ran our model with seven different fitting periods ranging from 4 to 10 years duration. The limits of the 95% CIs for excess mortality for 2020–23 fell between −3.4% and +2.4% and contained zero for all but one choice of fitting period. Thus, our results provide robust evidence that, regardless of the choice of baseline, there was no more than 2.4% cumulative excess mortality between 2020 and 2023, and any net excess that did occur cannot be confidently distinguished from zero.

We have not attempted to apply our method to estimate excess mortality in countries other than New Zealand, as this is beyond the scope of the study. Nevertheless, our estimated excess mortality of 204 (−222, 573) deaths per million people for 2020–23 ranks amongst the lowest in the world. Estimated excess mortality for 2020–23 for most OECD countries (with a few notable exceptions, including Australia) was in the range of 1500–4500 per million [7–9].

Gibson [12] argued that, by extrapolating raw death counts and not accounting for the abrupt drop in population growth in 2020, the Karlinksy and Kobak method [7] systematically overestimated expected mortality and therefore underestimated excess deaths. However, although population growth slowed dramatically during the pandemic, this was largely a result of reduced migration, which occurs predominantly in younger groups [24]. The population over 65 years of age (which is only around 15% of the total population but accounts for 80% of deaths) continued to grow at a rate similar to before the pandemic (see online supplementary material, Fig. S2). By controlling for total population size but not for age, Gibson’s method introduced a systematic bias and consequently underestimated expected mortality for 2020–22 by ∼4%. The main advantage of the method of Karlinsky and Kobak [7] is that it can be applied to a large set of countries to provide a meaningful and useful basis for international comparisons. Despite not controlling for population size or age, the Karlinksy and Kobak estimate was within the 95% CIs from our model. This is because the pre-pandemic trend in death counts was driven more by population ageing (which continued throughout the pandemic) than by population growth (which did not).

Our study has a number of important limitations. Due to reporting delays, there could be ∼40 deaths that occurred in 2023 that do not yet appear in the data. Hence, our results could underestimate excess deaths for 2023 by around 40, but we note this difference is much smaller than the model CIs.

We have compared estimates of all-cause excess mortality with Covid-19-attributed deaths. It is likely that the pandemic differentially impacted various causes of death at different times. This could be investigated by estimating excess mortality associated with different causes of death [18]. However, we have not attempted this due to a lack of up-to-date data on causes of death other than Covid-19.

Our model lumped all deaths over 95 years of age into a single age class. This was necessary because population size data were only available at this level. The size of the over-95-year age group increased from 5130 in 2014 (Q1) to 8530 in 2023 (Q4). It is likely that the age-distribution within this group became increasingly elderly over time, which would influence the group-specific mortality rate. This is mitigated by the inclusion of an interaction term in the quasi-Poisson model, which allows a group-specific linear trend in the mortality rate. Nevertheless, the model cannot explicitly account for any shift in the age distribution within the group that may have occurred between 2019 and 2023.

Covid-19 in New Zealand disproportionately affected Māori, Pacific Peoples, and people living with high levels of deprivation [25–27]. This mirrors well-documented international trends [28, 29], whereby social determinants of health intersect with factors affecting people’s ability to take protective measures, such as working from home [30]. Our analysis did not include socioeconomic variables such as ethnicity or deprivation because the required data were not available. Investigating how excess mortality in 2022 and 2023 was distributed across these axes and how this relates to the distribution of Covid-19 deaths is an important aim for future research.

Ethics approval

Ethics approval was not required for this research as it only used routinely collected administrative data, which were de-identified and aggregated prior to use.

Author contributions

M.J.P.: conceptualization, methodology, software, validation, formal analysis, writing—original draft, writing—review and editing. P.S.: conceptualization, methodology, software, validation, data curation, writing—review and editing. R.L.: methodology, writing—review and editing. M.J.P. will act as guarantor for the paper.

Supplementary data

Supplementary data is available at IJE online.

Conflict of interest: None declared.

Funding

None declared.

Data availability

Data and code to reproduce the analysis are available at https://dx.doi.org/10.5281/zenodo.15107131. The results shown in the article used raw, unrounded data on death counts. The raw data cannot be published due to privacy concerns relating to small counts. To preserve confidentiality, the repository linked above contains a data set where death counts were randomly rounded (see online supplementary material Section S1.1 for details). Running the code on this dataset will produce results that are similar to those in the article, but not identical and with broader confidence intervals due to the added noise.

Use of artificial intelligence tools

Artificial intelligence tools were not used in this work.