Skip to main content
NCBI home page
American Journal of Epidemiology logoLink to American Journal of Epidemiology
. 2024 Oct 28;194(9):2560–2569. doi: 10.1093/aje/kwae422

Spatial and demographic heterogeneity in excess mortality in the United States, 2020-2023: a multimodel approach

Sasikiran Kandula 1,✉, Anja Bråthen Kristoffersen 2, Gunnar Rø 3, Marissa LeBlanc 4,5, Birgitte Freiesleben de Blasio 6,7
PMCID: PMC12409136  PMID: 39489507

Abstract

In this study, we assessed the overall impact of the COVID-19 pandemic in the United States between 2020 and 2023 through estimates of excess all-cause mortality. Monthly mortality rates over a 19-year period, stratified by age, sex, and state of residence, were used to predict expected mortality for the pandemic years. A combination of models—2 timeseries, a spatial random-effects and a generalized additive—was used to better capture uncertainty. Results indicate that the national excess mortality for the United States decreased in 2023 to 157 000 (95% prediction interval: 35 000-282 000) from 502 000 (436 000-567 000), 574 000 (484 000-666 000) and 377 000 (264 000-484 000) during the years 2020-2022, respectively. Unlike in previous years, deaths with COVID-19 as the underlying cause of death possibly accounted for all excess deaths during 2023. While for the older age groups (>75 years), the year 2020—before vaccines were available—had the highest excess mortality rate, and the 2 younger age groups had the highest excess mortality in 2021. In each age group, women were estimated to have consistently lower excess mortality than men. West Virginia had the highest age-standardized excess mortality among all states in 2021 and 2022. Our findings demonstrate the value of a multimodel approach in capturing heterogeneity in excess mortality.

Keywords: COVID-19, excess mortality, multimodel combination, forecast verification, heterogeneity in disease burden

Introduction

The COVID-19 pandemic has caused large-scale disruptions globally, and estimating its short- and long-term societal health effects is important. One commonly used measure of pandemic burden is mortality, often limited to deaths with a confirmatory positive test for COVID-19. In the United States, limiting mortality estimates to deaths explicitly certified to be from COVID-19 could result in underestimating the pandemic’s impact due to several reasons, including: limited testing capacity during the initial months of the pandemic; limitations in cause-of-death certifications during periods of high pandemic severity; and possible discounting of indirect effects of the pandemic such as those from restricted access to preventive medical care and emergency procedures, or from increased drug overdoses.1-4 The magnitude of the underestimation can also differ by demographic group (eg, race, age, sex, economic precarity), geography, and time (eg, different waves/phases of the pandemic5,6) due to the recognized inequities in access to testing and critical care.7,8

All-cause excess mortality estimates, defined as the difference between observed all-cause deaths and deaths expected in the absence of a pandemic, can help quantify more fully the effects of the pandemic. Excess mortality estimates have been used to assess impacts of famines,9-11 wars,12 extreme weather events,13,14 climate change,15 and seasonal influenza.16 Admittedly, excess death estimates are an incomplete measure of the pandemic’s total impact, as they do not capture intermediate effects of the pandemic that did not lead to death, such as effects of economic distress including homelessness,17 food insecurity,18 impact on child development,19 harm to mental health,20,21 and the prolonged effects following infection (postacute sequelae of COVID-19)22; nevertheless, they are a more comprehensive estimate of mortality due to the pandemic than cause-specific mortality.

Multiple estimates of excess mortality from the COVID-19 pandemic have been previously reported. A comprehensive analysis from the World Health Organization (WHO) estimated 4.47 (95% credible interval (CI), 3.9-5.1) million excess deaths globally in 2020 and 10.36 (95% CI, 9.1-12) million cumulative excess deaths by the end of 2021,23-25 with considerable per capita differences across WHO regions. These estimates were 2.74 (95% CI, 2.4-3.1) times higher than reported deaths from COVID-19. Multiple studies with narrower focus on select countries, regions, and pandemic waves also exist.26-29 In the United States, excess mortality estimates were reported as early as June 2020,30,31 and the US Centers for Disease Control and Prevention’s (CDC) National Center for Health Statistics (NCHS) provided regular updates of estimates for national and state-level excess mortality through September 2023.32 Estimates for specific population subgroups (eg, age, race, sex, ethnicity etc.),6,33-37 states,38,39 and counties40-43 have also been reported, noting significant differences by demographic characteristics and geography.

Given multiple and sometimes discordant excess mortality estimates, it is natural to ask which estimates are “correct” (per an error or loss function), but this is not possible to ascertain as the outcome depends on an unobservable quantity—deaths that would have occurred in the absence of a pandemic. Goodness-of-fit measures (a model’s ability to predict observations it is trained on) are useful, but evaluations under condition of cross-validation or out-of-sample testing (a model’s ability to predict observations it is not trained on) are needed to better assess model skill. To the best of our knowledge, few prior studies have undertaken and/or reported validation results in the context of pandemic-associated excess mortality.24,44 Similarly, when deviations of observed deaths from expected deaths are estimated, it is necessary to assess whether the deviation could have happened by chance. In other words, distributional predictions of expected deaths need to be available to assess uncertainty in excess mortality estimates. Consequently, out-of-sample skill metrics of distributional predictions rather than of point estimates (such as absolute error) are appropriate, but these too are rarely reported.

A second related consideration is the need to reduce reliance on estimates from a single model. Multiple studies across several domains have shown that a single model of an essentially unknown data generation process cannot sufficiently capture the different sources of uncertainty, and combining predictions from multiple models can help mitigate concerns of model and parameter misspecification.45,46 More recently, combination approaches have been used in operational real-time forecasts of COVID-19 outcomes.47,48 We believe that multiple models of all-cause mortality are necessary to have more reliable estimates of expected and thereby excess mortality.

In this study, we add to the existing literature on excess mortality associated with COVID-19 in the United States, while attempting to address some of these previously mentioned concerns, in 3 significant ways: (1) we employed 3 distinct and diverse models to independently forecast mortality and assessed their skill under temporal cross-validation; (2) we used model averaging to combine estimates from the 3 models and demonstrate that the forecast skill under cross-validation improved over that of the 3 individual models; the combination model can thus serve as a single source of plausibly more reliable estimates for the pandemic period; and (3) we extended the projection period through the end of 2023, and therefore provide more up-to-date estimates of excess mortality in the United States. Specifically, using strata-specific state-level models, we report excess mortality due to COVID-19 between 2020 and 2023 for:

  • annual all-cause excess mortality in the United States nationally and stratified by age and sex;

  • annual all-cause excess mortality for the 50 states and the District of Columbia (DC), overall and stratified by age and sex;

  • monthly all-cause national estimates of excess mortality for the different age and sex strata, and associations with monthly certified COVID-19 deaths.

Data and methods

Data sources

Annual and monthly counts of all-cause deaths for years 2001-2023 were obtained from the public US National Vital Statistics System, CDC WONDER.49,50 Counts were stratified by the decedent’s age at time of death (<65 years, 65-74 years, 75-84 years, 85+ years), sex (male and female), and state of residence at the time of death. To account for changes in population size in each strata over time, mortality rates (deaths per 100 000 population) were estimated using strata-specific annual population estimates from the US Census Bureau.51 The interface suppressed instances with fewer than 10 deaths, and this affected a small fraction of instances (n = 13; 0.006%); linear interpolation was used to impute missing values.

Annual strata-specific counts and rates of deaths that were explicitly attributed to COVID-19 were identified using International Classification of Diseases-1052 underlying cause of death code U07.1 and also retrieved from CDC WONDER.

Methods

Let Inline graphic denote the number of all-cause deaths during month t in location s among the population in the kth strata (combination of age and sex of the decedent), Inline graphicthe time series of mortality counts up to month t, Inline graphic= (Inline graphic,⋯, Inline graphic), and Inline graphic the corresponding population size. A location- and strata-specific model at month t was fit using Inline graphic and Inline graphic and used to obtain predictions of expected deaths 1- to h-months ahead: (Inline graphic,⋯, Inline graphic). Here, horizon h = 48, |s| = 52 and |k| = 15. To capture prediction uncertainty, we rely on distributional predictions and the prediction is specified by estimates at 9 quantile levels: Inline graphic, where Inline graphic.

Component models

Three different methods were used to generate predictions and are described in detail in Appendix text S1. In selecting the models, we tried to balance model parsimony, data requirements, ability to capture trend and seasonality, robust estimation of uncertainty, and a preference for methods that have been previously used in similar settings. Briefly, we used (1) TS-combo: standard timeseries methods ARIMA and exponential trend smoothing (ETS), both designed to capture trend and seasonality in the input and have been used for short-term forecasts of multiple pathogens47,53,54; (2) GAMM: a Bayesian form of one of the methods underlying WHO’s estimates of COVID-19 excess mortality that modeled expected mortality as a combination of an annual trend (a thin-plate spline) and within year seasonality (a cyclic cubic spline)24,25; and, (3) INLA: a Bayesian spatial model that in addition to trend and seasonality, incorporated a spatial random effect component for capturing spatial dependence between states.26,42,43 Implementations of all models were based on standard R55 packages.56-61

Combination model

Extending the previously mentioned notation, if Inline graphic denotes model m’s h-month ahead prediction at the α-quantile level for location s and strata k, the corresponding estimate of the combination model was calculated as a simple average:

graphic file with name DmEquation1.gif

This combination approach has been previously used in multiple collaborative efforts, including forecasts of COVID-19 weekly hospitalizations and deaths in the United States47 and Europe.48

Forecast generation

For TS-combo and GAMM methods, we separately trained location- and strata-specific models (ie, 52 locations Inline graphic 15 strata), while for the spatial INLA approach we trained 15 strata-specific models. All models were trained using monthly data from January 2001 to December 2019, and distributional predictions for expected deaths were generated for 48 months between January 2020 and December 2023. While projections for 2024 are possible, observational data for the year were provisional at the time analysis was undertaken (additional concerns around projections at long horizons are discussed later). Monthly estimates of expected mortality at the different α-quantile levels were obtained from posterior distributions in the case of the 2 Bayesian approaches (GAMM and INLA) and from simulated paths of the trajectories (n = 1000) at each month of the projected period for TS-combo. Similarly, the posteriors/trajectories were used to aggregate monthly estimates to annual estimates, and in the case of the spatial model, to obtain US national estimates from state-level estimates. Distributional predictions of monthly and annual excess mortality were calculated from the distributions obtained after subtracting observed deaths from expected deaths per each sample/trajectory. Excess mortality was assumed to be statistically significant when the observed all-cause deaths were higher than the upper bound of the 95% prediction interval of expected deaths (α = 0.975). This approach can also detect instances when the observed deaths are lower than expected deaths.

To adjust for changes in population size over time, population was specified as an offset in GAMM and INLA models, while the TS-combo model was trained on monthly mortality rates (deaths per 100 000 population) rather than raw counts. Additionally, to adjust for difference in age distributions across states and by sex, expected and excess death estimates were standardized using the direct method with the US national age distribution in 2020 as the reference population (Appendix text S2).

Temporal cross validation

To inspect model estimates of expected deaths during the years before the pandemic, we performed a temporal cross-validation exercise. For each of the years 2010-2017, we trained the models with data up to (but not including) the candidate year and projected mortality 3 years ahead. From the quantile distribution of the predicted mortality rates and observed mortality in pre-pandemic years, we calculated 3 measures of forecast skill, both to aid comparison of the models and for selecting a primary model (Appendix text S3):

  1. Weighted interval score (WIS): This is a strictly proper scoring measure62 for quantile predictions, and can be interpreted as a sum of 3 penalties—for dispersion (the width of the prediction interval), overprediction (observed value below lower bound of the interval), and underprediction. It is negatively oriented (the lower the better) and has a lower bound of 0 and no upper bound.63

  2. Bias: This is considered to be a more robust measure than an interval scoreand has clear bounds of −1 and 1; it is used to detect the tendency of the forecast distribution to under- or overpredict.64 An unbiased forecast would have a value of 0.

  3. Coverage deviation: Interval coverage is calculated as the proportion of observations that fall within a given prediction interval, and when the forecast distribution is described at multiple confidence levels (ie, prediction intervals), coverage deviation is calculated as the average across all available intervals. As above, an ideal forecast would have 0 deviation, and the direction of deviation could aid detection of under- or overprediction.64 As the averaging cancels out oppositely signed deviations at α-levels, an absolute alternative to the metric was also considered (Appendix text S3).

Summary measures of model skill reported in the next section are an average across all cross-validation periods, locations, demographic strata, and when relevant, prediction horizon.

Results

Cross-validation and model selection

The WIS of the combination model is lower (10.62) than the individual models overall, indicating the best model calibration. Of the 3 component models, TS-combo had the lowest WIS (11.39) and was characterized by an approximately linear increase in penalty for dispersion with increasing forecast horizon (Figure 1). This trend was not discernible for the GAMM model, but as a counterpoint, it had higher coverage deviation at larger horizons (ie, the uncertainty of GAMM estimates did not increase sufficiently with increasing horizon while increasing excessively for TS-combo). Dispersion and coverage deviation of the INLA model shared some of the characteristics of the other 2 models. All models, including the combination model, were negatively biased and indicated miscalibrated estimates for winter months. Interval coverage of the TS-combo model was superior to the other 2 models, and the combination model matched TS-combo. The combination model and TS-combo had good interval coverage overall and at most horizons, while the other 2 models had lower coverage particularly at longer horizons (Figure S1). Empirical coverage was lower than nominal coverage at all quantile levels, with difference higher at smaller α levels. Metrics for individual states are largely consistent with these findings (Figure S2). On the whole, as the combination model appeared to have the best skill and moderate the deficiencies of the individual models, it was chosen as the primary model for reporting estimates in this section; estimates from individual models are included in Data S2.

Figure 1.

Figure 1

Open in a new tab

Temporal cross-validation measures. Each panel shows a calculated metric (row) for a model (column) at different prediction horizons (x-axis). The horizontal dashed line indicates the overall metric across all horizons. WIS, dispersion, under- and over-prediction have a zero lower bound and are negatively oriented (the lower the better). Coverage deviation and bias are bound between −1 and 1 and ideally zero.

Excess mortality estimates

National

There were an estimated 502 000 (95% prediction interval: 436 000-567 000) excess deaths in the US nationally during 2020, and 574 000 (484 000-666 000), 376 000 (264 000-484 000) and 157 000 (35 000-282 000) during 2021-2023, respectively. For comparison, 351 000 deaths were recorded with COVID-19 as the underlying cause of death during 2020, and 417 000, 187 000, and 49 000 during 2021-2023, respectively. Corresponding estimates of age-standardized excess mortality rates (per 100 000 population) for 2020-2023 are 157 (134-181), 183 (152-215), 130 (92-167), and 66 (22-109).

Monthly estimates indicate statistically significant excess during the first 22 months of the pandemic and in a majority of the months in 2022 (Figure 2). Monthly observed deaths during most of 2023, though elevated relative to median expected deaths, were within the 95% prediction interval. During months of relatively low mortality burden, estimated excess deaths were in line with deaths from COVID-19 as the underlying cause (ie, it is unlikely that there were deaths beyond those directly attributed to COVID-19), but noticeable divergence occurred during the second halves of 2020 and 2021.

Figure 2.

Figure 2

Open in a new tab

Monthly mortality in the United States. (top) Orange bands show the estimated expected monthly mortality in the absence of a pandemic (darker region: interquartile range; lighter region: 95% prediction interval). The red data points show observed monthly all-cause mortality. The blue shaded regions show the corresponding uncertainty of the excess monthly mortality (secondary y-axis, right), and the black data points represent deaths with COVID-19 as the underlying cause of death. Data points are filled in if they are statistically significant. Observed all-cause deaths within the lighter orange band can be interpreted as absence of statistically significant excess mortality in that month; reported COVID-19 deaths within the lighter blue band can be interpreted as an absence of excess mortality beyond what was directly attributed to COVID-19. (bottom) Historical mortality rate (per 100 000 population) in the 15 years before the pandemic plus 4 pandemic years (black), and ensemble projections during the pandemic period (orange bands, as in top plot).

Excess mortality was estimated in all age groups during each of the 4 years (except for 85+ year group during 2023), with estimates increasing with age (Figure 3). The difference between the estimated excess rates in the youngest and the oldest age groups was largest, over 40-fold, in 2020 and decreased in later years. For the 2 older age groups (75-84 years, 85+ years) the year 2020, before vaccines were available, had the highest excess mortality rate, while in the 2 younger age groups highest annual excess mortality was estimated for 2021. In each age group, women were estimated to have consistently lower excess mortality than men, with the difference larger during 2020 and 2021 in those younger than 75 years. These differences appear insensitive to the choice of the model (Figure S3). This difference was also noticed nationally after age-standardization to adjust for higher longevity of American women than men (Figure S4).

Figure 3.

Figure 3

Open in a new tab

Annual excess mortality rate (per 100 000 population) in the United States stratified by age and sex. In each age group (panel), the darker box shows the interquartile range of the estimates, and the lighter region represents the 95% prediction interval. A 95% prediction interval that crosses the y = 0 line is not statistically significant. Text on y = 0 line report median estimate. Top row shows unstratified estimate for the whole age group, and bottom row reports sex-stratified estimates. Note that the y-axis of each panel has a different range/scale.

State

Age-standardized state-level estimates indicated considerable temporal and spatial heterogeneity in the magnitude of excess mortality (Figure 4), but over 45 states were estimated to have had statistically significant excess mortality in each of the first 3 years of the pandemic, and 15 states during 2023. Mississippi and New Mexico were among the 5 states with the highest excess rate in each of the first 3 years of the pandemic; West Virginia had the highest excess rate in 2021 and 2022.

Figure 4.

Figure 4

Open in a new tab

Annual state-level age-standardized excess mortality rates (per 100 000 population). States with positive excess estimates (more deaths than expected) are shown in color (per legend), and states with no statistically significant excess are not colored.

Additional differences emerge when estimates were analyzed by age group (Figure S5). As previously noted, excess mortality in younger groups was higher in 2021 than in 2020, and this increase is more prominent in the Southern and South Atlantic states, namely, West Virginia, Kentucky, Tennessee, Mississippi, Alabama, and Oklahoma. A majority of these states continued to have high excess mortality rates in 2022, particularly in individuals 75-84 years of age; and unlike other age groups, the excess mortality in this group persisted into 2023. Further west, New Mexico and Arizona had high excess mortality in all 4 years in individuals younger than 65 years.

Comparing excess mortality to mortality from COVID-19 showed that in most states and years, there were more deaths than those directly attributable to COVID-19 and the magnitude of divergence varied (Figure 5). For example, while New York had one of the highest excess mortality rates in 2020, there was reasonable agreement with deaths attributed to COVID-19 (blue data point close to the diagonal), and this continued to be the case for 2021-2023. On the other hand, in West Virginia, Arizona, New Mexico, and a few other states, in addition to high excess mortality, there was indication of excess mortality from causes other than COVID-19 or that were coded as such (data points further away from the diagonal). Comparison of excess estimates with COVID-19 deaths at monthly resolutions showed that these discrepancies were consistent over extended periods rather than in a few select months (Figure S6).

Figure 5.

Figure 5

Open in a new tab

Comparison of excess mortality estimates in each state with COVID-19 certified deaths. Each panel plots the annual age-standardized excess mortality rate (per 100 000 population) in a state (y-axis) against mortality rate from COVID-19 (x-axis). The point denotes the median estimate, and the error bars represent the 95% prediction interval. Data points above the horizontal line y = 0 indicate excess mortality, and those above the diagonal (y = x) indicate a higher excess mortality than directly attributed to COVID-19. Arrows connecting data points show year-to-year transitions.

Examining year-to-year transitions within each state identified 2 broad, nonexhaustive categories: states in which there was an incremental decrease in excess mortality from 2020 to 2023 exemplified by New York, New Jersey, Connecticut, Illinois and the Dakotas; and states where these decreases were preceded by an increase from 2020 to 2021, as seen in most states in the south (Alabama, Florida, Georgia, Carolinas, etc.) and Mountain west (Wyoming, Montana, Nevada, Idaho).

Discussion

Our results show excess mortality during the first 4 years of the pandemic and considerable spatiotemporal and demographic heterogeneity in pandemic effects in the United States. Results also indicate that this heterogeneity extended to deaths not explicitly attributed to COVID-19.

Our primary focus in this analysis has been to make a case for moving away from dependence on a single model (and its assumptions) by using a combination of multiple disparate models, and to provide evidentiary support of reliability through forecast verification. While we reported summary findings from the estimates, the raw outputs from the models (Data S2) can aid secondary analyses, for example, to investigate associations between excess mortality and population socioeconomic status or underlying health conditions, as well as with public health measures implemented in 2020 and vaccine uptake post-2020, and cross-correlations with COVID-19 case, hospitalization, and mortality rates. While some of these associations have been reported for the early stages of the pandemic, excess mortality could continue to be an important outcome to track in the postacute phase of the pandemic with increasing prevalence of long-COVID and its uncertain impact over the life course, and other collateral effects. Similarly, it may be interesting to investigate the value of a multimodel approach in estimating cause-specific excess mortality from malignant neoplasms or cardiovascular diseases, as well as other causes.65-67

Our choice to train the models with monthly data was motivated by data availability, but excess mortality estimates at weekly resolutions could be of more real-time operational value and more suitable for studying temporal dependence of excess mortality with direct COVID-19 outcomes such as cases and hospitalizations. However, a public data source of historical weekly all-cause mortality at the different population stratifications is unavailable. A related barrier for generating real-time excess mortality estimates is the delays in mortality reporting that make it necessary for models to correct for potentially incomplete reporting in more recent weeks. In the United States, the CDC published weekly excess mortality estimates through September 2023, and we compared annual estimates from the combination approach reported here with CDC’s estimates.68 The two estimates were highly correlated (Spearman ρ = 0.95, n = 208 location-year combinations), with the CDC’s estimates generally lower than the median estimate of the combination model (Appendix text S4).

A continued use of the approach described here to estimate excess mortality for 2024 and beyond may not be advisable. The training period in our approach ends in 2019, and as is apparent in Figure 1 and Figure S1, the model skill degraded with increasing forecast horizon. An extension of the training window beyond 2019 and into parts of the pandemic period cannot be indefinitely avoided. The US CDC, EuroMOMO, and UK Health Security Agency have taken steps in this direction,69-71 but this raises difficulties in interpreting excess estimates. Two alternatives exist: to interpret excess deaths strictly as a consequence of the pandemic, which would require models to ignore mortality beyond March 2020, and accept the unreliability of excess estimates at longer horizons; or, acknowledge that the pandemic altered pre-pandemic mortality trends and COVID-19 is now an endemic respiratory pathogen like influenza, thus allowing models to use more up-to-date mortality data and interpreting excess estimates as those beyond “normal” COVID-19 activity. When there is no clear justification to choose between these options, reporting estimates from models fit with different training windows is perhaps to be preferred.

The models included in this study did not exploit dependencies between different demographic groups, and only one of the models accounted for spatial dependence. Unifying, hierarchical model forms simultaneously informed by mortality across different states and demographic groups may have better predictive skill and need to be investigated. Such alternative approaches may also yield coherent estimates (for example, national estimates of expected mortality equal sum of states’ mortality) unlike the approach used here. More specialized combination approaches that can learn from historical accuracy of individual models and exploit their full posterior distributions (rather than quantile summaries) may yield further improvements and need to be explored.45,72-74 Note that the use of the mean of the component estimates, as was done in this study, is referred to as model averaging in some domains with the terms combination and ensemble reserved for more specialized approaches.

Use of large age groupings, necessitated by limited access to historical all-cause mortality data, results in an inability to account for changes in age distribution within each age group. For example, an upward trend in mortality of individuals aged 75-84 years could be an effect of this group skewing older over the last 2 decades and neither accounting for change in the size of this group nor age-standardization effectively controls for this. Use of smaller strata, especially for the elderly population, may alleviate this concern. Additional demographic stratifications by population race/ethnicity and smaller age grouping in the under-65 population are essential, and models described here can be reused if data to support such analyses were available.

Caution is required when interpreting and comparing excess mortality across states. The models accounted for changes in a state’s population over the study period when estimating expected mortality rates, and the post hoc age-standardization would allow for identifying differences in these estimates beyond those stemming from a difference in age structure of the state populations. Mortality during the pandemic years was also age-standardized, but this adjustment may be insufficient given the complex interplay of population structure and COVID-19 transmission dynamics and case severity and could have possibly conflated differences in direct and indirect effects of the pandemic among the different age groups.

Retrospective predictions generated for the pre-pandemic period have shown that deviations between expected and observed deaths tend to be larger during the winter months, with the models often underestimating mortality. These deviations need not imply sub-optimal forecast skill and cause-specific excess mortality assessments, for example from influenza and other respiratory diseases, may help explain some of these discrepancies.

Acknowledgments

We thank the editor and anonymous reviewers for insightful and constructive feedback.

Supplementary Material

Web_Material_kwae422
web_material_kwae422.zip (1.4MB, zip)

Contributor Information

Sasikiran Kandula, Department of Methods Development and Analytics, Norwegian Institute of Public Health, Oslo, Norway.

Anja Bråthen Kristoffersen, Department of Methods Development and Analytics, Norwegian Institute of Public Health, Oslo, Norway.

Gunnar Rø, Department of Methods Development and Analytics, Norwegian Institute of Public Health, Oslo, Norway.

Marissa LeBlanc, Department of Methods Development and Analytics, Norwegian Institute of Public Health, Oslo, Norway; Oslo Centre for Biostatistics and Epidemiology, University of Oslo, Oslo, Norway.

Birgitte Freiesleben de Blasio, Department of Methods Development and Analytics, Norwegian Institute of Public Health, Oslo, Norway; Oslo Centre for Biostatistics and Epidemiology, University of Oslo, Oslo, Norway.

Supplementary material

Supplementary material is available at the American Journal of Epidemiology online.

Funding

None declared.

Conflict of interest

The authors declare that they have no conflicts of interest to disclose.

Disclaimer

The views presented are solely of the authors and do not necessarily represent those of their affiliated institutions.

Data availability

All-cause mortality data and model estimates are available on a public repository at https://github.com/fhi-kan/excess_mortality_us. This includes (1) Data S1 (input dataset): annual and monthly all-cause mortality from CDC WONDER; (2) Data S2 (output estimates dataset): annual all-cause excess mortality estimates from all component models and the combination model, nationally and for all combinations of age, sex and location; and (3) Data S3 (validation dataset): cross-validation scores (overall, by prediction horizon, and location).

References

  • 1. Hacker  KA, Briss  PA, Richardson  L, et al.  COVID-19 and chronic disease: the impact now and in the future. Prev Chronic Dis.  2021;18:(E62). 10.5888/pcd18.210086 [DOI] [Google Scholar]
  • 2. Piquero  AR, Jennings  WG, Jemison  E, et al.  Domestic violence during the COVID-19 pandemic-evidence from a systematic review and meta-analysis. J Crim Just.  2021;74:101806. 10.1016/j.jcrimjus.2021.101806 [DOI] [Google Scholar]
  • 3. Roy  CM, Bollman  EB, Carson  LM, et al.  Assessing the indirect effects of COVID-19 on healthcare delivery, utilization and health outcomes: a scoping review. Eur J Public Health.  2021;31(3):634-640. 10.1093/eurpub/ckab047 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Wadhera  RK, Shen  C, Gondi  S, et al.  Cardiovascular deaths during the COVID-19 pandemic in the United States. J Am Coll Cardiol.  2021;77(2):159-169. 10.1016/j.jacc.2020.10.055 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Siegel  M, Critchfield-Jain  I, Boykin  M, et al.  Actual racial/ethnic disparities in COVID-19 mortality for the non-Hispanic black compared to non-Hispanic white population in 353 US counties and their association with structural racism. J Racial Ethn Health Disparities.  2021;9(5):1697-1725. 10.1007/s40615-021-01109-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Zalla  LC, Mulholland  GE, Filiatreau  LM, et al.  Racial/ethnic and age differences in the direct and indirect effects of the COVID-19 pandemic on US mortality. Am J Public Health.  2022;112(1):154-164. 10.2105/AJPH.2021.306541 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Dalva-Baird  NP, Alobuia  WM, Bendavid  E, et al.  Racial and ethnic inequities in the early distribution of US COVID-19 testing sites and mortality. Eur J Clin Invest.  2021;51(11):e13669. 10.1111/eci.13669 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Mullachery  PH, Li  R, Melly  S, et al.  Inequities in spatial accessibility to COVID-19 testing in 30 large US cities. Soc Sci Med.  2022;310:115307. 10.1016/j.socscimed.2022.115307 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Greenough  PR. Prosperity and misery in modern Bengal: The famine of 1943-1944. USA: Oxford University Press; 1982. [Google Scholar]
  • 10. De Waal  A. Famine mortality: a case study of Darfur, Sudan 1984-5. Popul Stud.  1989;43(1):5-24. 10.1080/0032472031000143826 [DOI] [Google Scholar]
  • 11. Boyle  PP, Gráda  CÓ. Fertility trends, excess mortality, and the great Irish famine. Demography.  1986;23(4):543-562. 10.2307/2061350 [DOI] [PubMed] [Google Scholar]
  • 12. Burnham  G, Lafta  R, Doocy  S, et al.  Mortality after the 2003 invasion of Iraq: a cross-sectional cluster sample survey. Lancet.  2006;368(9545):1421-1428. 10.1016/S0140-6736(06)69491-9 [DOI] [PubMed] [Google Scholar]
  • 13. Santos-Burgoa  C, Sandberg  J, Suárez  E, et al.  Differential and persistent risk of excess mortality from hurricane Maria in Puerto Rico: a time-series analysis. Lancet Planetary. Health. 2018;2(11):e478-e488. 10.1016/S2542-5196(18)30209-2 [DOI] [Google Scholar]
  • 14. Rooney  C, McMichael  AJ, Kovats  RS, et al.  Excess mortality in England and Wales, and in greater London, during the 1995 heatwave. J Epidemiol Community Health.  1998;52(8):482-486. 10.1136/jech.52.8.482 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Gasparrini  A, Guo  Y, Sera  F, et al.  Projections of temperature-related excess mortality under climate change scenarios. Lancet Planetary Health.  2017;1(9):e360-e367. 10.1016/S2542-5196(17)30156-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Iuliano  AD, Roguski  KM, Chang  HH, et al.  Estimates of global seasonal influenza-associated respiratory mortality: a modelling study. Lancet. 2018;391(10127):1285-1300. 10.1016/S0140-6736(17)33293-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Benfer  EA, Vlahov  D, Long  MY, et al.  Eviction, health inequity, and the spread of COVID-19: housing policy as a primary pandemic mitigation strategy. J Urban Health.  2021;98(1):1-12. 10.1007/s11524-020-00502-1 [DOI] [Google Scholar]
  • 18. Gundersen  C, Hake  M, Dewey  A, et al.  Food insecurity during COVID-19. Appl Econ Perspect. Policy.  2021;43(1):153-161. 10.1002/aepp.13100 [DOI] [Google Scholar]
  • 19. Dreyer  BP. Let us Be vigilant: COVID-19 is poised to obliterate gains in healthy child development globally. Pediatrics.  2020;146(3). 10.1542/peds.2020-012591 [DOI] [Google Scholar]
  • 20. Czeisler  MÉ, Lane  RI, Wiley  JF, et al.  Follow-up survey of US adult reports of mental health, substance use, and suicidal ideation during the COVID-19 pandemic, September 2020. JAMA Netw Open.  2021;4(2):e2037665. 10.1001/jamanetworkopen.2020.37665 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Pfefferbaum  B, North  CS. Mental health and the COVID-19 pandemic. N Engl J Med.  2020;383(6):510-512. 10.1056/NEJMp2008017 [DOI] [PubMed] [Google Scholar]
  • 22. Nalbandian  A, Sehgal  K, Gupta  A, et al.  Post-acute COVID-19 syndrome. Nat Med.  2021;27(4):601-615. 10.1038/s41591-021-01283-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. World Health Organization . Global excess deaths associated with COVID-19, January 2020 - December 2021. 2022.
  • 24. Knutson  V, Aleshin-Guendel  S, Karlinsky  A, et al.  Estimating global and country-specific excess mortality during the COVID-19 pandemic. Ann. Appl Stat.  2023;17(2):1353-1374. 10.1214/22-AOAS1673 [DOI] [Google Scholar]
  • 25. Msemburi  W, Karlinsky  A, Knutson  V, et al.  The WHO estimates of excess mortality associated with the COVID-19 pandemic. Nature.  2023;613(7942):130-137. 10.1038/s41586-022-05522-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Konstantinoudis  G, Cameletti  M, Gómez-Rubio  V, et al.  Regional excess mortality during the 2020 COVID-19 pandemic in five European countries. Nat Commun.  2022;13(1):1-11. 10.1038/s41467-022-28157-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Kontis  V, Bennett  JE, Rashid  T, et al.  Magnitude, demographics and dynamics of the effect of the first wave of the COVID-19 pandemic on all-cause mortality in 21 industrialized countries. Nat Med.  2020;26(12):1919-1928. 10.1038/s41591-020-1112-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Wang  H, Paulson  KR, Pease  SA, et al.  Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020-21. Lancet. 2022;399(10334):1513-1536. 10.1016/S0140-6736(21)02796-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Nørgaard  SK, Vestergaard  LS, Nielsen  J, et al.  Real-time monitoring shows substantial excess all-cause mortality during second wave of COVID-19 in Europe, October to December 2020. Eurosurveillance.  2021;26(2):2002023. 10.2807/1560-7917.ES.2021.26.1.2002023 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30. Rossen  LM, Branum  AM, Ahmad  FB, et al.  Excess deaths associated with COVID-19, by age and race and ethnicity—United States, January 26-October 3, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(42):1522-1527. 10.15585/mmwr.mm6942e2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. Weinberger  DM, Chen  J, Cohen  T, et al.  Estimation of excess deaths associated with the COVID-19 pandemic in the United States, March to May 2020. JAMA Intern Med.  2020;180(10):1336-1344. 10.1001/jamainternmed.2020.3391 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Excess Deaths Associated with COVID-19  [Internet]. National Center for Health Statistics. 2022. Accessed September 12, 2022. https://www.cdc.gov/nchs/nvss/vsrr/COVID19/excess_deaths.htm [Google Scholar]
  • 33. Cronin  CJ, Evans  WN. Excess mortality from COVID and non-COVID causes in minority populations. Proc Natl Acad Sci.  2021;118(39):e2101386118. 10.1073/pnas.2101386118 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Rossen  LM, Ahmad  FB, Anderson  RN, et al.  Disparities in excess mortality associated with COVID-19—United States, 2020. MMWR Morb Mortal Wkly Rep.  2021;70(33):1114-1119. 10.15585/mmwr.mm7033a2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Ruhm  CJ. Excess deaths in the United States during the first year of COVID-19. Prev Med.  2022;162:107174. 10.1016/j.ypmed.2022.107174 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Faust  JS, Krumholz  HM, Du  C, et al.  All-cause excess mortality and COVID-19-related mortality among US adults aged 25-44 years, march-July 2020. JAMA.  2021;325(8):785-787. 10.1001/jama.2020.24243 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Riley  AR, Chen  Y-H, Matthay  EC, et al.  Excess mortality among Latino people in California during the COVID-19 pandemic. SSM Popul Health.  2021;15:100860. 10.1016/j.ssmph.2021.100860 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Woolf  SH, Chapman  DA, Sabo  RT, et al.  Excess deaths from COVID-19 and other causes, march-April 2020. JAMA.  2020;324(5):510-513. 10.1001/jama.2020.11787 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Zhang  Y, Chang  HH, Iuliano  AD, et al.  Application of Bayesian spatial-temporal models for estimating unrecognized COVID-19 deaths in the United States. Spat. Stat.  2022;50:100584. 10.1016/j.spasta.2021.100584 [DOI] [Google Scholar]
  • 40. Stokes  AC, Lundberg  DJ, Elo  IT, et al.  COVID-19 and excess mortality in the United States: a county-level analysis. PLoS Med.  2021;18(5):e1003571. 10.1371/journal.pmed.1003571 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41. Ackley  CA, Lundberg  DJ, Ma  L, et al.  County-level estimates of excess mortality associated with COVID-19 in the United States. SSM Popul Health.  2022;17:101021. 10.1016/j.ssmph.2021.101021 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Paglino  E, Lundberg  DJ, Zhou  Z, et al.  Monthly excess mortality across counties in the United States during the COVID-19 pandemic, March 2020 to February 2022. Sci Adv.  2023;9(25):eadf9742. 10.1126/sciadv.adf9742 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Kandula  S, Keyes  KM, Yaari  RA, et al.  Excess mortality in the United States, 2020-21: county-level estimates for population groups and associations with social vulnerability. medRxiv.  2024. 10.1101/2024.01.14.24301290 [DOI] [Google Scholar]
  • 44. Schöley  J. Robustness and bias of European excess death estimates in 2020 under varying model specifications. MedRxiv.  2021. 10.1101/2021.06.04.21258353 [DOI] [Google Scholar]
  • 45. Wang  X, Hyndman  RJ, Li  F, et al.  Forecast combinations: an over 50-year review. Int J Forecast.  2023;39(4):1518-1547. 10.1016/j.ijforecast.2022.11.005 [DOI] [Google Scholar]
  • 46. Petropoulos  F, Hyndman  RJ, Bergmeir  C. Exploring the sources of uncertainty: why does bagging for time series forecasting work?  Eur J Oper Res.  2018;268(2):545-554. 10.1016/j.ejor.2018.01.045 [DOI] [Google Scholar]
  • 47. Cramer  EY, Ray  EL, Lopez  VK, et al.  Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States. Proc Natl Acad Sci.  2022;119(15):e2113561119. 10.1073/pnas.2113561119 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Sherratt  K, Gruson  H, Johnson  H, et al.  Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations. Elife.  2023;12:e81916. 10.7554/eLife.81916 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49. Centers for Disease Control and Prevention, National Center for Health Statistics . National vital statistics system, mortality 1999-2020 on CDC WONDER online database [Internet]. 2024. Accessed February 23, 2024. http://wonder.cdc.gov/ucd-icd10.html
  • 50. Centers for Disease Control and Prevention, National Center for Health Statistics . National vital statistics system, provisional mortality on CDC WONDER online database [Internet]. 2024. Accessed February 23, 2024. http://wonder.cdc.gov/mcd-icd10-provisional.html
  • 51. US Census Bureau . Annual estimates of the civilian population by single year of age and sex for the United States and states [Internet]. 2022. Accessed December 18, 2023. https://www2.census.gov/programs-surveys/popest/datasets/2020-2022/state/asrh/sc-est2022-agesex-civ.csv
  • 52. World_Health_Organization . The International Statistical Classification of Diseases and Health Related Problems ICD-10: Tenth Revision. Vol. Volume 1: Tabular List. World Health Organization; 2004. [Google Scholar]
  • 53. Kandula  S, Yamana  T, Pei  S, et al.  Evaluation of mechanistic and statistical methods in forecasting influenza-like illness. J R Soc Interface.  2018;15(144):20180174. 10.1098/rsif.2018.0174 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54. Johansson  MA, Apfeldorf  KM, Dobson  S, et al.  An open challenge to advance probabilistic forecasting for dengue epidemics. Proc Natl Acad Sci.  2019;116(48):24268-24274. 10.1073/pnas.1909865116 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55. R Core Team . R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2022. [Google Scholar]
  • 56. Goodrich  B GJ, Ali  I & Brilleman  S. Rstanarm: Bayesian applied regression modeling via Stan. 2.21.14 ed  2023.
  • 57. Hyndman  R AG, Bergmeir  C, Caceres  G, et al. Forecast: forecasting functions for time series and linear models. 8.20 ed  2023.
  • 58. Lindgren  F, Rue  H. Bayesian spatial modelling with R-INLA. J Stat Softw.  2015;63(19):1-25. 10.18637/jss.v063.i19 [DOI] [Google Scholar]
  • 59. O’Hara-Wild  M, Hayes  A. Distributional: vectorised probability distributions. R package version 0.2. 1. 2020.
  • 60. O’Hara-Wild  M, Hyndman  R, Wang  E, et al. Fable: forecasting models for tidy time series 2021. 3 ed  2021.
  • 61. Rue  H, Martino  S, Chopin  N. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. J R Stat Soc B. Stat Methodol.  2009;71(2):319-392. 10.1111/j.1467-9868.2008.00700.x [DOI] [Google Scholar]
  • 62. Gneiting  T, Raftery  AE. Strictly proper scoring rules, prediction, and estimation. J Am Stat Assoc.  2007;102(477):359-378. 10.1198/016214506000001437 [DOI] [Google Scholar]
  • 63. Bracher  J, Ray  EL, Gneiting  T, et al.  Evaluating epidemic forecasts in an interval format. PLoS Comput Biol.  2021;17(2):e1008618. 10.1371/journal.pcbi.1008618 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64. Bosse  NI, Gruson  H, Cori  A, et al.  Evaluating forecasts with scoringutils in R. arXiv preprint  arXiv:220507090. 2022. 10.48550/arXiv.2205.07090 [DOI] [Google Scholar]
  • 65. Kansagra  AP, Goyal  MS, Hamilton  S, et al.  Collateral effect of COVID-19 on stroke evaluation in the United States. N Engl J Med. 2020;383(4):400-401. 10.1056/NEJMc2014816 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66. Richards  M, Anderson  M, Carter  P, et al.  The impact of the COVID-19 pandemic on cancer care. Nat. Cancer.  2020;1(6):565-567. 10.1038/s43018-020-0074-y [DOI] [Google Scholar]
  • 67. Solomon  MD, McNulty  EJ, Rana  JS, et al.  The COVID-19 pandemic and the incidence of acute myocardial infarction. N Engl J Med. 2020;383(7):691-693. 10.1056/NEJMc2015630 [DOI] [PubMed] [Google Scholar]
  • 68. National Center for Health Statistics . Excess deaths associated with COVID-19 [Internet]. 2023. Accessed February 25, 2024. https://data.cdc.gov/d/xkkf-xrst
  • 69. National Center for Health Statistics . Excess deaths associated with COVID-19. Technical Notes. 2023. Accessed March 1, 2024. Available from:  https://www.cdc.gov/nchs/nvss/vsrr/COVID19/excess_deaths.htm#techNotes
  • 70. EuroMOMO Bulletin . Week 21, 2023. Accessed March 1, 2024. https://www.euromomo.eu/bulletins/2023-21
  • 71. Office for National Statistics (ONS) . Estimating excess deaths in the UK, methodology changes. February 2024. Accessed March 10, 2024. https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/causesofdeath/articles/estimatingexcessdeathsintheukmethodologychanges/february2024
  • 72. Ray  EL, Reich  NG. Prediction of infectious disease epidemics via weighted density ensembles. PLoS Comput Biol.  2018;14(2):e1005910. 10.1371/journal.pcbi.1005910 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73. Wattanachit  N, Ray  EL, McAndrew  TC, et al.  Comparison of combination methods to create calibrated ensemble forecasts for seasonal influenza in the US. Stat Med.  2023;42(26):4696-4712. 10.1002/sim.9884 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74. Gneiting  T, Ranjan  R. Combining predictive distributions. Electron. J Stat.  2013;7(7):1747-1782. 10.1214/13-EJS823 [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Web_Material_kwae422
web_material_kwae422.zip (1.4MB, zip)

Data Availability Statement

All-cause mortality data and model estimates are available on a public repository at https://github.com/fhi-kan/excess_mortality_us. This includes (1) Data S1 (input dataset): annual and monthly all-cause mortality from CDC WONDER; (2) Data S2 (output estimates dataset): annual all-cause excess mortality estimates from all component models and the combination model, nationally and for all combinations of age, sex and location; and (3) Data S3 (validation dataset): cross-validation scores (overall, by prediction horizon, and location).

Articles from American Journal of Epidemiology are provided here courtesy of Oxford University Press

RESOURCES