Hunger relief: A natural experiment from additional SNAP benefits during the COVID-19 pandemic

Summary

Background

COVID-19 has directly affected millions of people. Others have been indirectly affected; for example, there has been a startling increase in hunger brought about by the pandemic. Many countries have sought to relieve this problem through public policy. This research examines the effectiveness of enhanced Supplemental Nutrition Assistance Program (SNAP) benefits in the U.S. to alleviate hunger.

Methods

Using a biweekly cross-sectional survey and corresponding population weights from the U.S. Census Bureau, we estimate the effects of enhanced SNAP benefits on hunger in the U.S. as measured by food insufficiency. We use a Bayesian structural time series analysis to predict counterfactual values of food insufficiency. We supplement these findings by examining the effect of enhanced SNAP benefits on observed visits to a food pantry network in a midsized U.S. city.

Findings

Our primary finding estimates that nationwide a total 850,000 (95% credible interval 0·24–1·46 million) instances of food insufficiency were prevented per week by the 15 percent increase in SNAP benefits enacted in January 2021. Secondarily, we find similar effects associated with SNAP benefit increases and local food pantry visits. Specifically, enhanced SNAP benefits resulted in fewer visits to the food pantry network than were predicted in the counterfactual model.

Interpretation

These results not only indicate that the policies enacted to mitigate hunger caused by the COVID-19 pandemic helped, but also quantifies how much these benefits helped on a national scale. As a result, policymakers can use this data to benchmark future policy actions at scale.

Funding

None.

Research in context.

Evidence before this study

Previous research published in peer-reviewed journals and by the USDA has shown participation in SNAP is associated with less food insecurity. However, the COVID-19 pandemic has presented a unique set of hunger-based challenges to policymakers as well as presented new and unique data sets. Policymakers enacted common sense changes, for example boosting SNAP benefits to mitigate hunger. Additionally new survey tools were launched to measure the effect of COVID-19 on the U.S. population. The combination of these two responses to COVID-19 provides a new natural experiment as well as nove measurements of hunger that were repeated frequently throughout the worst of the pandemic.

Added value of this study

We use the natural experiment of increased SNAP benefits in conjunction with a new U.S. Census survey providing information on food insufficiency and SNAP benefits. This large-scale survey is designed to be nationally representative. In addition, the survey is sensitive to changes in population hunger in that food insufficiency is measured over a short period of time (seven-days) and bi-weekly. This allows us to quantify the instances of hunger prevented through the change in SNAP policy with ‘real world’ data on a granular level previously unavailable. The robustness of these findings are then checked using an alternate measure of hunger, visits to a local food pantry network, using observational data that overcomes some of the issues inherent in questionnaire-based research.

Implications of all the available evidence

We find additional SNAP benefits reduced instances of food insufficiency during the COVID-19 pandemic. Rigorously demonstrating tangible value of increasing SNAP benefits is important in the fight against hunger. Further, by quantifying the national instances of hunger prevented per week by the 15% increase in SNAP benefits, we provide policymakers a benchmark to evaluate the return on investment for future social safety net policies designed to mitigate hunger.

Alt-text: Unlabelled box

Introduction

COVID-19 has directly affected millions of Americans. The Center for Disease Control reports over 62 million cases have been reported in the U.S. as of 13 January, 2022 (covid.cdc.gov/covid-data-tracker/). However, the direct effects of COVID-19 are not the full toll of the disease. Indirect effects of COVID-19 such as disruptions to health systems and decreased access to food are contributing to problems in public health. Fore and colleagues¹ argue that many of the responses to mitigate COVID-19, e.g. physical distancing and school closures, have strained national food systems worldwide causing a decline in nutritional quality of diets.

As food insecurity is highly linked to negative health outcomes,² it is one of the most widespread public health issues in the U.S.; a problem exacerbated by COVID-19.^3,4 According to Feeding America⁵ just prior to the pandemic the U.S. was experiencing the lowest rates of food insecurity in decades where 35·2 million total individuals (10·7 million children) lived in a food-insecure household. The same report projected food insecurity jumped to 45 million total individuals (15 million children) in 2020.⁵ Other studies reach similar conclusions; COVID-19 worsened the problem of food insecurity in the U.S., especially in households with children.^3,6 This is a particular problem in terms of public health as hunger is linked to issues like nutrient insufficiency, mental health issues, and disease.7, 8, 9, 10

What makes some countries, like the U.S., less vulnerable to issues of hunger stemming from COVID-19 is the resources available to tackle such issues through public policy actions. Both direct and indirect public policy actions can improve access to food. Recent research suggests increasing the minimum wage could indirectly lead to hunger mitigation by not just increasing food for hungry households, but also increasing the purchase of healthy food too.¹¹ An example of direct action in the U.S. is Supplemental Nutrition Assistance Program (SNAP), federal assistance that provides funds for food to those in need. To address the problem of hunger, SNAP benefits were increased by 15% in January 2021. SNAP funds are allocated according to the number of people in a household, so after this increase the typical maximum benefit a household of four people could receive was $782 per month.

Such public policy actions are specifically enacted to mitigate hunger. Yet, increases in SNAP benefits have detractors who are concerned about cost and dependency.¹² For example, the Wall Street Journal editorial board referred to a proposed increase in benefits as “a recipe for a weaker and fatter America”.¹³ Regardless, hunger is still a problem. A study conducted for the United States Department of Agriculture (USDA) of 2017 SNAP spending showed over half the total monthly benefits are spent in the first week.¹⁴ This potentially indicates that SNAP benefits alone are insufficient to meet the recipient needs.¹⁵ So, if current levels of food assistance are insufficient, then increasing food assistance should reduce the problem of hunger. The purpose of this research is to examine if COVID-19 SNAP benefit increases decreased hunger. And if so, to quantify how much.

Previous research examining issues such as food security have shown a positive effect associated with SNAP. For example, Mabli and Ohls estimated SNAP participation decreased food insecurity by 6–17%.¹⁶ While other estimates indicate that receiving SNAP benefits reduces the likelihood of food insecurity by as much as 30%.¹⁷ However, such studies address the question of whether or not participation in SNAP helps decrease issues of hunger, such as food insecurity. This is a different public policy question than our study which addresses the SNAP benefit amount. That is, policymakers have tools to expand the population covered by SNAP as well as the benefits received by SNAP participants. Opportunities to study changes in SNAP benefits are rarer as changes to SNAP benefits are less frequent. One study of the 2009 American Recovery and Reinvestment Act's (ARRA) average 16% increase in SNAP benefits showed an associated 2.2 percentage point decrease in food insecurity for SNAP-eligible households.¹⁸ Another study of the ARRA did not find any effect on food security for SNAP-eligible children.¹⁹ However, when these ARRA benefits were allowed to expire in 2013, there was an estimated 7.6% increase in food insecurity in SNAP participating households.²⁰

A major priority in public health research is to provide policy- and decision-makers evidence and information from situations not artificially constructed and conforming to the ‘real world’; one source of such evidence is through a natural experiment.²¹ We study the effects of SNAP benefit increases in such a situation. Further, assessing the overall effectiveness of COVID-19 SNAP benefit increases at a national level is a scenario where a randomized controlled trial i.e., the gold standard of causal research, is not feasible. In situations where random assignment is not possible, it becomes necessary to turn to observational data.²² The remainder of this paper describes how we use the natural experiment of increased marginal SNAP benefits. First, we describe the measure of food insecurity from a national survey conducted by the U.S. Census Bureau and how SNAP benefits have changed over time (Study 1). Next, we describe the method we use to construct counterfactuals to compare with actual responses. We then describe a robustness check using observed outcomes in a local area (Study 2). Finally, we discuss the ramifications of these results.

To aid the reader, we have included a table of common acronyms used throughout this manuscript in Table 1. The order of the acronyms are listed in order in which they appear.

Table 1.

Common acronyms.

Acronyms	Definition
COVID-19	Coronavirus disease 2019
SNAP	Supplemental Nutrition Assistance Program
USDA	United States Department of Agriculture
ARRA	American Recovery and Reinvestment Act of 2009
HPS	Household Pulse Survey
EBT	Electronic Benefits Transfer (card for SNAP benefits)
MCMC	Markov chain Monte Carlo
DMARC	Des Moines Area Religious Council food pantry network

Methods

Study 1: household pulse survey

Food insufficiency measurement

The U.S. Census Bureau deployed the Household Pulse Survey (HPS) in an effort to help inform state and federal policy makers as they respond to the evolving pandemic. The HPS is a meant to provide real-time, highly granular measurements on the social and economic wellbeing of U.S. households in the wake of the COVID-19 pandemic. The instrument is a 20 minute online survey collected and disseminated on a weekly to bi-weekly basis, starting in April 2020 and continuing throughout this manuscript's writing. Each weekly or bi-weekly survey is designed to be a stand-alone representation of the U.S. households at that point in time. Importantly, this survey is weighted to be a nationally representative sample of adults in the U.S. that can be used to make inference about the population's wellbeing.

Among the questions in this survey are several having to do with food insufficiency. Of particular interest is question FD1:

Getting enough food can also be a problem for some people. In the last 7 days, which of these statements best describes the food eaten in your household? Select only one answer.

• ○Enough of the kinds of food (I/we) wanted to eat (1)
• ○Enough, but not always the kinds of food (I/we) wanted to eat (2)
• ○Sometimes not enough to eat (3)
• ○Often not enough to eat (4)

We define hunger as those experiencing food insufficiency, specifically responses of (3) or (4) to FD1.²³ Nagata and colleagues indicate “Food insufficiency [is] often the most extreme form of food insecurity”.²⁴ This measure was designed in collaboration with the USDA's Economic Research Service and is “intended to assess rapid changes over time” as opposed to the longer time frame measure of food insecurity which asks about a 12-month perioda. Aside from being slow to change, a 12-month measure of food insecurity also risks increasing response error as some participants may not remember problems that long ago. As the question of food insufficiency is repeatedly measured throughout 2020 and 2021, we have access to fine grained changes in a measurement of hunger not previously available to researchers. Responses to this question, weighted appropriately using survey weights, are aggregated in order to obtain a time series of nationally representative estimates of the proportion of households experiencing food insufficiency. For ease of reporting, we then translate this proportion to the count of households experiencing food insufficiency.

Previous studies examining SNAP and hunger have used panel data to link households joining the SNAP program to decreasing 30-day food insecurity, however small sample sizes along with attrition patterns of panel data in this analysis caused responders to be partially biased in the direction of more food-secure households.²⁵ So, while the repeated cross-sectional nature of the HPS cannot link respondents from measurement-to-measurement, it does overcome the potential issue of attrition. Further, a major benefit of HPS is the collection waves are collected in rapid succession giving a granular view of the data. As a result, and unlike any previous analysis, HPS data can show how rapidly SNAP benefit changes effect food insufficiency.

SNAP benefits changes and food insufficiency

To be eligible for SNAP benefits, participants need a gross income at or below 130% of the federal poverty level and net income at or below the poverty level. Residents of Alaska and Hawaii are exceptions with higher income limits. Maximum benefits are determined by household size, such that greater benefits are given to households with more members. However, the total allotment is the maximum benefits less 30% of net monthly household income. Each month SNAP benefits are loaded to an Electronic Benefits Transfer (EBT) card, like a debit card, which can be used to purchase a restricted set of goods e.g., not alcohol. Since the start of the pandemic, several changes have been made to the SNAP program. Two impacts of importance are:

• •April 2020: All SNAP beneficiaries received the maximum total benefit for their household size, regardless of their net monthly household income.
• •January 2021: SNAP benefits increase by 15% for all beneficiaries.

Measuring an effect of the first impact is not feasible with the HPS because of the lack of pre-period data, i.e., the starting date of the HPS coincides with the implementation date of the first increase to benefits. Our goal is to quantify the causal impact of the second increase to benefits, which occurred in January 2021. Data are between 19 August, 2020 and 4 August 2021, aggregating a total of 2,882,158 responses over 23 surveys.

Constructing counterfactuals to estimate causal impact

The data resulting from the HPS, though rich, is questionnaire based. Ideally, causal inference is made in an experimental setting where a treatment is randomly assigned to units, therefore isolating treatment effects from any effects due to other underlying characteristics. When random assignment is not possible, a difference-in-differences technique is traditionally used to mimic an experimental setting and approximate a causal effect. For examples of difference-in-differences methods applied in similar settings refer to Hudak et al.¹⁹ or Katare and Kim.²⁰ However, this technique has limitations. As described by Brodersen and colleagues,²⁶ standard difference-in-difference techniques ignore autocorrelation. In addition, they often assume the effect of an intervention is a static location shift. In reality, we might expect an impact to have a temporary, lagged, or otherwise-nonlinear effect which are nuances that would be lost under the inappropriate assumptions.

Brodersen and colleagues²⁶ proposed a Bayesian structural time series model as a more flexible extension to standard difference-in-difference approaches in which one models a counterfactual series (the series of interest having not experienced the intervention) before the intervention, predicts the counterfactual series after the intervention, and proceeds to compare the predicted counterfactual to the observed (factual) in the post-intervention period. The counterfactual can be predicted using various sources of information. First, information regarding the general behavior of the series of interest, pre-intervention, such as seasonality, local level, and error variability. Second, one or more variables that are related to the response of interest, but not affected by the impact. Bayesian structural time series models have been used to quantify causal impacts in multiple areas of study, often when a natural experiment has taken place. See Brodersen and colleagues²⁶ for an application measuring the impact of an advertising campaign at Google, De Vocht and colleagues²⁷ for an application measuring the impact of local alcohol licensing changes on hospital admission and crime rates, and Feroze²⁸ for an investigation into the effect of lockdowns on COVID-19 outcomes.

We use this approach and estimate what food insufficiency numbers across the U.S. would have looked like, had no SNAP benefit increase occurred. We compare predicted counterfactuals to reported counts of food insufficiency responses. To construct the predictive model for the counterfactual, we refine our series of interest as well as make use of another HPS question regarding SNAP participation [“Do you or does anyone in your household receive benefits from the Supplemental Nutrition Assistance Program (SNAP) or the Food Stamp Program? Select only one answer.” Yes (1), No (2)]. The prevalence of food insufficiency among non-SNAP-beneficiaries is significantly related to the prevalence of food insufficiency among SNAP-beneficiaries. A pre-intervention simple linear regression model regressing the latter series onto the former series results in a p-value of 0·0197 and an R-squared of 0·51. Thus, we use food insufficiency prevalence among non-SNAP beneficiaries to predict the counterfactual series of food insufficiency prevalence among SNAP beneficiaries, post-intervention. Then, inference can be made by comparing the post-impact counterfactual predictions to the realized factual food insufficiency prevalence among SNAP beneficiaries. The estimand of interest, then, is the change to the number of SNAP beneficiaries experiencing food insufficiency as a result of the additional SNAP benefits. The validity of this approach depends firstly on the assumption that the intervention did not affect the proportion of non-SNAP beneficiaries experiencing food insufficiency.b Further, we assume that the structure of the model for the response of interest remains constant throughout the post-intervention period.

In this analysis, we employ a Bayesian structural time series model using the CausalImpact package in R.²⁹ Let $y_t$ represent the count of SNAP beneficiaries reporting food insufficiency at time t. Let $x_t$ represent the proportion of non-SNAP beneficiaries reporting food insufficiency at time t. We create a structural time series model, using $x_t$ to predict y_t, which can be described as:

$$y_t={\mu }_t+{\beta }_1{x}_t+{ϵ}_t^y{ϵ}_t^y\sim N\left(0,{\sigma }_y^2\right)$$ (1)

$${\mu }_t={\mu }_{t-1}+{ϵ}_t^{\mu }{ϵ}_t^{\mu }\sim N\left(0,{\sigma }_{\mu }^2\right).$$ (2)

Eq. (2) describes the model for the time-varying level, ${\mu }_t$. This term adds flexibility as well as addresses larger predictive uncertainty due to the autocorrelation present in the data. We use the default weakly informative $Inverse-Gamma\left(\alpha ,\beta \right)$c distributions as prior distributions on the error variance, ${\sigma }_y^2$, and the level random walk variance, ${\sigma }_{\mu }^2$. $\alpha$ is taken to be 0·01 to reflect a low ‘prior sample size’ (minimizing the effect the prior has on the posterior) while $\beta$ is set to $0·01\frac{1}{T-1}\sum {\left(y_t-\overline{y}\right)}^2$, thus incorporating the scale of the underlying data, which can vary by application. While 0·01 is subjectively set, the package authors note larger values can result in unrealistically large prediction intervals. The prior distribution on the scalar coefficient ${\beta }_1$ is set according to Zellner's g-prior. That is, we impose a normal distribution centered around zero, imposing minimal shrinkage towards zero. More details about these concepts can be found in Brodersen and colleagues.²⁶

Model results provide a point estimate (via posterior means) of the counterfactual ${\widehat{y}}_t={\widehat{\mu }}_t+{\widehat{\beta }}_1{x}_t$, where parameters are estimated using all the data up until the intervention. A natural semi-parametric estimator of the change in the count of SNAP beneficiaries reporting food insecurity as a result of the intervention is then $y_t-{\widehat{y}}_t$, the additive difference between the factual count and the counterfactual (post intervention). Importantly, we also have access to full posterior distributions and, thus, 95% credible intervals for $y_t-{\widehat{y}}_t$. If, for times t occurring after the intervention, these posterior distributions exist primarily less than zero, that lends evidence the intervention decreased the prevalence of food insufficiency among SNAP beneficiaries.

Study 2: local robustness check

The HPS is a rich source of data that allows us to make projections to the U.S. population. However, these data are not without limitation. Foremost, these are self-reported measures potentially subject to response bias. For example, due to the sensitive nature of the question, a respondent may not report experiencing food insufficiency or receiving SNAP benefits, even if they did. In addition, the survey was rapidly developed with response rates ranging from 5·3% to 10·3% potentially resulting in nonresponse error.³⁰ While statistical weighting helps mitigate such issues, it may not fully eliminate them. Finally, this data comes from a relatively short time frame. Because HPS data collection began with the onset of COVID-19, we are unable to make seasonal adjustments to the data or examine policy effects in early 2020.

To address these potential issues, we conduct a secondary analysis using data from a smaller geographic area using a different measure of hunger. Specifically, we use shopping visits to the Des Moines Area Religious Council's (DMARC) food pantry network of 14 separate food pantry sites; if a person uses a food pantry in the network, we designate that as an instance of hunger. Other research into hunger has shown a large increase in Google search terms like “foodbank” as a result of COVID-19,³¹ potentially highlighting the role that food charities play in pandemic hunger relief. Prior to COVID-19, DMARC network served an average of 19,948 individuals every month (www.dmarcunited.org/annual-report/). The greater Des Moines, IA area has a population of approximately 780,000 people, an unemployment rate approximately 1·9 percentage points lower than the national average, the cost of living in the area is 11·9% below the national average, but the median household income is approximately $5800 higher than the rest of the country.³² While this data is not representative of the entirety of U.S. population, this is observational data which overcomes issues like response bias associated with self-reported data. Further overcoming other potential data collection issues, data exists prior to the pandemic permitting both seasonal adjustments and examination of multiple public policy impacts. Finally, the data reflect the population of DMARC clients and therefore avoids issues from nonresponse error.

The DMARC data (Figure 2)d consists of the number of weekly visits made by SNAP beneficiaries and non-SNAP beneficiaries from 01 January, 2017 to 01 August, 2021. We use these two series as described in Study 1; the number of post-intervention visits made by SNAP beneficiaries is the factual whereas the number of visits made by non-beneficiaries predicts the counterfactual. However, the DMARC data allows us to further split these series by whether the family reports a child currently resides within their household, suggesting two separate analyses: one for households with children and one without. As a second difference, the several years of pre-intervention data allows for seasonal adjustments. We add a 52-week annual cycle, which accounts for several contributors to seasonal variability including holidays, seasonal unemployment, and cyclical shopping habits. This adds 51 binary indicators to the matrix of covariates. Because, with the longer time series, we are able to incorporate a high dimension of covariate data, we now impose a spike-and-slab prior on the coefficients corresponding to each of the covariates.³³ Practically, this means important signals are allowed to stay large while smaller signals can be shrunk exactly to zero. Statistically, this means that each coefficient is modelled as a mixture between a discrete point mass at 0 (the “spike”) and a diffuse normal distribution, allowing for significantly non-zero values (the “slab”). Of additional importance is that through the MCMC (Markov chain Monte Carlo) samples, the model effectively accomplishes Bayesian model averaging by averaging over the posterior probabilities with which each combination of covariates should be present in the model. The model can then be written as

$$y_t={\mu }_t+\sum {\beta }_j{x}_{jt}+{ϵ}_t^y{ϵ}_t^y\sim N\left(0,{\sigma }_y^2\right)$$ (3)

$${\mu }_t={\mu }_{t-1}+{ϵ}_t^{\mu }{ϵ}_t^{\mu }\sim N\left(0,{\sigma }_{\mu }^2\right)$$ (4)

$$\begin{array}{l}{\beta }_j={\delta }_j{\gamma }_j\\ {\delta }_j\sim Bernoulli\left(\pi \right)\\ {\gamma }_j\sim N\left(0,{\sigma }_{\gamma }^2\right),\end{array}$$ (5)

Figure 2.

The weekly number of visits by SNAP-beneficiaries and non-SNAP-beneficiaries to the DMARC food pantry network.

where ${\sigma }_{\gamma }^2$ is set proportional to ${\sigma }_y^2$, following Zellner's g-prior.

Model validation check

A model with high out-of-sample predictive power should be expected to yield counterfactual predictions that are very close to the observed counterfactual series before the intervention has taken place. That is, there should be no significant difference between the prediction and the truth when no impact has occurred to potentially shift one of the series. We run the same analysis as described in the previous section, but replacing the impact time with randomly selected dates from before any SNAP-interventions occurred and summarize the results. Several years of pre-intervention data allows us to compute these checks for Study 2, while a similar approach is not feasible for Study 1 due to the short pre-intervention period.

Results

Study 1: household pulse survey

The CausalImpact package uses an MCMC Gibbs sampler to sample from the joint posterior of unknown parameters. Using 10,000 samples from this posterior distribution we assess convergence of the MCMC chain using visual inspection of trace plots as well as calculation of effective sample sizes. Since samples from the posterior distribution are not independent, but instead serially related, effective sample sizes account for autocorrelation in order to give a more realistic view of how many independent ‘data points’ are effectively available with which to draw inference. Generally, 100 can be taken as a lower bound for the number of samples needed to draw inference. Trace plots of relevant parameters can be found in the appendix in Figures A.1–A.3. These show stationarity across the 10,000 draws, with no evidence of lack of convergence. Effective sample size is 10,000 for ${\beta }_1$ and ${\sigma }_y^2$ and 4138 for ${\sigma }_{\mu }^2$.

Figure A.1.

Trace plot of coefficient corresponding to ${\mathit{x}}_{\mathit{t}}$ in HPS model.

Figure A.3.

Trace plot of ${\mathit{\sigma }}_{\mathit{\mu }}^2.\mathit{in}\mathit{HPS}\mathit{model}$.

Comparison summaries of model-based counterfactual ${\widehat{y}}_t$and factual $y_t$ can be seen in Figure 1 while panel 1 of Table 2 provides numerical summaries of model results. In the three months following the increase to SNAP benefits occurring in January 2021, the counterfactual suggests an average of 11·09 million individuals experiencing food insufficiency per week without the intervention (95% credible interval 10·48–11·70 million). Instead, the realized number was 10·24 million which, notably, is below the lower bound of the credible interval. Our best estimate over this time period is the 15% increase in SNAP benefits reduced the total number of food insufficiency incidents by approximately 850,000 per week out of approximately 41·98 million individuals receiving SNAP benefits.

Figure 1.

Top panel shows the number of SNAP beneficiaries reporting food insufficiency (black line) along with the predicted counterfactual (blue line) with shaded pointwise 95% credible intervals. The middle panel shows estimates and 95% credible intervals for the difference between the factual and counterfactual. The bottom panel shows the cumulative difference.

Table 2.

Numerical summaries, including posterior means and 95% credible intervals, of model results from Study 1 (top panel) and Study 2 (bottom panel).

Impact Date		Effect Type	Observed (Factual)	Predicted (Counterfactual)	Predicted 95% CI	Effect (Observed - Predicted)	Effect 95% CI
			Study 1: HPS Estimates of Food Insufficiency
Jan-2021		Average	10,240,620	11,091,068	(10,484,539, 11,696,581)	-850,448	(-1,455,961, -243,919)
		Cumulative	51,203,100	55,455,339	(52,422,695, 58,482,903)	-4,252,239	(-7,279,803, -1,219,595)
			Study 2: DMARC Food Pantry Visits
Apr-2020	Child	Average	910	1234	(1093, 1372)	-324	(-463, -183)
		Cumulative	15,464	20,975	(18,582, 23,329)	-5511	(-7865, -3118)
	No Child	Average	385	415	(361, 469)	-30	(-84, 24)
		Cumulative	6546	7061	(6136, 7972)	-515	(-1,426, 410)
Jan-2021	Child	Average	630	768	(591, 946)	-138	(-315, 39)
		Cumulative	10,716	13,056	(10,047, 16,078)	-2,340	(-5362, 669)
	No Child	Average	293	362	(291, 428)	-69	(-135, 2)
		Cumulative	4978	6152	(4950, 7273)	-1,174	(-2295, 28)

We additionally consider cumulative effects in terms of the number of instances of hunger. In the three months following the intervention, HPS recorded 51·20 million instances of hunger. This represents a 7·7 percent decrease in incidents of food insufficiency nationally; without the intervention, we would have expected a total of 55·46 million instances (95% credible interval 52·42–58·48 million) over this post-intervention period. Further examining the cumulative effects, we see little change in instances of hunger in the initial period following the intervention; however, we see consistent significant decrease (95% credible interval excludes zero) beginning in the 17 February HPS survey wave. This means that the increase in SNAP benefits had a relatively rapid effect of observably decreasing hunger within two months. It is important to note that the survey is conducted on a bi-weekly basis but the measure of food insufficiency deals with the last seven days, meaning the count of food insufficiency is under-reporting cumulative incidence of food insufficiency prevented by increasing SNAP benefits. As a result, our estimates of the cumulative effects of the intervention are conservative in nature. In total, this evidence suggests that the single policy action increasing SNAP benefits prevented millions of instances of hunger.

Study 2: local robustness check

Using DMARC data, we measure the impact of both SNAP benefit interventions described previously and highlighted in Figure 2 by vertical lines. For the first intervention occurring in April 2020, we measure the impact only for the first three months post-intervention. In measuring the effect of the second intervention occurring in January 2021, we account for potential effects of the first intervention using a binary indicator variable.

Trace plots and effective sample sizes are computed for each model. Effective sample sizes for each parameter of interest are consistently estimated to be sufficient. The second panel of Table 2 displays numerical summaries of model results. Figure 3 displays model results for families reporting (orange) and not reporting (blue) a child in the household, as of the intervention date 01 April, 2020. In the first three monthse post intervention, the average weekly number of families with children visiting the pantry was 910 whereas the counterfactual was predicted to be an average of 1,234; over 300 (l-463 – -183) visits more per week than what was ultimately observed after the intervention. Cumulatively speaking, this amounts to 5511 (-7865 – -3118) fewer visits to the food pantry by families with children within the three-month span. Importantly, the effect of the intervention appears to be heterogeneous across families. In particular, for families not reporting a child in the household, we estimate a weekly decrease of 30 visits after the intervention; however, zero is well-within the 95% credible interval of the intervention effect (-84 – 24).

Figure 3.

Top panel shows estimates and 95% credible intervals for the difference between the factual and counterfactual. The bottom panel shows the cumulative difference. Date of intervention is April 01, 2020.

Figure 4 displays the model results examining the second impact, as was done in Study 1 with the HPS. In this case, for families with children, the probability of a causal effect after the second impact is smaller than measured after the first. Notably, for families with children, we estimate a weekly visit decrease of 138 – from a projected counterfactual of 768 visits to the factually observed 630 visits - on average, with a 95% credible interval of (-315–39); a result directionally consistent with the HPS analysis. For families without children, we estimate a weekly visit decrease of 69 (-135–2). Cumulatively, this corresponds to an estimated total of approximately 3500 fewer visits after the January 2021 impact. The posterior probability that the impact decreased panty usage for households with (without) children is 0·93 (0·97 f).

Figure 4.

Top panel shows estimates and 95% credible intervals for the difference between the factual and counterfactual. The bottom panel shows the cumulative difference. Date of intervention is 01 January, 2021.

As a model validation exercise, we selected 12 equally-spaced dates from 2019 – before any SNAP interventions had taken place – to serve as “impact” dates and reran the above analysis for households with and without children. In 20 out of the 24 validation analyses, we found a null effect in that 0 was comfortably within the prediction interval for a 3-month cumulative period.

Discussion

Hunger has been highlighted as key indirect problems caused by COVID-19 across the globe¹. The aim of this work was to determine to what extent, if any, additional SNAP benefits during the COVID-19 pandemic contributed to hunger relief. We focus our analysis on the U.S. and find there was a benefit of the government intervention during the pandemic reducing hunger; specifically, additional SNAP benefits reduced instances of food insufficiency.

We estimate the additional SNAP benefits beginning January 2021 prevented 850,000 incidents (nationwide) of food insufficiency per week through the end of March 2021. While these benefits continued further into the year, we limited the impact period to three months in an effort to avoid conflating factors. As this measure of food insufficiency is based on self-reported data, we conduct further robustness checks using observational data. We estimate that the April 2020 increase in SNAP benefits prevented over 5,000 visits (Des Moines, IA) over three months to the DMARC food pantry network. Similarly, our best estimate for the 15% increase in SNAP benefits in January 2021 prevented 3,500 additional visits (Des Moines, IA) to the DMARC food pantry over three months. Further, we show this reduction in visits is most pronounced in families with children, a finding counter to prior research on the SNAP increase with the ARRA that did not find any effect on food security for SNAP-eligible children.¹⁹

There are three particular strengths of this research. First, our analyses are multi-level (national vs local) and with multiple methods: communication (i.e. survey data) and observational (i.e. pantry usage). Second, in both cases, we have a counterfactual that, by definition, should not be impacted by SNAP benefit increases. Third, our modeling technique employs a flexible time series model rather than the standard difference-in-differences approach. In the absence of a randomized controlled trial, this is some of the strongest evidence to support the value of increased SNAP benefits on hunger during the COVID-19 pandemic.

Potential limitations to this research include generalizability of the results. Food purchasing behaviors are differentially affected by micro- and macroeconomic shocks,³⁴ this means the nature of the crisis could play a role in how people respond to relief efforts to ease hunger. So, the enhanced SNAP benefits enacted as a result of the COVID-19 pandemic may not be as effective in other situations. Further, as we model counterfactuals instead of observing a control group, our conclusions are based on the validity of model assumptions. That is, we are assuming that the model structure used to represent $y_t$ in the pre-intervention period remains to be a quality representation after the intervention. For example, error variability patterns as well as seasonal effects are assumed to remain constant. Violation of this would bring results into question. In addition, identifying “false positive” signals – while relatively unlikely - is possible, as demonstrated in the validation exercises.

Conclusion

Norman Borlaug said “Food is the moral right of all who are born into this world” in his Nobel Lecture.³⁵ But beyond this moral appeal, hunger is a matter of public health as food insufficiency is associated with both physical and mental health issues.¹⁰ Hunger was exacerbated in 2020 when food insufficiency more than doubled compared to the last economic crisis to hit the U.S., i.e. the Great Recession.⁴ Our research demonstrates that SNAP benefit increases prevented a large number of incidences of food insufficiency which means it may also be an instrumental tool in addressing other physical and mental health issues too. For example, the link between poor mental health and food insufficiency is diminished among those that receive free food.²⁴ By studying food insufficiency, and not food insecurity like previous research, we are able to show that this increase in SNAP benefits had a rapid effect on preventing hunger. Plainly stated, providing more SNAP funds results in a significant decrease in people reporting they did not have enough to eat in the last seven days.

These results have implications for both researchers and policymakers. First, while SNAP is a one of the strongest programs in preventing hunger in the U.S., the decrease in food insufficiency linked with a SNAP benefit increase may indicate the value of increasing the benefits further. As policymakers have the tools to both expand who is covered by SNAP and the benefit amount from SNAP, this research can be used as a building block to study the sufficiency of SNAP benefits. Second, by quantifying the estimated number of incidents of food insufficiency prevented by increasing benefits during COVID, we give policymakers values to evaluate the return on spending as well as to provide benchmarks to evaluate the effectiveness of future policy actions. Third, SNAP benefit increases have a rapid effect on preventing hunger, i.e. noticeable effects on food insufficiency within two months, highlighting how useful the action is in policymakers’ toolkit. As the infrastructure for distributing SNAP benefits already exists, increasing SNAP benefits should be one of the first actions policymakers take when tackling the problem of hunger broadly.

Contributors

Both authors were involved in the conceptualization, study design and methodology. LF performed data analysis and visualization. Both authors were involved in writing the manuscript.

Data sharing statement

HPS data and summary tables are available via the U.S. Census (https://www.census.gov/programs-surveys/household-pulse-survey.html). DMARC visit data summarized by week is available on request to the corresponding author.

Declaration of interests

We declare no competing interests.

Appendix

Figure A.1, Figure A.2, Figure A.3

Figure A.2.

Trace plot of ${\mathit{\sigma }}_{\mathit{y}}^2\mathit{in}\mathit{HPS}\mathit{model}.$