How Much Does the Supplemental Nutrition Assistance Program Reduce Food Insecurity?

Abstract

Nearly 15% of all U.S. households and 40% of near-poor households were food insecure in 2009. The Supplemental Nutrition Assistance Program (SNAP) is the cornerstone of federal food assistance programs and serves as the first line of defense against food-related hardship. This paper measures the effectiveness of SNAP in reducing food insecurity using an instrumental variables approach to control for selection. Our results suggest that receipt of SNAP benefits reduces the likelihood of being food insecure by roughly 30% and reduces the likelihood of being very food insecure by 20%.

The United States has one of the highest standards of living in the world, yet nearly 15% of all U.S. households and 40% of its near-poor households (below 130% of the poverty threshold) were food insecure in 2009 (Nord et al. 2010). Food insecurity has been connected with an array of negative outcomes, including poor health among children, lower academic achievement, and depression (Oberholser and Tuttle 2004). The Supplemental Nutrition Assistance Program (SNAP, formerly called the Food Stamp Program) is the largest food-assistance program in the United States and is the cornerstone of the federal food-assistance programs. It serves as the first line of defense against hunger and is designed to reduce food-related hardship, such as food insecurity. A key policy question is: how effective is SNAP in reducing food insecurity? Understanding the effectiveness of SNAP in meeting its goal is important for state SNAP administrators as they make changes to their programs.

Identifying the extent to which SNAP reduces food insecurity is complicated by the fact that households that do and do not receive SNAP benefits can differ in systematic ways. Households that are most needy and food insecure are more likely to be eligible for and to take up SNAP benefits, so simple comparisons of food insecurity for those who do and do not receive SNAP benefits are likely to find better outcomes for those who do not receive SNAP benefits. Selection of more needy households into SNAP makes it difficult to identify a causal relationship between SNAP participation and food insecurity.

This article measures SNAP’s effectiveness in reducing food insecurity using a dummy endogenous variable model with instrumental variables to control for selection bias. Changes in state SNAP policies and rules in the 1990s and 2000s provide exogenous variation, which we use to identify the model and control for selection into the program. The federal government began to give states flexibility to change SNAP policies in the mid- to late-1990s. These changes culminated in the Farm Security and Rural Investment Act of 2002 (the Farm Bill), which provides broader flexibility to states to set SNAP policies (and rules). Additional flexibility was subsequently provided. This variation in SNAP policies across states and over time provides the identifying variables (i.e., instruments) for the analysis. Household-level data come from the nationally representative, longitudinal 1996, 2001, and 2004 Survey of Income and Program Participation (SIPP) panels. State-level SNAP policy data come primarily from the Food Stamp Program State Rules Database.¹

We examine two measures of food-related hardship. One measure captures whether households are food insecure, while the second captures a higher degree of hardship and identifies households that are very food insecure.² Results from models that do not control for the endogeneity of SNAP receipt show that SNAP receipt is associated with higher food insecurity (both measures). This finding is consistent with the self-selection of more needy and food-insecure households into SNAP. However, instrumental variable models that control for the endogeneity of SNAP receipt suggest a different relationship. In these models, the receipt of SNAP benefits is found to reduce the likelihood of being food insecure and very food insecure.

Relevant Literature and Contribution

There is a growing body of literature examining SNAP participation and food insecurity. This literature uses a mix of methods and finds a mix of results. Studies have found that SNAP participants are more likely than nonparticipants to be food insecure or insufficient (Alaimo et al. 1998; Cohen et al. 1999; Jensen 2002; Ribar and Hamrick 2003; Wilde and Nord 2005). Other studies have found that SNAP participation has no statistically significant effect on food insecurity or insufficiency (Gibson-Davis and Foster 2006; Gundersen and Oliveira 2001; Huffman and Jensen 2008). These studies acknowledge concerns about selection into SNAP and several take steps to address this selection. For example,Wilde and Nord (2005) use a panel data approach, Gibson-Davis and Foster (2006) use propensity score matching (and caution against it), and Gundersen and Oliveira (2001) use an instrumental variable (IV) approach and simultaneous probit model for SNAP participation and food insufficiency.³ Gundersen and Oliveira (2001), in one of the early studies that address selection, use an imputed measure of stigma (shop at store where unknown) as the instrument to identify the food-insecurity equation. In their naïve model that does not control for selection, they find that SNAP receipt statistically significantly increases food insufficiency. The coefficient on SNAP receipt remains positive in their IV model, although the large standard error (SE) makes the coefficient statistically insignificant at conventional levels.

While numerous studies find no evidence that SNAP reduces food-related hardship, several studies find evidence that SNAP is associated with reduced food insecurity or insufficiency (Bartfeld and Dunifon 2006; Borjas 2004; DePolt, Moffitt, and Ribar 2009; Nord and Golla 2009; Yen et al. 2008).⁴ These studies examine different populations and/or use a variety of data and methods.

Bartfeld and Dunifon (2006), Borjas (2004), and Nord and Golla (2009) use the Current Population Survey (CPS) to estimate the relationship between program participation and food insecurity. Borjas focuses on immigrants, Bartfeld and Dunifon examine households with children, and Nord and Golla (2009) focus on all households that receive SNAP benefits. The analytic approach of these three articles also differs. Bartfeld and Dunifon (2006) use hierarchal regression and find that low-income and near-poor families in states with higher SNAP participation rates are less likely to be food insecure. Borjas, on the other hand, uses an IV approach and finds that increased immigrants’ public assistance participation (cash benefits, SNAP, or Medicaid) reduces immigrants’ food insecurity. This IV approach controls for selection into SNAP with an instrument for public assistance participation that captures the generosity of states’ immigrant eligibility rules after the 1996 federal welfare reform. Nord and Golla (2009) construct a synthetic panel using monthly data to examine household food insecurity before and after SNAP receipt. They find that food insecurity falls by roughly one-third after entry into SNAP.

Yen et al. (2008) and DePolt, Moffitt, and Ribar (2009) use smaller data sets that are not representative of the U.S. population and find results that may be explained in part by their datasets. Yen et al. (2008) use data from the 1996–97 National Food Stamp Program Survey, which is a survey of roughly 2,200 SNAP participants and income-eligible nonparticipants. DePolt, Moffitt, and Ribar (2009) use data from the Three-City Study (Boston, Chicago, and San Antonio), which includes roughly 2,500 families with children who had incomes below 200% of the federal poverty level at the initial interview (in 1999) and includes two follow-up interviews (in 2000–2001 and 2005).⁵

Both of these studies use methods to address the endogeneity of SNAP receipt. Unlike much of the literature, however, the studies find that SNAP participation is associated with lower food hardship in descriptive statistics or models that do not control for the self-selection into SNAP.^6,7 DePolt, Moffitt, and Ribar (2009) explain that this finding could be due to additional control variables in the Three-City Study data that are often not available in other data sets. When DePolt, Moffitt, and Ribar (2009) use Chamberlain’s quasi–fixed-effect model to control for unobserved family characteristics that may affect both SNAP receipt and food hardship, they find a similar, although generally stronger, negative relationship. Yen et al. (2008) use IV models to control for the endogeneity of SNAP participation and find that SNAP participations lowers the severity of food insecurity.

A working paper by Kreider et al. (2009) uses partial identification bounding methods to provide bounds on the impact of SNAP receipt on children’s food insecurity. Using a sample of roughly 4,400 near-poor children from the 2001–2006 nationally representative National Health and Nutrition Examination Survey, their analysis provides evidence that SNAP participation improves children’s food security and has positive effects on other child health outcomes.

Our study advances the literature on the relationship between SNAP participation and food-related hardship by using the recent, large, and nationally representative SIPP dataset from the late 1990s to 2005 and taking advantage of variation in state SNAP policies to control for selection into SNAP. In addition to using nationally representative and more recent data with new policy variation, our data cover multiple years and thus enable us to include year and state dummy variables in our empirical model to better control for potential endogeneity than earlier studies.⁸ Our analysis shows that state SNAP rules and policies are important determinants of SNAP participation, and our findings provide evidence that SNAP reduces food-related hardship. Given inherent difficulties in identifying causal effects where individuals self-select into programs (such as SNAP), the conclusions of any single analysis cannot be taken as definitive. The solid methodology and robust findings in this study make a strong contribution to a growing body of literature that finds SNAP reduces food insecurity.

Conceptual Model

Below we discuss the determinants of food insecurity and SNAP participation, followed by a discussion of the instrumental variables for SNAP participation.

Determinants of Food Insecurity

At the micro level, food insecurity is a function of earned income, public and private transfers, and household composition and food needs (e.g., household size, age/gender composition).⁹ Each is chosen, to some degree, by household members. Because our primary focus is on the role that SNAP plays in food insecurity, we model food insecurity as a function of SNAP participation and the reduced-form determinants of earned income, public and private transfers, and household composition and food needs. The reduced-form determinants and their hypothesized effects are based on human capital theory (Becker 1975) and Becker’s (1991) theory of the demand for children. State and year dummy variables and economic characteristics are included in our empirical model to control for macro-level variables.

Additional children in the household (especially young children) are hypothesized to increase food insecurity through their negative effect on wage labor hours and positive effect on household food needs. Additional working-age adults in the household are hypothesized to increase household labor supply (and earnings) and decrease food insecurity. However, if these additional adults do not work, then food insecurity can increase with the number of adults. Having a disabled person in the household is hypothesized to decrease labor supply (and earnings) because the individual may be unable (or limited in his or her ability) to work, and because another household member’s work hours may be limited by his or her need to care for the disabled individual. Increases in human capital are hypothesized to increase income, and thus, decrease food insecurity. Being young, a minority, a noncitizen, and/or female is hypothesized to lower income through the negative effects of such status on wages and thus increase food insecurity. Finally, improvements in the state of the economy are hypothesized to increase household income (through their positive effect on wages and the hours household members can choose to work) and reduce food insecurity. These variables provide the reduced-form control variables for our empirical model.

Hypothesized effect of SNAP participation on food insecurity

SNAP participation can have a direct mechanical effect on household food insecurity, as well as an indirect behavioral effect. The direct effect is hypothesized to reduce household food insecurity, while the indirect effect is hypothesized to increase household food insecurity.

SNAP provides direct support to households so that the household can purchase food. Because the program transfers resources to households, we hypothesize that the direct mechanical effect of SNAP participation is to reduce food insecurity. On the other hand, the availability of additional resources to purchase food could lead SNAP-participating households to reduce their labor supply and, thus, earnings. For example, household members might choose to reduce their labor supply in order to receive a larger benefit or become eligible for the program. Ceteris paribus, reduced earnings could lead to reduced food purchases and increased food insecurity. Thus, we hypothesize that the indirect behavioral effect of SNAP participation is to increase food insecurity. Overall, however, we expect the direct effect to dominate the indirect effect,¹⁰ and hypothesize that SNAP participation leads to lower levels of food insecurity.

Determinants of SNAP Participation

Participation in SNAP is affected by demographic and household characteristics and by the rules of the program. These program rules determine whether a family is eligible to participate, as well as the costs and benefits of program participation. Eligibility is a prerequisite for participation in any means-tested program. In some cases, family members can change their behavior to meet eligibility requirements (e.g., reduce earnings below the required threshold), while in other cases this is not possible (e.g., become a nonimmigrant to avoid eligibility restrictions on immigrants). Program rules can also affect the cost (pecuniary and nonpecuniary) of participation. For example, biometric technology (typically fingerprint imaging), which is used by some states to reduce multiple participation fraud, can increase the costs of participation. Program rules that lower the cost of participation are hypothesized to increase program participation, while program rules that increase the cost of participation are hypothesized to decrease program participation.

Instruments for SNAP Participation

Our estimation approach uses state SNAP program rules to identify the IV model. Program rules that are strong instruments are those that affect SNAP participation but do not directly lead to different levels of food insecurity conditional on SNAP participation status. We identify four IVs for our analysis: use of biometric technology, outreach spending, full immigrant eligibility, and partial immigrant eligibility.

Biometric technology is hypothesized to increase the costs of SNAP participation and thus decrease participation. We measure outreach spending on a per capita basis, where the target population consists of those living below 150% of the poverty threshold who are not food stamp recipients. Higher outreach spending by states is hypothesized to increase participation via an increase in the number of SNAP applicants (due to increased knowledge about SNAP). Finally, more lenient immigrant eligibility rules are hypothesized to increase SNAP participation among immigrants. States identified as having “full immigrant eligibility” are those in which all legal non-elderly adult immigrants who meet other program requirements are eligible for federal benefits or state-funded food assistance. States identified as having “partial immigrant eligibility” are those in which some, but not all, legal non-elderly adult immigrants who meet other program requirements are eligible for federal benefits or state-funded food assistance.

For our model to be identified, these policy changes cannot be endogenous to the processes under investigation. Our model design helps alleviate this problem by including state dummy variables (which control for time-invariant unobservable heterogeneity across states) and year dummy variables (which control for unobservable heterogeneity across years). A priori reasoning suggests that states were not setting policies in response to food insecurity, and so the policies are less likely to be endogenous; states were setting policies (during the time period covered by the analysis) largely in response to changes in federal policy. Still, we present tests, descriptive statistics, and qualitative evidence regarding the exogeneity of our variables below.

We can test the exogeneity of our IVs, but the test requires the assumption that at least one instrument is exogenous. We present qualitative research and descriptive statistics to argue that it is reasonable to assume that the immigrant eligibility rules are exogenous in the context of our model (which includes state and year dummy variables). Also, this assumption is consistent with Borjas’s (2004) study of immigrant food insecurity, which uses state immigrant eligibility rules to identify his IV model.

Evidence based on a survey of state officials (in all fifty states and the District of Columbia) and subsequent analyses does not suggest that changes in immigrant eligibility rules, which in part resulted from federal policy changes in the 1996 welfare reform legislation, were in response to trends in food hardship (Zimmermann and Tumlin 1999). The authors find that lenient immigrant eligibility rules were implemented by more generous and wealthier (higher per capita income) states. This is consistent with findings from the welfare reform literature that suggests that state policy changes to the Temporary Assistance to Needy Families program were not implemented in response to changes in hardship (Holcomb and Martinson 2002).

Analyses of our data show that states that expanded SNAP eligibility to immigrants included those that had both below and above average levels of food insecurity. Also, some states that extended benefits to immigrants also implemented policies that can deter SNAP participation, such as use of biometric technology (e.g., Massachusetts, Texas). While these descriptive statistics do not definitively answer the question of whether immigrant eligibility policy variables are exogenous, they complement findings in the qualitative literature.¹¹ The quality of our instruments is further discussed below.

Empirical Model

Our empirical model uses an IV approach to control for the endogeneity of SNAP participation. We estimate a bivariate probit model with an endogenous dummy variable, using state SNAP policies as instrumental variables. As discussed in the conceptual model, some SNAP policies are hypothesized to affect food insecurity only through their effect on SNAP participation, and these variables identify the model. With this approach, only the effects of SNAP participation that are correlated with these SNAP program rules are included in the causal effect of participation.

We measure the total (direct and indirect) effect of SNAP participation on food insecurity using a dummy endogenous variable model (Heckman 1978) with IVs. Our model consists of two equations: one equation relating food insecurity to SNAP participation and a second, reduced-form equation describing SNAP participation as a function of state program rules. The two equations are as follows:

$$\begin{array}{lll}{Y}_{\mathrm{ist}}^{\ast }& =& \beta {\mathrm{SNAP}}_{\mathrm{ist}}+X_{\mathrm{ist}}{\gamma }_2+E_{\mathrm{st}}{\gamma }_3\\ & & +{\mu }_s+{\tau }_t+{\epsilon }_{\mathrm{ist}}^Y\end{array}$$ (1)

where Y_ist = 1 if $Y_{\mathrm{ist}}^{\ast }>0$ and Y_ist = 0 otherwise; and

$$\begin{array}{lll}{\mathrm{SNAP}}_{\mathrm{ist}}^{\ast }& =& Z_{\mathrm{st}}{\delta }_1+X_{\mathrm{ist}}{\delta }_2+E_{\mathrm{st}}{\delta }_3\\ & & +{\mu }_s+{\tau }_t+{\epsilon }_{\mathrm{ist}}^S\end{array}$$ (2)

where SNAP_ist = 1 if ${\mathrm{SNAP}}_{\mathrm{ist}}^{\ast }>0$ and SNAP_ist = 0 otherwise.

In this model, Y_ist is an indicator variable measuring whether household i in state s at month t is food insecure (or very food insecure). SNAP_ist is an indicator variable for whether household i in state s at month t participates in SNAP. The coefficient on SNAP participation (β) captures the total effect of participation, including both the direct effect of participation and the indirect effect through, for example, changes in labor supply.

The remaining explanatory variables in the equations of our primary specification are drawn from the conceptual framework described above.¹² X_ist is a vector of variables controlling for individual-level and household-level characteristics (age, race and ethnicity, noncitizen immigrant, educational attainment, number of children and adults in household, female- and male-headed household, disabled person in household, and metropolitan status).¹³ The vector Z_st represents the instruments and includes four specific state SNAP policies (biometric technology, outreach spending per capita, partial immigrant eligibility, and full immigrant eligibility).¹⁴ E_st is a vector of time-varying variables controlling for economic conditions (state monthly unemployment rate, state monthly employment/population ratio, state annual per capita income, and quarterly gross GDP). Finally, μ_s is a set of state dummy variables, τ_t is a set of year dummy variables, and ${\epsilon }_{\mathrm{ist}}^Y$ and ${\epsilon }_{\mathrm{ist}}^S$ are the error terms. State and year dummy variables are included in all equations to control for state- and year-specific unobservable factors that affect SNAP participation or food insecurity.¹⁵ To account for potential serial correlation in the error term, we cluster our SEs by state.

We estimate a bivariate probit model because the dependent variables in equations (1) and (2) are binary—food insecure or very food insecure (yes/no) and SNAP participation (yes/no). We assume the error terms are draws from a bivariate normal distribution with mean zero and variance of one, and estimate the equations simultaneously using a bivariate probit model. The correlation coefficient is $\rho =\mathrm{Cov}({\epsilon }_{\mathrm{ist}}^Y,{\epsilon }_{\mathrm{ist}}^S)$. If ρ ≠ 0, then the error terms are correlated and probit estimation of equation (1) ignoring equation (2) will yield inconsistent estimates of the parameters due to the endogeneity of SNAP participation.

The bivariate probit model is appropriate for the dichotomous nature of the dependent variable and endogenous participation regressor, but it places parametric structure on the data generating process (e.g., the errors are drawn from a bivariate normal distribution).¹⁶ Our model also restricts β—the effect of SNAP participation on food insecurity—to be the same across households. The implications we draw from our results are conditional on these assumptions. Kreider et al. (2009) use partial identification bounding methods, which impose relatively weak assumptions, to provide bounds on the impact of SNAP receipt on children’s food insecurity. Although our approach differs from Kreider et al.’s (2009) along several dimensions, our estimate of β falls within their estimated bounds. Future studies might consider allowing the estimated effect of SNAP participation on food insecurity to vary across households.

The ability of our bivariate probit model to correct for the endogeneity of SNAP receipt depends on the explanatory power of the instruments in the SNAP receipt equation (equation (2)) and on whether it is appropriate to exclude the instruments from the food-insecurity equation. Our set of instruments (biometric technology, outreach spending, partial immigrant eligibility, and full immigrant eligibility) has good predictive power in the SNAP receipt equation, with a joint test of significance indicating that the instruments are jointly statistically significant at the 1% level. Following Evans and Schwab (1995) and Yoruk (2009), we test the exogeneity of our instruments using a linear IV model. Tests to evaluate the quality of IVs are well developed for the linear framework. Based on Hansen’s J-test, we do not reject the null hypothesis that the instruments are exogenous.¹⁷ We also examine the strength of our instruments using the Kleibergen–Paap statistic, which provides evidence that the IVs identify the model; we reject the null hypothesis that the model is underidentified.¹⁸ Taken together, these characteristics suggest that the instruments influence food insecurity only through their effect on SNAP participation.

In addition to the bivariate probit model, we estimate a naive probit model of the effect of SNAP receipt on food insecurity (equation (1)). If food-insecure households are more likely to become SNAP participants conditional on their observed characteristics, the estimated coefficients from this model will be biased. A comparison of the probit and bivariate probit model results highlights the importance of correcting for this selection.

Study Population

Selection of the study population is an important element of this study, as an inappropriate study population could lead to biased estimates. Because the study focuses on SNAP participation and food insecurity, one might select the study population to include only households eligible for SNAP. Defining the study population this narrowly has a drawback, however. Focusing only on the SNAP-eligible population excludes households that can slightly alter their behavior to become eligible for benefits (i.e., it excludes households at the margin). Ashenfelter (1983), for example, argues that if the elasticity of labor supply does not equal zero, the pool of persons that should be examined as eligible for a program is larger than those who would actually qualify for the program under current income and asset limits. The concern with limiting the sample to the SNAP eligible population is that it results in a sample of households that are disproportionately more likely to alter their behavior to become eligible for SNAP benefits. Carrying out our analysis on a select group of households may produce biased estimates. As a result, we carry out our primary analyses on a more broadly defined study population and then conduct sensitivity tests with a more restricted population.

Our primary study population includes low-income households defined as being below 150% of the poverty threshold and having readily available assets of less than or equal to $4,000 or $5,000 if at least one household member is age 60 or older.¹⁹ We also carry out robustness checks on a secondary population that more closely mimics the SNAP-eligible population—households with incomes below 130% of the poverty threshold and readily available assets of less than or equal to $2,000 or $3,000 if at least one household member is age 60 or older.²⁰

Data

Individual-level data for the analysis come from the 1996, 2001, and 2004 SIPP panels. Each of these SIPP panels contains a nationally representative (noninstitutional) sample of between 36,000 and 46,000 households whose members are interviewed at four-month intervals about the previous four months (these four-month intervals are referred to as waves).²¹ In addition to collecting monthly data on many demographic and economic characteristics, the SIPP includes “topical modules” that ask periodic questions about a variety of topics, including material well-being and asset holdings. The timeframe covered by the topical modules varies by topic area and question.

A key strength of the SIPP is its monthly data on SNAP participation, income, and household composition. At each interview, data are collected on these and other variables for each of the preceding four months. SNAP benefits are received monthly, not annually, so the monthly SIPP data allow participation to be examined over the same time period that benefits are received. All household-level characteristics identified in the conceptual framework are available in the SIPP.

Household food insecurity is self-reported and constructed using a series of questions available in the adult well-being topical module. These questions ask respondents about the household’s food-related hardship over the prior four months and are asked once in each of the three SIPP panels.²² Combining the 1996, 2001, and 2004 SIPP panels provides information on whether households are food insecure and very food insecure in three separate years—1998 (April to October), 2003 (February to August), and 2005 (February to August). While the SIPP does not provide this information on a more regular basis (e.g., annually), the SIPP data do provide these food-insecurity measures in years when state SNAP policies were changing and in strong and weak economic times. It is this variation that allows us to identify our empirical model.

Our food-insecurity measures take account of whether households have enough food to eat and whether households are able to afford balanced meals. Five questions in the SIPP topical module are used to generate our two indicators of household food-related hardship:

1. The food that you bought just didn’t last and you didn’t have money to get more. Was that [this statement] often, sometimes, or never true for you in the last four months?

2. You couldn’t afford to eat balanced meals. Was that [this statement] often, sometimes, or never true for you in the last four months?

3. In the past four months did you or the other adults in the household ever cut the size of your meals or skip meals because there wasn’t enough money for food?

4. In the past four months did you or the other adults in the household ever eat less than you felt you should because there wasn’t enough money to buy food?

5. In the past four months did you or the other adults in the household ever not eat for a whole day because there wasn’t enough money for food?

These five questions are used in conjunction with the methodology developed by the USDA’s Economic Research Service (Nord 2006) to generate our two indicators of food-related hardship.²³ Our first measure identifies households that have low or very low food security, while the second is a more severe measure that identifies households that have very low food security.²⁴ We refer to these levels of food hardship as “food insecure” and “very food insecure,” respectively.²⁵

These two food-insecurity measures capture households’ experiences over the full SIPP wave (i.e., a four-month period), while household characteristics such as SNAP participation, income, and household structure are available each month of the wave.²⁶ Household characteristics in each month of the wave can influence households’ food insecurity, because the outcomes are measured over the four-month period. Thus, households are included in our sample up to four times—once for each month in the wave that food insecurity is measured.²⁷

A potential weakness of the SIPP involves concerns about the underreporting of SNAP receipt. Estimates suggest that the SIPP underreports SNAP receipt by 7% to 19% (Bitler, Currie, and Scholz 2003; Bollinger and David 1997; Cody and Tuttle 2002), which is somewhat lower than the Current Population Survey of the U.S. Census Bureau and the Bureau of Labor Statistics.²⁸ Analyses suggest that SNAP reporting error is due primarily to SNAP recipients who do not report SNAP benefit receipt, not nonrecipients who report receiving SNAP benefits. Using SIPP data matched to administrative data, for example, Bollinger and David (1997) find that nearly all of the SNAP reporting error stems from recipients who do not report receiving benefits (12%); only 0.3% of nonrecipients report receiving SNAP (i.e., report a false positive).

The underreporting of SNAP receipt has the potential to bias the results.²⁹ However, if the SNAP classification error is uncorrelated with the instruments, then estimates from the standard IV model are unbiased (Wooldridge 2003). A priori reasoning does not suggest that state SNAP policies are correlated with the underreporting of SNAP receipt, so our estimates are likely not biased as a result of this underreporting. Also, Kreider et al. (2009) estimate bounds on the impact of SNAP receipt on children’s food insecurity under a series of misreporting assumptions. Their estimated bounds generally widen as the level of misreporting increases, but the bounds are substantially tighter under the assumption of no false positives versus arbitrary misreporting. Focusing on their results that assume no false positives—a reasonable assumption given findings from the literature—our estimate of the effect of SNAP participation on food insecurity falls within or near their bounds.³⁰ Further investigation of classification error is a potential topic for future research. Our analysis uses the SIPP weights to help account for attrition, nonresponse, and a complex sample design.

Among our sample of low-income households, 24.4% were food insecure and 10.3% were very food insecure (table 1). Comparisons of these outcomes for households that do and do not participate in SNAP show higher rates of food insecurity among SNAP-recipient households. While 35.6% of SNAP-recipient households are food insecure, 19.9% of nonparticipating households are food insecure. Similarly, the proportion of households that are very food insecure is higher among SNAP-participating than SNAP-nonparticipating households—15.4% and 8.3%, respectively. The higher rates of food-related hardship among SNAP participants suggest that these households are more needy and are more likely to self-select into SNAP.

Table 1.

Low-Income Households’ Food-Related Hardship and SNAP Participation, Households with Income below 150% of Poverty Threshold^a

Variable	All Households	SNAP Participants	SNAP Nonparticipants
Food Insecure	24.4%	35.6%	19.9%
Very Food Insecure	10.3%	15.4%	8.3%
SNAP Receipt	28.6%	100%	0.0%
Number of Observations	65,269	20,197	45,072

Note: All percentages are weighted.

^aSample includes households with income below 150% of the poverty threshold who have liquid assets below $4,000, or below $5,000 if one member of the household is age 60 or older.

SNAP receipt is quite prevalent among low-income households, with roughly one-quarter (28.6%) of our sample receiving SNAP benefits. As compared with non-SNAP recipients, SNAP-recipient households tend to be younger, minority, less educated, and female headed, and to have more children and include a disabled member. Food-insecure households have these same tendencies, as compared with households that are food secure.³¹

To control for changes in the economy, the SIPP data are supplemented with (a) state monthly unemployment rates, (b) state monthly employment/population ratio, (c) state annual per capita income, and (d) quarterly GDP from the U.S. Department of Commerce (2008).

Measures of the state-specific SNAP rules—biometric technology, outreach spending per capita, and immigrant eligibility—come largely from USDA’s Food Stamp Program State Rules Database. This database contains data through only December 2004, so we used additional documents provided to us by the USDA to update these variables through 2005. We merge SNAP rules from this database with household-level SIPP data by state, year, and month.

Five states used biometric technology at some point over the study period. Two of the states made changes to their policies—California began to use biometric technology, and Massachusetts ceased using biometric technology in the early 2000s. Twenty-five states changed their level of outreach spending over the analysis period, with nineteen of these states making multiple changes in their level of spending. Average state per capita outreach spending has varied over the study period from a low of less than 1 cent per person to a high of 4.5 cents per person. Finally, thirty-five states began or ceased providing benefits for some, but not all, legal non-elderly adult immigrants (partial immigrant eligibility), and five states made changes to whether they provided benefits to all legal non-elderly adult immigrants (full immigrant eligibility). Over the analysis period, for example, two states (California and Wisconsin) moved from not providing to providing full immigrant eligibility, two states (Massachusetts and Rhode Island) shifted from providing to not providing full immigrant eligibility, and one state (Minnesota) had changes in both directions.³² In the analysis, the state-level immigrant eligibility variables are interacted with an indicator of whether the household includes noncitizen immigrants.

Results

Below we present results from our primary specification for the food insecure and very food insecure outcomes. Next, we present additional specifications that highlight the robustness of the findings.

Food Insecure

The naive probit model results, which do not control for the endogeneity of SNAP receipt, show that SNAP receipt is associated with higher food insecurity. The estimated coefficient on SNAP participation suggests that participating in SNAP is associated with an 8.6-percentage-point increase in the probability of being food insecure (table 2, model 1). This result is consistent with the self-selection of more needy and food-insecure households into SNAP.

Table 2.

Estimates of the Effects of SNAP Participation on the Likelihood of Being Food Insecure and Very Food Insecure, Households with Income below 150% of Poverty Threshold^a

Explanatory Variable	Food Insecure				Very Food Insecure
	Probit (1)		Bivariate Probit (IV) (2)		Probit (3)		Bivariate Probit (IV) (4)
	Coeff/SE	Marginal Effect	Coeff/SE	Marginal Effect	Coeff/SE	Marginal Effect	Coeff/SE	Marginal Effect
SNAP Receipt	0.275*** (0.028)	0.086	−0.582*** (0.091)	−0.162	0.208*** (0.033)	0.034	−0.268*** (0.062)	−0.039
Rho			0.509*** (0.054)				0.284*** (0.035)
Number of Observations	65,269		65,269		65,269		65,269

Note: The unit of observation is a household-month. Robust standard errors are presented within parentheses. Standard errors are adjusted for clustering by state. All models include controls for age, age squared, noncitizen immigrant, black, Hispanic, other non-white race, no high school degree, high school degree only, number of children in household, number of adults in household, female-headed household, male-headed household, disabled person in household, and metropolitan area; state unemployment rate, state employment-population ratio, state per capita income, and gross domestic product; and state and year dummy variables. Instrumental variables are biometric technology, outreach spending per capita, and immigrant eligibility rules (i.e., all legal immigrants eligible interacted with noncitizen and some legal immigrants eligible interacted with noncitizen).

^aSample includes householdes with income below 150% of the poverty threshold who have liquid assets below $4,000, or below $5,000 if one member of the household is age 60 or older.

^***p < 0.01,

^**p < 0.05,

^*p < 0.1.

Results from the bivariate probit model, which controls for the endogeneity of SNAP receipt, suggest a different relationship. The receipt of SNAP benefits is found to reduce the likelihood of food insecurity by 16.2 percentage points (table 2, model 2).^33,34 Nearly one-quarter of our low-income sample is food insecure, so these results suggest that SNAP has a substantial effect on households’ food insecurity and is achieving what the program was designed to do—reduce food-related hardship. To further put this number in context, we use the marginal effect along with the SNAP recipients’ level of food insecurity to estimate the percent (versus percentage point) decline in food insecurity implied by our model. Our summary statistics show that 35.6% of SNAP recipients are food insecure. The bivariate probit model estimates suggest that SNAP recipients’ food insecurity would be 16.2 percentage points higher (51.8%), if SNAP benefits were not available. The decrease in the likelihood of food insecurity from 0.518 without the SNAP program to 0.356 with the program suggests that SNAP receipt reduces food insecurity by 31.2%. This result relies on the assumptions underlying the bivariate probit model and restricts the effect to be the same across households. Also, if the SNAP classification error is correlated with our state policy instruments, then this reporting error could impact our estimated effect. Nonetheless, the magnitude of this decline is consistent with the findings of Nord and Golla (2009), whereby the likelihood of being very food insecure falls by roughly one-third when households begin receiving SNAP benefits, and also falls within the bounds estimated by Kreider et al. (2009).

A comparison of the SNAP receipt coefficients from the probit and bivariate probit models suggests that controlling for selection into SNAP is important for disentangling the effect of SNAP receipt on food insecurity. The model that does not control for the endogeneity of SNAP receipt shows that SNAP participation is associated with increased food insecurity, while the model that does control for the endogeneity shows that SNAP participation reduces food insecurity. Further, the correlation coefficient from the bivariate probit model indicates a positive and statistically significant correlation between unobservables that affect SNAP receipt and food insecurity (ρ = 0.509, SE = 0.054).

As discussed above, the validity of our IV model depends on the quality of the instruments. Under the assumption that one of the four instruments (immigrant eligibility) is exogenous, we are able to test the exogeneity of the other three instruments. Using Hansen’s J-test, we conclude that the instruments are exogenous (we do not reject the null hypothesis that the instruments are exogenous, p = 0.89). Using the Kleibergen–Paap statistic, we test whether the IVs identify the model and reject the null hypothesis that the model is underidentified (p = 0.0002). Each of the four instruments has the anticipated sign and statistically significantly (p < 0.1) affects SNAP receipt (table 3), and a joint test for significance of the four instruments indicates that they are jointly statistically significant at the 1% level (χ²(4) = 15.8, p = 0.003). The coefficients and SEs of the instruments from the reduced-form SNAP participation equation are presented in table 3 (column 1).

Table 3.

Bivariate Probit (IV) Estimates of the Effect of SNAP Participation on the Likelihood of Being Food Insecure, Households with Income below 150% of Poverty Threshold^a

Explanatory Variable	SNAP Participation	Food Insecure
SNAP participation		−0.582*** (0.091)
Instruments-State Food Stamp Rulesb
Biometric technology	−0.269*** (0.095)
Outreach spending per capita	0.402* (0.228)
All legal immigrants eligible × noncitizen immigrant	0.370** (0.201)
Some legal immigrants eligible × noncitizen immigrant	0.312* (0.180)
Demographic Characteristics
Age	−0.016*** (0.004)	0.021*** (0.005)
Age squared	0.000** (0.000)	−0.000*** (0.000)
Noncitizen immigrant	−0.414** (0.173)	−0.015 (0.055)
Race/Ethnicity (Omitted: White, non-Hispanic)
Black, non-Hispanic	0.383*** (0.028)	0.290*** (0.033)
Hispanic	0.214* (0.120)	0.220*** (0.074)
Other, non-Hispanic	0.294*** (0.078)	0.144** (0.072)
Educational Attainment (Omitted: More than high school)
Less than high school	0.462*** (0.037)	0.282*** (0.030)
High school only	0.223*** (0.031)	0.115*** (0.025)
Number of children in household	0.264*** (0.015)	0.088*** (0.014)
Number of adults in household	−0.045** (0.015)	0.008 (0.021)
Household Structure (Omitted: Two adult-headed household)
Female-headed household	0.666*** (0.029)	0.410*** (0.046)
Male-headed household	0.240*** (0.042)	0.296*** (0.033)
Disabled person in household	0.793*** (0.036)	0.614*** (0.032)
Metropolitan area	−0.093** (0.037)	0.029 (0.037)
Economic Characteristics
State monthly unemployment	−0.078 (0.219)	−0.012 (0.207)
State monthly employment-population ratio	−6.398 (22.07)	−10.76 (20.15)
State annual per capita income (in $100s)	−0.001 (0.002)	0.003 (0.002)
Quarterly GDP (in trillions)	−0.069 (0.095)	−0.236** (0.096)
Year
1998	−0.460** (0.181)	−0.338* (0.186)
2003	−0.249*** (0.080)	−0.332*** (0.096)
Constant	6.584 (22.339)	10.54 (20.12)
Rho	0.509*** (0.054)
Number of Observations	65,269

Note: The unit of observation is a household-month. Robust standard errors are presented within parentheses. Standard errors are adjusted for clustering by state.

^aSample includes householdes with income below 150% of the poverty threshold who have liquid assets below $4,000, or below $5,000 if one member of the household is age 60 or older.

^bA joint test for significance of the four instruments indicates that they are jointly statistically significant at the one percent level (χ²(4) = 15.8, p = 0.003). The model includes state dummy variables.

^***p < 0.01,

^**p < 0.05,

^*p < 0.1.

Many household demographic characteristics are important determinants of food insecurity (table 3, column 2). Food insecurity increases with age until age 33 and then decreases with age. Households headed by minorities and persons with limited education are more likely to be food insecure. The more children a household has, the more likely it is to be food insecure. Female-headed and male-headed households are more likely than households headed by two adults to be food insecure. Finally, having a disabled person in the household is associated with a higher likelihood of food insecurity. The state unemployment rate and employment/population ratio do not affect food insecurity, although a stronger economy as measured by quarterly GDP is found to reduce food insecurity.

Very Food Insecure

Findings from our analysis of the relationship between SNAP participation and the likelihood of being very food insecure show a similar pattern. The model that does not control for selection into SNAP finds a positive, statistically significant relationship between SNAP receipt and being very food insecure, while the model that does control for selection finds that SNAP receipt statistically significantly reduces the likelihood of being very food insecure (table 2, models 3 and 4). Fewer households are very food insecure than food insecure (10.3% vs 24.4%, respectively), and the magnitudes of the estimated coefficients are consistent with this lower prevalence. The bivariate probit model results suggest that SNAP receipt reduces the likelihood of being very food secure by 3.9 percentage points. Translating this percentage point decline into a percent decline (as done above for food insecurity), we find that SNAP reduces the likelihood of being very food insecure by 20.2%.³⁵ This is lower than the roughly 30% decline found for food insecurity, although it is still substantial.

Like our analysis of food insecurity, the correlation coefficient indicates a positive and statistically significant relationship between unobservables that affect SNAP receipt and being very food insecure (ρ = 0.284, SE = 0.035). Also, we again find that Hansen’s J-test suggests that the instruments are exogenous (p = 0.53) and the Kleibergen–Paap statistic leads us to reject the null hypothesis that the model is underidentified (p = 0.0002). Each of the four instruments has the anticipated sign and statistically significantly (p < 0.1) affects SNAP receipt, and a joint test for significance of the four instruments indicates that they are jointly statistically significant at the 1% level (χ²(4) = 14.9, p = 0.005).³⁶

Additional Specifications

We estimate over fifteen additional model specifications, with three discussed above in the Empirical Model section and the remainder discussed below. First, we test whether the results are sensitive to the choice of control variables, for both the food insecure and very food insecure outcomes. Although these covariates are based on our conceptual model, we want to ensure that our findings are not sensitive to particular covariates. We estimate a series of seven models; each model starts with the primary specification and drops one group of control variables: (a) age, (b) race/ethnicity, (c) educational attainment, (d) household composition (number of adults and children, household structure), (e) disabled person in household, (f) metropolitan area, and (g) state economic characteristics. The estimated values of β are always negative and statistically significantly different from zero at the 1% level (as in the primary specification) and are similar to the magnitudes reported above.

We estimate additional models that add household income as measured by the income-to-needs ratio (i.e., household income relative to the federal poverty threshold). Again, the estimated values of β are similar to the magnitudes from the primary specifications and remain statistically significantly different from zero at the 1% level. The robustness of the results when household income is added to the model likely results from the inclusion of the reduced form determinants of income in the model and that the analysis is restricted to a relatively low income population.

As discussed above, we estimate our model on somewhat different samples to test the sensitivity of our results. First, we examine households with incomes below 130% of the poverty threshold, which more closely mimics the SNAP eligibility criteria. Analyses based on this more disadvantaged population show very similar results. The naive models that do not control for selection into SNAP find a positive relationship between SNAP receipt and food insecurity, while models that control for selection find that SNAP receipt reduces food insecurity. The bivariate probit results suggest that SNAP receipt reduces the likelihood of being food insecure by 27.8% and reduces the likelihood of being very food insecure by 18.2%.³⁷ These declines are very similar to what was found for the broader population of households with incomes below 150% of the poverty threshold.

We also examine whether our findings are sensitive to having each household in the study sample for up to four months, by estimating the model on a sample that includes each household only once.³⁸ Results from the bivariate probit model (estimated with this smaller sample) suggest that SNAP receipt statistically significantly (at the 1% level) reduces the likelihood of being food insecure by 16.0 percentage points and the likelihood of being very food insecure by 3.9 percentage points. These estimated effects are nearly identical to those presented in table 3 (16.2 and 3.9 percentage points, respectively).

We also estimate models on a sample that includes households that have imputed food-insecurity or SNAP-receipt data, and again find very similar results. In this case, the bivariate probit model results suggest that SNAP receipt statistically significantly (at the 1% level) reduces the likelihood of being food insecure by 16.0 percentage points and being very food insecure by 4.1 percentage points.

In addition to testing sensitivity to different study populations, we examine another measure of food-related hardship—food insufficiency—which has also been examined in the literature. Food insufficiency captures a relatively severe level of food hardship and is more similar to our very-food-insecure than to our food-insecure measure. Food-insufficient households are those that report sometimes or often not having enough to eat.³⁹ Food insufficiency is not our primary outcome; food insecurity is considered to be a stronger measure and is the measure used in USDA official statistics. Among households with incomes below 150% of the poverty threshold, 6.9% are food insufficient (compared with 10.3% that are very food insecure). Results from the bivariate probit model suggest that SNAP participation statistically significantly (at the 1% level) reduces food insufficiency by 2.7 percentage points, or by 19.4%. This is nearly identical to our finding that SNAP reduces the likelihood of being very food insecure by 20.2%. The specification checks discussed here and in the empirical model suggest that our results are robust to additional model specifications.

Discussion and Conclusion

Using nationally representative SIPP data from the late 1990s and early to mid-2000s and strong IV models, this study provides evidence that SNAP reduces households’ food-related hardships. We find that SNAP participation reduces the likelihood of being food insecure, very food insecure, and food insufficient. How much does SNAP reduce food-related hardship? The results suggest that the effect of the program is sizable. Results from our primary specification suggest that participation in SNAP reduces the likelihood of being food insecure by 16.2 percentage points (31.2%) and reduces the likelihood of being very food insecure by 3.9 percentage points (20.2%). Results from our specification tests show similar declines. Further, we find that SNAP receipt reduces food insufficiency by about 20%. While our results are based on assumptions underlying the bivariate probit model and restrict the effect of SNAP on food insecurity to be the same across households, these estimated effects provide evidence that SNAP is meeting its key goal of reducing food-related hardship.

Given inherent difficulties in identifying causal effects where individuals self-select into programs (such as SNAP), the conclusions of any single analysis cannot be taken as definitive. This study contributes nationally representative findings from models designed to control for self-selection to a growing body of literature that finds SNAP reduces food insecurity. For example, this study provides results consistent with Nord and Golla’s recent study (2009) using CPS data, which finds that food insecurity falls by roughly one-third after entry into SNAP. Our results are also consistent with analyses by DePolt, Moffitt, and Ribar (2009), Kreider et al. (2009), and Yen et al. (2008) that find that SNAP reduces food hardship using data from the National Food Stamp Program Survey, the National Health and Nutrition Examination Survey, and the Three-City Study, respectively.

It is important for policymakers and program administrators to understand the effectiveness of their programs so they can better serve low-income households and those experiencing food-related hardship. The results of this study suggest that program administrators can improve the well-being of households by increasing their enrollment in SNAP. Prior research suggests that this can be accomplished by making SNAP program rules more lenient and by expanding outreach (e.g., Bartlett, Burstein, and Hamilton 2004; Ratcliffe, McKernan, and Finegold 2008; Yen et al. 2008; Ziliak, Gundersen, and Figlio 2003). In addition, easing SNAP rules is a cost-efficient way for states to increase SNAP participation and improve the well-being of residents, as the federal government pays roughly half of the programs’ administrative costs and the full cost of benefits. States, however, should weigh concerns about program fraud and abuse and federal resources in deciding whether and which SNAP policies to ease.