The potential impact of Supplemental Nutrition Assistance Program (SNAP) restrictions on expenditures: a systematic review

Abstract

Objective

To systematically review the potential impact of reducing the set of Supplemental Nutrition Assistance Program (SNAP)-eligible foods (e.g. not allowing purchase of sugar-sweetened beverages with SNAP benefits) on expenditures for restricted foods.

Design

The impact on food expenditures of a $US 1 reduction in available SNAP benefits can be used to estimate the impact of restrictions on SNAP-eligible foods. An electronic search of EconPapers, AgEcon Search, EconLit, WorldCat, ProQuest Dissertations and Theses, PubMed and NALDC, and a snowball search were conducted to obtain a sample of studies up to March 2015 that estimate the impacts of SNAP and other income on household food expenditures. The studies were classified according to study population, study design and whether they attempted to correct for major study design biases.

Setting

Estimates were extracted from fifty-nine published and unpublished studies.

Subjects

US households.

Results

Fifty-nine studies were found, yielding 123 estimates of the impact of SNAP benefits on food expenditures and 117 estimates of the difference in impacts between SNAP benefits and other income. Studies correcting for or mitigating study design biases had less estimate variation. Estimates indicate that expenditures on the restricted item would decrease by $US 1·6 to $US 4·8 if $US 10 of SNAP benefits would have otherwise been spent, with a median overall impact of $US 3.

Conclusions

The present literature suggests that restrictions on SNAP-eligible items may result in a small but potentially meaningful decrease in SNAP expenditures for restricted items. Further research is needed to evaluate whether this would translate into improvements in diet quality.

Some studies have shown that participants in the Supplemental Nutrition Assistance Program (SNAP) suffer more from obesity than non-participants^{( 1 , 2 )}, possibly due to poor diet quality^{( 3 )}. Policy proposals to address these issues typically consist of either incentivizing healthier food options^{( 4 )} or restricting the set of foods that can be bought with benefits^{( 5 )}. The effectiveness and impact of restrictions, however, is a subject of considerable debate. Whether restrictions will actually lead to a decrease in purchasing and consumption of restricted foods depends on how much a decrease in SNAP purchasing power leads to a corresponding decrease in food expenditures. An individual may simply decide to offset the decrease in SNAP purchasing power by using other income to purchase any restricted items. Thus, the extent to which SNAP participants are willing to substitute other income for SNAP income determines the ability of restrictions to reduce purchasing of restricted foods.

Over fifty studies, appearing between 1974 and 2014 and using a variety of methods, have estimated the extent of this substitution. However, the most recent review of this literature included studies only up to 2001^{( 6 )}. Additionally, that review as well as earlier reviews^{( 7 – 9 )} did not undertake a comprehensive search of the literature and did not seek to systematically summarize the heterogeneity in results. In the present study we therefore seek to systematically summarize all available previous literature that measures the substitution between SNAP and other income, as well as investigate sources of heterogeneity and bias. In so doing, we will identify and emphasize results from studies that mitigate the bias from their respective study designs. Our analysis will point to what the current literature can tell us about the potential impact of SNAP restrictions on expenditures for restricted items.

To illustrate the impact of a decrease in SNAP purchasing power, we use the example of an individual who prior to a restriction spent $10 (all dollar values in the present paper are US dollars) out of SNAP income on soda (see, for example, the Appendix to Basu et al.^{( 10 )}). After the soda restriction is implemented, the individual can use no SNAP income to purchase soda: the benefits available to purchase soda drop to $0. They may, however, decide to use some of their other income to purchase soda. If the individual decides not to use any of their own money, their expenditures on soda will drop from $10 pre-restriction to $0 post-restriction. If, however, they decide to spend $6 out of their other income to purchase soda, their expenditures will drop from $10 pre-restriction to $6 post-restriction. For illustrative purposes, we will use the example of a soda restriction throughout the present paper. The concepts and analysis, however, can be used for any restricted item. The amount that expenditures drop due to a restriction is an empirical question that determines the effectiveness of the restriction in disincentivizing the purchase of less healthy food items. In the present paper we summarize the prior literature estimating the extent to which food expenditures on a restricted item can be expected to decrease due to a restriction.

Methods

Definition of effect sizes of interest

Using the older term ‘food stamps’ to designate SNAP income, we refer to the decrease in expenditures induced by decreasing benefits available to purchase the restricted item by $1 as the ‘marginal propensity to spend out of food stamps’ (MPSFS). This is the hypothetical case of an individual who prior to a restriction spends $1 of food stamps on soda. While post-restriction no food stamps can be used to purchase soda, the individual may choose to use other income on soda. Thus the MPSFS measures, on a per-dollar basis, the extent to which an individual substitutes other income for food stamps. An MPSFS of 0·4 means that, if benefits available to purchase soda decrease by $1, soda expenditure will decrease by $0·40 – the individual will use $0·60 from other income sources to purchase soda. Similarly, we refer to the decrease in expenditures induced by a $1 decrease in other (non-SNAP) income as the ‘marginal propensity to spend out of income’ (MPSInc). Restated in their more common – but theoretically equivalent – forms, the MPSFS is the increase in expenditures caused by increasing food stamps by $1 and the MPSInc is the increase in expenditures caused by increasing other income by $1.

Most of the relevant literature estimates the MPSFS and MPSInc for total food expenditures or at-home food expenditures; few studies estimate these effects for individual foods. Thus our primary effect size of interest will be the MPSFS for total or at-home food expenditures, which conceptually measures the extent to which total food expenditures decrease/increase when food stamps decrease/increase by $1. We will, however, also briefly consider the few studies that estimate the MPSFS for various food categories.

We also use a second effect size of interest: the difference between the MPSFS and the MPSInc. The difference is important to consider for two reasons. First, the MPSFS estimates the potential food expenditure response to a restriction on the types of foods that can be purchased with SNAP. While the MPSFS then directly measures the potential expenditure response to a restriction, the underlying question is: how interchangeable are SNAP income and other income for participants? Another indication of how interchangeable households perceive these two income sources is the difference between the MPSFS and the MPSInc. The difference measures the additional decrease in expenditure resulting from lost income being in the form of food stamps instead of other income. If households perfectly substitute other income for SNAP income, there should be no additional decrease caused by income being in the form of food stamps, and so no difference between the MPSFS and the MPSInc. In this case SNAP restrictions should have no influence on food purchasing decisions.

The second reason for also considering the difference between the MPSFS and the MPSInc relates to the development of the literature. Since SNAP benefits are restricted to food expenditures, SNAP already has a degree of restrictiveness built in. Thus, a related though distinct question asks how food expenditures would change if participants were given unrestricted cash instead of SNAP benefits. In the early 1990s, the US Department of Agriculture commissioned cashout experiments to answer this question. For the cashout experiments, a random sample of food stamp recipients was given cash instead of food stamps in certain parts of the USA. These experiments allow more direct observation of the different usage of SNAP and cash benefits. However, since they measure cash benefits which households may use differently from non-benefit income, the cashout experiments ask a slightly different question from the one we wish to address. The existence of cashout results has not stopped the growth of the literature measuring substitution between food stamps and other income, due partly to the amount of time that has passed since the experiments, the conflicting results even among experiments and their inherently local nature. Due to the similarity between our question of interest and that of the cashout experiments, we include studies based on the cashout experiments in our analysis. As we will explain below, we are able to obtain only the difference MPSFS – MPSInc from some cashout studies. Therefore, to allow for comparison with other studies, we use MPSFS – MPSInc as our second effect size of interest. While MPSFS – MPSInc does not directly measure the expenditure response to a restriction as does the MPSFS, in our data the MPSFS explains 87 % of the variation in the difference, and so the effect sizes are measuring very closely related phenomena. A smaller MPSFS as well as a smaller difference indicate a smaller impact of SNAP restrictions on food expenditures, and thus a likely smaller impact on diet and health. Since we will refer to the terms MPSFS, MPSInc and difference (MPSFS – MPSInc) throughout the present paper, for ease of reference Table 1 contains short explanations of each.

Table 1.

Summary of terms

Marginal propensity to spend food stamps (MPSFS)	The decrease in a household’s total (SNAP plus non-SNAP) food expenditures induced by decreasing available SNAP benefits by $1. A lower value indicates less influence of a reduction in SNAP benefits on household food expenditures
Marginal propensity to spend income (MPSInc)	The decrease in a household’s total (SNAP plus non-SNAP) food expenditures induced by decreasing other income by $1. A lower value indicates less of an influence of a reduction in income on household food spending
Difference (MPSFS – MPSInc)	The additional decrease in expenditure resulting from lost income being in the form of food stamps instead of other income. A lower value indicates greater substitutability between food stamps and other income

SNAP, Supplemental Nutrition Assistance Program.

Search strategy

We searched for published or unpublished studies that estimate a relationship between receipt of food stamps/SNAP and household at-home or total food expenditures using micro-level (usually household-level) data. To be included, a study had to report an MPSFS or the difference MPSFS – MPSInc. We also included studies that reported an estimate from which a marginal propensity to spend or a difference (MPSFS – MPSInc) could be inferred or calculated, and when necessary we performed the necessary calculations. The cashout studies only allow for a difference (MPSFS – MPSInc) to be calculated^{( 11 )}. We did not require that the standard error of the marginal propensity be reported or obtainable, since this would result in non-random sample attrition. If a working paper duplicated the analysis of a published study, only the published results were included in our data. If a working paper presented a different analysis from what was subsequently published, both studies were included. We included all possible effect sizes from each study unless the study explicitly preferred a specific analysis over the others.

The search began by using Google Scholar to perform backwards and forwards citation searches from an initial list of studies reported in four reviews^{( 6 – 9 )}. In turn, we performed backwards and forward citation searches on the studies identified by the initial search and so on until no further studies were identified. This resulted in a ‘snowball’ sample. We then performed keyword searches in Google Scholar and in major online databases (EconPapers, AgEcon Search, EconLit, WorldCat, ProQuest Dissertations and Theses, PubMed and the National Agricultural Library Digital Collections). Keywords we used were combinations of ‘food stamps/SNAP/food assistance’ AND ‘food expenditures/food consumption/cashout/marginal propensity to spend’, as well as ‘determinants of food expenditures/consumption’. When the search yielded more than twenty pages of results (e.g. for Google Scholar), we used only the first twenty pages to identify studies to include in our analysis. We did not set a limit on the publication date for any of the searches. The main search occurred in June and July 2013, and was updated in March 2015. The electronic search and study coding were conducted by one author (J.C.) and the results were reviewed by another author (T.K.M.B).

Analysis

In addition to the effect sizes, we collected and coded detailed information on the characteristics of the analysis that resulted in the specific effect size. A study can provide more than one effect size. We calculated the MPSFS – MPSInc for studies reporting both a non-negative MPSFS as well as a non-negative MPSInc.

We do not require a standard error to be reported along with the sample size. Thus, instead of traditional meta-analytic methods we use graphical displays and summary statistics to investigate the impact of study characteristics on effect sizes. Due to the number of dimensions on which studies can vary, we focus on three very general dimensions: research design, population and treatment of biases. Investigating whether and how each study corrects for the biases inherent in its design will further allow us to summarize the results of the ‘best’ studies, with the assumption that studies which correct for design biases are on average superior to those which do not. In addition to the three general dimensions, to investigate whether effect sizes may be different for different foods, we summarize the studies that estimate effect sizes for specific food groups.

Using the detailed information on the method of analysis, we classify each effect size into one of four research designs, depending on the data used and the type of estimation. Studies can report or allow estimation of effect sizes using more than one research design. Thus, the research design is at the level of the estimate.

‘Participant/non-participant’ designs use regression techniques or (in one instance) a simple comparison of means to estimate the impact on household food expenditures of participating in SNAP. This impact on food expenditures can be converted into an approximate MPSFS at the mean if the average SNAP benefits level is reported. We include differences-in-differences studies in this category.

‘Non-cashout dose–response’ designs directly estimate marginal propensities to spend out of food stamps – and usually other income – using regression analysis on observational (usually cohort) data. They do so by including in the analysis a continuous measure of SNAP participation – the amount of SNAP benefits a household receives. The MPSInc for non-cashout dose–response studies is the marginal propensity to spend out of non-SNAP income.

‘Cashout dose–response’ designs use similar empirical methods as non-cashout dose–response studies with data from the cashout experiments to estimate the marginal propensities of interest. For these studies, the MPSInc is the marginal propensity to spend out of cash benefits.

‘Cashout’ designs use experimental or quasi-experimental data and simple comparisons of means to estimate the effect of cashout on food expenditures. Fraker et al.^{( 11 )} show that the cashout impact can be converted into the difference between the MPSFS and the marginal propensity to spend out of cash benefits for a linear dose–response model. We perform the necessary calculations using reported average benefit sizes and, for the sake of comparability, assuming that the MPSInc can be approximated by the marginal propensity to spend out of cash benefits. When possible, we include four effect sizes from each cashout study that vary on two dimensions: whether the food expenditure measure includes food purchased from outside the home, and whether the expenditure is normalized to account for household size. Note that due to the randomization, the cashout experiment results have a causal interpretation only in terms of the impact of cashout on food expenditures which can be transformed into an average difference in marginal propensities. Neither marginal propensity is an experimental result.

We define study population as the combination of data set and the population of interest. Many studies use the same data sets, but may focus on different populations of interest. Populations of interest are classified as one of: all households, SNAP-eligible households, SNAP participant households, and other. Examples of other populations are low-income households or Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)-eligible households. To describe the study population, we use a graphical display of effect sizes across an exhaustive list of combinations of common data sets and populations of interest. We have no a priori reason for believing that there would be any differences in effect size for different study populations, with one exception. ‘Constrained’ households spend only their SNAP benefits on food – they do not spend their own money. If, given the same amount in cash, they would spend less on food, then receipt of SNAP induces them to spend more on food than they otherwise would have. Their choices are thus considered ‘constrained’. Using the example of a soda restriction, constrained households are those that would be willing to purchase soda only using SNAP benefits pre-restriction. If given cash, they would likely not spend as much on soda and instead use the money for other purchases. ‘Unconstrained’ households would use both SNAP benefits and other income to purchase soda. If given cash, we would not expect unconstrained households to change their purchasing behaviour. We would therefore expect restrictions to have a greater impact on the purchasing behaviour of constrained households. Thus we also compare the effect sizes of the studies that take into account the difference between constrained and unconstrained households with the studies that do not.

After comparison by research design and population, we assess the quality of the methods behind the effect sizes. Each research design entails its own systematic biases (explained in the Results section). For each design, we summarize the effect sizes by whether and how these biases are addressed. The effect sizes will vary on a number of other dimensions as well: whether and how household size is accounted for, survey weighting, and other dimensions. Given the variety of methods used to address these issues, as well as the fact that there is often no accepted ‘better’ way to do so, we will not discuss these issues in detail here.

Finally, we summarize the effect sizes for expenditures on individual foods or food groups instead of for food expenditures in general. In this summary we focus on variation between effect sizes within each study, and for two reasons do not emphasize the levels of the food group-specific effect sizes. First, the sample size of studies which report estimates of the effect size for individual food groups is very small. Second, the food group’s MPSFS measures the impact that decreasing total food stamps has on expenditures for that item. This is slightly different from a restriction, which decreases only the amount of food stamps available to purchase the restricted item. Thus, while within-study variation can point to differences in the substitutability of food stamps and other income in purchasing decisions for different food groups, the level of MPSFS may not be informative in the case of a restriction. We use the appropriate PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to report the methods and results of our systematic review.

Results

Research design

We found a total of fifty-nine studies that estimate one or both of our effect sizes of interest. Figure 1 shows the process of study selection. Since many studies estimate multiple effect sizes, these studies give us 123 MPSFS estimates and 117 estimates of the difference between MPSFS and MPSInc. Table 2 summarizes the research designs, populations and effect sizes for the fifty-nine studies^{( 10 , 12 – 69 )}.

Fig. 1.

Selection process flowchart

Table 2.

Overview of studies and effect sizes

Reference	Research design	Data set	Population	MPSFS	MPSInc	Difference
Allen and Gadson( 12 )	DR	NFCS-LI	Eligible	0·241	0·119	0·122
Allen and Gadson( 12 )	DR	NFCS-LI	Eligible	0·295	0·076	0·219
Arcia et al.( 13 )	DR	Other	Other	−0·130	0·120
Arcia et al.( 13 )	DR	Other	Other	−0·040	0·110
Arcia et al.( 13 )	PNP	Other	Other	0·040*	0·037	0·003
Arcia et al.( 13 )	PNP	Other	Other	0·065*	0·062	0·003
Basiotis et al.( 14 )	PNP	NFCS-LI	Eligible	0·212*	0·090	0·122
Basiotis et al.( 15 )	PNP	NFCS-LI	Eligible	0·325*	0·100	0·225
Basiotis et al.( 15 )	DR	NFCS-LI	Eligible	0·174	0·100	0·074
Basu et al.( 10 )	DR	NHANES	Part.	0·35
Beatty and Tuttle( 16 )	PNP	CEX-I	Other	0·310
Beatty and Tuttle( 17 )	PNP	CEX-I	Other	0·48
Benus et al.( 18 )	DR	PSID	All	0·860	0·050	0·810
Blanciforti( 19 )	PNP	NFCS-PR	Eligible	0·328*	0·097	0·231
Blanciforti( 19 )	PNP	NFCS-PR	Eligible	0·173*	0·107	0·066
Blanciforti( 19 )	DR	NFCS-PR	Part.	0·333	0·034	0·299
Blanciforti( 19 )	DR	NFCS-PR	Part.	0·415	0·031	0·384
Boonsaeng et al.( 20 )	PNP	CEX-D	All	0·315
Breunig and Dasgupta( 21 )	DR	San Diego CO	Unconstrained Part.	0·298	0·057	0·241
Breunig and Dasgupta( 21 )	DR	San Diego CO	Unconstrained Part.	0·492	0·059	0·433
Breunig and Dasgupta( 21 )	DR	San Diego CO	Unconstrained Part.	0·042	0·049	−0·007
Breunig and Dasgupta( 22 )	CO: DR	San Diego CO	Unconstrained Part.	0·078	0·269	−0·191
Breunig and Dasgupta( 22 )	CO: DR	San Diego CO	Unconstrained Part.	0·393	0·235	0·158
Breunig and Dasgupta( 22 )	CO: DR	San Diego CO	Unconstrained Part.	0·416	0·221	0·195
Breunig and Dasgupta( 22 )	CO: DR	San Diego CO	Unconstrained Part.	0·034	0·406	−0·372
Bruich( 23 )	PNP	Other	All	0·30
Cage( 24 )	DR	CEX-I	Other	0·21*	0·130	0·080
Cage( 24 )	DR	CEX-I	Other	0·23*	0·20	0·030
Chavas and Yeung( 25 )	DR	CEX-D	Other	0·175	0·069	0·107
Chavas and Yeung( 25 )	DR	CEX-D	Other	−0·035	0·0732
Chavas and Yeung( 25 )	DR	CEX-D	Other	−0·420	0·113
Chavas and Yeung( 25 )	DR	CEX-D	Other	0·561	0·029	0·531
Chavas and Yeung( 25 )	DR	CEX-D	Other	0·370	0·126	0·245
Chavas and Yeung( 25 )	DR	CEX-D	Other	1·351	0·042	1·309
Chavas and Yeung( 25 )	DR	CEX-D	Other	0·756	0·086	0·669
Chavas and Yeung( 25 )	DR	CEX-D	Other	0·966	0·082	0·884
Chen( 26 )	DR	NFCS-LI	Eligible	0·197	0·111	0·086
Chen( 26 )	DR	Other	Eligible	0·226	0·154	0·072
Clarkson( 27 )	PNP	Other	Part.	0·729*	0·330	0·399
Cohen and Young( 28 )	CO: DR	WA CO	Part.	0·238	0·028	0·210
Cohen and Young( 28 )	CO: DR	WA CO	Part.	0·297	0·196	0·101
Cohen and Young( 28 )	CO	WA CO	Part.			0·359*
Cohen and Young( 28 )	CO	WA CO	Part.			0·357*
Cohen and Young( 28 )	CO	WA CO	Part.			0·338*
Cohen and Young( 28 )	CO	WA CO	Part.			0·201*
Daponte et al.( 29 )	DR	Other	Low income	0·280	0·020	0·260
de los Santos( 30 )	DR	NFCS-LI	Eligible	0·298*
de los Santos( 30 )	PNP	NFCS-LI	Eligible	0·405*
de los Santos( 30 )	PNP	NFCS-LI	Eligible	0·246*
Devaney and Fraker( 31 )	CO: DR	NFCS-PR	Eligible	0·213	0·226	−0·013
Devaney and Fraker( 31 )	CO: DR	NFCS-PR	Eligible	0·268	0·21	0·058
Devaney and Fraker( 32 )	DR	NFCS-LI	Eligible	0·424	0·084	0·34
Devaney and Fraker( 32 )	DR	NFCS-LI	Eligible	0·212	0·069	0·143
Davis et al.( 33 )	DR	Other	Low income	0·758	0·226	0·532
Davis et al.( 33 )	DR	Other	Low income	0·876	0·295	0·581
Davis and Werner( 34 )	DR	ASSETS CO	Part.	0·064	0·012	0·052
Davis and Werner( 34 )	DR	ASSETS CO	Part.	0·192	0·025	0·167
Davis and Werner( 34 )	DR	ASSETS CO	Part.	0·074	0·005	0·069
Davis and Werner( 34 )	CO	ASSETS CO	Part.			0·33*
Davis and Werner( 34 )	CO	ASSETS CO	Part.			0·32*
Davis and Werner( 34 )	CO	ASSETS CO	Part.			0·28*
Davis and Werner( 34 )	CO	ASSETS CO	Part.			0·26*
Fraker et al.( 35 )	CO	AL CO	Part.			−0·027*
Fraker et al.( 35 )	CO	AL CO	Part.			−0·034*
Fraker et al.( 35 )	CO	AL CO	Part.			0·004*
Fraker et al.( 35 )	CO	AL CO	Part.			0·013*
Fraker et al.( 36 )	CO: DR	AL CO	Part.	0·307	0·311	−0·004
Fraker et al.( 36 )	CO: DR	AL CO	Part.	0·329	0·341	−0·012
Fraker et al.( 37 )	DR	CSFII	WIC-eligible	0·050	0·110	−0·060
Fraker et al.( 37 )	DR	CSFII	WIC-eligible	0·066	0·058	0·008
Fraker et al.( 37 )	DR	CSFII	WIC-eligible	0·290	0·088	0·202
Fraker et al.( 37 )	DR	CSFII	WIC-eligible	−0·160	0·073
Hoagland( 38 )	PNP	CEX-D	All	0·570
Hollonbeck et al.( 39 )	CO: DR	Elderly CO	Eligible	0·110	0·170	−0·060
Hoynes and Schanzenbach( 40 )	PNP	PSID	Other	0·296	0·098	0·198
Hoynes and Schanzenbach( 40 )	PNP	PSID	Other	0·163	0·087	0·076
Hu and Knaub( 41 )	DR	PSID	Eligible	1·287	0·130	1·157
Hymans and Shapiro( 42 )	DR	PSID	All	0·290†	0·230	0·060
Johnson et al· ( 43 )	DR	NFCS-LI	Eligible	0·170	0·060	0·110
Kisker and Devaney( 44 )	PNP	NFCS-LI	Eligible	0·112*
Knaub( 45 )	DR	PSID	Part.	0·574	0·073	0·502
Knaub( 45 )	DR	PSID	Part.	0·206	0·049	0·157
Knaub( 45 )	DR	PSID	Part.	0·452	0·078	0·375
Knaub( 45 )	DR	PSID	Part.	0·325	0·073	0·253
Kramer-LeBlanc et al.( 46 )	DR	CSFII	Part.	0·349	0·066	0·283
Kramer-LeBlanc et al.( 46 )	DR	CSFII	Part.	0·345	0·079	0·266
Lane( 47 )	PNP	Other	Eligible	0·375
Levedahl( 48 )	DR	Other	Part.	0·521	0·01	0·511
Levedahl( 48 )	DR	Other	Part.	0·461	−0·003
Levedahl( 49 )	DR	NSFC-LI	Unconstrained Part.	0·414	0·100	0·314
Levedahl( 49 )	DR	NSFC-LI	Unconstrained Part.	0·316	0·114	0·202
Levedahl( 49 )	DR	NSFC-LI	Unconstrained Part.	0·475	0·078	0·397
Levedahl( 49 )	DR	NSFC-LI	Unconstrained Part.	0·816	0·230	0·586
Levedahl( 49 )	DR	NSFC-LI	Unconstrained Part.	0·440	0·118	0·322
Levedahl( 49 )	DR	NSFC-LI	Unconstrained Part.	0·288	0·094	0·194
Levedahl( 49 )	DR	NSFC-LI	Unconstrained Part.	0·500	0·099	0·401
Levedahl( 49 )	DR	NSFC-LI	Unconstrained Part.	0·688	0·189	0·499
Levedahl( 50 )	DR	San Diego CO	Unconstrained Part.	0·263	0·066	0·197
Lin et al.( 51 )	DR	Other	Part.	0·203	0·037*	0·166
Long( 52 )	DR	Other	Other	−0·125	0·037
Moffit( 53 )	CO: DR	NSFC-PR	Eligible	0·161	0·208	−0·047
Moffit( 53 )	CO: DR	NSFC-PR	Eligible	0·114	0·150	−0·036
Morgan( 54 )	DR	PSID	All	−0·051	0·036
Neenan and Davis( 55 )	DR	Other	Part.	0·450	0·060	0·390
Ohls et al.( 56 )	CO: DR	San Diego CO	Part.	0·301	0·192	0·109
Ohls et al.( 56 )	CO: DR	San Diego CO	Part.	0·277	0·108	0·169
Ohls et al.( 56 )	CO	San Diego CO	Part.	0·222*
Ohls et al.( 56 )	CO	San Diego CO	Part.	0·190*
Ohls et al.( 56 )	CO	San Diego CO	Part.	0·205*
Ohls et al.( 56 )	CO	San Diego CO	Part.	0·191*
Price( 57 )	PNP	CEX-D	All	0·418	0·057*	0·361
Price( 57 )	PNP	CEX-D	Part.	0·457	0·057*	0·400
Price( 57 )	DR	CEX-D	Part.	0·453*	0·057*	0·396
Ranney and Cushman( 58 )	DR	Other	Part.	0·597*	0·104	0·493
Reese et al.( 59 )	PNP	Other	All	0·420
Reese et al.( 59 )	PNP	Other	All	0·350
Reese et al.( 59 )	PNP	Other	All	0·810
Rush et al.( 60 )	DR	Other	WIC-eligible	−0·040	−0·003
Rush et al.( 60 )	DR	Other	WIC-eligible	0·180	0·279	−0·099
Rush et al.( 60 )	DR	Other	WIC-eligible	−0·120	0·004
Rush et al.( 60 )	DR	Other	WIC-eligible	0·190	0·171	0·019
Salathe( 61 )	PNP	CEX-D	Eligible	0·3625	0·061	0·302
Salathe( 61 )	PNP	CEX-D	Eligible	0·220	0·083	0·137
Salathe( 62 )	PNP	CEX-D	Eligible	0·233	0·0614	0·172
Senauer and Young( 63 )	DR	PSID	Part.	0·264	0·073	0·191
Senauer and Young( 63 )	DR	PSID	Part.	0·327	0·050	0·277
Smallwood and Blaylock( 64 )	DR	NFCS-LI	Eligible	0·233	0·099	0·134
West( 65 )	DR	CEX-D	All	0·470	0·098*	0·372
West( 65 )	DR	CEX-D	Part.	0·169*	−0·030*
West( 65 )	DR	CEX-D	Eligible	0·550	0·107*	0·443
West and Price( 66 )	DR	Other	Other	0·297	0·050	0·247
West and Price( 66 )	DR	Other	Other	0·432	0·066*	0·366
West and Price( 66 )	DR	Other	Other	0·610	0·107*	0·503
West and Price( 66 )	DR	Other	Other	0·149	−0·030*
West et al.( 67 )	PNP	Other	Eligible	0·306	0·134*	0·173
Whitmore( 68 )	CO	San Diego CO	Unconstrained Part.			0·133*
Whitmore( 68 )	CO	San Diego CO	Constrained Part.			0·552*
Whitmore( 68 )	CO	AL CO	Unconstrained Part.			−0·127*
Whitmore( 68 )	CO	AL CO	Constrained Part.			0·244*
Wilde and Ranney( 69 )	CO: DR	San Diego CO	Part.	0·221*	0·099*	0·123
Wilde and Ranney( 69 )	CO: DR	AL CO	Part.	0·356*	0·424*	−0·068
Wilde and Ranney( 69 )	CO: DR	San Diego CO	Part.	0·276*	0·351*	−0·075
Wilde and Ranney( 69 )	CO: DR	AL CO	Part.	0·219*	0·098*	0·121

MPSFS, marginal propensity to spend on food out of food stamps; MPSInc, marginal propensity to spend on food out of normal income; DR, non-cashout dose–response; PNP, participant/non-participant; CO, cashout; CO: DR, cashout dose–response; NFCS-LI, Nationwide Food Consumption Survey – Low Income; NHANES, National Health and Nutrition Examination Survey; CEX-I, Consumer Expenditure Survey – Interview; PSID, Panel Survey of Income Dynamics; NFCS-PR, Nationwide Food Consumption Survey – Puerto Rico; CEX-D, Consumer Expenditure Survey – Diary; San Diego CO, San Diego Cashout Demonstration; WA CO, Washington Family Independence Program Evaluation (cashout); ASSETS CO, Alabama ASSETS Evaluation (cashout); AL CO, Alabama Cashout Demonstration; CSFII, Continuing Survey of Food Intakes by Individuals; Elderly CO, Elderly/SSI Cashout Demonstration; Eligible, households/individuals eligible for SNAP; Part., SNAP participants/participant households; All, all households/individuals in data set; WIC, Special Supplemental Nutrition Program for Women, Infants, and Children; SNAP, Supplemental Nutrition Assistance Program.

Note the difference was only calculated when both MPSFS and MPSInc were positive.

^*From authors’ calculations.

^†From Fraker^{( 8 )}; MPSFS calculation unclear or unavailable.

Figure 2 shows box-and-whisker plots of the effect sizes for each study design. The non-cashout dose–response studies have the largest variability for both effect sizes. However, the median MPSFS for each research design hovers around 0·3. There is more variability in the median difference between MPSFS and MPSInc, although the medians for each research design range from close to 0 to 0·3. Cashout dose–response studies appear to have a substantially lower difference.

Fig. 2.

Box-and-whisker plots of effect sizes according to research design. The left and right edges of the box represent the first and third quartiles (interquartile range); the line within the box represents the median; the left and right whiskers represent the minimum and maximum values; outliers are excluded. (MPSFS, marginal propensity to spend on food out of food stamps; MPSInc, marginal propensity to spend on food out of normal income)

Study population

Figures 3 and 4 display the effect sizes for an exhaustive set of combinations of data sets and populations. Focusing on categories with more than five effect sizes, with the exception of results from the Consumer Expenditure Survey, effect sizes from studies using SNAP-eligible households appear to have less variation than those from studies using other populations of interest.

Fig. 3.

Comparison of MPSFS between studies using the San Diego Cashout Demonstration data (SP); the Panel Study of Income Dynamics using data for all households (PA), SNAP participants (PP) or other households (PO); the Nationwide Food Consumption Survey – Low Income supplement using SNAP-eligible households (NE); the Consumer Expenditure Survey Diary data using all households (CA), SNAP-eligible households (CE) and SNAP participants (CP); and other data sets using all households (OA), SNAP-eligible households (OE), SNAP participants (OP) and other households (OO). (MPSFS, marginal propensity to spend on food out of food stamps; SNAP, Supplemental Nutrition Assistance Program)

Fig. 4.

Comparison of MPSFS – MPSInc between studies using the San Diego Cashout Demonstration data (SP); the Panel Study of Income Dynamics using data for all households (PA), SNAP participants (PP) or other households (PO); the Nationwide Food Consumption Survey – Low Income supplement using SNAP-eligible households (NE); the Consumer Expenditure Survey Diary data using all households (CA), SNAP-eligible households (CE) and SNAP participants (CP); and other data set using all households (OA), SNAP-eligible households (OE), SNAP participants (OP) and other households (OO). (MPSFS, marginal propensity to spend on food out of food stamps; MPSInc, marginal propensity to spend on food out of normal income; SNAP, Supplemental Nutrition Assistance Program)

Despite the important difference between constrained and unconstrained households, many studies do not differentiate between these types of household. In our sample, eight studies account for the difference between constrained and unconstrained households: three non-cashout dose–response studies, one cashout dose–response study, one cashout study and one participant/non-participant study restrict their SNAP sample to just unconstrained households, and one non-cashout dose–response study and one cashout dose–response study incorporate the difference between constrained and unconstrained into their statistical model. These eight studies yield twenty-three MPSFS estimates and twenty-four difference estimates. Four studies (ten MPSFS estimates) use San Diego cashout data with SNAP participants, one study (two MPSFS estimates) uses the Panel Survey of Income Dynamics (PSID) with SNAP participants, two studies (ten MPSFS estimates) use another data set with SNAP participants, and the other study uses another data set with SNAP-eligible households.

Figure 5 shows the effect sizes from these studies. Even among the studies that restrict the sample, there is great variation, although most have a real difference between marginal propensities. The (cashout and non-cashout) dose–response studies that report little or negative difference (or small MPSFS) do so for a specific sub-sample, or for only one cashout experiment and not another. The cashout study^{( 68 )} compares cashout impacts of unconstrained and constrained households and finds that the average cashout impacts are the result of large changes in constrained household purchasing instead of unconstrained household purchasing. The average MPSFS for all studies that account in some way for the difference between constrained and unconstrained households is 0·38, and the average difference is 0·20.

Fig. 5.

Comparison of effect sizes between studies that account for the difference between unconstrained and constrained households by either restricting the sample to just unconstrained households (R) or incorporating the difference in the statistical model (M). (MPSFS, marginal propensity to spend on food out of food stamps; MPSInc, marginal propensity to spend on food out of normal income)

Impact of systematic biases

Participant/non-participant studies compare SNAP participants with non-participants to estimate a total impact on food expenditures of getting food stamps. The underlying assumption is that participants would spend the same as non-participants if they were not receiving food stamps. This counterfactual might be mistaken: participants might differ systematically from non-participants such that they would have different expenditure levels even without food stamps^{( 7 )}. While some of this can be controlled for using covariates in a regression, many of the differences may remain unobserved and could bias any estimated impact.

Of the twenty-four participant/non-participant studies, only seven correct for this selection bias: three using formal modelling of the selection process (or instrumental variables) and four using natural or quasi-experimental methods. Ten of the resultant seventeen MPSFS estimates come from studies using a sample of eligible households from the Nationwide Food Consumption Survey – Low Income (NFCS-LI) or another data set; the rest come from a variety of other populations and data sets. Figure 6 compares the MPSFS values of the participant/non-participant studies that correct for sample selection with those that do not. The estimates from studies that correct in some way are clustered in the middle of those that employ no correction. Those that correct also have a similar range of values to others from studies that use SNAP-eligible populations (with the exception of the Consumer Expenditure Survey; Fig. 3). Thus it is possible that the study population in addition to selection correction impacted the effect sizes. If we assume that studies which correct for sample selection are on average preferable to those that do not, the best MPSFS estimates from participant/non-participant studies range from 0·16 to 0·48, with a mean of 0·30. Only one participant/non-participant study which corrects for sample selection also allows for a calculation of the difference between MPSFS and MPSInc^{( 40 )}; from this study we can calculate two difference estimates of 0·076 and 0·198, although the study cannot reject that the marginal propensities are the same.

Fig. 6.

Comparison of MPSFS between participant/non-participant studies that correct for sample selection and those that do not. (MPSFS, marginal propensity to spend on food out of food stamps)

Non-cashout dose–response studies estimate the marginal impact of SNAP benefits and so can use samples of participants and non-participants, or only participants. When they include both groups, these studies have similar problems to participant/non-participant studies: those on SNAP are likely different in unobservable ways from those not on SNAP. Twenty non-cashout dose–response studies use samples comprising SNAP participants and some group of other people (e.g. eligible non-participants). Of these twenty studies, only four corrected for selection bias – each by statistical modelling of the selection process. The four studies that correct give seven MPSFS and difference estimates, four of which are from studies that use a sample of SNAP-eligible households from the NFCS-LI.

When they include only SNAP participants, studies do not suffer from selection bias. Since they include the amount of SNAP benefits in the statistical analysis, however, they suffer from a different problem. The amount of benefits a household receives by definition depends on the household size and income after taking into account deductions. Most of these sources of variation are factors chosen by the household, and thus likely are related to other unmeasured factors that determine household food expenditures^{( 7 )}. A SNAP benefits variable is thus endogenous to the regression model, violating a central assumption of regression analysis and biasing the estimated marginal impact of food stamps even without sample selection.

While the problem of endogeneity is always present in dose–response models, it has been argued that the best dose–response methods use very flexible regression functional forms^{( 50 )}. One particular functional form, used first by Senauer and Young^{( 63 )} and recommended by Levedahl^{( 50 )}, uses benefit levels to construct a household’s total income and estimates the relationship between (log of) food expenditures and (log of) total income and the proportion of total income that comes from SNAP.

Figure 7 shows the effect sizes of non-cashout dose–response studies, split up by their sample (just participants or participants and others) and whether they correct for sample selection bias. The studies that correct and those that use just participants have less variation than those that use other samples but do not correct. On average, though, the studies using just participants have higher MPSFS and larger differences than those that use some participation correction: the mean MPSFS for the group with just participants is 0·40, while the mean for the uncorrected is 0·23 and the mean for the corrected is 0·30.

Fig. 7.

Comparison of effect sizes between non-cashout dose–response studies using a sample of only food stamp participants (P), participants and others without sample bias correction (NC), and participants and others with sample bias correction (C). (MPSFS, marginal propensity to spend on food out of food stamps; MPSInc, marginal propensity to spend on food out of normal income)

Figure 8 displays the effect sizes of the non-cashout dose–response studies that use the Senauer and Young (SY) functional form and other flexible functional forms, compared with those studies that use the linear form. We restrict the estimates shown to just those with no sample selection bias (so from samples of just participants or from studies using correction) to display the effect of only one source of bias. Two effect sizes are from data that include non-participants and designs that correct for sample selection. Twelve MPSFS estimates are from models using the SY specification. These estimates come from studies using data on SNAP participants from the San Diego cashout data, PSID or other data sets. The SY specification has the least variation, although the sample size of those using other flexible functional forms is too small to compare. The mean MPSFS of the displayed estimates with the SY specification is 0·32, and the mean difference of the displayed estimates with the SY specification is 0·24.

Fig. 8.

Comparison of effect sizes between non-cashout dose–response studies without sample bias that use the linear, Senauer and Young (SY), or other flexible functional form. (MPSFS, marginal propensity to spend on food out of food stamps; MPSInc, marginal propensity to spend on food out of normal income)

With three exceptions, cashout dose–response studies use samples comprising entirely of SNAP participants, so they do not suffer from sample selection. Of the three exceptions, two do not correct for sample selection bias. All of the cashout dose–response studies do, on the other hand, suffer from bias due to endogeneity in similar ways to the non-cashout dose–response studies. Figure 9 displays the effect sizes of the studies that do not suffer from sample selection bias, broken down by the flexibility of the functional form. Eight of these cashout dose–response effect sizes come from studies using the San Diego cashout demonstration, four from the Alabama cashout demonstration, two from the Washington Family Independence Program (FIP) demonstration and two from an analysis of the Puerto Rico cashout experience. Unfortunately, the sample size of those that use the SY or other flexible functional forms is too small to make meaningful comparisons. While acknowledging that there is a considerable amount of variability in this category, even with the small sample size, which could make a single summary statistic misleading, we report the mean effect sizes: the mean MPSFS is 0·24 and the mean difference is 0.

Fig. 9.

Comparison of effect sizes between cashout dose–response studies that use the linear, Senauer and Young (SY), or other flexible functional form. (MPSFS, marginal propensity to spend on food out of food stamps; MPSInc, marginal propensity to spend on food out of normal income)

In order for the cashout dose–response studies to be directly comparable to the non-cashout studies, we have to assume that households treat cash benefits the same as other income. Of the eight tests that we found of the equality between the marginal propensity to spend out of cash benefits and the marginal propensity to spend out of other income, five could not reject equality. In our data of cashout dose–response studies, the median difference between the marginal propensity to spend out of cash benefits and out of other income is 0·127 (mean 0·155) – the marginal propensity to spend out of cash benefits is larger at the median by 0·127. Thus studies using cashout data likely provide conservative estimates of the difference between MPSFS and MPSInc.

Effect sizes for food groups

Table 3 compares the effect sizes for studies that allow different marginal propensities for different food groups. It includes only those expenditure categories that are most directly comparable across studies. Within each study, the MPSFS for vegetables is lower than that for meat or for bakery products. This differential is substantial for mainly one study^{( 57 )}. Similarly, among the cashout studies the largest difference between SNAP income and other income is apparent for meat and to an extent soda. Again, however, this differential is not very large.

Table 3.

Effect sizes for individual food categories

		MPSFS
Reference	Research design	Meat	Cereals/bakery products	Vegetables	Sweets
Arcia et al.( 13 )	Dose–response	0·02	0	−0·03
Blanciforti( 19 )	Participant/non-participant	0·05	0·02	0·01	0
Chavas and Yeung( 25 )	Dose–response	0·05	0·08	0·03	0·02
Price( 57 )	Participant/non-participant	0·18	0·06	0·05
Reese et al.( 59 )	Participant/non-participant	0·15
Salathe( 61 )	Participant/non-participant	0·03	0·02	0·02	0
		MPSFS – MPSInc
Reference	Research design	Meat	Vegetables	Soda	Sweets
Cohen and Young( 28 )	Cashout	0·03	0·02	0·03	0·01
Fraker et al.( 35 )	Cashout	0	0	0	0
Ohls et al.( 56 )	Cashout	0·04	0·01	0·02	0·01

MPSFS, marginal propensity to spend on food out of food stamps; MPSInc, marginal propensity to spend on food out of normal income.

For Chavas and Yeung^{( 25 )}, ‘meat’ refers to beef and veal; for Price^{( 57 )}, ‘meat’ refers to meat, poultry and fish; for Salathe^{( 61 )}, ‘meat’ refers to beef and veal. For Blanciforti^{( 19 )} and Salathe^{( 61 )}, ‘cereals and bakery products’ is defined as just bakery products. For Cohen and Young^{( 28 )}, ‘meats’ refers to low-cost meats; for Ohls et al.^{( 56 )}, ‘meats’ refers to high-cost meats.

Discussion

The present systematic review is the first to provide a comprehensive overview of the literature on the substitution between SNAP benefits and other income. In addition, we attempted to identify the best studies by summarizing the results those that have attempted to correct for biases resulting from the study design. While we identified studies that use different methods to account for the biases, there remained much heterogeneity in the results. In most cases, however, there was smaller variation in the studies that tried to correct for potential biases.

Participant/non-participant studies that correct for sample selection bias have the smallest variation: estimates of the MPSFS are between 0·16 and 0·48, with a mean of 0·30. (We have only two difference estimates for this group of studies, with a mean difference 0·14.) This range overlaps somewhat with the higher end of MPSInc estimates, so an individual study might not find them to be statistically different^{( 40 )}. Non-cashout dose–response studies that do not suffer from sample selection bias and that use more flexible functional forms have a mean MPSFS of 0·32 and mean difference of 0·24. Cashout dose–response studies – which potentially underestimate the marginal propensity to spend out of non-benefit income – that have mitigated both sample selection bias and problems with variation have substantial variation, but a mean MPSFS of 0·24 and mean difference of 0. Thus, the average MPSFS values by study design of the studies that try to account for biases range from 0·24 to 0·32, and the average difference values range from 0 to 0·24, with 0 being a possible underestimate. The average MPSFS values thus lie in the middle of the participant/non-participant range of 0·16 to 0·48.

Complementary observations can be made by summarizing studies that account for the difference between unconstrained and constrained households, and the cashout studies. Studies that account for unconstrained v. constrained households have a mean MPSFS of 0·38 and a mean difference of 0·20. If we ignore the difference between unconstrained and constrained households, and use the cashout results preferred by most researchers^{( 11 , 21 )}, the ‘best’ cashout results show a difference between MPSFS and MPSInc of between 0·11 and 0·22. Thus we find that SNAP income is not perfectly substitutable with other income, but that the difference is also small.

Although few studies estimated MPSFS or MPSFS – MPSInc for different food groups, those that did estimate them suggest small differences in the marginal propensities between foods. None of these studies, however, accounted for the biases inherent to their respective research designs. They also did not test the statistical significance of the different food group MPSFS values. Therefore, while suggestive, the number and quality of these studies mitigate against putting much weight on their specific estimates.

A final consideration relates our original motivation with the methodologies used by the studies considered here. In the Introduction we considered a soda restriction that effectively cut the benefits available to purchase soda to $0. Only one study^{( 23 )} estimates the MPSFS in the explicit context of a benefit decrease. Although theoretically equivalent, there may be reason to believe that an MPSFS derived from a benefit increase (as in, for example, Beatty and Tuttle^{( 16 , 17 )}) would be different from an MPSFS derived from a decrease in benefit. Given the other differing characteristics between these studies, however, we cannot say whether the effect size from one study is larger than from the other solely due to this reason.

Our range for the MPSFS of 0·16–0·48 can be used directly to estimate the expenditure impact of a restriction. We can consider a restriction on sugar-sweetened beverages. One sample^{( 70 )} of SNAP households was found to spend on average $6·72 of SNAP benefits per month on sugar-sweetened beverages. A restriction can be expected to decrease monthly expenditures on sugar-sweetened beverages in this sample by $1·07 to $3·23 per month (from 6·72×0·16 and 6·72×0·48). An avenue for further research would be to use this range in more extensive modelling that estimates what individuals will substitute for sugar-sweetened beverage expenditures and the resulting dietary quality and obesity impacts.

Our study highlights difficulties with the present state of the literature. Many of the studies do not correct for the systematic biases discussed above, and those that do vary on a great number of other dimensions. This variation leads to a wide range of estimates, which makes much summary discussion of the literature difficult. The problem of summarizing the literature points to both the necessity of this analysis, as well as its necessary limits.

Conclusion

Our analysis has summarized the likely expenditure impact from restricting the set of SNAP-eligible foods. These estimates point to imperfect substitutability between SNAP income and other income in household purchasing decisions, which means restrictions may reduce the purchase of restricted items to some extent. Based on our analysis of the literature, we would expect restrictions to decrease total expenditures on the restricted items by approximately $1·6 to $4·8 if without the restriction an individual would have otherwise used $10 worth of SNAP benefits to purchase the item. This suggests that restrictions on food items that may be purchased with SNAP may have a small to moderate effect on household purchasing of restricted foods.

In addition, the heterogeneity in estimates uncovered herein points to the need to measure these impacts using experimental trials of SNAP restrictions. Such a trial would provide a more reliable measure of the effect of a restriction on the purchasing of restricted foods (e.g. extent to which sugar-sweetened beverage purchasing decreases if its purchase using SNAP benefits is not allowed). The trial would also allow for measuring substitutions arising from the restriction, and so directly examine the restriction’s impact on diet quality and obesity.