Non-Linear Effects of Soda Taxes on Consumption and Weight Outcomes

Abstract

The potential health impacts of imposing large taxes on soda to reduce the incidence of obesity have been of interest for over a decade. While estimates of the effects of existing soda taxes with low rates suggest little impact, recent proposals suggest that large taxes may be effective in reducing weight because of non-linear consumption responses or threshold effects. This paper tests this hypothesis in two ways. First, we estimate non-linear effects of taxes using the range of current rates. Second, we leverage the sudden, relatively large soda tax increase in two states during the early 1990s combined with new synthetic control methods useful for comparative case studies. Our findings suggest virtually no evidence of non-linear or threshold effects.

Introduction

Rates of obesity in the developed world have increased rapidly over the past several decades (Ogden et al., 2006). One set of explanations for this large change in population health is the falling price of food coupled with increases in sedentary lifestyles (Lakdawalla and Philipson 2009, Cutler, Glaeser, & Shapiro, 2003). Since price changes have occurred differentially across types of food, with “low quality” calorie-dense foods falling more quickly than fruits and vegetables, both researchers and policymakers have proposed policies to offset these price differentials to blunt the obesity increase (e.g., Brownell and Frieden, 2009).

Large taxes on soda, and more recently the broader category of sugar-sweetened beverages (SSBs), are often proposed because soda is one of the largest categories of energy intake in the US and because soda consumption may represent “empty calories” devoid of nutritional content (Jacobson and Brownell, 2000; Block, 2004). Since there have been no such SSB taxes enacted in the US, to our knowledge, several studies have examined previously enacted soda taxes to inform both soda and SSB taxation policy. Indeed, soda taxation has a long history in the US, having been implemented since at least 1920, and 19 states taxed soda in 2006 (New York Times, 1920; Fletcher, Frisvold, and Tefft, 2010a). However, until recently, such taxes have primarily been used as a revenue source rather than for any potential health benefits.

To determine the impact of soda or SSB taxes on obesity, one strand of the literature estimates or uses previous estimates of price elasticities, often from across-state (or city) variation, and then uses these estimates to predict the effects of tax increases of around 20 percent. These studies often suggest large impacts on soda consumption and obesity rates (e.g. Finkelstein et al., 2010; Smith, Biing-Hwan, and Jonq-Ying, 2010; Wang et al. 2012). Limitations of these approaches include the potential that unobserved state/area characteristics that are determinants of obesity are correlated with the soft drink tax rate, the use of household rather than individual level data, the focus on price rather than tax effects, simplistic assumptions about the relationships between changes in caloric intake and changes in obesity, and the failure to fully explore substitution effects. For example, Wang et al. (2012) assume that 60% of the reduction in soda calories from soda tax rate increases would not be compensated with other calories.

Several papers have attempted to address subsets of these limitations in the literature, and the estimates from these papers are typically smaller than the estimates in papers that fail to account for the empirical issues. For example, Dharmasena and Capps (2012) and Duffey et al. (2010) find smaller effects in specifications that account for substitution patterns compared to specifications that do not; Lin et al. (2011) finds smaller effects when implementing a dynamic model of weight change in response to changes in caloric intake. Harding and Lovenheim (2014) account for the endogeneity of expenditures and prices using an instrumental variables approach and find that a nutrient-specific tax, e.g. a sugar tax, has a larger effect on caloric intake than does an equivalent product tax, e.g. a soda tax.

In contrast to studies that use household consumption measures and price variation, studies using data on individual level consumption and within-state variation in actual tax rates have found no net measureable effects on population weight. For example, Fletcher, Frisvold, and Tefft (2010a) find that increases in soft drink tax rates decrease soda consumption among children, but do not influence total caloric intake, as children increase their consumption of other high-calorie beverages. This finding is consistent with a similar lack of effects for adults (Fletcher, Frisvold and Tefft, 2010b). Other research taking this approach finds mixed results, demonstrating that average weight in some high risk populations may be more susceptible to soda taxes (Sturm et al., 2010).

The divergent policy implications of the various approaches taken suggests that there remains disagreement on the possibility that new soda taxes may succeed where current taxes have not. Indeed, one concern with the ability of the results from some previous studies to predict the consumption response to large taxes, such as the 18 percent tax proposed in New York in 2008,¹ and a potential reason for the differences in the results from the various strands of literature is that the existing soda tax rates are too low to be meaningful to most consumers since the average tax rate in 2006 was approximately 5 percent (Sturm et al., 2010; Todd and Zhen, 2010). Implicit in this argument is that substitution effects would also exhibit a threshold effect, where at high enough soda tax rates, individuals would substitute towards no beverages or low-calorie alternatives (e.g. water).

In this paper we attempt to examine the plausibility of the effectiveness of high taxes to reduce adult weight outcomes. We pursue this question using two complementary datasets and empirical approaches. First, we implement a two-way state and year fixed effects difference-in-differences specification and examine whether there is evidence of non-linear effects through the incorporation of low order polynomials (square, cubic, quartic). This analysis utilizes National Health and Nutrition Examination Surveys (NHANES) data to estimate effects on reported consumption and caloric intake of soda and other beverages as well as measured height and weight for a nationally representative sample of adults between 1989 and 2006. This analysis will help us understand whether there appears to be any evidence of threshold effects at the upper end of the current rates available in the data.

Our second empirical strategy examines two case studies, where in the early 1990s, Ohio and Arkansas enacted legislation that substantially increased soda taxes. Using newly developed synthetic control methods, we construct a useful control, “a counterfactual Ohio”, to examine whether there is any evidence that this large tax increase affected population health. We utilize Behavioral Risk Factor Surveillance System (BFRSS) data, which includes state identifiers, was constructed to allow for state-representative estimates, and includes reported body mass index but has no information on caloric intake; thus, we examine the reduced form effect of taxation on weight with these data.² The key advantage of the BRFSS is the ability to examine the weight trajectory of Ohio and Arkansas residents during the large soda tax increase in early 1990s compared with individuals in other control states.

In summary, none of our results suggest non-linear effects of soda taxes on population weight. Both sets of complementary analyses and research designs point to linear effects and the likelihood of important substitution effects when reacting to soda taxes. Thus, this paper contributes to the literature through the emphasis on the impacts of taxes, as opposed to prices, and the evaluation of the impacts of large soft drink taxes. Taxes are directly policy-relevant and are more likely to be exogenous than prices (Gruber and Frakes, 2006), and evaluating whether impacts of “large” taxes might be different than the impacts of “small” taxes within a causal framework is novel to this literature.

Is There a Non-Linear Effect of Soda Taxation?

Empirical Strategy

We begin our empirical examination by exploring potential non-linear effects of soda taxation on multiple diet and weight outcomes using the NHANES data. A typical specification found in the literature is:

$$Y_{\mathit{ist}}={\beta }_0{X}_{\mathit{ist}}+{\beta }_1{T}_{st}+{\mu }_s+{\tau }_t+{\epsilon }_{\mathit{ist}}$$ (1)

where Y is an outcome (e.g. body mass index) for individual i residing in state s at time t. Time can be calculated using years or year/quarters. X is a set of social, economic, and demographic characteristics, μ is a set of state-level fixed effects, τ is a set of time fixed effects, and ε is an idiosyncratic error term. T is the key independent variable of interest, indicating the state-level net tax rate on soda (compared to other consumption such as water or juice). To examine whether there may be non-linear effects in the range of our data as an indication of whether higher rates may be more effective, we supplement equation (1) by including a polynomial in the tax rate:

$$Y_{\mathit{ist}}={\beta }_0{X}_{\mathit{ist}}+{\sum }_{j=1}^K{\beta }_j{T}_{st}^j+{\mu }_s+{\tau }_t+{\epsilon }_{\mathit{ist}}$$ (2)

where K=1,2,3 or 4. We estimate equation (2) using ordinary least squares with heteroskedasticity-robust standard errors that allow for clustering within states. The impact of soft drink taxes is identified in equations (1) and (2) from variation within states over time. Consistent with this identifying assumption, the results reported below are generally robust to including time-varying state characteristics (the lagged state mean adult BMI, the state cigarette tax, and the lagged state unemployment rate) that could be correlated with states’ decisions to change the tax rate because of concerns related to population health or economic conditions.^3,4,5

Data: NHANES 1989–2006 and Soda Tax Rates

NHANES data include a series of nationally representative surveys administered by the National Center for Health Statistics (NCHS) of the Centers for Disease Control and Prevention (CDC) to assess the health and nutritional status of the civilian, non-institutionalized population.⁶ NHANES III includes nearly 34,000 respondents and was conducted between 1988 and 1994.⁷ In 1999, the NHANES program changed to consist of a nationally representative sample of about 5000 persons each year; however, the sampling design remained similar to NHANES III.

NHANES data contain information on body mass index (BMI)⁸, soft drink and other beverage consumption, and demographic characteristics. Using the detailed consumption and nutrient data from the 24-hour dietary recall, we construct measures of the total calories consumed, the total calories consumed from soft drinks, and the total grams of soft drinks consumed.⁹ To explore the possibility of substitution effects, we calculate the total caloric intake from non-soda beverages, which includes coffee, tea, milk, juice, sports, and juice-like drinks. We merge NHANES III data with the NHANES 1999–2006 data.¹⁰ We restrict the sample to adults ages 18 and above with non-missing height and weight or soft drink consumption information.

As shown in Table 1, the average total caloric intake of adults is 2233 calories per day and soda represents 130, or more than 5 percent, of these calories even though only 59 percent of adults consumed any soda. Given that the consumption of all other non-alcoholic beverages leads to only 200 calories per day, the consumption of soda represents a significant portion of the daily caloric intake of beverages. Additionally, as shown in Table 1, the average BMI is 27.9, 64 percent of adults are overweight, and 30 percent are obese.

Table 1.

Descriptive Statistics, NHANES 1989 – 2006

	Mean	Standard Error	Sample Size
Obese	0.302	0.004	34294
Overweight	0.639	0.004	34294
Underweight	0.020	0.001	34294
Log(BMI)	3.306	0.002	34294
BMI	27.921	0.051	34294
Total calories	2232.528	8.192	36196
Calories from soft drinks	130.357	1.892	36196
Consumed any soft drinks	0.591	0.004	36196
Grams of soft drink consumption	467.790	5.508	36196
Calories from non-soft drink beverages	200.804	1.890	36196
Dietary recall is based on a weekday	0.650	0.004	36391
Female	0.518	0.004	37712
Age	44.844	0.114	37712
Black	0.112	0.002	37712
Other race/ethnicity	0.171	0.003	37712
White	0.717	0.003	37712
Soft drink tax rate	2.590	0.022	37712

Notes: Descriptive statistics are weighted using the NHANES survey weights.

State of residence information in NHANES is available through the Census and NCHS Research Data Centers, which allows us to merge our state-level tax information with the individual level data. States currently tax soft drinks through excise taxes, sales taxes, and special exceptions to food exemptions from sales taxes.¹¹ For this paper, we define the soft drink tax as the tax on soft drinks net of taxes on other food and beverage items since proposed taxes typically target a single beverage (category) rather than change the price of all foods and beverages, as in the case of a general sales tax change (for more details on the compilation of soft drink taxes and the calculation of the defined variable, see Fletcher, Frisvold, and Tefft, 2010b). The average annual soft drink tax rate between 1989 and 2006 was small, varying between 1% and 3%. Nearly half of all states taxed soft drinks in any given year and, among states with a tax, the average rate was not more than approximately 5%. The maximum tax during this period was 12 percent.

Results

Tables 2 through 5 display estimates from equation (2) for calories from soft drinks, calories from non-soda beverages, total calories, and body mass index.¹² Each table includes estimates from separate specifications for each value of K and display the F statistic and corresponding p-value for the null hypothesis that the soft drink tax rate coefficients are jointly equal to zero. The Bayesian Information Criterion (BIC) is also shown and is used to determine the appropriate model specification.

Table 2.

Calories from Soft Drinks: Ages 18–90

	Linear	Quadratic	Cubic	Quartic
Soft Drink Tax Rate	1.566 (2.447)	0.569 (5.866)	−6.717 (11.575)	−2.377 (15.066)
Soft Drink Tax Rate^2		0.102 (0.408)	2.347 (2.788)	−0.293 (8.448)
Soft Drink Tax Rate^3			−0.149 (0.171)	0.290 (1.420)
Soft Drink Tax Rate^4				−0.022 (0.071)
Observations	35940	35940	35940	35940
F Statistic	0.410	0.062	0.399	0.275
P-Value	0.526	0.804	0.674	0.843
BIC	484050.8	484061.2	484070.6	484081.1

Notes: Each column represents a separate regression. Heteroskedasticity-robust standard errors that allow for clustering within states are in parentheses. The F statistic and corresponding p-value are based on the null hypothesis that the coefficient for the soft drink tax rate and all higher order polynomial terms are jointly equal to zero. Additional variables include female, age, age squared, black, other race, whether the food diary is from a weekday, state, year, and quarter. All regressions utilize NHANES survey weights.

Source: NHANES 1989 – 2006

Table 5.

BMI: Ages 18–90

	Linear	Quadratic	Cubic	Quartic
Soft Drink Tax Rate	0.007 (0.093)	−0.075 (0.207)	0.524 (0.430)	−0.680 (0.634)
Soft Drink Tax Rate^2		0.010 (0.017)	−0.169* (0.099)	0.529 (0.389)
Soft Drink Tax Rate^3			0.012* (0.006)	−0.103 (0.066)
Soft Drink Tax Rate^4				0.006* (0.003)
Observations	37018	37018	37018	37018
F Statistic	0.006	0.323	6.338	11.745
P-Value	0.937	0.573	0.004	1.63E-05
BIC	239824.6	239834.4	239837.4	239842.3

Notes: Each column represents a separate regression. Heteroskedasticity-robust standard errors that allow for clustering within states are in parentheses. The F statistic and corresponding p-value are based on the null hypothesis that the coefficient for the soft drink tax rate and all higher order polynomial terms are jointly equal to zero. Additional variables include female, age, age squared, black, other race, whether the food diary is from a weekday, state, year, and quarter. All regressions utilize NHANES survey weights.

Source: NHANES 1989 – 2006

As shown in Table 2, using a linear specification of the soft drink tax rate variables suggests that the relationship between soft drink taxes and calories from soft drinks is small in magnitude and not statistically significant for adults. Based on the BIC values, a linear specification is preferred, but the soft drink tax coefficients are not jointly significant in any of the specifications.

The results examining whether changes in soft drink tax rates influence the caloric intake of substitute beverages are shown in Table 3. Similar to the results shown in Table 2, the BIC values suggest that a linear specification is preferred. The results from the linear specification in Table 3 show that a one percentage point increase in the soft drink tax would increase caloric intake from non-soda beverages by 7.5 calories. Although this result suggests that there could be substitution effects among adults in response to a soda tax, this estimate is statistically significant only at the 10 percent level and is not robust to including additional state characteristics.

Table 3.

Calories from Non-Soda Beverages (includes milk, coffee, tea, juice, and juice drinks)

	Linear	Quadratic	Cubic	Quartic
Soft Drink Tax Rate	7.458* (3.703)	18.552*** (5.492)	16.260 (11.895)	47.692** (19.308)
Soft Drink Tax Rate^2		−1.133 (0.408)	−0.427 (3.530)	−19.544 (12.491)
Soft Drink Tax Rate^3			−0.047 (0.229)	3.138 (2.231)
Soft Drink Tax Rate^4				−0.157 (0.115)
Observations	35940	35940	35940	35940
F Statistic	4.06	7.73	4.15	9.73
P-Value	0.0515	0.0086	0.0239	0.0001
BIC	496316.3	496320.7	496331.1	496338.9

Notes: Each column represents a separate regression. Heteroskedasticity-robust standard errors that allow for clustering within states are in parentheses. The F statistic and corresponding p-value are based on the null hypothesis that the coefficient for the soft drink tax rate and all higher order polynomial terms are jointly equal to zero. Additional variables include female, age, age squared, black, other race, whether the food diary is from a weekday, state, year, and quarter. All regressions utilize NHANES survey weights.

Source: NHANES 1989 – 2006

Table 4 displays the results for total caloric intake. For this outcome, a linear specification is again preferred, and the results in the first column show that a one percentage point increase in the soft drink tax rate increases total caloric intake by 27.7 calories per day for adults. An important conclusion, though, is that this evidence demonstrates that large increases in soft drink taxes are unlikely to reduce total caloric intake. Consistent with this conclusion, as shown in Table 5, the estimate of the impact of soft drink taxes on BMI is small in magnitude and not statistically significant.¹³

Table 4.

Total Calories: Ages 18–90

	Linear	Quadratic	Cubic	Quartic
Soft Drink Tax Rate	27.683** (12.555)	24.938 (26.081)	−80.275* (39.802)	79.635 (80.230)
Soft Drink Tax Rate^2		0.280 (2.208)	32.691*** (8.255)	−64.564 (46.605)
Soft Drink Tax Rate^3			−2.158*** (0.486)	14.044* (7.763)
Soft Drink Tax Rate^4				−0.799** (0.384)
Observations	35940	35940	35940	35940
F Statistic	4.862	0.016	11.834	14.583
P-Value	0.034	0.90	0.000111	2.26E-06
BIC	593492.3	593502.7	593502.7	593508.6

Notes: Each column represents a separate regression. Heteroskedasticity-robust standard errors that allow for clustering within states are in parentheses. The F statistic and corresponding p-value are based on the null hypothesis that the coefficient for the soft drink tax rate and all higher order polynomial terms are jointly equal to zero. Additional variables include female, age, age squared, black, other race, whether the food diary is from a weekday, state, year, and quarter. All regressions utilize NHANES survey weights.

Source: NHANES 1989 – 2006

Case Studies of Two Large Soda Tax Changes

In addition to our analysis of non-linear effects above, we also examine case studies that leverage the large changes in taxes found in our data. We estimate the impact of the large tax changes in Arkansas and Ohio enacted in the early 1990s using BRFSS data by first estimating a difference-in-differences specification comparing weight outcome changes in each of these states to changes in all other states without changing tax rates, states with similar average BMI in the year prior to the tax change, and states within the same region during the same period. Then, to improve the comparability of the control group, we construct a synthetic cohort from weighted state averages by matching on a broad set of characteristics of states prior to the implementation of the large tax. We selected the Arkansas and Ohio tax changes because they were among the largest, most visible tax changes, and in the case of Ohio there was sufficient pre-treatment data on height and weight available in BRFSS to conduct a synthetic control analysis.

The Arkansas tax was enacted in a special legislative session in December 1992 by Governor Jim Guy Tucker (who replaced Bill Clinton after he won the 1992 presidential election). In this session, Arkansas passed the equivalent of two cents per 12 ounces tax on soda, which at the time was the largest soda tax increase in modern US history, to our knowledge. The proceeds were earmarked for Medicaid, which was in severe deficit. In November 1994, soda manufacturers collected enough signatures to hold a referendum on the tax, but it was defeated (55%) (Smith, 2009). The tax remains in effect today to raise revenue for Arkansas’ Medicaid Trust Fund (Tucker, 2013).

In late December 1992, Ohio Governor Voinovich enacted a 1-cent per 12-ounce excise tax on soda, on top of the five percent sales tax, and an equivalent excise tax on other containers, syrup, and the canisters that restaurants (e.g. fast food outlets) utilize. Unlike most current efforts but similar to past soda taxes, this new tax was used to help balance the state budget during the 1990’s recession (Kilborn 1993). Like Arkansas, the Ohio state constitution has a balanced budget amendment, so that fluctuations in budget revenue from the recession had to be counteracted with either increased revenue collections or reduced services/expenditures. The proceeds of this tax went to the general state fund and were not earmarked for any particular use. The tax was repealed in November 1994 through an amendment of the state constitution.

Data: BRFSS 1989–1996

The Behavioral Risk Factor Surveillance System (BFRSS) is conducted annually by state and U.S. territory health departments and the Centers for Disease Control to monitor population health risks. An important advantage of the BRFSS sample is that it allows for the construction of representative annual state-level aggregate statistics using provided sample weights. For the variables of primary interest in this analysis, BRFSS includes the self-reported height and weight of each respondent, which we use to calculate BMI and dichotomous measures of overweight and obese.¹⁴ Additional demographic and economic characteristics that we use as control variables in the regressions below and as predictors in the synthetic control analysis, aggregated by state and year, include sex, race/ethnicity, schooling, the state’s mean age, the state’s cigarette tax rate, and the lagged state’s unemployment rate.

In order to match the first year that Arkansas reported height and weight values in BRFSS, we analyze the sample beginning in 1991 and ending in 1996. Since the tax in Ohio was repealed at the end of 1994, we were also able to estimate whether there was a separate repeal effect in that state. This allows two pre-tax and four tax years for Arkansas and two pre-tax, two tax, and two post-tax years for Ohio. Descriptive statistics for all respondents in states without tax changes during the sample period (those states which are candidate control states), Arkansas, and Ohio are reported in Table 6. The most consistent and relevant difference between Arkansas and Ohio, and the no tax change sample is that the two states of interest have a slightly higher mean BMI and obesity prevalence.

Table 6.

BRFSS Descriptive Statistics

	States without tax changes N= 423,750		Arkansas N= 7,762		Ohio N= 7,537
	Mean	Std Dev	Mean	Std Dev	Mean	Std Dev
BMI	25.953	4.788	26.153	4.903	26.127	4.872
Overweight	0.526	-	0.541	-	0.548	-
Obese	0.165	-	0.184	-	0.178	-
Male	0.492	-	0.490	-	0.496	-
Age	44.558	17.691	46.143	18.159	44.995	18.095
Black	0.095	-	0.127	-	0.082	-
Hispanic	0.066	-	0.018	-	0.019	-
High School Grad	0.682	-	0.643	-	0.700	-
College Grad	0.240	-	0.176	-	0.212	-
State Unemployment rate, lagged	6.112	1.491	6.023	0.865	6.147	0.815
State cigarette tax	25.750	13.623	28.198	5.379	20.999	3.000
State soft drink tax rate (net of food)	1.961	2.344	9.711	4.384	6.951	2.776

Notes: Mean estimates are calculated using BRFSS survey weights.

Source: BRFSS, 1991–1996

Empirical Strategy: Traditional Difference-in-Differences

We first estimate separate difference-in-differences specifications of the impact of the Arkansas and Ohio tax changes that are similar to the NHANES analysis above, except that this analysis focuses more specifically on the effects of these individual tax changes. We estimate:

$$Y_{\mathit{ist}}={\beta }_0{X}_{\mathit{ist}}+{\beta }_1{\mathit{HighTaxXPost}}_{st}+{\mu }_s+{\tau }_t+{\mu }_s\ast t+{\epsilon }_{\mathit{ist}}$$ (4)

where the variables are defined as above, except that HighTaxXPost_st is equal to one during 1993 and 1994 for the analysis of Ohio, equal to one between 1993 and 1996 for the analysis of Arkansas, and equal to zero during all other years. Thus, β₁ represents the impact of a large tax on weight outcomes. Equation (4) also includes state-specific time trends, in addition to state and year fixed effects, to control for any time-varying characteristics that evolve linearly.¹⁵

Results

In the first set of regressions we compare Ohio and Arkansas each to the full set of U.S. states that do not experience a change in the state soft drink tax rate over the sample period. Next, following Callison and Kaestner (2012), we restrict the comparison group for each to include states with statistically indistinguishable average BMI values in 1991, the initial sample period.¹⁶ For Arkansas, 21 of the 35 states without tax changes during the time period satisfied this criterion, and for Ohio 10 of the 35 states satisfied it. Our last set of comparison groups are other states in each treatment state’s Census Division: for Arkansas, this is the West South Central Division including Louisiana, Oklahoma, and Texas, and for Ohio, this is the East North Central Census Division including Indiana, Illinois, Michigan, and Wisconsin (Louisiana was excluded because its soft drink tax rate changed over the sample period). By varying the comparison group we are in part anticipating the synthetic control analysis below by highlighting the challenge of constructing an appropriate comparison group.

As shown in Table 7, the estimates vary between Arkansas and Ohio and depend on the comparison group of states. For example, the sign of the estimates for BMI for Arkansas change from a decrease of 0.278 kg/m² when comparing Arkansas to no tax change states versus a statistically significant increase of 0.152 kg/m² when comparing Arkansas to states in its Census Division. Because these specifications control for fixed and trend differences across states, the differences based on comparison group argue for a more careful selection procedure.

Table 7.

Difference-in-differences Analysis of Large Soft Drink Tax Effects

Control Group	BMI			Overweight			Obese
	No tax changes	Matched on 1991 BMI	Census Division	No tax changes	Matched on 1991 BMI	Census Division	No tax changes	Matched on 1991 BMI	Census Division
Arkansas, entire period	−0.278*** (0.061)	−0.231*** (0.067)	0.152** (0.018)	−0.025*** (0.005)	−0.021*** (0.006)	0.039* (0.010)	−0.033*** (0.005)	−0.028*** (0.006)	−0.034** (0.007)
N	431,512	260,359	27,182	431,512	260,359	27,182	431,512	260,359	27,182
Ohio, entire period	0.065*** (0.022)	0.078* (0.038)	0.069 (0.042)	0.010*** (0.002)	0.011*** (0.003)	0.002 (0.003)	0.001 (0.002)	0.004 (0.002)	0.001 (0.004)
N	431,287	127,241	57,040	431,287	127,241	57,040	431,287	127,241	57,040
Ohio, enactment (1991–1994)	0.451*** (0.043)	0.644*** (0.062)	0.457* (0.197)	0.051*** (0.004)	0.058*** (0.008)	0.045*** (0.005)	−0.003 (0.004)	0.017*** (0.005)	0.000 (0.014)
N	271,964	81,614	35,984	271,964	81,614	35,984	271,964	81,614	35,984
Ohio, repeal (1993–1996)	−0.321*** (0.053)	−0.352** (0.114)	−0.391 (0.256)	−0.033*** (0.005)	−0.025** (0.009)	−0.048*** (0.009)	−0.004 (0.004)	−0.013 (0.011)	−0.009 (0.025)
N	301,382	88,185	39,410	301,382	88,185	39,410	301,382	88,185	39,410

Notes: each cell represents a separate regression, and the coefficients are the diff-in-diff coefficient comparing years in which a soft drink tax was in effect to years in which it was not, relative to the designated control group. Heteroskedasticity-robust standard errors that allow for clustering within states are in parentheses. Additional variables include male, age, black, Hispanic, high school grad, college grad, lagged state unemployment rate, state cigarette tax, indicators for state, year, and quarter, and state-specific time trends.

Source: BRFSS 1991 – 1996

Additionally, although the Arkansas tax appears to reduce both BMI and obesity prevalence when compared to no tax change states, the tax according to the entire, enactment, and repeal period results increased BMI in Ohio (while having no significant effect on obesity prevalence). Overall, even though there are a number of statistically significant coefficient estimates arising from a traditional diff-in-diff framework, they are not robust to the selection of comparison group or across enactment and repeal.

Empirical Strategy: Synthetic Controls

In order to conduct a more targeted analysis with the goal of recovering the treatment effect, we pursue additional methods described in Abadie, Diamond, and Hainmuller (2010) that allow for the construction of a data driven “optimal” control group. Whereas the specification above requires an assumption about the best set of control states, the synthetic control method constructs a weighted average of all the potential control states that most closely match the treatment state on pre-treatment characteristics and trends. Strengths of the synthetic control method compared to traditional diff-in-diff methods include: (1) in practice it is often difficult to find a single untreated unit (state) or set of units that approximates the characteristics of the treatment unit, so that this method reduces the discretion in the choice of comparable units; (2) this method safeguards against extrapolation because the weights can be restricted to be positive and sum to one; and (3) the method is transparent because we can show the relative contribution of each control unit to the counterfactual exercise.

We only analyze Ohio in this context because of the suitability of data available in BRFSS. Prior to 1993, BRFSS includes height and weight measures for Arkansas in 1991 only, while these measures are reported for Ohio in all prior years. Since the synthetic control method both requires a strongly balanced panel for each of the treatment and comparison states and because the synthetic control is best constructed when matched to earlier trends, the single year available for Arkansas is not suitable for this analysis. In our analysis of the Ohio tax, we use data from the 1989 through 1994 waves of BRFSS. Since the Ohio tax under study was effective at the end of 1992 and repealed at the end of 1994, we consider 1993 and 1994 to be the treatment years, and we build the synthetic control using the four years prior to the treatment period.¹⁷ The tradeoff for including additional pre-treatment years of data is that the empirical method requires a strongly balanced panel and fewer states asked respondents about their height and weight in earlier waves; beginning in 1989 there are 39 eligible states and beginning in 1984 there are only 15 eligible states.

The basic idea of the synthetic control method is to create a match for the outcomes of the intervention state (Ohio) by weighting the control states to match the intervention state outcome before the intervention. With a set of J optimal weights, an estimate for the effect of a large tax is then given by:

$${\widehat{\alpha }}_{1t}=Y_{1t}-\sum _{j=2}^{J+1}{w}_j^{\ast }{Y}_{jt}$$ (7)

where j denotes all states other than Ohio and w_j* denotes the optimal weight for state j. In order to create the weights, we estimate:

$$\sum _{j=2}^{J+1}{w}_j{Y}_{jt}={\delta }_t+{\theta }_t\sum _{j=2}^{J+1}{w}_j{Z}_j+{\lambda }_t\sum _{j=2}^{J+1}{w}_j{u}_j+\sum _{j=2}^{J+1}{w}_j{\epsilon }_{jt}$$ (6)

where δ_t is an unknown common factor with constant factor loadings across units (analogous to τ_t above), Z is a vector of observed covariates (analogous to X above) with associated unknown parameters θ_t, λ_t is a vector of unobserved common factors (analogous to μ_s above) with a vector of associated unknown factor loadings u_i. Note that adding the assumption that λ_t is constant for all t to equation (5) would create a typical diff-in-diff estimator (Abadie, Diamond, and Hainmuller, 2010).

In order to create the weights, we use the set of pre-treatment state level characteristics described in the Data section, including the average of the variable of interest (BMI, overweight, or obese) during the pre-treatment period, to match the control states to the treatment state.^18,19 Descriptive statistics for Ohio, its synthetic controls, and states without tax changes during the pre- and post-treatment periods are presented in Table 8. In order to calculate standard errors, a permutation method is used that repeats the analysis for each state in the sample, assigning treatment status to each control state as placebo tests. The basic intuition is that if there is an identifiable effect of the policy in question then the treated state must deviate more from its synthetic control than would states that are not treated (i.e. placebos) from their controls. This allows us to estimate whether the effect of the treatment is large relative to the effect estimated for a state chosen at random. In other words, the procedure allows us to calculate the distribution of the estimated effect of the placebo interventions.

Table 8.

Synthetic Control Descriptive Statistics

	BMI			Overweight			Obese
	Ohio	Synthetic Control	States Without Tax Changes	Ohio	Synthetic Control	States Without Tax Changes	Ohio	Synthetic Control	States Without Tax Changes
Pre-treatment (1989–1992)
Weight Variable	25.237	25.235	24.967	0.468	0.467	0.444	0.127	0.126	0.120
Weight Variable, 1989 only	24.880	24.950	24.745	0.426	0.429	0.420	0.115	0.115	0.107
Weight Variable, 1990 only	25.035	25.073	24.907	0.458	0.457	0.439	0.107	0.114	0.112
Weight Variable, 1991 only	25.486	25.370	25.005	0.481	0.479	0.450	0.149	0.140	0.118
Weight Variable, 1992 only	25.549	25.546	25.210	0.506	0.504	0.468	0.136	0.136	0.127
Male	0.480	0.481	0.487	0.480	0.483	0.487	0.480	0.480	0.487
Age	44.068	44.540	43.918	44.068	44.401	43.918	44.068	45.226	43.918
Black	0.085	0.057	0.076	0.085	0.060	0.076	0.085	0.056	0.076
Hispanic	0.014	0.063	0.048	0.014	0.023	0.048	0.014	0.040	0.048
High School Grad	0.854	0.782	0.835	0.854	0.813	0.835	0.854	0.851	0.835
College Grad	0.189	0.188	0.217	0.189	0.183	0.217	0.189	0.224	0.217
State unemployment rate, lagged	6.000	8.185	5.556	6.000	7.407	5.556	6.000	4.918	5.556
State cigarette tax	18.000	22.891	20.171	18.000	21.582	20.171	18.000	31.829	20.171
Post-treatment (1993–1994)
Weight Variable	25.563	25.814	25.415	0.499	0.519	0.488	0.152	0.149	0.139
Weight Variable, 1993 only	25.390	25.668	25.345	0.483	0.527	0.482	0.143	0.151	0.135
Weight Variable, 1994 only	25.736	25.960	25.485	0.514	0.512	0.493	0.160	0.147	0.143

Notes: Mean values for Ohio and states without tax changes are state level (unweighted) means while synthetic control values are calculated using the method described in Abadie et al (2010).

Figure 1 shows aggregate measures of state-level BMI over time for Ohio (the treated state), its synthetic control, and the average of all sample states.²⁰ All three averages increased over the sample period by approximately 0.7 to 0.9 points. Using the method described above, the synthetic control was calculated to be a weighted average of Alabama (0.3%), Texas (36.7%), and West Virginia (63%). Mean BMI in Ohio notably declined in 1993, which is consistent with the hypothesis that the large soft drink tax contemporaneously reduced mean BMI. However, mean BMI rebounds and reaches a new high in 1994, which defies a consistent explanation in terms of either contemporaneous or lagged tax effects.

Figure 1.

Average Annual State-level BMI: Sample States, Ohio, and its Synthetic Control

Note: the synthetic control includes a weighted average of Alabama (0.3%), Texas (36.7%), and West Virginia (63%).

In order to formally evaluate the changes in BMI during the treatment period in Figure 1, we calculates two sets of p-values following Abadie, Diamond, and Hainmuller (2010). First, we compare Ohio with its synthetic control during the treatment period only by calculating the mean squared prediction error (MSPE) for 1993 only and also for 1993 and 1994. Then, the analogous post-MSPEs were calculated for all of the other states (as placebos) and the post-MSPEs were ordered. The proportion of placebo post-MSPEs that are greater than or equal to the treated post-MSPE in each case serves as the p-value. These are reported in Table 9. For 1993, the p-value is 0.06, significant at the 10% level. When calculating the MSPE also including 1994, the p-value rises to 0.16 offering weaker evidence that the implementation of Ohio’s soft drink tax influenced state-level BMI.

Table 9.

P-values for Ohio Synthetic Control Tests

	Post-treatment MSPE	Post/pre-treatment MSPE ratio
BMI
1993 only	0.06*	0.34
1993 & 1994	0.16	0.35
Overweight
1993 only	0.09*	0.28
1993 & 1994	0.19	0.26
Obese
1993 only	0.69	0.88
1993 & 1994	0.48	0.84

Notes: P-values are calculated by ordering the mean squared prediction error (ratio) and calculating the proportion of placebo values that are greater than or equal to the treated value.

It is possible that the drop in BMI in 1993 is significant only because the overall synthetic control “fit” is poor for Ohio. To account for this issue, Abadie, Diamond, and Hainmuller (2010) suggest an alternative method for calculating p-values. We follow this and calculate p-values that are based on the post- to pre-treatment MSPE ratio rather than the post-MSPE alone. This measure reflects the magnitude of the deviation of the treated state from its control during the treatment period relative to its deviation in the pre-treatment period. The p-values based on this measure grow larger and insignificant for Ohio, up to 0.34 when including just 1993 and 0.35 when including both 1993 and 1994. Thus, our main finding is that, as we use techniques that are able to more closely match Ohio with a suitable control group, the evidence of an effect of a large soda tax on BMI levels becomes less convincing

Figure 2 shows analogous results for overweight prevalence in Ohio. Overweight prevalence steadily increased across the time period, by about 4 or 5 percentage points. Also, the synthetic control appears to be a good match. In this case, the synthetic control is comprised of Michigan (19.4%), Oklahoma (50.5%), and West Virginia (30.1%). The post-MSPE p-values are a statistically significant 0.09 when testing 1993 only and 0.28 when testing both 1993 and 1994. Accounting for the pre-treatment period fit by again calculating the post-to-pre-MSPE increases the p-values to 0.19 and 0.26, respectively. Figure 3 shows results for obesity prevalence. The post-MSPE p-values in this case are 0.69 when testing 1993 only and 0.88 when testing both 1993 and 1994. Accounting for the pre-treatment period fit, the post-to-pre-MSPE changes the p-values to 0.48 and 0.84, respectively, implying that we cannot detect a significant weight effect due to a large soft drink tax increase.

Figure 2.

Annual Overweight Prevalence: Sample States, Ohio, and its Synthetic Control

Note: the synthetic control includes a weighted average of Michigan (19.4%), Oklahoma (50.5%), and West Virginia (30.1%).

Figure 3.

Annual Obesity Prevalence: Sample States, Ohio, and its Synthetic Control

Note: the synthetic control includes a weighted average of Michigan (11.7%), Pennsylvania (34%), and South Dakota (54.3%).

Conclusion

Although the evidence from the literature suggests that current small (and often hidden) taxes on soda do not have detectable impacts on population weight, much less is known about the potential for effects of large, noticeable taxes. Several states and cities are currently considering raising soda taxes to 1-cent per ounce, which would be the largest increases in US history. Based on current evidence it is difficult to predict the likely effectiveness of these large tax increases on use and weight outcomes. This paper presents the first examination in the literature that attempts to answer this question by estimating potential non-linear effects of soda taxes on consumption and weight outcomes. This question is addressed with two complementary approaches and datasets.

First, we examine whether there is any evidence of non-linear effects of current soda tax rates, with the idea that if very large taxes could have relatively larger effects, then we should see evidence consistent with this hypothesis based on the larger tax rates in our data, which reach 12%. However, using a variety of specifications, we find no evidence of effects on use or weight for a nationally representative sample of adults.

Our second approach uses a new comparative case-study method that leverages the sudden and large tax increase found in Ohio in the early 1990s. This method creates a “synthetic Ohio” based on a weighted average of states that are most similar to Ohio’s population body mass index before the tax was raised. Outside of simulation methods, this is the most informative approach to understanding the potential impact of recently proposed taxes, and it suggests very little evidence that the large tax imposed in Ohio had any detectable effect on population weight.

Together, our results cast serious doubt on the assumptions that proponents of large soda taxes make on its likely impacts on population weight. Together with evidence of important substitution patterns in response to soda taxes that offset any caloric reductions in soda consumption (Fletcher et al. 2010a), our results suggest that large changes to policy proposals relying on large soda taxes to be a key component in reducing population weight may be required to successfully reduce weight.

An important limitation of our approach is that we are unable to directly assess the effects of some large tax proposals, such as the previously discussed 18% tax proposed in New York, since there are no previous examples of tax rates that are that large in the US. However, as has been discussed in previous work, there may be other health benefits (besides weight) that may make the taxation of soda an important health policy, though these benefits have yet to be fully elucidated. In addition, Arkansas’ Medicaid Trust Fund illustrates that a soda tax can be successfully implemented as a mechanism to offset health care costs attributable to obesity or other associated health conditions.