Does Information Matter? The Effect of the Meth Project on Meth Use among Youths

Abstract

Are demand-side interventions effective at curbing drug use? To the extent demand-side programs are successful, their cost effectiveness can be appealing from a policy perspective. Established in 2005, the Montana Meth Project (MMP) employs a graphic advertising campaign to deter meth use among teens. Due to the MMP’s apparent success, seven other states have adopted Meth Project campaigns. Using data from the Youth Risk Behavior Surveys (YRBS), this paper investigates whether the MMP reduced methamphetamine use among Montana’s youth. When accounting for a preexisting downward trend in meth use, effects on meth use are statistically indistinguishable from zero. These results are robust to using related changes of meth use among individuals without exposure to the campaign as controls in a difference-in-difference framework. A complementary analysis of treatment admissions data from the Treatment Episode Data Set (TEDS) confirms the MMP has had no discernable impact on meth use.

“In 2005 Montana had one of the highest rates of methamphetamine use in the country, and all of the problems that go with it…An aggressive public awareness campaign was the answer.”

The Economist

I. Introduction

The annual economic burden of methamphetamine (“meth”) in the U.S. was recently estimated to be $23.4 billion; this translates into roughly $26,000 for each individual who used meth in the past year or around $74,000 for each dependent user (RAND 2009). Methamphetamine use is the dominant drug problem in Western and Midwestern United States (Rawson et al. 2002). Based on a 2007 survey of law enforcement officials, 47% of county sheriffs reported meth as their number one drug problem. That was more than marijuana (22%), cocaine (21%), and heroin (2%) combined (National Association of Counties (NACO) 2007). In 2005, approximately 4.3% of the U.S. population and around 4.5% of high school seniors reported having used methamphetamines (National Institute on Drug Abuse (NIDA) 2006). In communities plagued by meth use, addicts place a substantial burden on healthcare facilities, county jails, and state and federal penitentiaries (Gonzales 2006; NACO 2006). Taxpayers bear significant medical and dental costs associated with incarcerated meth users (Sullivan 2006). Because of the consequences associated with methamphetamines, it is important to understand whether or not interventions aimed at reducing use have a causal influence on consumption.

There are primarily three methods to decreasing drug use: enforcement, treatment, and prevention (Dobkin and Nicosia 2009). Enforcement efforts generally take the form of government intervention and target the supply-side of drug markets. Drug treatment is a demand-side intervention that aims at reducing use and rehabilitating current users. Prevention, also a demand-side intervention, commonly takes the form of raising awareness and providing information to potential and current users through education programs, community action, and anti-drug campaigns. This paper studies the prevention mechanism by examining the effectiveness of an anti-methamphetamine campaign. In 2005, Montana adopted a graphic advertising campaign, the Montana Meth Project (MMP), with the intent of curbing meth use. The objective of the MMP was and remains to educate Montana’s youth about the harmful consequences of meth use. Methamphetamine abuse has been a major problem for Montana. In 2006, roughly 50 percent of the jail population was incarcerated for meth-related offenses and over half of the parents whose children were in foster homes used meth (McGrath 2007).

A large literature has been devoted to examining the impact of advertising and health campaigns on the use of harmful substances. The majority of this research has focused on the effects of tobacco and alcohol advertising bans. Along similar lines, other research has examined the influence that anti-substance publicity and campaigning has had on substance use. The results concerning these types of prevention tactics are far from decisive.¹ Educational programs designed to deter individuals from using tobacco, alcohol, and drugs have also been a popular form of prevention. Results regarding the efficacy of these programs are mixed as well.²

This paper makes at least three important contributions to the literature. First, the effectiveness of the MMP has not been empirically scrutinized in a rigorous fashion. This study fills that gap by investigating the impact of the MMP campaign on teen meth use. Second, to the author’s knowledge, this is the first paper to evaluate a demand-side intervention that specifically targets the market for methamphetamines.³ Third, and perhaps most notably, the findings in this paper have important implications for understanding policy efficacy in illegal drug markets. A demand-side intervention, such as the MMP, represents an extremely low cost program when compared to supply-side involvements that have been shown to have only temporary effects (see below). To the extent such a program works, the cost effectiveness is appealing from a policymaker's perspective. The annual operating budget of the MMP is approximately $2.5 million, while the economic costs of methamphetamine in Montana have been estimated in the range of $200 to $300 million per year (Stanford GSB 2009; MT Department of Justice 2009).

The MMP has received significant praise since its inception in 2005. In 2006, the MMP campaign was cited by the White House as a model prevention program for the nation (montanameth.org 2009). Due to its apparent success, the campaign has been adopted by seven other states since 2007.⁴ Popular press accounts and legislators have attributed large decreases in teen meth use to the MMP (see, e.g., “The Antidrug Lord” 2008; “Graphic Ads” 2008; McCulloch 2009). A recent follow-up report on progress, prepared by the Montana Department of Justice, cites a decrease in meth use among teens of 44.6 percent since 2005 (McGrath 2008). However, Erceg-Hurn (2008) suggests the negative findings have been supported by poor methodology and misrepresented by the MMP. In particular, Erceg-Hurn (2008) cites the lack of an adequate control group from baseline and criticizes the MMP for ignoring preexisting downward trends when citing the success of their program. Yet, Erceg-Hurn (2008) omits any type of formal statistical analysis to support his criticisms.

To assess the impact of the MMP on teen meth use, this paper uses data from the 1999–2009 Youth Risk Behavior Surveys (YRBS).⁵ Initial results illustrate that rates of meth use were roughly 1.5 to 4 percentage points lower after the adoption of the MMP. However, when accounting for a preexisting downward trend in meth use, effects on meth use become small and statistically insignificant. These null findings are robust to using the related changes of meth use among individuals in states without exposure to the campaign as controls in a difference-in-difference empirical framework. ⁶ A separate analysis of admission reports from the Treatment Episode Data Set (TEDS) confirms the MMP has had no discernable impact on meth use. These findings suggest that other factors, such as increased policing efforts that preceded the MMP, are more likely to have contributed to the decrease in the use of methamphetamines.

The remainder of this paper is organized as follows: Section II describes the background of the Montana Meth Project; Section III describes the YRBS data; Section IV lays out the empirical strategy; Section V discusses the results; Section VI analyzes treatment admissions data; Section VII concludes.

II. Methamphetamine Interventions and the Montana Meth Project

Methamphetamine is used in a variety of forms and can be smoked, snorted, injected, or ingested orally. As a powerful stimulant, immediate effects of use include increased wakefulness and physical activity and decreased appetite. Hyperthermia and convulsions can occur with an overdose and, if not treated promptly, can lead to death. Long-term effects may include addiction, memory loss, psychosis, violent behavior, mood disturbances, severe dental problems, and weight loss. Chronic abuse has also been linked to changes in brain structure that lead to reduced motor speed and impaired verbal learning (NIDA 2006).

The individual outcomes and economic consequences listed above have motivated multiple supply-side government interventions. Evaluations of these interventions generally conclude the programs have only temporary effects.⁷ Reuter and Caulkins (2003) emphasize the importance of quantifying the costs of these interventions. They stress that regulatory burdens and limits on the range of products available for therapeutic use need to be considered.

An alternative to supply-side interventions are programs that target consumers. The Montana Meth Project, a non-profit organization, introduced a large-scale, statewide antimethamphetamine campaign in 2005. The MMP set about approaching methamphetamines as a consumer product-marketing issue with the goal of providing Montana teens with information on the negative consequences of meth use. In particular, the MMP’s objective was to:

“•Increase the perceived risk and decrease the perceived benefit of trying meth so that perceptions reflected accurate information about the drug;

•Promote dialogue about the drug between parents and teens, as such dialogue has been shown to decrease illicit drug use; and

•Stigmatize use, making meth use socially unacceptable, just as cigarette smoking has become socially unacceptable in recent decades”

(Siebel and Mange 2009).

Focus group sessions held prior to the launching of the campaign indicated that a majority of teens believed meth to be a “party drug” without the addictive consequences associated with drugs like heroin (Siebel and Mange 2009). The MMP campaign aimed at changing these misperceptions.

The primary element of the MMP campaign has been the use of graphic advertisements.⁸ Users are depicted as “unhygienic, dangerous, untrustworthy, and exploitive” (Erceg-Hurn 2008). Ads rely on explicit images that include “illustrations of the decay of users’ bodies, young girls selling their bodies to older men for meth, violent criminal behavior committed by meth-hungry teens, and groups of meth users leaving their friends to die” (Siebel and Mange 2009). Individuals can be exposed to the campaign several different ways. First, statewide advertisements air as television commercials. Second, radio ads portray the risks of use with actual stories from Montana teens who disclose their personal experiences with meth. Lastly, print ads are featured in high school newspapers and on billboards across the state. Representing the largest advertiser in Montana, the MMP campaign consisted of 45,000 television ads, 35,000 radio ads, 10,000 print impressions, and 1,000 billboards statewide from September 2005 to September 2007.⁹ Evaluation of the campaign suggests the advertisements reach 70 to 90 percent of the state’s teenage population three times per week (methproject.org 2009).

The MMP was initially a privately funded campaign. Today, the campaign is financed through state and federal dollars and private contributions.¹⁰

III. Data

The data used in this paper come from the Montana and National Youth Risk Behavior Surveys (YRBS). For the analysis that considers Montana separately, the data cover the period 1999–2009. When individuals from the national sample are included as controls, the analysis is restricted to the period 1999–2007. This restriction is made because the 2009 national data are not yet available. Due to missing values, there is complete information for slightly over 13,800 individuals in the Montana YRBS and approximately 61,100 individuals in the National YRBS. Response rates for both surveys were very similar across the sample time frame. Below is a brief description of the YRBS data.

National Youth Risk Behavior Surveys

The national surveys are conducted every other year by the Centers for Disease Control and Prevention (CDC) and provide data on U.S. high school students. The primary purpose of the YRBS is to gather information on youth activities that influence health. Each survey contains a battery of questions that gauge the use of alcohol, tobacco, and other drugs. Other survey questions address daily behaviors such as eating habits, physical exercise, and TV watching. The YRBS data have been used by economists to study a wide range of topics concerning policy evaluations and youth behavior.¹¹ Though intended to be nationally representative, not all 50 states are represented in any given year the survey has been conducted. For example, students from Montana have not been included in the national surveys. As a result, information from the state-specific Montana survey is augmented with the national data to analyze the difference between changes in Montana meth use and meth use in other states. The data provide student demographic characteristics and self-reported information on participation in risky activities. Restricted use state-identified versions of the National YRBS are used.

Montana Youth Risk Behavior Surveys

In addition to the National YRBS, state surveys are conducted by state education and health agencies. The questionnaires used at the state-level mirror the national surveys. More specifically, the questions used in the analysis below were worded exactly the same for the national and Montana surveys. Similar to the national surveys, the state surveys are conducted every other year and are aimed at collecting information on high school students. For Montana, the YRBS began including questions pertaining to meth use in 1999.

IV. Empirical Strategy

To estimate the effect of the MMP on meth use among Montana’s youth, this paper employs two approaches. The first approach relies on within-Montana variation in meth use to identify the effect of the MMP. This evaluation focuses on meth use before and after the MMP was implemented in 2005. This first-difference approach is estimated by the following equation:

$${\mathrm{Y}}_{\mathit{\text{it}}}=\alpha +{\mathbf{X}}_{\mathit{\text{it}}}{\mathit{\beta }}_{\mathit{1}}+{\beta }_{\mathit{2}}{\text{AfterMMP}}_t+{\epsilon }_{\mathit{\text{it}}}$$ (1)

where i indexes the individual and t indexes the year.

In equation (1), Y refers to the binary response of whether or not the individual reports having ever used meth. In particular, survey respondents were asked: “During your life, how many times have you used methamphetamines (also called speed, crystal, crank, or ice)?” X is a vector of the individual characteristics described in Table 2. AfterMMP is a dummy variable for observations after the implementation of the MMP. The coefficient of interest, β₂, measures the impact of the MMP campaign on the meth use of Montana’s youth. Equation (1) is estimated with weighted least squares where age-by-race populations for the state of Montana are used as weights.¹²

Table 2.

Descriptive statistics for MT and National YRBS data: independent variables

Variable	National YRBS means	Montana YRBS means
Age	16.189	16.069
Male	0.479	0.492
White	0.445	0.853
Black	0.221	0.008
Other race	0.334	0.139
Freshman	0.238	0.277
Sophomore	0.248	0.273
Junior	0.255	0.236
Senior	0.256	0.204
Ungraded	0.002	0.011
Depressed	0.294	0.255
Regular smoker	0.144	0.166
Drink often	0.139	0.174
Marijuana	0.426	0.407
Other drugs	0.089	0.080
TV less than 1 hr.	0.206	0.319
TV 1 to 3 hours	0.537	0.581
TV 4 hours plus	0.258	0.100
Seat belt often	0.600	0.663
Sports participation	0.542	0.620
Passenger of drinking driver	0.320	0.343
Resource	…	0.119

Notes: (1) N = 61,133 (national sample). N = 13,832 (Montana sample). (2) Sample period is 1999–2007 for national YRBS. Sample period is 1999–2009 for Montana YRBS. (3) The Depressed variable is equal to 1 if respondent felt so sad/hopeless that he/she stopped doing usual activities during the past 12 months, and equal to 0 otherwise. (4) The Regular smoker variable is equal to 1 if respondent has ever smoked at least one cigarette per day for 30 days, and equal to 0 otherwise. (5) The Drink often variable is equal to 1 if respondent has had at least one drink of alcohol on 6 or more days of the past month, and equal to 0 otherwise. (6) The two drug use variables are equal to 1 if respondent has ever used during his/her lifetime, and equal to 0 otherwise. (7) The TV variables describe the amount of TV watched on an average school day. (8) The Seat belt often variable is equal to 1 if respondent wears a seat belt "Most of the time" or "Always" when riding in a car driven by someone else, and equal to 0 otherwise. (9) The Sports participation variable is equal to 1 if respondent played on at least one sports team during the past 12 months, and equal to 0 otherwise. (10) The Passenger of drinking driver variable is equal to 1 if, during the past month, the respondent has ridden in a vehicle driven by someone who had been drinking alcohol, and equal to 0 otherwise. (11) The Resource variable is equal to 1 if respondent has received help from a resource teacher, speech therapist or other special education teacher, and equal to 0 otherwise. The information for the Resource variable was only available in the Montana YRBS. As a result, this variable was only included in the first-difference estimations.

Equation (1) does not account for unobserved Montana-specific changes that may have influenced meth use. One way to better control for unobserved variables is to include a control group that is plausibly uninfluenced by the MMP. The control group used in this paper consists of teens from states other than Montana and, thus, who are likely to be unaffected by the MMP.¹³ Identification in this framework relies on the assumption that meth use among individuals from other states tracks the trend of use among Montana's youth except the out-of-state individuals are not subject to the meth campaign. More specifically, the control group provides instructive counterfactuals for what would have happened to the rate of meth use among teens from Montana had they not been subjected to the MMP. This difference-in-difference (DD) approach is estimated by the following equation:

$${\mathrm{Y}}_{\mathit{\text{ist}}}=\alpha +{\mathbf{X}}_{\mathit{\text{ist}}}{\mathit{\beta }}_{\mathit{1}}+{\beta }_{\mathit{2}}{\text{MT}}_{\mathit{\text{st}}}+{\beta }_{\mathit{3}}{\text{AfterMMP}}_t+{\beta }_{\mathit{4}}({{\text{MT}}_{\mathit{\text{st}}}}^{*}{\text{AfterMMP}}_t)+{\mathbf{S}}_{\mathit{s}}{\mathit{\beta }}_{\mathit{5}}+{\epsilon }_{\mathit{\text{ist}}}$$ (2)

where i indexes the individual, s indexes the state, and t indexes the year. The variable MT is a dummy variable equal to one if the individual is from Montana. S_s represents a vector of state fixed effects that control for differences in states that are common across years.¹⁴ The remaining variables are described as above. The interaction term coefficient, β₄, represents the difference-in-difference estimate of the effects of the MMP on meth use among Montana’s youth. If the MMP decreases meth use, then we expect β₄ to be negative.

All DD models are estimated by weighted least squares where state-specific age-by-race populations are used as weights.¹⁵ Models are estimated with least squares for ease of interpretation; however, the interpretation of the DD results is similar when probit models are used to explicitly model the dichotomous nature of the dependent variable. Following Bertrand et al. (2004), standard errors are clustered at the state-level.

V. Results from YRBS Data

Descriptive statistics

Table 1 presents descriptive statistics of the dependent variable used in the analysis. Rates of meth use from the North Dakota and Wyoming YRBS data are also reported in Table 1. In results discussed below, North Dakota and Wyoming are considered as alternative control states. For visual convenience, Figure 1a plots the means from Table 1 for Montana and the national sample. In Figure 1a, “meth use” is defined as having ever used meth during one’s lifetime.¹⁶ It is immediately clear that self-reported meth use has been trending downward throughout the sample time frame for Montana’s youth. For the national sample, the downward trend holds for all years after 2001. For Figure 1a, it appears that meth use fell slightly more for individuals in Montana after the introduction of the MMP than for youths elsewhere. However, the downward trend for Montana meth use is fairly smooth throughout the sample period; this is contrary to what one would expect if the MMP has had a causal influence on meth use. Most importantly, these data illustrate the importance of controlling for preexisting trends. Figures 2a–2d illustrate trends in meth use for subsamples of the population on which one might expect the treatment effect to vary. Figures 2a and 2b separate the sample by sex, while Figures 2c and 2d consider differences by age. Each figure portrays similar trends to those shown in Figure 1a.

Table 1.

Descriptive statistics for MT, National, ND, and WY YRBS data: dependent variable

Variable	1999		2001		2003	2005		2007	2009
Montana
Meth (ever)	0.131		0.125		0.094	0.078		0.047	0.031
SE	0.006		0.006		0.006	0.005		0.003	0.004
N	2881		2835		2706	2947		3864	1786
				Pre-MMP (1999–2005)			Post-MMP (2007–2009)
		Meth (ever)		0.107			0.042
		SE		0.003			0.003
National sample
Meth (ever)	0.073		0.089		0.067	0.058		0.044	…
SE	0.002		0.002		0.002	0.002		0.002	…
N	15076		13075		14947	13498		13568	…
				Pre-MMP (1999–2005)			Post-MMP (2007)
		Meth (ever)		0.072			0.044
		SE		0.002			0.002
North Dakota
Meth (ever)	0.106		0.092		0.087	0.051		0.045	0.030
SE	0.007		0.007		0.007	0.005		0.005	0.004
N	1795		1583		1648	1712		1693	1787
				Pre-MMP (1999–2005)			Post-MMP (2007–2009)
		Meth (ever)		0.084			0.037
		SE		0.003			0.003
Wyoming
Meth (ever)	0.121		0.108		0.120	0.089		0.062	0.052
SE	0.008		0.006		0.008	0.006		0.005	0.004
N	1625		2724		1522	2448		2145	2813
				Pre-MMP (1999–2005)			Post-MMP (2007–2009)
		Meth (ever)		0.107			0.056
		SE		0.003			0.003

Note: The standard error of the mean is reported below each annual estimate of the average rate of having ever used methamphetamines.

Figure 1.

Figure 2.

Table 2 presents descriptive statistics for the remainder of the variables used in the regression analysis. The YRBS data are limited in their content of individual characteristics in that only age, sex, race, and grade are incorporated in all years of the survey. Additional variables are included to control for individual preferences, personality, and risk preference. For example, whether or not an individual is a regular smoker or frequently wears a seat belt when riding in a vehicle proxy propensity towards risk. Sports participation is included to proxy attachment to school and community (Chatterji et al. 2004). Other variables included in the model describe the use of alcohol and other drugs, depression, whether the individual has been in a vehicle with a driver who was under the influence of alcohol, and whether the individual sees a resource teacher at school. Lastly, because MMP anti-meth ads air on Montana television channels, the amount of television the respondent reports watching on an average school day is included to proxy exposure to the campaign’s television component.¹⁷ It is important to note these variables may be endogenous if they are a function of the same unobserved factors that influence meth use. However, as long as they are not correlated with the MMP campaign, their inclusion will not bias the MMP coefficients. Evidence from Table 4 suggests the use rates of other drugs were not influenced by the MMP campaign.

Table 4.

Difference-in-difference estimates for use of other drugs.

	(1) Marijuana	(2) Inhalants	(3) Cocaine	(4) Heroin
MT	0.066* (0.033)	−0.204*** (0.008)	−0.197*** (0.012)	−0.013 (0.019)
AfterMMP	−0.014 (0.011)	0.015** (0.006)	−0.003 (0.007)	−0.009* (0.005)
MT*AfterMMP	−0.000 (0.010)	0.003 (0.006)	−0.007 (0.007)	0.000 (0.005)
State trends	YES	YES	YES	YES
R2	0.297	0.149	0.225	0.075
N	74095	73726	74948	74218

Notes: (1) Sample is 1999–2007 National and Montana Youth Risk Behavior Surveys. (2) Each column represents a separate regression. (3) All regressions control for age, sex, race, grade, and include dummy variables describing depression, smoking, drinking, other drug use behavior, hours spent watching TV, seat belt use, sports participation, whether the individual has been the passenger of a drunk driver, state of residence, and state-specific linear trends. (4) State race- and age-specific populations are used as weights. (5) Standard errors are in parentheses and are clustered at the state-level. (6)

^*significant at 10% level;

^**significant at 5% level;

^***significant at 1% level.

Further reference of Table 2 illustrates the Montana and national samples are similar along many dimensions. Yet, several characteristics are quite different. For example, Montana youth appear to be more physically active in that they spend less time watching TV and are more likely to participate in sports than individuals from the national sample. The primary difference between the two samples is the distribution of respondents by race.

First-difference results

Table 3 illustrates first-difference regression results for meth use. Here, the dependent variable indicates whether or not the respondent has ever used methamphetamines. Column 2 repeats the results from Column 1 with the exception that individual-specific controls are included. In Columns 1 and 2, the coefficient estimates on the AfterMMP variable are negative and significant at the 1% level. The estimate in Column 2 suggests that meth use declined by 4.1 percentage points after the introduction of the MMP. For further perspective, this represents an approximate 38% reduction from the pre-MMP mean of self-reported meth use.

Table 3.

First-difference and difference-in-difference estimates for methamphetamine use.

	First-difference estimates			Difference-in-difference estimates
	(1) Baseline Sample: 1999–2009	(2) Baseline Sample: 1999–2009	(3) Controlling for Preexisting Linear Trend: 1999–2009	(4) Baseline Sample: 1999–2007	(5) Controlling for Preexisting Linear Trend: 1999–2007
Meth use
MT	---	---	---	−0.012*** (0.002)	0.103*** (0.027)
AfterMMP	−0.058*** (0.004)	−0.041*** (0.003)	−0.009 (0.006)	−0.026*** (0.005)	−0.015* (0.008)
MT*AfterMMP	---	---	---	−0.015*** (0.004)	0.005 (0.008)
Individual- specific controls	NO	YES	YES	YES	YES
Trends	NO	NO	YES	NO	YES
R2	0.012	0.422	0.424	0.391	0.392
N	13832	13832	13832	73885	73885

Notes: (1) Sample is 1999–2009 Montana Youth Risk Behavior Surveys for first difference results. Sample is 1999–2007 National and Montana Youth Risk Behavior Surveys for DD results. (2) Each column is a separate regression. (3) Regression models in Columns 2–5 control for age, sex, race, grade, and include dummy variables describing depression, smoking, drinking, other drug use behavior, hours spent watching TV, seat belt use, sports participation, whether the individual sees a resource teacher at school, and whether the individual has been the passenger of a drunk driver. (4) For the first-difference estimates, "Trends" refers to a simple linear trend. For the DD estimates, "Trends" refers to state-specific linear trends. (5) Race- and age-specific populations are used as weights. (6) Standard errors are in parentheses and are clustered at the state-level for the DD results. (7)

^*significant at 10% level;

^**significant at 5% level;

^***significant at 1% level.

Column 3 of Table 3 explicitly takes into consideration the negative trend of meth use among Montana's youth. In particular, this regression includes an independent variable that takes on the value of one in 1999, three in 2001, five in 2003, and so forth. If the MMP has a causal impact on meth use, then one would expect to observe decreases in meth use relative to trend after the campaign was introduced in 2005. When controlling for a preexisting linear trend, the coefficient estimate on AfterMMP is no longer statistically significant at a conventional level. Furthermore, the magnitude of the estimate is substantively small. Again, this result should come at no surprise given the foreshadowing of Figure 1a.¹⁸ It is important to note the inclusion of individual-specific controls does not alter the main results of this analysis, further supporting the research design.¹⁹

Difference-in-difference results

Table 3 also presents the difference-in-difference results based on estimation of equation (2). The DD estimator is shown in the third row as the coefficient estimate on the interaction term MT*AfterMMP. The baseline specification in Column 4 illustrates a negative and significant coefficient estimate for MT*AfterMMP. A reduction of 1.5 percentage points represents roughly a 14% decrease in meth use among Montana's youth from the pre-MMP mean. The size of the decrease in meth use is considerably lower for the baseline DD estimate than the baseline first-difference estimates. Because the DD estimator compares the changes in meth use of Montana youths with the changes in use among individuals from other states, the large difference between the first-difference and DD estimates is clearly due to the fact the aforementioned negative trend was prevalent in the national sample as well as in Montana. Column 5 incorporates state-specific linear time trends. Here, the interaction term coefficient becomes positive and is nowhere near significant. Similar to the first-difference estimate that considers the downward trend, the magnitude of the coefficient is substantively small. Overall, the results from Table 3 provide little support for a decrease in meth use due to the MMP.²⁰

Other substances

Although the MMP campaign specifically targets the use of methamphetamines, it is possible users (or potential users) of other substances are influenced by the advertisements. To examine if the MMP has had an impact on the use of other drugs, Table 4 considers binary indicators as dependent variables for marijuana, household inhalants, cocaine, and heroin.²¹ For marijuana, the dummy variable represents whether or not the respondent reports having used the substance at least once during the past 30 days. For household inhalants, cocaine, and heroin, the dummy variable indicates whether or not the substance has ever been used during the respondent’s lifetime. All difference-in-difference coefficient estimates provide strong evidence the MMP has had no influence on the rates of use of other substances.

Subsamples of youths and the selective recruitment hypothesis

Column 1 of Table 5 estimates equation (2) for subsamples of youths by demographic characteristics. Each cell represents a separate regression where the dependent variable indicates whether or not the respondent has ever used methamphetamines. Only the DD coefficient estimates are reported for each regression. All regressions control for state-specific trends. When considering the entire sample, it is possible that important effects on subgroups go undetected. Column 1 of Table 5 addresses this concern by estimating separate equations for whites, nonwhites, males, females, and youths by age. DD results for nonwhites and younger individuals actually indicate an increase in use after the adoption of the MMP campaign. The results in Column 1 provide little evidence that the MMP reduced use among subgroups of the general youth population in Montana.

Table 5.

Difference-in-difference results for subsamples of youths and selective recruitment.

	(1) DD estimates (coefficient on MT*AfterMMP)		(2) DD estimates (coefficient on MT*AfterMMP
Subsamples of youths		Selective recruitment
Whites	0.008 (0.010) N=38069	No binge drinking past month	−0.002 (0.007) N=51998
Nonwhites	0.010*** (0.003) N=35816	Not a regular cigarette smoker	0.002 (0.008) N=62849
Males	0.005 (0.009) N=35579	Have not driven under influence of alcohol in past month	0.007 (0.007) N=63847
Females	0.003 (0.008) N=38306	Have not carried a weapon in past month	0.005 (0.008) N=60182
Age < 17	0.011** (0.005) N=42820	Wear a seat belt often	−0.001 (0.007) N=44875
Age ≥ 17	−0.007 (0.014) N=31065

Notes: (1) Sample is 1999–2007 National and Montana Youth Risk Behavior Surveys for DD results. (2) Each cell represents a separate regression. (3) All regressions control for age, sex, race, grade, and include dummy variables describing depression, smoking, drinking, other drug use behavior, hours spent watching TV, seat belt use, sports participation, whether the individual has been the passenger of a drunk driver, state of residence, and state-specific linear trends. (4) State race- and age-specific populations are used as weights. (5) Standard errors are in parentheses and are clustered at the state-level. (6)

^*significant at 10% level;

^**significant at 5% level;

^***significant at 1% level.

Column 2 of Table 5 investigates the selective recruitment hypothesis.²² It is possible that youths who display relatively less risky behaviors are the most likely to be influenced by the MMP. Column 2 tests this hypothesis by estimating the effect of the MMP on meth use among subsamples who report not participating (or participating less) in certain risky behaviors. Column 2 follows the format of Column 1 where each cell represents a separate regression that includes state-specific linear trends. In particular, Column 2 considers samples of individuals who report having not binge drank in the last 30 days, never regularly smoked cigarettes, not driven under the influence of alcohol in the past month, not carried a weapon in the past month, and worn a seat belt often when riding in a vehicle.²³ All coefficient estimates are statistically insignificant. The results in Column 2 do not support the selective recruitment hypothesis.

Exposure to the campaign among MT youth

By the MMP’s own estimates, the anti-meth advertisements reach over 70 percent of the state’s teenage population three times per week (methproject.org 2009). Yet, it is likely some individuals are exposed to the campaign more than others.²⁴ Unfortunately, campaign exposure is unobserved in the data. County- and school-level identifiers would help control for exposure to certain elements of the campaign (e.g. exposure to billboards), but these indicators are not available due to confidentiality issues.

As an alternative measure of campaign exposure, Table 6 considers interactions between the AfterMMP indicator and the variables that describe the amount of TV the respondent reports watching on an average school day. Admittedly, the amount of TV watched by a MT teen is a very crude measure of exposure. There is no indication as to how many of the hours of TV watched are spent on MT-specific stations that air the meth ads. However, it seems reasonable to assume that youths who watch more TV are more likely to see the anti-meth commercials.

Table 6.

Teen meth use and television exposure for Montana youth.

	Controlling for Preexisting Linear Trend: 1999–2009
Meth use
AfterMMP	−0.013* (0.008)
TV_1to3hrs	−0.002 (0.005)
TV_4hrs_plus	−0.010 (0.008)
AfterMMP*TV_1to3hrs	0.004 (0.007)
AfterMMP*TV_4hrs_plus	0.012 (0.012)
Trend	−0.006*** (0.001)
R2	0.424
N	13832

Notes: (1) Sample is 1999–2009 Montana Youth Risk Behavior Surveys. (2) Less than one hour of TV watching per day is the reference. (3) Each column is a separate regression. (4) All regression models control for age, sex, race, grade, and include dummy variables describing depression, smoking, drinking, other drug use behavior, hours spent watching TV, seat belt use, sports participation, whether the individual sees a resource teacher at school, and whether the individual has been the passenger of a drunk driver. (5) Race- and age-specific populations are used as weights. (6) Standard errors are in parentheses. (7)

^*significant at 10% level;

^**significant at 5% level;

^***significant at 1% level.

The equation estimated for Table 6 is identical to equation (1) except for the inclusion of the interactions between the AfterMMP variable and the TV dummies. Here, the interaction terms represent DD estimators that exploit the temporal variation of the MMP campaign and the within-MT variation of TV hours watched by the individual respondents. The interaction term coefficient estimates indicate that youths who watch one to three hours or four hours or more of TV per day were no less likely to try meth after the adoption of the campaign than were individuals who reported watching less than one hour of TV per day.

Robustness to comparison group

A potential issue with the DD results above is the use of the national sample as the control group. As previously noted, the Montana youth appear different from youth in the national sample along several dimensions. Most apparent is the difference in racial composition between the two groups.

To test the robustness of the results to the specification of the comparison group, data from the North Dakota and Wyoming Youth Risk Behavior Surveys are used. North Dakota and Wyoming border and have similar populations to that of Montana. Additionally, these two states have high levels of youth meth use that are more comparable to rates in Montana than are rates in the national sample (see Table 1). An added benefit to using the North Dakota and Wyoming YRBS is that data for 2009 is available.²⁵ This allows for examining a longer post-treatment period than is feasible with the national sample. Lastly, by utilizing Wyoming data for 2009, it is possible to observe youth meth use one year after Wyoming started its own Meth Project that was modeled to mimic the Montana campaign.

Because teens from two different states at two different times were exposed to a Meth Project, this analysis is based upon the following modification of equation (2):

$${\mathrm{Y}}_{\mathit{\text{ist}}}=\alpha +{\mathbf{X}}_{\mathit{\text{ist}}}{\mathit{\beta }}_{\mathit{1}}+{\beta }_{\mathit{2}}\text{Meth}\_{\text{Project}}_{\mathit{\text{st}}}+{\mathbf{S}}_{\mathit{s}}{\mathit{\beta }}_{\mathit{3}}+{\mathbf{T}}_{\mathit{t}}{\mathit{\beta }}_{\mathit{4}}+{\epsilon }_{\mathit{\text{ist}}}$$ (3)

where Meth_Project is equal to one if state s has a Meth Project campaign during year t and is equal to zero otherwise. X is a vector of the individual characteristics described in Table 2 and S and T represent state and year fixed effects, respectively.²⁶ β₂ is the coefficient of interest and is analogous to the coefficient on the interaction term in equation (2).

Figure 1b illustrates trends in meth use for Montana, North Dakota, and Wyoming. All three states exhibit downward trends in meth use during the sample time frame. In the main specification that includes state-specific linear trends, the coefficient estimate on Meth_Project is small in magnitude and statistically insignificant (β₂ = −0.008, s.e. = 0.009). For the sake of brevity, a table is not reported for these results but is available upon request. The estimates confirm the Meth Project has had no discernable impact on methamphetamine use.²⁷

VI. Analysis of the Treatment Episode Data Set

Methamphetamine treatment admissions data from the Treatment Episode Data Set (TEDS) are used to compliment the YRBS analysis. Drug treatment providers that receive federal funding are required to submit data to TEDS. For each admission, data on whether the patient tested positive for meth upon arrival is recorded. With the data from TEDS, it is possible to construct yearly, age- and sex-specific methamphetamine admission rates for each state. Existing research suggests treatment admissions data from TEDS serve as a useful proxy for the total number of methamphetamine users in the general population (Cunningham et al. 2010).

There are at least three benefits to using the TEDS data. First, they provide an objective measure of meth use as opposed to the self-reported data from the YRBS. Second, using annual data from 1995 to 2008, it is possible to observe a longer sample time frame than is feasible with the YRBS data. Moreover, the TEDS data are compiled annually, whereas the YRBS data are collected every other year. Lastly, it is possible to examine whether the MMP has had an impact on adult populations. Though the MMP specifically targets teenagers in their campaign, it is possible older individuals are influenced by the anti-meth ads.²⁸ This is of concern because the prison population and foster care caseloads are affected by meth use (Cunningham et al. 2010).

To estimate the effect of the MMP on methamphetamine admission rates, this paper estimates a model that mirrors the YRBS difference-in-difference research design presented above. Specifically, the following equation is estimated:

$${\mathrm{Y}}_{\mathit{\text{ast}}}=\alpha +{\mathbf{X}}_{\mathit{\text{ast}}}{\mathit{\beta }}_{\mathit{1}}+{\beta }_{\mathit{2}}{\text{MT}}_{\mathit{\text{st}}}+{\beta }_{\mathit{3}}{\text{AfterMMP}}_t+{\beta }_{\mathit{4}}({{\text{MT}}_{\mathit{\text{st}}}}^{*}{\text{AfterMMP}}_t)+{\mathbf{S}}_{\mathit{s}}{\mathit{\beta }}_{\mathit{5}}+{\mathbf{T}}_{\mathit{t}}{\mathit{\beta }}_{\mathit{6}}+{\mathbf{\text{Trend}}}_{\mathit{s}}+{\epsilon }_{\mathit{\text{it}}}$$ (3)

where a indexes whether the observed admission rate is for males or females, s indexes the state, and t indexes the year.

In equation (3), the dependent variable is the natural logarithm of the sex-specific methamphetamine admissions rate per 100,000 of the relevant population.²⁹, ³⁰ X is a vector of characteristics that includes a dummy indicating whether the observed admissions rate is for males or females, the state unemployment rate, the state average per capita income, and the state percentage of the population that is black. MT and AfterMMP are defined as above. S and T are state and year fixed effects, respectively. Lastly, Trend represents state-specific time trends. State populations are used as weights and standard errors are clustered at the state-level (Bertrand et al. 2004)

Table 7 presents descriptive statistics of the data used in the TEDS analysis. Because of the issues mentioned above that are associated with finding an appropriate control group, results were considered where MT was systematically compared to all other U.S. states, all other Western U.S. states, and all other Mountain states.³¹ As Table 7 makes clear, methamphetamine admission rates are very high in Montana relative to elsewhere. This is especially the case for the older age groups.

Table 7.

Descriptive statistics: Treatment episode data, 1995–2008.

	Montana	All states (excl. MT)	Western U.S. states (excl. MT)	Mountain U.S. states (excl. MT)
Meth admission rate, age 15–17	225.74	62.05	179.02	128.27
Meth admission rate conditional on no prior treatment, age 15–17	102.60	35.73	116.02	85.77
Meth admission rate, age 18–20	481.00	94.39	252.05	216.35
Meth admission rate conditional on no prior treatment, age 18–20	217.08	50.71	150.64	120.93
Meth admission rate, age 21–29	705.70	136.03	377.04	285.82
Meth admission rate conditional on no prior treatment, age 21–29	238.38	66.17	201.90	137.99
Unemployment rate	4.569	5.101	5.688	4.642
Income per capita (2000 dollars)	24563.86	30035.63	30858.04	28124.07
Percent black	0.005	0.125	0.056	0.035

Notes: (1) Sample is 1995–2008 Treatment Episode Data Set (TEDS). (2) Sample means are state/year averages. (3) Admission rates are rates per 100,000 of the specified age group population. (4) Western U.S. states are Arizona, California, Colorado, Idaho, Nevada, New Mexico, Oregon, Utah, Washington, and Wyoming. Mountain U.S. states are Arizona, Colorado, Idaho, Nevada, New Mexico, Utah, and Wyoming.

A problem with the TEDS data is the inability to observe whether the patient has had prior treatment episodes for meth use. This is of particular concern in light of extremely high recidivism rates among individuals seeking treatment for methamphetamine addiction (National Drug Intelligence Center 2006). The TEDS data does contain, however, information on the number of previous treatment episodes the patient has received in any drug or alcohol program.³² Table 7 also reports mean admission rates conditional on no previous treatment. Across all samples, conditional admission rates are significantly less than unconditional rates. This statistical artifact is consistent with the highly addictive nature of methamphetamines (Winslow et al. 2007). For conciseness, only regression results for unconditional admissions rates where all other U.S. states serve as controls are reported. The findings are robust to using conditional rates and the Western and Mountain U.S. states separately as controls. These results are available from the author upon request.

Table 8 presents the DD coefficient estimates of interest from the TEDS analysis.³³ Each column represents a separate regression. Column 1 illustrates results for 15 to 17 year-olds, those youths who are of an age most similar to the YRBS sample. Columns 2 and 3 represent estimates for 18 to 20 and 21 to 29 year-olds, respectively.³⁴

Table 8.

Meth admissions by age group. Treatment episode data analysis, 1995–2008.

	(1) 15 to 17 year-olds	(2) 18 to 20 year-olds	(3) 21 to 29 year-olds
MT*AfterMMP	0.040 (0.228)	−0.141 (0.235)	−0.102 (0.235)
State FE	YES	YES	YES
Year FE	YES	YES	YES
State trends	YES	YES	YES
R2	0.878	0.895	0.913
N	1359	1368	2736

Notes: (1) Sample is 1995–2008 Treatment Episode Data Set (TEDS). (2) Each column is a separate regression. (3) Control group consists of all other U.S. states. (4) All regression models also control for sex, the state unemployment rate, state income per capita, and the percent of the state population that is black. (5) State populations are used as weights. (6) Standard errors are in parentheses and are clustered at the state-level. (7)

^*significant at 10% level;

^**significant at 5% level;

^***significant at 1% level.

While two of estimates in Table 8 are negative in sign, none are statistically significant. For all estimates, the standard errors are quite large. The null findings for the 15–17 year-olds help confirm the results from the YRBS analysis. The Montana Meth Project appears to have had no influence on the meth use of high school aged individuals. The same holds for persons who are in transition from high school to adulthood (i.e. 18–20 year-olds) and for young adults (i.e. 21–29 year-olds).

VII. Conclusion

Methamphetamine use is widely prevalent across the United States and poses a considerable public health threat. Government supply-side interventions to the methamphetamine market have shown to have only temporary effects on meth-related behavior. Moreover, these programs have been criticized due to the regulatory burdens they impose and their potential for limiting the range of products available for legitimate therapeutic use. An alternative to supply-side interruptions, demand-side programs aim to prevent meth use by educating individuals on the adverse consequences associated with methamphetamines.

In 2005, the state of Montana adopted a graphic advertising campaign, the Montana Meth Project (MMP), with the intent of curbing meth use among Montana’s youth. What initially began as a privately funded campaign, the MMP is currently financed through state and federal dollars and private contributions. With an annual operating budget of approximately $2 to $3 million, the MMP represents a potentially low cost alternative to supply-side interventions.

To evaluate the effectiveness of the MMP on teen meth use, this paper uses data from the 1999–2009 Youth Risk Behavior Surveys (YRBS). When accounting for a preexisting downward trend in meth use, effects on meth use become small and statistically insignificant. These results are robust to using the related changes of meth use among youths without exposure to the campaign as controls in a difference-in-difference specification. A complementary analysis of treatment admissions data confirms the MMP had no effect on meth use.

These results suggest the Montana Meth Project’s campaign did not contribute to the decrease in meth use among Montana’s youth. From a policy perspective, this research is important because it highlights the ineffectiveness of a campaign that is widely regarded as successful. It is vital for future research to determine whether or not other factors that preceded the MMP contributed to the decrease in teen meth use. For example, drug task forces were committed to seizing clandestine meth labs during the years prior to the introduction of the antimeth campaign (McGrath 2008). Subsequent studies may also benefit from focusing on trying to quantify trends in sentiment toward methamphetamine that pre-date the MMP. Lastly, this study calls for future research to focus on the determinants of youth meth use so as to better guide the allocation of resources towards effective policies.