Abstract
Background
This study examined the construct validity of state survey-based prevalence estimates of binge drinking.
Methods
State prevalence estimates of past 30-day binge drinking for 1999, 2001, 2003, 2005 and 2007 were obtained from published reports or public use data for the National Survey on Drug Use and Health (NSDUH), Behavioral Risk Factor Surveillance Survey (BRFSS), and Youth Risk Behavior Survey (YRBS). Construct validity was assessed in 2009 by examining correlations between these survey estimates and, for corresponding years, state per capita alcohol consumption levels (based on sales data for beer, wine and spirits) and the percentage of drivers in fatal motor vehicle crashes with a blood alcohol concentration (BAC) of at least .08.
Results
88% of the correlations between state survey-based binge drinking estimates and per capita alcohol sales data were statistically significant and moderate to strong (r ≥ .30, range =.16–.60). Similarly, 86% of the state survey binge drinking estimates were moderately or strongly correlated with the percentage of drivers in fatal crashes with BAC ≥ .08 (range: .11–.60).
Conclusions
Results suggest that state survey-based estimates of binge drinking have construct validity, and therefore can be used to investigate relationships between state alcohol policies and other state characteristics and the prevalence of this behavior.
Introduction
National epidemiological surveys indicate that binge drinking remains prevalent in the U.S., particularly among young adults. Binge drinking has been defined as five or more drinks for males and four or more drinks females within a two-hour period1, but several ongoing national surveys define binge drinking for males and females more simply as five or more consecutive drinks per occasion within a 30-day or two week period.2,3 The 2008 National Survey on Drug Use and Health (NSDUH) indicated that 34% of 18 to 20-year-olds, 46% of 21 to 25-year-olds, and 43% of 26 to 29-year-olds engaged in binge drinking during the 30 days prior to the date on which they were surveyed.2 The 2008 Monitoring The Future (MTF) Survey indicated that 8% of 8th graders, 16% of 10th graders, and 25% of 12th graders engaged in binge drinking during the prior two weeks.3 Although surveys such as the NSDUH and MTF are invaluable as tools to monitor national trends in heavy alcohol use, state-level estimates of binge drinking are potentially more useful for assessing the effects of alcohol control policies that are known to vary considerably across states.4
However, the validity of state-level binge drinking estimates is of potential concern due to methodological limitations of the surveys from which such estimates are derived, including the NSDUH, Behavioral Risk Factor Surveillance Survey (BRFSS), and Youth Risk Behavior Survey (YRBS). Of particular concern are variations in state survey response rates, which have often been less than optimal. For example, the overall response rate across the states for the 2006–07 NSDUH was 66.7% (range: 52.4 – 76.3%),5 while for the 2007 BRFSS it was 50.6% (range: 26.9 – 65.4%),6 and for the 2007 YRBS it was 68.0% (range: 60.0 – 90.0%).7 Although post-stratification weights are typically used to adjust for under- or over-representation of demographic subgroups in state respondent samples, such adjustments may only provide a partial correction for sample bias. Binge drinkers may thus be under-represented in state respondent samples.
State-level estimates derived from the NSDUH are based on both in-home computer assisted self interview data from respondents aged 12 and older in each state, and on state sociodemographic characteristics that are used in predictive models.5 State respondent samples are combined for rolling two-year periods (e.g., 2006–07, 2007–08) to increase sample sizes and precision of prevalence estimates. NSDUH prevalence estimates for binge drinking among youth are consistently lower than those derived from other surveys (e.g., MTF, YRBS) due to differences in mode of survey administration (e.g., in-home versus school-based), but NSDUH alcohol use trends have generally paralleled those of other national youth surveys.8
The BRFSS is a household telephone survey of adults that is intended to provide state estimates for a variety of health-related behaviors, including binge drinking.6 BRFSS response rates have been steadily declining, particularly for younger, less educated adults.9–11 This trend may contribute to under-representation of binge drinkers in state respondent samples, as this behavior is most prevalent among young adults.
The YRBS is a school-based, self-administered survey of 9th through 12th graders that is intended to provide state estimates of health-related behaviors among adolescents, including binge drinking.12 Not all states participate in the YRBS, and even within those that do so, a substantial percentage of schools sampled for the YRBS choose not to participate.12 Further, the YRBS also does not represent youth who have dropped out of school or are otherwise unavailable on the day of survey administration, and who are disproportionately likely to be heavy or episodic drinkers.13
Although the questions included in these surveys regarding past 30-day heavy alcohol use are generally accepted as valid and reliable, less is known about the validity of state prevalence estimates that are derived from them. To date, validity assessments have been limited to comparisons among alcohol use prevalence estimates obtained from different surveys. For example, a strong correlation (r = .82, p < .001) was observed between the 1999–2001 NSDUH and the 1999–2001 BRFSS14 in regards to estimates of the state-level prevalence of past 30-day binge drinking among adults. However, no comparisons have been reported for specific age groups within the adult samples. Nelson et al.15 found moderate to strong correlations between 1993–2005 BRFSS and YRBS state prevalence estimates for past 30-day alcohol use (range: .35–.68, p < .01) and binge drinking (range: .24–.60, p < .01). Stronger correlations have been observed between BRFSS young adult and YRBS binge drinking estimates. In an earlier study, Nelson et al.16 found BRFSS adult binge drinking estimates for 40 states were positively correlated with binge drinking among college students who participated in the Harvard College Alcohol Survey (r = .43, p < 0.01). A similar association was observed between heavy drinking estimates for college students and BRFSS non-college 18 to 24-year-olds who resided in the same state (0.45, p < 0.01). These findings provide some support for the validity of state binge drinking prevalence estimates that have been derived from these surveys, since they are based on different respondent samples and survey methods. However, because they are all based on self-report measures of binge drinking, the observed correlations may be at least partly attributable to shared method variance.17 Stronger evidence for the validity of state binge drinking prevalence estimates would be provided if they were associated with state alcohol consumption and intoxication data from archival sources that are not subject to the various biases noted above.
The purpose of this study is to examine whether state-level prevalence estimates of binge drinking that are based on survey data are associated with alcohol sales and with the percentage of drivers in fatal motor vehicle crashes with a blood alcohol concentration (BAC) at or above .08, the legal limit of intoxication in all states. Positive associations between these record-based measures and state survey estimates of binge drinking would provide evidence for the construct validity of the latter.18, 19 Construct validity is supported by associations between measures of constructs that should be related in theory. Thus, we would expect to see higher prevalence rates of binge drinking in states where more alcohol is sold per capita, even though such sales constitute an imperfect measure of alcohol consumption in each state because some portion of alcohol is bought and consumed by visitors or tourists. We would also expect to see, as an outcome of higher state prevalence rates of binge drinking, a higher percentage of motor vehicle crashes in which the driver had a BAC ≥ .08, even though alcohol-involved motor vehicle crashes can also be influenced by other factors such as state drinking and driving policies.
Methods
Data Sources
All survey and archival data for this study were obtained from published online sources for 1999, 2001, 2003, 2005, and 2007. These years were selected because state prevalence estimates for binge drinking were first reported for the NSDUH in 1999, and because the YRBS is conducted biennially in odd years.
National Survey on Drug Use and Health (NSDUH)
NSDUH prevalence estimates for past 30-day binge drinking (5 or more consecutive drinks) were obtained for the 50 states and the District of Columbia (DC) from tables in appendices of published reports.20 These tables include prevalence estimates for the total sample and the following age groups: 12+, 12–17, 18–25, and 26+. Prevalence estimates for 12 to 20-year-olds also were included after 2001. With the exception of 1999, state and DC prevalence estimates are based on respondent samples that were combined for two-year periods (e.g., 2000–01, 2002–03, etc.). Correlations between annual binge drinking estimates for the total NSDUH samples ranged from .78 to .91 (p < .01). More detail regarding the methodology used for generating state-level binge drinking estimates, including sample weighting procedures, is provided in NSDUH reports.5
Behavioral Risk Factor Surveillance System (BRFSS)
BRFSS prevalence estimates for any past 30-day binge drinking (5 or more consecutive drinks ) were obtained for all 50 states and DC from electronic data files downloaded from the BRFSS website.6 Prevalence estimates were generated for the total adult samples, underage respondents (18–20 year olds), and young adults (18–24 year olds) using sample weights provided in the BRFSS data sets. Correlations between annual binge drinking estimates for the total BRFSS samples ranged from .77 to .92 (p < .01). More detail regarding the methodology used for generating state-level binge drinking estimates, including sample weighting procedures, is provided in BRFSS reports.6
Youth Risk Behavior Survey (YRBS)
YRBS state and DC prevalence estimates for any past 30-day binge drinking (5 or more consecutive drinks within a couple of hours) were obtained from the Morbidity and Mortality Weekly Report.21 Prevalence estimates were reported for all 9th through 12th graders who participated in the YRBS. The number of participating states varied from 22 to 40 across the five years. Correlations between annual binge drinking estimates for the YRBS samples ranged from .68 to .94 (p < .01). More detail regarding the methodology used for generating state-level binge drinking estimates, including sample weighting procedures, is provided in YRBS reports.21
Brewers Almanac
State and DC per capita sales data for beer, wine, and spirits were obtained from annual Brewers Almanacs published online.22 These figures represent gallons of beer, wine, and spirits shipped from producers to wholesalers in each state and DC, divided by the total population. We computed total annual per capita alcohol sales for each state and DC by summing the values for beer, wine, and spirits.
Fatal Accident Reporting System (FARS)
Annual data for the percentage of drivers involved in fatal motor vehicle crashes with BAC ≥ .08 were obtained from the online FARS reporting system.23 These figures were reported for all 50 states and DC.
Data Analysis
All analyses for this study were conducted in 2009. Pearson product-moment correlations were used to assess the degree of association between state survey-based estimates of binge drinking, per capita alcohol sales, and percentage of drivers in fatal crashes with BAC ≥ .08. Correlations of .10–.29 were considered weak, .30–.49 moderate, and ≥ .50 strong.24
Results
As shown in Table 1, 26 (52%) of the 50 correlations between state survey estimates of binge drinking and per capita alcohol sales were moderate, while 18 (36%) were strong and six (12%) were weak. Correlations were strongest for NSDUH and BRFSS samples, especially among adults at least 25 years old. The weakest correlations were observed for the YRBS and for BRFSS underage respondents in 2001 and 2005.
Table 1.
Correlations between state survey estimates of binge drinkinga and per capita alcohol salesb by year and age group
| Per capita alcohol sales | ||||||
|---|---|---|---|---|---|---|
| Survey | 1999 | 2001 | 2003 | 2005 | 2007 | |
| NSDUH | ||||||
| Total sample | .43** | .59** | .54** | .45** | .55** | |
| 12–17 | .43** | .44** | .41** | .28* | .46** | |
| 12–20 | --- | --- | .41** | .38** | .30* | |
| 18–25 | .28* | .48** | .42** | .47** | .51** | |
| 26+ | .47** | .60** | .55** | .43** | .55** | |
| BRFSS | ||||||
| Total sample | .52** | .51** | .53** | .56** | .49** | |
| 18–20 | .47** | .21 | .39** | .16 | .49** | |
| 18–24 | .42** | .39** | .44** | .41** | .30* | |
| 25+ | .53** | .51** | .51** | .57** | .50** | |
| YRBS | .25 | .34* | .34* | .32* | .23 | |
Five or more consecutive drinks on at least one occasion in the past 30 days. NSDUH - National Survey on Drug Use and Health; BRFSS - Behavioral Risk Factor Surveillance Survey; YRBS - Youth Risk Behavior Survey.
State per capita alcohol sales data were obtained from the Brewers Almanac published by the Beer Institute.
p<.05,
p<.01
When examined separately, state per capita beer and spirits sales were more strongly associated with state survey estimates of binge drinking than were per capita wine sales (results not reported in tables). Across the five years, 43 (86%) of the 50 correlations between state binge drinking survey estimates and per capita beer sales were moderate to strong, compared to 31 (62%) of spirits sales, and 15 (30%) of wine sales.
Results displayed in Table 2 indicate that 36 (72%) of the correlations between state survey estimates of binge drinking and motor vehicle crashes with driver BAC ≥ .08 were moderate, while seven (14%) were strong and an equal number were weak. The weakest correlations were again observed for the YRBS and BRFSS underage respondents.
Table 2.
Correlations between state survey estimates of binge drinkinga and percent of drivers in fatal crashes with BAC ≥ .08b by year and age group
| Percent of drivers in fatal crashes with BAC ≥ .08 | ||||||
|---|---|---|---|---|---|---|
| Survey | 1999 | 2001 | 2003 | 2005 | 2007 | |
| NSDUH | ||||||
| Total sample | .40** | .60** | .59** | .50** | .42** | |
| 12–17 | .49** | .46** | .47** | .27 | .40** | |
| 12–20 | --- | --- | .53** | .40** | .25 | |
| 18–25 | .31* | .48** | .56** | .48** | .39** | |
| 26+ | .39** | .60** | .54** | .49** | .40** | |
| BRFSS | ||||||
| Total sample | .32* | .49** | .42** | .44** | .43** | |
| 18–20 | .11 | .28* | .34* | .21 | .36** | |
| 18–24 | .26 | .42** | .46** | .30* | .45** | |
| 25+ | .32* | .47** | .36* | .45** | .35* | |
| YRBS | .34* | .31 | .45** | .34* | .27 | |
Five or more consecutive drinks on at least one occasion in the past 30 days. NSDUH - National Survey on Drug Use and Health; BRFSS - Behavioral Risk Factor Surveillance Survey; YRBS - Youth Risk Behavior Survey.
State fatal crash data were obtained from the Fatal Accident Reporting System maintained by the National Highway Traffic Safety Administration.
p<.05,
p<.01
Discussion
In this study we found that 88% of the associations between state survey-based estimates of past 30-day binge drinking were at least moderately (r ≥ .30) associated with per capita alcohol sales in their respective states, and that a similarly high proportion (86%) of these estimates were at least moderately associated with the percentage of drivers in fatal motor vehicle crashes who had a BAC of at least .08. These findings provide evidence for the construct validity of state-level estimates of binge drinking that are based on survey data. Since they were first published, there has been considerable concern as to the quality of these estimates, as they are subject to a variety of potential biases. These include highly variable sample sizes and response rates across the states, adjustments to increase the representativeness of survey samples and, in the case of the NSDUH, estimation procedures that are based on both respondent data and predictive models. Of particular importance, the associations reported here for the NSDUH and BRFSS are consistent and stable across five survey administrations at biennial intervals, from 1999 through 2007. The associations of YRBS data with the two archival outcomes specified are somewhat weaker, particularly in the last (2007) of the five years assessed, but most of these are still in the lower end (r = .30–.40) of the “moderate” range. These results do not necessarily mean that YRBS estimates of binge drinking are less valid than NSDUH or BRFSS estimates, since adolescents are less likely than adults to purchase and consume alcohol, and less likely to be involved in alcohol-related motor vehicle crashes. Of further interest, the association of per capita beer sales with binge drinking was considerably stronger than sales of spirits, which in turn were stronger than wine sales. This finding is not surprising, since beer is the alcoholic beverage mostly commonly consumed in heavy drinking episodes.25, 26
The strength and robustness of these findings suggest that survey-based state level binge drinking estimates may be used, with appropriate caution, in a variety of research contexts. For example, it may be helpful to examine associations between state-level policies related to alcohol, as summarized on the Alcohol Policy Information System (APIS) of the National Institute on Alcohol Abuse and Alcoholism, and binge drinking rates.27 Such an examination, which could include strategies to decrease supply through interdiction or reductions in outlet density or the social availability of alcohol (e.g., keg registration or social host laws), could provide evidence for researchers, practitioners and policy-makers to address a longstanding public health concern.
We capitalized on available data from all 50 states and the District of Columbia concerning state-level estimates from three discrete surveys, each with its own data collection methodology, and two record-based measures, over five biennial iterations beginning in 1999. As such, this study complements and extends previous published research that has examined associations among estimates of heavy drinking yielded by various surveys.15, 16 The associations we have reported were primarily in the moderate to strong range, which is what one would expect given the disparate nature of the measures of the constructs examined. In light of the various types of systematic error that are likely to be present in state-level survey and record based measures for alcohol related behaviors (e.g., recall error in self-reports of alcohol use, alcohol consumed outside of the state where it was sold), our findings may underestimate the strength of association between the prevalence of binge drinking, per capita alcohol sales, and alcohol-involved motor vehicle crashes. Nevertheless, our findings should increase confidence in the validity of state survey-level estimates of binge drinking, as well as their potential utility as a means to evaluate the effectiveness of statewide strategies to reduce this hazardous behavior.
Acknowledgements
This study was supported by grants from the National Institute on Alcohol Abuse and Alcoholism (NIAAA grant Nos. P60 AA006282 and R01 AA016584).
References
- 1.NIAAA. NIAAA council approves definition of binge drinking. NIAAA Newsletter. 2004;3:3. [Google Scholar]
- 2.SAMHSA. Results from the 2008 National Survey on Drug Use and Health: national findings. NSDUH Series H-36, HHS Publication No. SMA 09-4434. Rockville, MD: OAS; 2009. [Google Scholar]
- 3.Johnston LD, O'Malley PM, Bachman JG, Schulenber JE. Monitoring the Future national results on adolescent drug use: Overview of key findings, 2008. NIH Publication No. 09-7401. Bethesda, MD: NIDA; 2009. [Google Scholar]
- 4.Fell JC, Fisher DA, Voas RB, Blackman K, Tippetts AS. The relationship of underage drinking laws to reductions in drinking drivers in fatal crashes in the United States. Accid Anal Prev. 2008;40:1430–1440. doi: 10.1016/j.aap.2008.03.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hughes A, Sathe N, Spagnola K. State estimates of substance use from the 2006–2007 National Surveys on Drug Use and Health, Introduction and Appendix D. NSDUH Series H-35, HHS Publication No. SMA 09-4362. Rockville, MD: SAMHSA; 2009. [Google Scholar]
- 6.CDC. Behavioral Risk Factor Surveillance System Survey Data. 2009 April 30; http://www.cdc.gov/BRFSS/technical_infodata/surveydata.htm.
- 7.CDC. Behavioral Risk Factor Surveillance System Summary Data Quality Report, 2007. 2008 March 9; ftp://ftp.cdc.gov/pub/Data/Brfss/2007SummaryDataQualityReport.pdf.
- 8.Fendrich M, Johnson TP. Examining prevalence differences in three national surveys of youth: impact of consent procedures, mode, and editing rules. J Drug Issues. 2001;31:615–642. [Google Scholar]
- 9.Kempf AM, Remington PL. New challenges for telephone survey research in the twenty-first century. Annu Rev Public Health. 2007;28:113–126. doi: 10.1146/annurev.publhealth.28.021406.144059. [DOI] [PubMed] [Google Scholar]
- 10.Schneider KL, Lapane KL. Adult health behaviors. Public Health Reports. 2007;122:432. [PMC free article] [PubMed] [Google Scholar]
- 11.Voigt LF, Koepsell TD, Daling JR. Characteristics of telephone survey respondents according to willingness to participate. Am J Epidemiol. 2003;157:66–73. doi: 10.1093/aje/kwf185. [DOI] [PubMed] [Google Scholar]
- 12.CDC. Youth Risk Behavior Surveillance - United States, 2007. MMWR Publication Number 57: SS04. Atlanta, GA: 2008. Jun 6, [Google Scholar]
- 13.Flewelling RL, Paschall MJ, Ringwalt C. The epidemiology of underage drinking in the United States: an overview. In: Committee on Developing a Strategy to Reduce and Prevent Underage Drinking. In: Bonnie RJ, O'Connell ME, editors. Reducing underage drinking: a collective responsibility, background papers. Washington, DC: The National Academies Press; 2004. pp. 319–350. [Google Scholar]
- 14.Miller JW, Gfroerer JC, Brewer RD, Naimi TS, Mokdad A, Giles WH. Prevalence of adult binge drinking: A comparison of two national surveys. Am J Prev Med. 2004;27:197–204. doi: 10.1016/j.amepre.2004.05.004. [DOI] [PubMed] [Google Scholar]
- 15.Nelson DE, Naimi TS, Brewer RD, Nelson HA. State alcohol-use estimates among youth and adults, 1993–2005. Am J Prev Med. 2009;36:218–224. doi: 10.1016/j.amepre.2008.10.018. [DOI] [PubMed] [Google Scholar]
- 16.Nelson TF, Naimi TS, Bewer RD, Wechsler H. The state sets the rate: the relationship among state-specific college binge drinking, state binge drinking rates, and selected state alcohol control policies. Am J Public Health. 2005;95:441–446. doi: 10.2105/AJPH.2004.043810. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull. 1959;56:81–105. [PubMed] [Google Scholar]
- 18.Carmines EG, Zeller RA. Reliability and validity assessment. Beverly Hills, CA: Sage; 1979. [Google Scholar]
- 19.DeVellis RF. Scale development: theory and applications. Newbury Park, CA: Sage; 1991. [Google Scholar]
- 20.SAMHSA. State level data on alcohol, tobacco, and illegal drug use. 2008 June 19; http://www.oas.samhsa.gov/states.cfm.
- 21.CDC. YRBSS Publications: Morbidity & Mortality Weekly Report (MMWR) 2008 December 30; http://www.cdc.gov/HealthyYouth/yrbs/pubs_mmwr.htm.
- 22.Beer Institute. Brewers Almanac [Online] 2009 November 10; http://www.beerinstitute.org/statistics.asp?bid=200>.
- 23.NHTSA. Fatality Analysis Reporting System Encylopedia. 2009 November 10; http://www-fars.nhtsa.dot.gov/Main/index.aspx.
- 24.Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc.; 1988. [Google Scholar]
- 25.Naimi TS, Brewer RD, Miller JW, Okoro C, Mehrotra C. What do binge drinkers drink? Implications for alcohol control policy. Am J Prev Med. 2007;33:188–193. doi: 10.1016/j.amepre.2007.04.026. [DOI] [PubMed] [Google Scholar]
- 26.Rogers JD, Greenfield TK. Beer drinking accounts for most of the hazardous alcohol consumption reported in the United States. J Stud Alcohol. 1999;60:732–739. doi: 10.15288/jsa.1999.60.732. [DOI] [PubMed] [Google Scholar]
- 27.NIAAA. Alcohol Policy Information System. 2009 November 10; http://www.alcoholpolicy.niaaa.nih.gov/