Abstract
Background:
Valid measurement of alcohol use can be difficult in surveys, which are subject to biases like under-reporting and differential non-response. Still, monitoring trends, policy impacts, disparities, and related issues all require valid individual- and state-level drinking data collected over time. Here we propose a double-adjustment approach for improving validity of the Behavioral Risk Factor Surveillance System (BRFSS) alcohol measures.
Methods:
Validity analyses of the 1999–2016 BRFSS, a general population survey of US adults. Measures are aggregated to state level for N = 918 observations, single-adjusted for BRFSS methodologic changes, and double-adjusted by per capita consumption. Fixed effects models 1) assess predictive validity using adjusted BRFSS drink volume to predict mortality outcomes; 2) assess outcome validity using state-level alcohol taxes to predict adjusted BRFSS volume.
Results:
Neither the raw nor the single-adjusted BRFSS drinking measures were related to mortality in the expected direction, while double-adjusted BRFSS volume and 5+ days were significantly positively related to mortality, as expected. Spirits and beer taxes were not related to single-adjusted BRFSS drinking in the expected direction. However, spirits and beer taxes were both significantly related to double-adjusted BRFSS volume in the expected directions.
Conclusions:
Future studies should consider using the double-adjusted BRFSS measures to ensure the validity of drinking survey data in analyses where variation over time is considered.
Keywords: Behavioral Risk Factor Surveillance System, BRFSS, alcohol, per capita consumption, measurement, validity
INTRODUCTION
Valid measurement of alcohol use can be difficult in surveys, which are subject to biases like under-reporting and differential non-response (Boniface et al., 2014; Boniface et al., 2017). Still, monitoring trends, policy impacts, disparities, and related issues all require valid individual- and state-level drinking data collected over time. A number of US national surveys exist for these purposes, including the National Alcohol Survey (NAS), the National Survey on Drug Use and Health (NSDUH), and the Behavioral Risk Factor Surveillance System (BRFSS). These general population surveys are similar in reaching all 50 states and the District of Columbia (DC), over-sampling racial/ethnic minorities, and including volume and heavy occasion drinking questions. The surveys differ in their concentrations (e.g., alcohol, drugs, health behaviors), modes (e.g., telephone, in-person), sampling (e.g., random digit dial, address-based), and frequency (e.g., yearly, every five years).
The BRFSS, which the US Centers for Disease Control (CDC) has conducted annually since 1984, is a telephone survey focused on health behaviors, chronic conditions, and use of services (Centers for Disease Control and Prevention (CDC), 2015). According to the CDC, the BRFSS is the largest continuous health survey in the world, with >400,000 (18+) US adults completing interviews every year. Besides its reach, one of the primary advantages of the BRFSS is its frequency, as the yearly data permit close monitoring of trends and are especially useful for policy impact evaluations compared to most other national surveys, which are less frequent. However, a systematic review (Pierannunzi et al., 2013) concluded that the reliability and validity of the 2004–2011 BRFSS substance use measures were only moderate compared to other national surveys. For example, the prevalence of binge drinking at both national and state levels are consistently lower in BRFSS than in the NSDUH (Pierannunzi et al., 2013).
A validity study evaluating concordance of binge drinking from the combined 1999 and 2001 BRFSS surveys and the NSDUH estimated the national prevalence of binge drinking as 14.7% (95% CI: 14.5%−15.2%) in the BRFSS and 21.6% (95% CI: 21.2%−22.0%) in the NSDUH. The BRFSS state-level prevalence estimates were also significantly lower than NSDUH in 46 states and DC (Miller et al., 2004). Because the demographics of binge drinkers and wording of the binge questions were similar between the two surveys, the authors concluded that the differences were likely due to varying methodologies, i.e., the NSDUH was face-to-face and focused on drug use while the BRFSS was via telephone and focused on health behaviors (Miller et al., 2004).
Sales-based consumption estimates, such as per capita alcohol consumption (PCC), are considered more complete and objective than survey data (Greenfield and Kerr, 2008; Kerr and Greenfield, 2007). Sales figures cover the vast majority of consumption due to the low level of unrecorded alcohol use in the US (World Health Organization, 2014) and widespread availability of alcohol tax information, from which PCC estimates are derived. A comparison of US sales and BRFSS drinking data for 1993–2006 highlighted strong cross-sectional correlations, although the BRFSS estimates only accounted for a median of 22%−32% of sales across years (Nelson et al., 2010). While the BRFSS significantly underestimated volume, the cross-sectional associations between sales and BRFSS survey data were robust, suggesting that both sources provide important information (Nelson et al., 2010). A commentary on this paper specified a number of potential sources for under-coverage, including assumptions regarding drink alcohol content, the use of usual quantity/occasion in a past-month timeframe, and earlier survey content focused on health behaviors that might influence later responses regarding alcohol (Kerr, 2010). Importantly, this commentary also noted a gap in evaluating the consistency of trend correlations with sales data, meaning that the validity of BRFSS survey-based trend estimates remains unknown.
Under-coverage similarly affects international self-report surveys, with many only accounting for 30%−40% of official alcohol sales (Stockwell et al., 2016). In response, a small number of studies have proposed adjustments to survey data using per capita consumption data to ameliorate this problem. For example, a study investigating various distributions to model consumption within demographic subgroups in the NESARC concluded that triangulating alcohol survey data with per capita consumption data is crucial for validity, and suggested a routine application of an 80%−100% upwards adjustment to per capita consumption (Rehm et al., 2010). The suggested adjustment factor of 80% matches that recommended in a study comparing per capita consumption and data from cohort studies, which are more accurate and have less under-reporting than surveys (Stockwell et al., 2018). Similarly, an analysis of the 2008 Canadian Alcohol and Drug Use Monitoring, a survey meant to represent the population of Canada, found its coverage rate to be as low as 34% compared to per capita estimates; thus, the authors also recommended triangulating survey data with per capita consumption in any comparative analyses and intervention planning (Shield and Rehm, 2012). Likewise, a study applying the triangulation methods proposed by (Rehm et al., 2010) to British survey data found that the adjustments yielded more accurate estimates of consumption that were in turn more strongly associated with alcohol-attributable harms (Meier et al., 2013).
Although per capita adjustments have been used to correct surveys in countries like Canada and England, we know of no study applying these adjustments to US data. The BRFSS is the only series that collects alcohol data with sampling at the US state level, which corresponds with needs for analyses of state-level policies and subgroup analyses of alcohol-related mortality causes. Survey-based estimates of drinking are needed for analyzing differential impacts of alcohol polices across subgroups, as well as relationships between volume and health/mortality relevant to disparities (Zemore et al., 2018). Studies of alcohol-mortality relationships overwhelmingly show that alcohol volume is positively related to mortality, e.g., from unintentional injuries (Rehm et al., 2017), while prior studies of policy effectiveness show that stronger alcohol policies reduce drinking overall and among subgroups (Naimi et al., 2014; Xuan et al., 2015). Thus, the relationship between BRFSS alcohol volume should be significantly related to both mortality outcomes (as a predictor) and policy predictors (as an outcome). However, as a preliminary step in our research undertaking both of these types of analyses, we have found it necessary to evaluate and adjust state-level alcohol measures from the 1999–2016 BRFSS surveys to improve validity. We outline our adjustment approach here.
Current study
Because BRFSS alcohol measures are used as both predictor and outcome variables, our overarching aim is to compare how unadjusted, single-adjusted, and double-adjusted BRFSS alcohol measures are related to mortality outcomes and policy predictors. The “single-adjustment” accounts for BRFSS methodologic changes across surveys, while the “double-adjustment” further accounts for trends in per capita consumption. Specifically, we 1) adjust the state-level BRFSS drink volume estimates; 2) assess predictive validity using adjusted BRFSS estimates to predict mortality outcomes; and 3) assess outcome validity using state-level spirits and beer taxes to predict adjusted BRFSS volume estimates.
METHODS
Data and measures
Behavioral Risk Factor Surveillance System
Description of the BRFSS methodology can be found elsewhere (Ahluwalia et al., 2003). Using the 2016 BRFSS as the standard, the 30-day volume measure was based on: Frequency (“During the past 30 days, how many days per week or per month did you have at least one drinking of any alcoholic beverage?”), Usual Quantity (“On the days when you drank, about how many drinks did you drink on average?”), Binge Days (“How many times during the past 30 days did you have 5/4 (for men/women) or more drinks on an occasion?”). For those whose usual quantity was <5/4 drinks (for men/women), volume was calculated as (frequency days – binge days)*usual quantity + binge days * 6 drinks. This is very similar to the method used in (Dwyer-Lindgren et al., 2015). The only difference is that we used 6 drinks rather than maximum drinks to estimate the average binge volume, as maximum quantity question was not available for 2004 and earlier BRFSS and 6-drink is more conservative than maximum. For those whose usual quantity was ≥5/4 drinks, volume was calculated as frequency days*usual quantity. Mean volume (drinks/month) and binge days were created by state and year, and further by gender and age.
Per Capita Consumption
Per capita consumption (PCC) measures total ethanol from wine, beer, and liquor sales for each state, averaged by population aged 15 and older. PCC measures were recently estimated applying a sales-weighted mean % alcohol by volume (%ABV) (Kerr et al., 2006; Martinez et al., 2019). Recent work has shown that a time-varying measure of the percent alcohol by volume (%ABV) in the calculation of PCC improves precision (Martinez et al., 2019). Thus, PCC estimates based on time-variant %ABV, as used here, are a valuable resource with which to make corrections to potential under-coverage in survey data. Here, PCC was converted to standard drinks to compare with BRFSS-aggregate mean volume (one drink = 0.6 ounce pure ethanol).
Mortality outcomes
Mortality data were obtained from the National Center for Health Statistics Compressed Mortality File series (National Center for Health Statistics, 2004). Mortality rates (per 100,000) were derived separately for three age groups (20–34, 35–54, 55–74), and age-standardized within the subgroups. Total unintentional injuries were defined based on ICD 10 codes V01-X59 and Y85-Y86. Motor vehicle accident (MVA) deaths were from ICD 10 codes V02-V04, V09.0, V09.2, V12-V14, V19.0-V19.2, V19.4-V19.6, V20-V79, V80.3-V80.5, V81.0-V81.1, V82.0-V82.1, V83-V86, V87.0-V87.8, V88.0-V88.8, V89.0 and V89.2. Poisoning deaths were based on ICD 10 codes X40-X49, while other unintentional injury deaths were all unintentional injuries excluding MVA and poisoning.
Alcohol taxes
Tax rates for beer and spirits for 2000–2002 come from the Prevention Research Center’s Statewide Availability Data System (Ponicki, 2004). Tax rates for beer and spirits for 2003–2015 come from the National Institute on Alcohol Abuse and Alcoholism’s Alcohol Policy Information System (APIS) (National Institute on Alcohol Abuse and Alcoholism). This includes both specific excise taxes and ad valorem excise taxes, which are a percentage of the retail price. The annual NABCA Survey Book provides critical data on spirits mark-ups in states with government-controlled sales, thus enabling us to derive control state tax estimates absent from APIS, using the method described in (Kerr et al., 2014). Beer and spirits tax data for license states were accessed via APIS for the years 2003–2015 and verified against information from the Federation of Tax Administration; the latter was also the source for federal tax data for 2000–2003. All taxes are CPI-adjusted to 2012 dollars.
Adjustment approach: Single-adjusted
The alcohol questions in the BRFSS underwent several changes, as detailed previously (Dwyer-Lindgren et al., 2015). Until 2005, instead of using 5+/4+ as indicator of binge drinking for men and women separately, all respondents were asked “How many times during the past 30 days did you have 5 or more drinks on an occasion?” Furthermore, in 2011 cell phone surveys were incorporated, and weighting methods were modified applying a raking algorithm; the BRFSS uses iterative proportional fitting, or “raking” to adjust for demographic differences between persons who are sampled and the target population. Applying these new methods improved the coverage of more disadvantaged groups (e.g., lower income, lower education, and younger age), and prevalence estimates for some of the most salient indicators of poor health and risky health behaviors (Pierannunzi et al., 2012).
To calibrate 4+ days for women in 2005 and earlier based on the reported 5+ days, two steps were taken. The first step was to estimate the likelihood of any 4+ for women who reported no 5+ in past 30 days based on their Usual Quantity measure, comparing the prevalence of any 5+ and any 4+ before and after 2006 BRFSS for each usual quantity level. For example, when the usual quantity reported was three drinks, the prevalence of any 5+ and any 4+ was 35.3% and 49.0%, respectively. Then, the prevalence of any 4+ for those with a usual quantity of three drinks without 5+ was estimated as 21.2% ((49.0%−35.3%)/(1–35.3%)). Those women were then randomly assigned the value of 1 or 0 (for any 4+ or not), with a 21.2 % probability of being assigned 1. This was done for those having a usual quantity of 1–3 drinks, with all individuals whose usual quantity was 4+ assigned to 1. The second step was to calibrate the mean 4+ days for women in 2005, by comparing the difference between 5+ days (prior to 2006) and 4+ days (2006 and later) by their usual quantity. For example, for a female whose usual quantity was 4+ drinks, if she didn’t report 5+ but was just recoded to positive for any 4+, she would be assigned 3.4 days of 4+; if she did report 5+, her 4+ days would be her 5+ days plus 0.8 days. The rationale to assign 3.4 and 0.8 days was based on the empirical subsample averages of the number of 4+ days from women reporting none or any 5+ days, i.e., the average number of 4+ days among women from the 2006–2016 BRFSS was used to estimate the number of 4+ days for women without and with 5+ days from the 1999–2005 BRFSS (because the binge drink question changed from 5+ to 4+ in 2006). The adjustment procedure utilized the strong correlation between usual quantity and binge (about 0.5). It inflated the prevalence of any binge drinking for women, and also shifted up the mean binge drinking days, for BRFSS 2005 and earlier years. Note that the procedures above can only apply to adjustment of aggregate measures such as prevalence and mean.
To adjust for the BRFSS cell phone and raking changes in 2011, we first considered the significant differences observed only between 2010 and 2011. For example, the proportion of ≤high school graduate in total US from BRFSS was 38.9%, 37.9% and 44.5% in 2009, 2010 and 2011 respectively. Similarly, the prevalence of any 30-day binge drinking was 16.8%, 16.3% and 20.0% from 2009 to 2011. The following steps were taken to adjust for these differences in drinking and demographic aggregate measures in 2010 and earlier years. First, we substituted the measures in 2011 for those in 2010. Second, measures in 2009 and earlier were adjusted proportionally to maintain the same rate of change from year to year as in the unadjusted data. This can be illustrated by an example in which we adjust the 2009 binge prevalence by using the proportion of those with ≤high school education: make the 2010 prevalence the same as the 2011 mean, then apply the change rate in original 2009–2010 to the adjusted prevalence. Therefore, the adjusted 2010 prevalence of individuals with a high school education or less would be 44.5% (the 2011 prevalence), and since the 2009–10 change rate is 38.9/37.9=1.026, the adjusted 2009 prevalence would be 44.5%*1.026=45.7%. The adjusted prevalence would be 45.7%, 44.5% and 44.5% for 2009, 2010 and 2011, separately. The year 2008 and earlier can be adjusted applying the same change rate. This process of adjustment was applied to each sub-group (state by gender by race by age). We refer to the volume measure that was adjusted for methodologic changes in the survey as “single-adjusted.”
Adjustment approach: Double-adjusted
To address issues of under-coverage in the BRFSS, we then adjusted further the BRFSS-aggregate drinking measures (volume and 5+ days) for total sample and for each subgroup by the ratio between PCC and BRFSS mean volume at given state-year, after the adjustment for BRFSS methodologic changes. This “double-adjustment” corrects for both under-coverage and potential bias from the methodologic changes in 2011, and we refer to this measure as “double-adjusted.”
Figure 1 illustrates how double-adjusted volume was derived from single-adjusted volume, using the US national trend as an example (note that state-level data were used for analyses, national data are shown here for illustration). First, the ratio between PCC and BRFSS single-adjusted volume for the total population (the solid curve in Figure 1) was derived. This adjustment ratio was then multiplied by the single-adjusted volume for each subgroup (by gender, age and race/ethnicity) to derive the double-adjusted volume. For the total population, the double-adjusted volume is the same as PCC. For the men and women subgroups shown in Figure 1, applying the PCC adjustment factor not only shifted the curves upward, but also affected the trend of the curves over time.
Figure 1.
Double-adjusted vs. single-adjusted volume for total US, men, and women, 1999–2016
Validity analyses
The state-level panel data linking mortality rates and BRFSS state-aggregate drinking and demographic measures contained 918 observations (50 states and D.C. across 18 years 1999–2016). First, to assess predictive validity, semi-logged fixed effect models were used to estimate the relationships between unadjusted and adjusted BRFSS state-aggregate drinking measures and mortality rates from unintentional injuries, with mortality rates log-transformed. Fixed effect models explicitly control for cross-state heterogeneity by modeling the within-state change. Random effect models were also estimated in sensitivity analyses with results very similar to those from fixed effects (not shown). Finally, to assess outcome validity we examined the effect of beer- and spirits-specific taxes and outlet densities on volume measures using the fixed effect models. We compared the outcome validity of unadjusted, single-adjusted, and double-adjusted BRFSS mean volume estimates. The outcome validity analyses used 2000–2015 BRFSS data because only 2000–2015 tax data were available. All models included year indicators to control for secular trends common across states. All analyses were performed in STATA version 15 (StataCorp., 2017).
RESULTS
Preliminary analyses comparing BRFSS volume and PCC
First, fixed effect models compared unadjusted BRFSS-aggregate mean volume (in drinks/month), single-adjusted BRFSS mean volume accounting only for methodologic changes (i.e., incorporation of cell phones and raking in 2011), and state-level aggregate PCC in predicting mortality rates of total unintentional injury. The comparison of PCC to BRFSS mean volume was motivated by two observations: the substantial difference in trends across time between BRFSS volume and PCC, and the absence of an expected positive association between BRFSS-derived drinking measures and injury mortality rates. Figure 2 shows trends of the three volume measures, in drinks/month, for the total US. We tested linear trends of volume measures at the state level using fixed effect models. A significant increase in PCC was observed in most years 1999–2016, except for a drop in 2008–2010 during the recession. Significant downward trends over time (1999–2016) were observed for the BRFSS-raw volume and the single-adjusted volume. In contrast, significant upward trends were found for double-adjusted volume. These results at the state level are consistent to the volume trend shown in Figure 2 for the total US.
Figure 2.
Trend of alcohol consumption from Per Capita Consumption (PCC) and BRFSS-aggregate volume (before and after single-adjustment for methodologic changes) for total US, 1999–2016
The raw BRFSS mean volume also shows a clear upshift from 2010 to 2011, most likely due to the methodologic changes. PCC was almost flat between 2010 and 2011, justifying our methodologic adjustment approach to substituting 2011 volume for 2010. After the methodologic changes were adjusted, BRFSS mean volume showed a slight downward trend overall over time, clearly different from the mostly upward PCC trend. Figure 2 also shows the large under-coverage from the survey volume compared to sales data. The average BRFSS adjusted volume from 1999 to 2016 was 15.0 drinks/month, representing only 37% of the average PCC of 40.1 drinks/month.
Table 1 shows results of the preliminary analysis, with BRFSS raw mean volume, BRFSS single-adjusted mean volume, and BRFSS double-adjusted mean volume predicting log-rates of unintentional injuries. When only year dummies were included, BRFSS unadjusted volume had a significantly (unexpected) negative association with total unintentional injury deaths whereby unintentional injuries were lower when raw volume was higher. This inverse association with volume was even stronger when BRFSS aggregate demographic variables were included. The inverse association with volume appeared again using BRFSS single-adjusted volume as predictor. However, significant positive associations were observed with the double-adjusted BRFSS mean volume, consistent with prior literature. The estimated coefficient of 0.20, when both year dummies and BRFSS aggregate demographics were included in Model 2, translates to a 22% increase in total unintentional injury mortality rates for an increase of 1 drink/day on average based on PCC. Taken together, these findings underscore the need for the double-adjustment to account for both methodologic changes and under-coverage in the BRFSS, as described earlier.
Table 1.
Predicting log-transformed mortality rate (in 100,000) for total unintentional injuries by BRFSS state average volume, Per Capita consumption (PCC) and BRFSS state aggregate demographic variables (50 states and DC from 1999–2016, N=918)
| BRFSS raw volume2 | BRFSS volume adjusted for methodologisc changes2 (single-adjusted) | BRFSS adjusted for methodologic changes and PCC2 (double-adjusted) | ||||
|---|---|---|---|---|---|---|
| Coef | 95% CIs | Coef | 95% CIs | Coef | 95% CIs | |
| Model 11 | ||||||
| Alcohol volume (Drinks per day) | −0.20 | (−0.36, −0.04) * | −0.05 | (−0.19, 0.08) | 0.25 | (0.16, 0.35) *** |
| Model 21 | ||||||
| Alcohol volume (Drinks per day) | −0.30 | (−0.47, −0.13) ** | −0.18 | (−0.31, −0.04) * | 0.20 | (0.10, 0.31) *** |
| % Men | 8.17 | (5.03, 11.31) *** | 6.03 | (2.78, 9.29) *** | 4.51 | (1.20, 7.82) ** |
| % Age 18–25 | 0.70 | (0.00, 1.40)ƚ | 0.75 | (0.38, 1.12) *** | 0.51 | (0.14, 0.89) ** |
| % Age 60+ | 0.47 | (−0.47, 1.41) | 1.15 | (0.24, 2.07) * | 0.74 | (−0.18, 1.67) |
| % African Americans | −0.01 | (−0.55, 0.54) | 0.73 | (0.20, 1.27) ** | 0.74 | (0.21, 1.27) ** |
| % Hispanics | −0.82 | (−1.33, −0.30) ** | −0.07 | (−0.49, 0.35) | 0.00 | (−0.42, 0.41) |
| % ≤ HS grad | 1.16 | (0.60, 1.72) *** | 0.00 | (−0.45, 0.45) | −0.08 | (−0.53, 0.37) |
| % ≥ College grad | 0.54 | (−0.13, 1.22) | 1.01 | (0.13, 1.89) * | 0.71 | (−0.17, 1.59) |
| % Unemployed | −0.44 | (−1.27, 0.38) | −0.20 | (−1.00, 0.59) | 0.20 | (−0.59, 1.00) |
| % Not in labor | −0.55 | (−1.09, −0.02) * | −0.91 | (−1.40, −0.43) *** | −0.80 | (−1.28, −0.32) ** |
| % Married | 0.05 | (−0.57, 0.67) | −0.86 | (−1.55, −0.17) * | −0.81 | (−1.50, −0.13) * |
| % Never married | −0.77 | (−1.55, 0.01)ƚ | −1.15 | (−1.73, −0.58) *** | −0.97 | (−1.54, −0.40) ** |
| % No health insurance | −0.06 | (−0.49, 0.37) | 0.35 | (−0.03, 0.73)ƚ | 0.27 | (−0.11, 0.65) |
Both models 1 and 2 control for year fixed effects
BRFSS demographics were entered in model 2 only
p<0.01
p<0.05
p<0.01
p<0.001
Coef = Coefficient estimates, CIs = Confidence Intervals
Predictive validity analyses
The next set of analyses assessed predictive validity. In order to confirm the predictive validity of PCC on its own, total unintentional injury mortality rates were first regressed on PCC. As expected, models showed that per capita consumption is significantly related to total unintentional injury mortality rates (coefficient = 0.20, 95% CI: (0.10, 0.31)). Table 2 shows results from fixed effect models predicting log-transformed total unintentional injury mortality rates from the single- and double-adjusted volume (drinks/day) and binge drinking measures. Models were fit for the total sample of state-years and by gender/age subgroups. The single-adjusted BRFSS mean volume estimates were not significantly related to mortality in the expected positive direction for any of the subgroups. By contrast, the double-adjusted mean volume and 5+ days showed significant positive effects on total unintentional injury mortality rates for the total population and for both men and women separately, as expected. Significant associations were also found for the 35–54 age group (men and women combined) and for men 35–54.
Table 2.
Predictive validity analyses: BRFSS volume and binge drinking, before and after adjusting for Per Capita Consumption (PCC), predicting log-transformed total unintentional mortality rate (in 100,000) in 50 states and DC from 1999–2016, N=9181
| BRFSS volume adjusted for methodologic changes (single-adjusted) |
BRFSS volume adjusted for methodologic changes and PCC (double-adjusted) |
|||
|---|---|---|---|---|
| Coef | 95% CIs | Coef | CIs | |
| Total | ||||
| Drinks per day | −0.18 | (−0.31, −0.04) * | 0.20 | (0.10, 0.31) *** |
| Binge days per week | 0.01 | (−0.26, 0.27) | 0.21 | (0.09, 0.34) *** |
| Men total | ||||
| Drinks per day | −0.04 | (−0.12, 0.05) | 0.11 | (0.05, 0.17) *** |
| Binge days per week | 0.09 | (−0.07, 0.25) | 0.13 | (0.06, 0.20) *** |
| Women total | ||||
| Drinks per day | −0.15 | (−0.39, 0.09) | 0.11 | (0.01, 0.20) * |
| Binge days per week | 0.34 | (−0.09, 0.78) | 0.24 | (0.08, 0.40) ** |
| 20–34, all genders | ||||
| Drinks per day | −0.01 | (−0.10, 0.08) | 0.02 | (−0.03, 0.07) |
| Binge days per week | 0.001 | (−0.16, 0.16) | 0.05 | (−0.02, 0.12) |
| 35–54, all genders | ||||
| Drinks per day | −0.04 | (−0.23, 0.16) | 0.08 | (0.01, 0.15) * |
| Binge days per week | 0.24 | (−0.10, 0.58) | 0.17 | (0.05, 0.28) ** |
| 55–74, all genders | ||||
| Drinks per day | 0.00 | (−0.17, 0.18) | 0.02 | (−0.04, 0.08) |
| Binge days per week | 0.13 | (−0.20, 0.47) | 0.04 | (−0.07, 0.16) |
| Men, 20–34 | ||||
| Drinks per day | −0.01 | (−0.06, 0.04) | 0.01 | (−0.02, 0.03) |
| Binge days per week | 0.003 | (−0.09, 0.09) | 0.02 | (−0.02, 0.06) |
| Men, 35–54 | ||||
| Drinks per day | −0.01 | (−0.12, 0.10) | 0.04 | (0.00, 0.08)ƚ |
| Binge days per week | 0.17 | (−0.03, 0.37) | 0.10 | (0.03, 0.17) ** |
| Men, 55–74 | ||||
| Drinks per day | 0.03 | (−0.08, 0.15) | 0.02 | (−0.02, 0.06) |
| Binge days per week | 0.08 | (−0.11, 0.28) | 0.03 | (−0.04, 0.09) |
| Women, 20–34 | ||||
| Drinks per day | 0.07 | (−0.15, 0.29) | 0.05 | (−0.04, 0.13) |
| Binge days per week | 0.15 | (−0.19, 0.49) | 0.09 | (−0.05, 0.22) |
| Women, 35–54 | ||||
| Drinks per day | −0.27 | (−0.62, 0.07) | −0.04 | (−0.16, 0.08) |
| Binge days per week | −0.42 | (−0.98, 0.14) | −0.11 | (−0.30, 0.09) |
| Women, 55–74 | ||||
| Drinks per day | −0.14 | (−0.50, 0.22) | −0.03 | (−0.15, 0.10) |
| Binge days per week | 0.33 | (−0.38, 1.05) | 0.07 | (−0.18, 0.31) |
All models control for year fixed effects and BRFSS demographic measures in Table 1
p<0.01
p<0.05
p<0.01
p<0.001
Coef = Coefficient estimates, CIs = Confidence Intervals
Double-adjusted = adjusted for both BRFSS methodologic changes and per capita consumption
In supplementary analyses, we then used fixed effect models to predict mortality rates from MVA, poisoning, and other unintentional injuries by the BRFSS volume and binge measures (please see Online Supplemental Table). Results were similar to those found for total unintentional injuries.
Outcome validity analyses
The final set of analyses assessed the validity of the BRFSS volume measures as outcomes. Table 3 displays results from fixed effects models predicting single-and double-adjusted BRFSS average drinks/day from spirits and beer taxes, where spirits and beer taxes are first modeled separately as predictors and then together in the same model. When modeled as separate predictors, neither spirits nor beer taxes were significantly related to single-adjusted BRFSS volume in the sample overall. However, both spirits and beer taxes were significantly related to double-adjusted volume in the expected direction. Furthermore, after the double-adjustment, the direction of the association between taxes and volume reversed from (counter-intuitive) positive to (expected) negative.
Table 3.
Outcome validity analyses: Predicting BRFSS volume per day before and after adjusted by PCC by US state-level policy variables, 2000–2015
| BRFSS volume adjusted for methodologic changes (single-adjusted) |
BRFSS volume adjusted for methodologic changes and PCC (double-adjusted) |
|||
|---|---|---|---|---|
| Coef | 95% CIs | Coef | 95% Cis | |
| Taxes modeled separately | ||||
| Total sample | ||||
| Spirits tax | 0.0016 | (−0.0001, 0.0034) | −0.0060 | (−0.0083, −0.0037) *** |
| Beer tax | −0.0029 | (−0.0080, 0.0022) | −0.0075 | (−0.0141, −0.0010) * |
| Men | ||||
| Spirits tax | 0.0028 | (−0.0003, 0.0058) | −0.0070 | (−0.0113, −0.0027) ** |
| Beer tax | −0.0022 | (−0.0110, 0.0066) | −0.0128 | (−0.0251, −0.0004) * |
| Women | ||||
| Spirits tax | 0.0007 | (−0.0004, 0.0018) | −0.0046 | (−0.0075, −0.0016) ** |
| Beer tax | −0.0022 | (−0.0054, 0.0010) | −0.0050 | (−0.0134, 0.0035) |
| Taxes modeled together | ||||
| Total sample | ||||
| Spirits tax | 0.0023 | (0.0004, 0.0042) * | −0.0058 | (−0.0082, −0.0034) *** |
| Beer tax | −0.0053 | (−0.0107, 0.0002) | −0.0015 | (−0.0084, 0.0054) |
| Men | ||||
| Spirits tax | 0.0035 | (0.0002, 0.0068) * | −0.0062 | (−0.0108, −0.0016) ** |
| Beer tax | −0.0060 | (−0.0154, 0.0035) | −0.0062 | (−0.0194, 0.0071) |
| Women | ||||
| Spirits tax | 0.0011 | (−0.0001, 0.0024) | −0.0045 | (−0.0077, −0.0013) ** |
| Beer tax | −0.0034 | (−0.0068, 0.0001) | −0.0002 | (−0.0093, 0.0088) |
p<0.05
p<0.01s
p<0.001
Coef = Coefficient estimates, CIs = Confidence Intervals
Double-adjusted = adjusted for both BRFSS methodologic changes and per capita consumption
When modeling spirits and beer taxes together as predictors, the direction of the association between spirits taxes and volume again reversed; spirits taxes had a significant positive relationship with single-adjusted volume and a significant negative association with double-adjusted volume. The direction and significance of the association between beer taxes did not change when regressed on single- vs. double-adjusted volume. Within gender subgroups, the pattern was the same for men as for the sample overall. Among women, the pattern was similar, though beer taxes were not significantly related to either single- or double-adjusted volume in any model.
DISCUSSION
We find that the BRFSS alcohol consumption variables under-estimate state-level mean volume. Adjustment for changes in BRFSS survey methodology over time only partially alleviates the under-coverage issue, and further adjustment using PCC estimates based on time-varying %ABV appears necessary for validity. As expected, the double-adjusted BRFSS mean volume and binge-drinking variables were consistently related to unintentional injury mortality rates while the unadjusted and single-adjusted volume and binge-drinking variables were not. The double-adjusted volume and binge-drinking variables were also significantly related to motor vehicle and other unintentional injury mortality rates, supporting the importance of the double-adjustment for predictive validity. Finally, both spirits and beer taxes had significant inverse relationships with the double-adjusted, but not the single-adjusted, BRFSS volume measures when modeled as separate predictors. In addition, the relationship between spirits taxes actually reversed when regressed on single- vs. double-adjusted BRFSS volume, further supporting the importance of the double-adjustment for outcome validity. Importantly, all of the tax estimates were in the expected negative direction when predicting the double-adjusted volume measures, even where the point estimates were not statistically significant. These patterns held for the sample overall and within gender subgroups, demonstrating the robustness of the double-adjustment. The pattern of associations between spirits and beer taxes and volume outcomes is also consistent with findings from National Alcohol Survey and PCC analyses (Subbaraman et al., in press).
The process of adjusting BRFSS data has been carried out successfully with other measures. For example, a study aiming to estimate national and state-level prevalence of obesity found that the prevalence of obesity and overweight was under-reported by 5.7 to 9.5 percentage points, that self-report biases were not consistent across demographic groups, and that appropriate weighting was necessary to correct for demographic factors such as gender and educational attainment (Liaw et al., 2005). Similarly, a study comparing tobacco use measures across national surveys found that the BRFSS (using the national median of the state BRFSS surveys) and NHIS estimates were statistically similar only after weighting the BRFSS (Klein et al., 2007). Though the current study is the first to our knowledge to apply corrections to BRFSS drinking data, a Spanish study using multiple national data sources to estimate average alcohol consumption concluded that relying on uncorrected estimates, especially self-reported consumption, for monitoring trends or other purposes is misleading (Sordo et al., 2016). The authors argued that self-reported consumption should be multiplied upwards by more than 85%−90%, and that the issues with the Spanish data most likely apply worldwide because under-reporting is common across cultures and locations.
In addition to ensuring estimates and associations can be accurately observed in a survey sample, our findings and those of other studies suggest adjustments are also important for sample sub-groups. A study extending early methodological work comparing BRFSS drink measures to the NHIS and NHANES found that while the BRFSS drink volume measures were similar to those from NHIS overall, BRFSS and NHIS volume and binge drinking estimates significantly differed when examining subgroups defined by gender, race/ethnicity, or education levels (Fahimi et al., 2008). Furthermore, overall binge drinking prevalence was significantly lower in the BRFSS compared to both the NHIS and the NHANES (Fahimi et al., 2008).
Strengths, limitations, and next steps
This is the first study to consider double-adjustments to BRFSS alcohol measures for improved predictive and outcome validity. Future studies should consider using the double-adjusted BRFSS measures as a standard practice prior to analysis to ensure the quality and utility of drinking survey data in any analyses where variation over time is considered. However, it is important to note that the state-level panel analysis could be influenced considerably due to changes in BRFSS methodology over time, as aggregate measures of drinking and demographics at the state level are main predictors in the models. Furthermore, the same ratio of PCC to BRFSS volume was applied to all subgroups, and it is likely that demographic subgroups differ in under-coverage; ongoing collaborations with the CDC are examining this issue. We also chose to examine mortality due to unintentional injuries here because deaths due to acute causes should be more responsive to immediate changes in population consumption than deaths due to 100% alcohol-attributable (e.g., alcoholic liver disease) or other chronic diseases (e.g., cirrhosis) with longer latent periods. We might expect stronger associations between double-adjusted survey measures and 100% alcohol-attributable or chronic mortality given Swedish results showing significant associations between per capita consumption and alcohol-related deaths of various kinds (e.g., drink driving, cirrhosis (Norström and Ramstedt, 2018)). We plan to examine 100% alcohol-attributable and chronic disease mortality in forthcoming analyses.
Supplementary Material
Acknowledgments
Declaration of Interest: This work was funded by NIAAA P50AA005595. Drs. Subbaraman and Kerr have received support for contracts and/or travel from the National Alcoholic Beverage Control Association.
REFERENCES
- Ahluwalia IB, Mack KA, Murphy W, Mokdad AH, Bales VS (2003) State-specific prevalence of selected chronic disease-releated characteristics—behavioral risk factor surveillance system, 2001. MMWR Morb Mortal Wkly Rep 52(SS08):1–80. [PubMed] [Google Scholar]
- Boniface S, Kneale J, Shelton N (2014) Drinking pattern is more strongly associated with under-reporting of alcohol consumption than socio-demographic factors: evidence from a mixed-methods study. BMC Public Health 14:1297 PMCID: PMC4320509\ [DOI] [PMC free article] [PubMed] [Google Scholar]
- Boniface S, Scholes S, Shelton N, Connor J (2017) Assessment of non-response bias in estimates of alcohol consumption: applying the continuum of resistance model in a general population survey in England. PLoS One 12(1):e0170892. PMCID: PMC5283659 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Centers for Disease Control and Prevention (CDC) (2015) Behavioral Risk Factor Surveillance System. 2015 Summary Data Quality Report [Accessed: 2017–05-01 Archived by WebCite® at http://www.webcitation.org/6q8yejPMs ], Atlanta, GA.
- Dwyer-Lindgren L, Flaxman AD, Ng M, Hansen GM, Murray CJL, Mokdad AH (2015) Drinking patterns in US counties from 2002 to 2012. American Journal of Public Health (AJPH) 105(6):1120–1127. PMCID: PMC4431075 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fahimi M, Link M, Mokdad A, Schwartz DA, Levy P (2008) Tracking chronic disease and risk behavior prevalence as survey participation declines: statistics from the Behavioral Risk Factor Surveillance System and other national surveys. Prev Chronic Dis 5(3):A80 PMCID: PMC2483564 [PMC free article] [PubMed] [Google Scholar]
- Greenfield TK, Kerr WC (2008) Alcohol measurement methodology in epidemiology: recent advances and opportunities. Addiction 103(7):1082–1099. PMCID: PMC2782942 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kerr WC (2010) Commentary on Nelson et al. (2010): The many sources of survey under-coverage. Addiction 105(9):1597–1598. [DOI] [PubMed] [Google Scholar]
- Kerr WC, Greenfield TK (2007) Methods of drink ethanol assessment for use in monitoring surveys of alcohol consumption, in International Research Symposium: Monitoring Alcohol and Other Drug Related Harm: Building systems to support better policy. May 7–10, Victoria, BC. [Google Scholar]
- Kerr WC, Greenfield TK, Tujague J, Brown SE (2006) The alcohol content of wine consumed in the US and per capita consumption: new estimates reveal different trends. Alcohol Clin Exp Res 30(3):516–522. [DOI] [PubMed] [Google Scholar]
- Kerr WC, Patterson D, Greenfield TK (2014) Spirits and Wine Tax Rates for the Control States: 2012 estimates based on retail price impact relative to license state pricing. National Alcohol Beverage Control Association (NABCA), Alexandria, VA. [Google Scholar]
- Klein JD, Thomas RK, Sutter EJ (2007) Self-reported smoking in online surveys: prevalence estimate validity and item format effects. Med Care 45(7):691–695. [DOI] [PubMed] [Google Scholar]
- Liaw Y-P, Huang Y-C, Lien G-W (2005) Patterns of lung cancer mortality in 23 countries: application of the Age-Period-Cohort model. BMC Public Health 5:22 PMCID: PMC555565 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Martinez P, Kerr WC, Subbaraman M, Roberts SCM (2019) New estimates of the mean ethanol content of beer, wine, and spirits sold in the United States show a greater increase in per capita alcohol consumption than previous estimates. Alcohol Clin Exp Res 43(3):509–521. PMCID: PMC6397099 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Meier PS, Meng Y, Holmes J, Baumgerg B, Purshouse R, Hill-McManus D, Brennan A (2013) Adjusting for unrecorded consumption in survey and per capita sales data: quantification of impact on gender- and age-specific alcohol-attributable fractions for oral and pharyngeal cancers in Great Britain. Alcohol Alcohol 48(2):241–249. [DOI] [PubMed] [Google Scholar]
- Miller JW, Gfroerer JC, Brewer RD, Naimi TS, Mokdad A, Giles WH (2004) Prevalence of adult binge drinking: a comparison of two national drinking surveys. Am J Prev Med 27(3):197–204. [DOI] [PubMed] [Google Scholar]
- Naimi TS, Blanchette J, Nelson TF, Nguyen T, Oussayef N, Heeren TC, Gruenewald P, Mosher J, Xuan Z (2014) A new scale of the U.S. alcohol policy environment and its relationship to binge drinking. Am J Prev Med 46(1):10–16. PMCID: PMC3878154 [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Center for Health Statistics (2004) Compressed Mortality File 1999–2002. National Center for Health Statistics, Hyattsville, MD. [Google Scholar]
- National Institute on Alcohol Abuse and Alcoholism Alcohol Policy Information System (APIS) [https://alcoholpolicy.niaaa.nih.gov], Bethesda, MD. [PMC free article] [PubMed]
- Nelson DE, Naimi TS, Brewer RD, Roeber J (2010) US state alcohol sales compared to survey data, 1993–2006. Addiction 105(9):1589–1596. [DOI] [PubMed] [Google Scholar]
- Norström T, Ramstedt M (2018) The link between per capita alcohol consumption and alcohol-related harm in Sweden, 1987–2015. J Stu Alcohol Drugs 79(4):578–584. [PubMed] [Google Scholar]
- Pierannunzi C, Hu SS, Balluz L (2013) A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004–2011. BMC Med Res Methodol 13:49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pierannunzi C, Town M, Garvin W, Shaw F,E, Balluz L (2012) Methodological changes in the behavioral risk factor surveillance system in 2011 and potential effects on prevalence estimates. MMWR Morb Mortal Wkly Rep 61(22):410–413. [PubMed] [Google Scholar]
- Ponicki WR (2004) Statewide availability data system II:1933–2003. Pacific Institute for Research and Evaluation, Prevention Research Center, Berkeley, CA. [Google Scholar]
- Rehm J, Gmel GE Sr., Gmel G, Hasan OSM, Imtiaz S, Popova S, Probst C, Roerecke M, Room R, Samokhvalov AV, Shield KD, Shuper PA(2017) The relationship between different dimensions of alcohol use and the burden of disease—an update. Addiction 112(6):968–1001. PMCID: PMC5434904 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rehm J, Kehoe T, Gmel G, Stinson F, Grant B, Gmel G (2010) Statistical modeling of volume of alcohol exposure for epidemiological studies of population health: the US example. Popul Health Metr 8:3 PMCID: PMC2841092 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shield KD, Rehm J (2012) Difficulties with telephone-based surveys on alcohol consumption in high-income countries: the Canadian example. Int J Methods Psychiatr Res 21(1):17–28. PMCID: PMC3561771 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sordo L, Barrio G, Bravo M, Villalbí JR, Espelt A, Neira M, Regidor E (2016) Estimating average alcohol consumption in the population using multiple sources: the case of Spain. Popul Health Metr 14:21 PMCID: PMC4890273 [DOI] [PMC free article] [PubMed] [Google Scholar]
- StataCorp. (2017) Stata Statistical Software: Release 15. StataCorp LLC, College Station, TX. [Google Scholar]
- Stockwell T, Zhao J, Greenfield TK, Li J, Livingston M, Meng Y (2016) Estimating under- and over-reporting of drinking in national surveys of alcohol consumption: identification of consistent biases across four English-speaking countries. Addiction 111(7):1203–1213. PMCID: PMC4899158 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stockwell T, Zhao J, Sherk A, Rehm J, Shield K, Naimi T (2018) Underestimation of alcohol consumption in cohort studies and implications for alcohol’s contribution to the global burden of disease. Addiction 113(12):2245–2249. [DOI] [PubMed] [Google Scholar]
- Subbaraman MS, Mulia N, Kerr WC, Patterson D, Karriker-Jaffe KJ, Greenfield TK (in press) Relationships between US state alcohol policies and alcohol outcomes: differences by gender and race/ethnicity. Addiction. [DOI] [PMC free article] [PubMed] [Google Scholar]
- World Health Organization (2014) Global Status Report on Alcohol and Health 2014 [Accessed: 2016–05-31 Archived by WebCite® at http://www.webcitation.org/6hvWyyErO], pp 376, Geneva, Switzerland.
- Xuan Z, Blanchette J, Nelson TF, Heeren T, Oussayef N, Naimi TS (2015) The alcohol policy environment and policy subgroups as predictors of binge drinking measures among US adults. Am J Public Health 105(4):816–822. PMCID: PMC4358175 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zemore SE, Karriker-Jaffe KJ, Mulia N, Kerr WC, Ehlers CL, Cook WK, Martinez P, Lui C, Greenfield TK (2018) The future of research on alcohol-related disparities across U.S. racial/ethnic groups: a plan of attack. J Stu Alcohol Drugs 79(1):7–21. PMCID: PMC5894859 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.