Skip to main content
Journal of Studies on Alcohol and Drugs logoLink to Journal of Studies on Alcohol and Drugs
. 2014 Jul;75(4):695–703. doi: 10.15288/jsad.2014.75.695

Nonresponse Bias in Survey Estimates of Alcohol Consumption and Its Association With Harm

Deborah A Dawson a,b,*, Risë B Goldstein b, Roger P Pickering b, Bridget F Grant b
PMCID: PMC4108608  PMID: 24988268

Abstract

Objective:

Selective nonresponse represents a major source of potential bias in survey-based estimates of alcohol consumption and its association with harm. This study examined whether consumption differs for respondents and nonrespondents after correcting for their sociodemographic differences.

Method:

This study compared baseline consumption among initial respondents who did (n = 34,653) and did not (n = 5,306) respond to a 3-year follow-up interview in a prospective study of the U.S. general population. Differences in consumption measures were presented before and after adjustment or sociodemographic differences, and interactions of nonresponse with consumption were assessed in models predicting various types of harm.

Results:

After we adjusted for sociodemographic differences and factored in the overall level of nonresponse (13.3%), the degree to which the prevalence of drinking was underestimated in the total population was only 1.6%, and the extent to which consumption was overestimated among drinkers lay in the range of 1.7% to 2.4%. There was no consistent evidence that nonresponse moderated the association between consumption and alcohol-related harm. Sociodemographic differentials in nonresponse generally matched those reported for cross-sectional studies in the literature.

Conclusions:

The extent of nonresponse bias in survey estimates of alcohol consumption should not affect drinking guidelines and planning for prevention and treatment programs. The findings of this study are supportive of study designs that have been used to assess nonresponse bias, including the use of registry data on alcohol-related harms and secondary nonresponse data from prospective studies.


The extent to which alcohol consumption among survey respondents is representative of the general population has important implications for the development of drinking guidelines, prevention programs, and treatment services. Selective nonresponse represents a major source of potential bias in survey-based estimates of alcohol consumption and its association with alcohol-related harm, and there is growing awareness that high response rates do not necessarily ensure representative survey data (Davern, 2013; Johnson and Wislar, 2012). There is evidence that non-response varies by sociodemographic characteristics known to be associated with consumption. Although age differentials have been inconsistent, cross-sectional health studies commonly have found elevated levels of nonresponse among men (Kypri et al., 2004; Meiklejohn et al., 2012; Søgaard et al., 2004), racial/ethnic minorities (Klein et al., 2011; Kypri et al., 2004; Meiklejohn et al., 2012; Søgaard et al., 2004), urban residents (Mäkelä, 2003; Søgaard et al., 2004), the unmarried (Korkeila et al., 2001; Lemmens et al., 1988; Zhao et al., 2009), and individuals of low socioeconomic status (Korkeila et al., 2001; Lorant et al., 2007; Meiklejohn et al., 2012; Søgaard et al., 2004; Zhao et al., 2009). Correlates of secondary nonresponse in multiwave studies generally have been similar to those in cross-sectional studies (Bijl et al., 1998; Caetano et al, 2003; de Graaf et al., 2013; Lamers et al., 2012; Wild et al., 2001), although these correlates have sometimes varied according to the type of attrition (e.g., refusal versus inability to contact; de Graaf et al., 2013; Eaton et al., 1992). Demographic sources of bias in survey-based estimates of consumption can be eliminated through the use of inverse probability weights and poststratification techniques that match survey respondents to the sociodemographic profile of the target population. However, questions remain as to whether alcohol consumption differs between respondents and nonrespondents even after correcting for sociodemographic differences.

Attempts to assess the association of nonresponse with alcohol consumption have used diverse study designs, including (a) targeted follow-up of initial nonrespondents (Hill et al., 1997; Lahaut et al., 2002; Lemmens et al., 1988), (b) comparison of baseline consumption among secondary respondents and nonrespondents in multiwave studies (Gmel, 2000; Wild et al., 2001), (c) comparison of early versus late response (Korkeila et al., 2001; Kypri et al., 2004; Lahaut et al., 2003; Meiklejohn et al., 2012; Zhao et al., 2009), and (d) comparison of survey respondents and nonrespondents in terms of registry-based data on harms thought to act as proxies for alcohol consumption because of the high correlation between the two (Ohlson and Ydreborg, 1985; Romelsjö, 1989). Although registry data are not available in the United States, some Nordic and other European countries have population-based registries that record health and social data at the individual level, using identification numbers that can be linked with survey respondents and nonrespondents.

The results of studies using these approaches have been inconsistent. Whereas some have found no association between consumption and nonresponse after adjusting for sociodemographic differences (Gmel et al., 2000; Kypri et al., 2004), others have reported significant associations of modest (Lahaut et al., 2002; Lemmens et al., 1988; Meiklejohn et al., 2012) or, less commonly, substantial (Wild et al., 2001) magnitude. Nonresponse has been linked with both an increase (Zhao et al., 2004) and a decrease (Lahaut et al., 2002; Lemmens et al., 1988) in the prevalence of drinking versus abstaining. The most consistently reported finding is that nonresponse is positively associated with occasional heavy drinking (Lemmens et al., 1988; Meiklejohn et al., 2012). There is no consensus as to the association of nonresponse with frequent heavy drinking and volume of consumption.

Limitations to existing studies based on targeted follow-up of nonrespondents include low response rates and different modes of survey administration at follow-up, both of which may confound consumption differences relative to initial respondents (Gmel, 2000; Link and Mokdad, 2005). Studies based on secondary nonresponse in multiwave studies or early versus late response in cross-sectional surveys are limited by the unknown extent to which these are adequate proxies for initial nonresponse. Attempts to infer nonresponse effects on consumption by means of external data on alcohol-related harm (which can be linked at the individual level to both respondents and nonrespondents) are limited by the possibility that associations between consumption and harm may differ by response status (Gray et al., 2013). We are unaware of any studies that have directly tested whether nonresponse moderates the association between consumption and harm, which would invalidate the use of linked external data for the estimation of nonresponse effects on consumption estimates. Irrespective of design, many studies have been based on small samples that lack the statistical power to detect modest nonresponse effects.

To address as many of these limitations and questions as possible, this study compared baseline consumption among initial respondents who did and did not respond to a 3-year follow-up interview in a prospective study of the U.S. general population. Because the baseline data for secondary respondents and nonrespondents were collected at the same time and using the same interview mode (personal interviews), many potential sources of confounding were eliminated. The large baseline sample (n = 43,093) increased the likelihood of having sufficient statistical power to detect modest consumption differences by secondary response status. Extensive baseline consumption data permitted assessment of nonresponse bias in the prevalence of drinking, prevalence of monthly or greater and weekly or greater heavy episodic drinking (HED), overall frequency of HED, largest quantity of drinks consumed in a single day, and average daily volume of ethanol consumption. In addition, available survey data permitted testing for interactions between nonresponse and consumption in predicting measures of harm that spanned diverse problem domains. All consumption and harm data were ascertained for the year preceding the baseline interview, ensuring comparability of their time reference periods.

The specific objectives of the study were threefold. The first was to compare sociodemographic differentials in secondary nonresponse with those reported for initial non-response in the literature, with an eye toward assessing the validity of using the former as a proxy for the latter in the estimation of nonresponse bias. The second was to estimate the extent of nonresponse bias in baseline consumption measures when using secondary nonresponse as a proxy for initial nonresponse. The third was to test whether secondary nonresponse moderated associations between consumption and harm, to address the validity of using external linked measures of harm as proxies for consumption among nonrespondents in the estimation of nonresponse bias.

Method

Sample

This study used data from the 2001–2002 Wave 1 National Epidemiologic Survey on Alcohol and Related Conditions (NESARC; Grant et al., 2003). The nationally representative baseline sample comprised 43,093 U.S. adults 18 years of age and older living in households and noninstitutional group quarters (response rate = 81.0%). Informed consent was obtained after potential respondents were informed in writing about the survey content, uses of the data, voluntary nature of participation, and confidentiality of identifiable survey information. The research protocol received full ethical review and approval. Approximately 3 years after the baseline interviews, reinterviews were attempted with all eligible respondents (i.e., excluding those who had died, had become institutionalized or were too ill to participate, or had left the United States or were on active military duty throughout the reinterview period). This analysis is based on baseline respondents eligible for reinterview (n = 39,959).

Measures

Secondary response status distinguished respondents (n = 34,653) and nonrespondents (n = 5,306) to the 3-year follow-up interview. Nonrespondents comprised refusals and cases that could not be located at follow-up. Although the NESARC data did not permit disaggregation of these two sources of attrition, prior longitudinal studies based on similar follow-up intervals have reported ratios of refusals to cases that could not be located that lie in the range of 6:1 (Lamers et al., 2012) to 9:1 (de Graaf et al., 2013). As the follow-up interval increases, the two groups approach parity (Eaton et al., 1992).

Sociodemographic characteristics included age, sex, race/ethnicity, marital status, educational attainment, family income, and urban/rural residence. All were measured as of the baseline interview.

Baseline consumption measures, reflecting the 12 months preceding the baseline interview, comprised (a) any past-year drinking (consuming one or more alcoholic drinks); (b) frequency of HED (five or more drinks in a day for men and four or more drinks in a day for women); (c) prevalence of monthly or greater HED; (d) prevalence of weekly or greater HED; (e) largest quantity of drinks consumed in a single day (based on an open-ended question for all alcoholic beverage types combined); and (f) average daily volume of ethanol consumption (based on overall frequency of drinking, usual and largest quantities of drinks, frequencies of drinking the largest quantity and five or more drinks, usual drink size, and ethanol content by volume, the latter based on the usual brand consumed). Details on the derivation of volume have been described elsewhere (Dawson, 2003). For this analysis, we used the larger of the sum of the beverage-specific volumes for coolers, beer, wine, and distilled spirits or the volume for all alcoholic beverages combined, which was assessed in a separate series of questions. To avoid undue influence of extreme outliers, average daily volumes of intake greater than 7.2 oz. were top-coded to 7.2, and the largest quantities of drinks greater than 24 were top-coded to 24. This affected only the top percentile of cases for each of these variables.

All measures of alcohol-related harm reflected the 12 months preceding the baseline interview. A diagnosis of alcohol dependence required endorsement of at least one symptom item for three or more of the seven criteria for alcohol dependence (based on criteria from the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition; American Psychiatric Association, 1994; Grant et al., 2004). Consistent with its conceptualization as a syndrome, two or more positive symptoms were required to satisfy the withdrawal criterion. Impaired driving was defined as more than once driving a motor vehicle after having had too much to drink, and alcohol-related fighting as getting into physical fights while or right after drinking. Fair or poor versus excellent, very good, or good health was based on a question on self-perceived general health. Injuries were defined to comprise any that required medical care or cutting down on usual activities for more than half a day. Getting fired or laid off was based on an item embedded in a series of questions assessing past-year stressors, and hypertension required respondent confirmation that the condition had been diagnosed by a health professional.

Analysis

First, we assessed variation in unweighted secondary nonresponse rates across baseline sociodemographic characteristics by means of chi-square tests. We then compared consumption measures for respondents and nonrespondents within categories of those characteristics, using t tests for differences of means and proportions. To preclude type I error resulting from the multiple comparisons performed for each consumption measure, we established an alpha level of .0025 for citing differences as statistically significant in the text. In addition, marginally significant differences (.0025 < p < .05) were noted in the tables. Next, we used a dichotomous measure of secondary nonresponse to predict each consumption measure in multiple regression models controlling for sociodemographic characteristics. Continuous consumption measures were logged to better satisfy model assumptions regarding their distribution and to yield multiplicative models comparable to the multiple logistic regression models used for dichotomous consumption outcomes. Log transformations for zero values were set to the log of half the lowest reported nonzero value. The beta parameters for nonresponse from the models predicting logged continuous consumption measures can be exponentiated to obtain the adjusted ratio of the consumption variable among non-respondents relative to respondents.

Finally, to address whether nonresponse moderated associations between consumption and harm, we used multivariate logistic regression models to predict each of the seven dichotomous harm measures on the basis of consumption, nonresponse, and their interaction, controlling for sociodemographic characteristics. To preclude type I error resulting from the multiple consumption measures tested for each harm outcome, an alpha level of .01 was established for statistical significance of the interactions between consumption and nonresponse, with marginally significant differences (.01 < p < .05) additionally noted in the tables. As a sensitivity analysis, we also explored the impact of controlling for any significant interactions of the sociodemographic characteristics with consumption.

The chi-square analyses of the unweighted secondary response rates were performed using SAS version 9.2 (SAS Institute, Inc., 2008); all other analyses of the weighted Wave 1 data used SUDAAN (Research Triangle Institute, 2008), which employs Taylor series linearization to adjust variance estimates for the characteristics of complex, multistage sample designs.

Results

Among baseline respondents eligible for reinterview, the secondary nonresponse rate was 13.3% and varied by sociodemographic characteristics—decreasing with age, education, and income and higher for men, most minorities, the divorced/separated and never married, and urban residents (Table 1). The proportion of past-year drinkers was significantly higher for respondents than nonrespondents in the total population and among individuals who were 25-44 years of age, married or cohabiting, had attended/completed college, and lived in urban areas (Table 2). In the total population of past-year drinkers, the remaining consumption measures were consistently higher for nonrespondents than respondents. Differences were greatest for monthly or greater HED, weekly or greater HED, and frequency of HED, with unadjusted ratios of 1.30, 1.33, and 1.37, respectively. Differences significant among all drinkers generally remained significant among drinkers who were male, were White, attended/completed college, and lived in urban areas. Marginally significant differences were observed within many additional subgroups.

Table 1.

Nonresponse rates at 3-year follow-up, by baseline sociodemographic characteristics: U.S. adults 18 years and older

Variable Non-response rate χ2 df p
Total 13.3
Age group, in years 332.5 3 <.0001
 18–24 18.8
 25–44 15.1
 45–64 11.0
 ≥65 9.0
Sex 32.9 1 <.0001
 Male 14.4
 Female 12.4
Race/ethnicity
 White 11.3 272.0 4 <.0001
 Black 13.7
 Native American 10.2
 Asian/Pacific Islander 19.9
 Hispanic 17.9
Marital status 218.1 3 <.0001
 Married/cohabiting 11.9
 Widowed 9.3
 Divorced/separated 14.2
 Never married 17.3
Education 83.6 2 <.0001
 Less than high school graduate 16.2
 High school graduate 13.9
 At least some college 12.0
Family income 102.9 2 <.0001
 <$35,000 15.0
 $35,000–$69,999 11.8
 $70,000 or more 11.3
Urban/rural residence 162.8 1 <.0001
 Urban 14.3
 Rural 8.7

Table 2.

Selected baseline alcohol consumption measures, by baseline sociodemographic characteristics and response status at 3-year follow-up: U.S. adults 18 years and older

Past-year consumption among drinkers
Past-year drinker (%)
Engaged in HEDa ≥monthly (%)
Engaged in HEDa ≥weekly (%)
Frequency of HEDa (days/year)
Largest no. drinks consumed in a day
Average daily volume of ethanol intake (oz.)
Population subgroup Resp. % (SE) Nonresp. % (SE) Resp. % (SE) Nonresp. % (SE) Resp. % (SE) Nonresp. % (SE) Resp. M (SE) Nonresp. M (SE) Resp. M (SE) Nonresp. M (SE) Resp. M (SE) Nonresp. M (SE)
Total 67.0 (0.6) 64.2 (0.9)* 20.7 (0.4) 26.9 (1.2)* 12.8 (0.3) 17.0 (0.8)* 21.5 (0.6) 29.4 (1.8)* 4.3 (0.1) 4.8 (0.1)* 0.565 (0.011) 0.694 (0.034)*
Age group, in years
 18–24 71.5 (1.2) 68.7 (1.8) 39.1 (1.2) 43.0 (2.7) 24.0 (1.0) 26.4 (2.1) 37.0 (1.7) 45.4 (4.5) 6.7 (0.2) 6.4 (0.3) 0.820 (0.036) 0.958 (0.085)
 25–14 73.8 (0.7) 69.1 (1.4)* 23.1 (0.6) 27.1 (1.5)§ 14.1 (0.5) 17.2 (1.2) § 23.0 (0.9) 30.3 (2.5)§ 4.8 (0.1) 5.2 (0.2)§ 0.540 (0.016) 0.682 (0.047)§
 45–64 65.3 (0.8) 59.9 (1.7)§ 14.8 (0.6) 19.5 (1.9)§ 9.4 (0.5) 12.9 (1.4)§ 17.6 (1.0) 20.6 (2.6) 3.5 (0.1) 3.7 (0.1) 0.538 (0.017) 0.586 (0.048)
 ≥65 48.6 (1.0) 44.2 (2.4) 6.0 (0.5) 6.5 (1.8) 3.7 (0.4) 3.8 (1.3) 8.1 (1.1) 8.5 (2.8) 2.2 (0.0) 2.2 (0.1) 0.435 (0.018) 0.426 (0.052)
Sex
 Male 73.3 (0.7) 70.4 (1.2)§ 26.7 (0.6) 34.4 (1.8)* 17.5 (0.5) 23.0 (1.4)* 29.7 (1.0) 40.1 (3.2)* 5.5 (0.1) 6.1 (0.2)* 0.797 (0.018) 0.957 (0.057)§
 Female 61.4 (0.8) 58.0 (1.1)§ 14.3 (0.5) 17.8 (1.3)§ 7.8 (0.3) 9.7 (1.1) 12.9 (0.6) 16.4 (2.0) 3.1 (0.0) 3.3 (0.1)§ 0.318 (0.009) 0.377 (0.032)
Race/ethnicity
 White 71.1 (0.7) 69.1 (1.2) 20.6 (0.5) 27.5 (1.4)* 12.5 (0.4) 18.1 (1.1)* 21.3 (0.7) 31.8 (2.4)* 4.4 (0.1) 5.0 (0.1)* 0.564 (0.013) 0.734 (0.043)*
 Black 53.6 (1.0) 57.0 (1.9) 18.3 (0.9) 21.3 (2.3)§ 13.1 (0.8) 16.2 (2.1) 22.5 (1.7) 29.5 (3.9) 3.6 (0.1) 3.8 (0.2) 0.627 (0.034) 0.778 (0.075)
 Native American 58.8 (2.9) 53.8 (7.9) 25.5 (2.8) 44.5 (9.0)§ 18.3 (0.3) 26.1 (8.0) 32.3 (5.6) 39.0 (10.6) 5.2 (0.4) 5.5 (0.9) 0.775 (0.104) 0.720 (0.222)
 Asian/Pacific Islander 49.3 (2.2) 48.2 (4.1) 14.6 (1.9) 16.8 (3.7) 8.7 (1.4) 8.2 (2.3) 14.3 (2.6) 11.8 (3.2) 3.4 (0.2) 3.6 (0.4) 0.396 (0.053) 0.392 (0.093)
 Hispanic 60.3 (1.0) 59.4 (1.7) 24.6 (1.2) 29.7 (2.6)§ 14.9 (0.9) 15.0 (1.5) 22.7 (1.5) 24.1 (3.0) 4.8 (0.1) 5.2 (0.3) 0.523 (0.029) 0.564 (0.063)
Marital status
 Married/cohabiting 67.6 (0.7) 62.9 (1.3)* 15.9 (0.4) 21.7 (1.5)* 9.4 (0.4) 12.2 (1.0)§ 16.3 (0.7) 20.4 (1.9)§ 3.8 (0.0) 4.3 (0.2)§ 0.476 (0.011) 0.531 (0.038)
 Widowed 42.4 (1.4) 37.1 (3.3) 8.9 (1.1) 6.6 (2.8) 5.2 (0.8) 4.2 (2.5) 10.9 (2.1) 7.8 (4.3) 2.4 (0.1) 2.4 (0.3) 0.414 (0.043) 0.359 (0.096)
 Divorced/separated 69.2 (0.8) 67.2 (2.0) 24.9 (1.0) 29.2 (2.3) 16.2 (0.8) 21.8 (2.1) § 30.6 (1.9) 39.1 (4.6) 4.7 (0.1) 5.0 (0.2) 0.714 (0.034) 0.873 (0.082)
 Never married 71.0 (0.8) 69.3 (1.4) 35.1 (1.0) 36.8 (2.0) 22.5 (0.9) 24.6 (1.6) 34.5 (1.4) 43.1 (3.7)§ 6.1 (0.1) 5.9 (0.2) 0.783 (0.028) 0.939 (0.065)§
Education
 <HS graduate 47.5 (1.0) 50.3 (2.0) 27.3 (1.3) 32.3 (3.0) 17.8 (1.1) 21.8 (2.6) 34.0 (2.6) 38.4 (4.7) 5.1 (0.2) 5.7 (0.3) 0.749 (0.047) 0.815 (0.088)
 HS graduate 61.5 (0.8) 63.4 (1.5) 23.6 (0.8) 29.6 (2.2)§ 15.1 (0.6) 19.5 (1.7)§ 25.7 (1.1) 33.1 (3.4)§ 4.6 (0.1) 5.0 (0.2) 0.606 (0.021) 0.696 (0.059)
 ≥Some college 74.6 (0.6) 69.9 (1.1)* 18.5 (0.5) 23.9 (1.2)* 11.0 (0.4) 14.3 (1.0)* 17.8 (0.6) 24.8 (2.0)* 4.1 (0.1) 4.5 (0.1)§ 0.519 (0.011) 0.660 (0.043)*
Family income
 <$35,000 57.8 (0.6) 57.8 (1.2) 25.4 (0.7) 32.2 (1.6)* 16.9 (0.6) 21.8 (1.2)* 28.4 (1.0) 39.8 (2.8)* 4.8 (0.1) 5.4 (0.2)* 0.658 (0.019) 0.853 (0.051)*
 $35,000–69,999 68.9 (0.7) 67.0 (1.5) 20.2 (0.6) 22.7 (1.6) 12.1 (0.5) 13.0 (1.3) 20.7 (1.0) 22.2 (2.7) 4.2 (0.1) 4.4 (0.2) 0.534 (0.018) 0.544 (0.052)
 ≥$70,000 79.2 (0.8) 75.1 (1.9)§ 15.8 (0.6) 22.5 (2.0)* 8.8 (0.5) 13.2 (1.4)§ 14.4 (0.8) 19.5 (2.4)§ 3.9 (0.1) 4.5 (0.2)§ 0.491 (0.016) 0.595 (0.051)
Urban/rural residence
 Urban 68.9 (0.7) 65.0 (1.0)* 19.9 (0.5) 26.6 (1.3)* 12.2 (0.4) 16.6 (0.9)* 20.3 (0.7) 28.6 (1.9)* 4.2 (0.1) 4.8 (0.1)* 0.547 (0.012) 0.693 (0.037)*
 Rural 59.8 (1.0) 58.5 (2.4) 24.3 (0.8) 15.2 (0.6) 28.8 (3.1) 19.9 (2.3)§ 27.0 (1.3) 35.5 (4.5) 4.9 (0.1) 5.1 (0.3) 0.642 (0.025) 0.702 (0.055)

Notes: HED = heavy episodic drinking; resp. = respondent; nonresp. = nonrespondents; HS = high school.

a

Heavy episodic drinking (i.e., consuming five or more drinks in a day for men or four or more drinks in a day for women).

*

Difference between respondents and nonrespondents p < .0025;

§

difference between respondents and nonrespondents .0025 < p < .05.

After we adjusted for sociodemographic factors (Table 3), nonrespondents were 12% less likely than respondents to be past-year drinkers (OR = 0.88). Among drinkers, the largest quantity of drinks consumed in a day did not differ by response status, but the odds of monthly or greater HED were 18% higher, the odds of weekly or greater HED were 16% higher, and the frequency of HED and average daily ethanol consumption were each 13% higher for nonrespondents than respondents. At the total population level, nonresponse associations with the prevalence of drinking and consumption among drinkers largely cancelled each other out, and monthly or greater HED, frequency of HED, and average daily ethanol consumption did not differ for respondents and nonrespondents. However, the maximum quantity of drinks was 4% lower among nonrespondents than respondents (p = .037).

Table 3.

Results of multivariate regression modelsa predicting selected baseline consumption measures as a function of non-response at follow-up: U.S. adults 18 years and older

Parameters for nonresponse vs. response at follow-up
Outcome consumption measure β (SE) p OR [95% CI]
Prevalence of past-year drinking in the total population -.129 (.043) .004 0.88 [0.81, 0.96]
Consumption among past-year drinkers
 % of drinkers who drank 5+/4+ drinks ≥monthly in past year .163 (.062) .011 1.18b [1.04, 1.33]
 % of drinkers who drank 5+/4+ drinks ≥weekly in past year .151 (.065) .023 1.16b [1.02, 1.32]
 Frequency of drinking 5+/4+ drinks (log) .119 (.051) .022 1.13b [1.02, 1.24]
 Maximum quantity of drinks consumed in a single day (log) .011 (.019) .575 N.S.
 Average daily ethanol consumption in ounces (log) .125 (.045) .008 1.13b [1.04, 1.24]
Consumption in the total population
 % of drinkers who drank 5+/4+ drinks ≥monthly in past year .109 (.056) .055 n.s.
 % of drinkers who drank 5+/4+ drinks ≥weekly in past year .110 (.062) .079 n.s.
 Frequency of drinking 5+/4+ drinks (log) .041 (.035) .249 n.s.
 Maximum quantity of drinks consumed in a single day (log) -.044 (.021) .037 0.96b [0.92, 0.1.00]
 Average daily ethanol consumption in ounces (log) -.105 (.065) .110 N.S.

Notes: OR = odds ratio; CI = 95% confidence interval; n.s. = not significant.

a

All model parameters adjusted for age, sex, race/ethnicity, marital status, educational attainment, family income, and urban/rural residence;

b

adjusted ratio of the consumption measure among nonrespondents relative to respondents.

In the multiplicative models used to predict the seven measures of harm (Table 4), all interactions between consumption and harm fell short of the p < .01 level of statistical significance; that is, the number of significant interactions was no greater than would be expected by chance. Three interactions were marginally significant (.01 < p < .05). Nonresponse had negative interactions with the largest number of drinks in predicting impaired driving (p = .012) and alcohol-related fighting (p = .041) and with the frequency of HED in predicting alcohol-related fighting (p = .026). When controls for interactions of consumption with sociodemographic characteristics were added to the models, there were no meaningful changes to the parameters presented in Table 4 (data not shown). The three interactions of marginal statistical significance remained marginally significant, and all nonsignificant interactions remained nonsignificant.

Table 4.

Results of multivariate regression modelsa predicting selected types of harm as a function of baseline consumption, nonresponse at follow-up, and the interaction of consumption with nonresponse: U.S. past-year drinkers 18 years and older

Engaged in HED ≥monthly
Engaged in HED ≥weekly
Frequency of HEDb (days/year)
Largest no. drinks consumed in a dayb
Average daily volume of ethanol intake (oz.)b
Type of harm β (SE) p β (SE) p β (SE) p β (SE) p β (SE) p
Alcohol dependence
 Main effect of consumption 2.530 (0.096) .000 2.416 (0.080) .000 0.638 (0.022) .000 2.139 (0.084) .000 1.067 (0.046) .000
 Main effect of nonresponse -0.296 (0.223) .189 -0.111 (0.150) .462 -0.221 (0.200) .273 0.334 (0.322) .303 -0.119 (0.121) .329
 Interaction 0.319 (0.283) .264 0.107 (0.216) .623 0.034 (0.055) .535 -0.147 (0.149) .329 -0.021 (0.087) .805
Impaired driving
 Main effect of consumption 2.319 (0.092) .000 2.128 (0.089) .000 0.569 (0.019) .000 1.988 (0.061) .000 0.860 (0.032) .000
 Main effect of nonresponse 0.013 (0.237) .955 -0.007 (0.159) .965 0.032 (0.223) .886 0.666 (0.304) .032 -0.232 (0.113) .043
 Interaction -0.265 (0.274) .336 -0.317 (0.221) .155 -0.076 (0.057) .186 -0.375 (0.145) .012§ -0.101 (0.076) .186
Alcohol-related fighting
 Main effect of consumption 2.779 (0.192) .000 2.358 (0.158) .000 0.677 (0.045) .000 2.119 (0.131) .000 0.998 (0.075) .000
 Main effect of nonresponse 0.328 (0.362) .369 0.516 (0.241) .036 0.854 (0.312) .008 1.399 (0.514) .008 0.185 (0.181) .311
 Interaction -0.273 (0.449) .545 -0.650 (0.387) .098 -0.207 (0.091) .026§ -0.504 (0.241) .041§ -0.234 (0.165) .163
Fair/poor health
 Main effect of consumption 0.150 (0.072) .040 0.276 (0.085) .002 0.041 (0.016) .011 0.016 (0.043) .708 -0.044 (0.015) .005
 Main effect of nonresponse -0.027 (0.084) .746 -0.032 (0.080) .691 -0.032 (0.078) .685 -0.040 (0.131) .759 0.030 (0.123) .806
 Interaction 0.036 (0.170) .833 0.064 (0.202) .751 0.015 (0.036) .682 0.024 (0.101) .811 0.015 (0.039) .707
Any injury
 Main effect of consumption 0.060 (0.051) .245 0.116 (0.065) .080 0.027 (0.011) .015 0.102 (0.031) .002 0.003 (0.010) .777
 Main effect of nonresponse -0.256 (0.086) .004 -0.215 (0.075) .005 -0.233 (0.078) .004 -0.282 (0.133) .038 -0.169 (0.081) .042
 Interaction 0.197 (0.154) .206 0.095 (0.158) .552 0.032 (0.032) .311 0.068 (0.081) .402 0.013 (0.032) .688
Fired/laid off from job
 Main effect of consumption 0.222 (0.074) .004 0.319 (0.081) .000 0.062 (0.015) .000 0.228 (0.042) .000 0.064 (0.015) .000
 Main effect of nonresponse -0.030 (0.102) .766 0.039 (0.093) .673 0.006 (0.094) .951 -0.125 (0.141) .378 0.112 (0.105) .291
 Interaction 0.219 (0.161) .179 0.046 (0.191) .813 0.030 (0.033) .357 0.124 (0.079) .121 0.039 (0.038) .307
Hypertension
 Main effect of consumption 0.201 (0.058) .001 0.266 (0.071) .000 0.039 (0.012) .002 0.052 (0.034) .123 -0.026 (0.011) .017
 Main effect of nonresponse -0.386 (0.092) .000 -0.340 (0.089) .000 -0.356 (0.087) .000 -0.508 (0.120) .000 -0.251 (0.132) .061
 Interaction 0.276 (0.176) .121 0.125 (0.202) .537 0.053 (0.034) .120 0.180 (0.093) .056 0.023 (0.037) .530

Notes: HED = heavy episodic drinking; no. = number.

a

All model parameters adjusted for age, sex, race/ethnicity, marital status, educational attainment, family income, and urban/rural residence;

b

logged value of consumption measure used in models.

§

Significance of interaction between consumption and nonresponse .01 < p < .05.

Discussion

After we adjusted for response-related differences in sociodemographic characteristics, which can be eliminated as a source of bias through appropriate adjustments to case weights, this study found a modest residual association of nonresponse with alcohol consumption. The odds of drinking were 12% lower among nonrespondents relative to respondents, and various measures of consumption among drinkers were 13%–18% higher among nonrespondents. Because of the compensating nature of these effects, no nonresponse effect was found for most measures of consumption at the total population level. The one exception indicated that the largest quantity of drinks consumed in a day was 4% lower among nonrespondents than among respondents. To put these findings into context concerning the overall level of bias in NESARC-based estimates of consumption, one must consider not only the adjusted differences between respondents and nonrespondents (expressed in ratio form in this analysis) but also the level of nonresponse. Given a secondary nonresponse rate of just 13.3%, the overall degree to which drinking prevalence was underestimated in the total population was only 1.6% (.12 × .133), and the extent to which consumption was overestimated among drinkers lay in the range of 1.7%–2.4%.

The degree to which this study’s results, based on secondary nonresponse in a longitudinal study, can be extrapolated to nonresponse bias in cross-sectional studies depends on the similarity of consumption differences associated with secondary and initial nonresponse. Although we are not aware of any data that can be directly brought to bear on this question, one factor supporting the validity of this study design is the similarity of sociodemographic differentials in these two types of nonresponse. The higher rates of secondary non-response associated with male sex, minority race/ethnicity, urbanicity, being unmarried, and lower levels of education and income that were found in this study are similar to those reported for initial nonresponse in numerous cross-sectional health studies, including both those that simply documented nonresponse differentials and those that used varying approaches to estimate resulting nonresponse bias (Korkeila et al., 2001; Kypri et al., 2004; Lorant et al., 2007; Mäkelä, 2003; Meiklejohn et al., 2012; Søgaard et al., 2004; Zhao et al., 2009). This by no means ensures comparable consumption differentials, as there is wide heterogeneity in consumption even within sociodemographic subgroups. However, other factors that might yield consumption differentials by response status (e.g., a desire to avoid admitting to heavy drinking or a sense that the survey is irrelevant among non-drinkers) arguably would have the same impact (in direction if not in magnitude) on initial and secondary survey participation. Moreover, the relatively short follow-up interval between the first and second wave of the NESARC probably contributed to fewer differences between initial and secondary nonrespondents than would be the case for a longer interval. Finally, a number of studies that have used different approaches to estimate nonresponse bias in survey estimates of consumption have reported similar findings in terms of the underestimation of drinkers versus abstainers (Lahaut et al., 2002; Lemmens et al, 1988) and the overestimation of occasional HED (Lemmens et al., 1988; Meiklejohn et al., 2012).

Interestingly, the association of nonresponse with weekly or greater HED in this study was of considerably smaller magnitude (adjusted OR = 1.16 vs. 1.83) than that reported by Wild et al. (2001), the one prior study with a sufficiently similar study design to make direct comparisons meaningful. The disparity in ORs likely reflects the fact that Wild et al. split nonresponse into two components, that is, the willingness to be reinterviewed (which was not associated with consumption) and actually being reinterviewed among those who were willing. Had nonresponse irrespective of willingness been examined, the results presumably would have been more similar to ours; pooling a nonsignificant association with one of a fairly large magnitude would have resulted in an association of lower magnitude. Our finding that the prevalence of drinking was lower among nonrespondents than respondents was consistent with the findings of two prior studies that used targeted follow-up of initial nonrespondents (Lahaut et al., 2002; Lemmens et al., 1988), but it contradicted Zhao et al. (2009), who found that late respondents were more likely to be past-year drinkers than early respondents. This discrepancy raises some doubt as to the argument that late responders and nonrespondents share the same biases with respect to reporting of consumption.

One of the important findings of this study was the lack of any consistent or compelling evidence that nonresponse moderated the association between consumption and harm, despite consideration of seven diverse harm measures that ranged from alcohol use disorder to social problems and medical conditions. The one consumption measure for which there were repeated suggestions of effect modification by nonresponse was the largest number of drinks consumed in a single day. For the outcomes of both impaired driving and alcohol-related fighting, this measure demonstrated negative interactions of marginal significance with nonresponse. This may signal the need for some caution in using registry-based external measures of alcohol-related harm to estimate the impact of nonresponse on this specific aspect of consumption. However, nothing in our findings invalidates the use of external data (i.e., data obtained from a source outside the survey that can be linked with survey respondents and non-respondents at the individual level) for assessing the effect of nonresponse on the more commonly ascertained measures of consumption, including volume of intake and frequency of HED. Notably, these are also the measures for which the most data exist as to their associations with different types of harm.

Although the findings of this study suggest that non-response has only a modest potential for biasing survey estimates of alcohol consumption, two additional design issues must be considered in interpreting the results. First, the NESARC-based consumption measures were based on an extensive series of questions that were corrected for internal inconsistency. Although the resulting estimates of volume and HED may differ substantially from those based on just a question or two, there is no strong reason to assume that this would differentially affect secondary respondents and nonrespondents. In addition, the NESARC harm measures were based on self-report and may differ from external measures of those same harms in sources such as medical records or population registries. Secondary nonrespondents could be overrepresentative of individuals who minimized their baseline reports of alcohol-related harm if those who felt compelled to do so were less willing to face a second round of questions. However, this would not necessarily affect the interaction between response status and consumption in predicting harm if these same nonrespondents also under-reported their baseline consumption.

In conclusion, data from a nationally representative sample of U.S. adults suggested that the impact of nonresponse on the accuracy of survey-based estimates of consumption was slight and that external measures of alcohol-related harm comprised a valid proxy for consumption in the assessment of nonresponse bias. These findings suggest that the use of survey estimates of alcohol consumption will provide an acceptable and relatively unbiased basis for evaluating drinking guidelines and prevention efforts and for assessing the need for and allocation of treatment resources. Although the aim of this article was to evaluate nonresponse bias in consumption estimates per se, this study also suggests future analyses that might be undertaken on the basis of associations between secondary nonresponse (in its own right rather than as a proxy for initial nonresponse) and consumption. These might include estimation of bias in drinking trends, the course of alcohol use disorders, and prospective health outcomes.

Footnotes

The study on which this article is based, the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC), is sponsored by the National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, U.S. Department of Health and Human Services, with supplemental support from the National Institute on Drug Abuse. This research was supported in part by the Intramural Program of the National Institutes of Health, National Institute on Alcohol Abuse and Alcoholism. The views and opinions expressed in this article are those of the authors and should not be construed to represent the views of any of the sponsoring organizations, agencies, or the U.S. government.

References

  1. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th ed. Washington, DC: Author; 1994. [Google Scholar]
  2. Bijl RV, van Zessen G, Ravelli A, de Rijk C, Langendoen Y. The Netherlands Mental Health Survey and Incidence Study (NEMESIS): Objectives and design. Social Psychiatry and Psychiatric Epidemiology. 1998;33:581–586. doi: 10.1007/s001270050097. [DOI] [PubMed] [Google Scholar]
  3. Caetano R, Ramisetty-Mikler S, McGrath C. Characteristics of non-respondents in a US national longitudinal survey on drinking and intimate partner violence. Addiction. 2003;98:791–797. doi: 10.1046/j.1360-0443.2003.00407.x. [DOI] [PubMed] [Google Scholar]
  4. Davern M. Nonresponse rates are a problematic indicator of nonresponse bias in survey research. Health Services Research. 2013;48:905–912. doi: 10.1111/1475-6773.12070. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Dawson DA. Methodological issues in measuring alcohol use. Alcohol Research & Health. 2003;27:18–29. [PMC free article] [PubMed] [Google Scholar]
  6. de Graaf R, van Dorsselaer S, Tuithof M, ten Have M. Sociodemographic and psychiatric predictors of attrition in a prospective psychiatric epidemiological study among the general population. Result of the Netherlands Mental Health Survey and Incidence Study-2. Comprehensive Psychiatry. 2013;54:1131–1139. doi: 10.1016/j.comppsych.2013.05.012. [DOI] [PubMed] [Google Scholar]
  7. Eaton WW, Anthony JC, Tepper S, Dryman A. Psychopathology and attrition in the epidemiologic catchment area surveys. American Journal of Epidemiology. 1992;135:1051–1059. doi: 10.1093/oxfordjournals.aje.a116399. [DOI] [PubMed] [Google Scholar]
  8. Gmel G. The effect of mode of data collection and of non-response on reported alcohol consumption: A split-sample study in Switzerland. Addiction. 2000;95:123–134. doi: 10.1046/j.1360-0443.2000.95112313.x. [DOI] [PubMed] [Google Scholar]
  9. Grant BF, Dawson DA, Stinson FS, Chou SP, Dufour MC, Pickering RP. The 12-month prevalence and trends in DSM-IV alcohol abuse and dependence: United States, 1991-1992 and 2001-2002. Drug and Alcohol Dependence. 2004;74:223–234. doi: 10.1016/j.drugalcdep.2004.02.004. [DOI] [PubMed] [Google Scholar]
  10. Grant BF, Kaplan K, Shepard J, Moore T. Source and accuracy statement for Wave 1 of the 2001-2002 National Epidemiologic Survey on Alcohol and Related Conditions. Bethesda, MD: National Institute on Alcohol Abuse and Alcoholism; 2003. [Google Scholar]
  11. Gray L, McCartney G, White IR, Katikireddi SV, Rutherford L, Gorman E, Leyland AH. Use of record-linkage to handle non-response and improve alcohol consumption estimates in health survey data: A study protocol. BMJ Open. 2013;3:e002647. doi: 10.1136/bmjopen-2013-002647. Retrieved from http://bmjopen.bmj.com/content/3/3/e002647.full?rss=1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hill A, Roberts J, Ewings P, Gunnell D. Non-response bias in a lifestyle survey. Journal of Public Health Medicine. 1997;19:203–207. doi: 10.1093/oxfordjournals.pubmed.a024610. [DOI] [PubMed] [Google Scholar]
  13. Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. Journal of the American Medical Association. 2012;307:1805–1806. doi: 10.1001/jama.2012.3532. [DOI] [PubMed] [Google Scholar]
  14. Klein DJ, Elliott MN, Haviland AM, Saliba D, Burkhart Q, Edwards C, Zaslavsky AM. Understanding nonresponse to the 2007 Medicare CAHPS survey. The Gerontologist. 2011;51:843–855. doi: 10.1093/geront/gnr046. [DOI] [PubMed] [Google Scholar]
  15. Korkeila K, Suominen S, Ahvenainen J, Ojanlatva A, Rautava P, Helenius H, Koskenvuo M. Non-response and related factors in a nation-wide health survey. European Journal of Epidemiology. 2001;17:991–999. doi: 10.1023/a:1020016922473. [DOI] [PubMed] [Google Scholar]
  16. Kypri K, Stephenson S, Langley J. Assessment of nonresponse bias in an internet survey of alcohol use. Alcoholism: Clinical and Experimental Research. 2004;28:630–634. doi: 10.1097/01.alc.0000121654.99277.26. [DOI] [PubMed] [Google Scholar]
  17. Lahaut VM, Jansen HA, van de Mheen D, Garretsen HF. Non-response bias in a sample survey on alcohol consumption. Alcohol and Alcoholism. 2002;37:256–260. doi: 10.1093/alcalc/37.3.256. [DOI] [PubMed] [Google Scholar]
  18. Lahaut VM, Jansen HA, van de Mheen D, Garretsen HF, Verdurmen JEE, van Dijk A. Estimating non-response bias in a survey on alcohol consumption: Comparison of response waves. Alcohol and Alcoholism. 2003;38:128–134. doi: 10.1093/alcalc/agg044. [DOI] [PubMed] [Google Scholar]
  19. Lamers F, Hoogendoorn AW, Smit JH, van Dyck R, Zitman FG, Nolen WA, Penninx BW. Sociodemographic and psychiatric determinants of attrition in the Netherlands Study of Depression and Anxiety (NESDA) Comprehensive Psychiatry. 2012;53:63–70. doi: 10.1016/j.comppsych.2011.01.011. [DOI] [PubMed] [Google Scholar]
  20. Lemmens PHHM, Tan ES, Knibbe RA. Bias due to non-response in a Dutch survey on alcohol consumption. British Journal of Addiction. 1988;83:1069–1077. doi: 10.1111/j.1360-0443.1988.tb00534.x. [DOI] [PubMed] [Google Scholar]
  21. Link MW, Mokdad AH. Effects of survey mode on self-reports of adult alcohol consumption: A comparison of mail, web and telephone approaches. Journal of Studies on Alcohol. 2005;66:239–245. doi: 10.15288/jsa.2005.66.239. [DOI] [PubMed] [Google Scholar]
  22. Lorant V, Demarest S, Miermans P-J, Van Oyen H. Survey error in measuring socio-economic risk factors of health status: A comparison of a survey and a census. International Journal of Epidemiology. 2007;36:1292–1299. doi: 10.1093/ije/dym191. [DOI] [PubMed] [Google Scholar]
  23. Mäkelä P. Impact of correcting for nonresponse by weighting on estimates of alcohol consumption. Journal of Studies on Alcohol. 2003;64:589–596. doi: 10.15288/jsa.2003.64.589. [DOI] [PubMed] [Google Scholar]
  24. Meiklejohn J, Connor J, Kypri K. The effect of low survey response rates on estimates of alcohol consumption in a general population survey. PLoS ONE. 2012;7(4):e35527. doi: 10.1371/journal.pone.0035527. Retrieved from http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0035527. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Ohlson C-G, Ydreborg B. Participants and non-participants of different categories in a health survey. A cross-sectional register study. Scandinavian Journal of Social Medicine. 1985;13:67–74. doi: 10.1177/140349488501300203. [DOI] [PubMed] [Google Scholar]
  26. Romelsjö A. The relationship between alcohol consumption and social status in Stockholm. Has the social pattern of alcohol consumption changed? International Journal of Epidemiology. 1989;18:842–851. doi: 10.1093/ije/18.4.842. [DOI] [PubMed] [Google Scholar]
  27. Research Triangle Institute. SUDAAN language manual, release 10.0. Research Triangle Park, NC: Author; 2008. [Google Scholar]
  28. SAS Institute, Inc. SAS/STAT 9.2 user’s guide. Cary, NC: Author; 2008. [Google Scholar]
  29. Søgaard AJ, Selmer R, Bjertness E, Thelle D. The Oslo Health Study: The impact of self-selection in a large, population-based survey. International Journal for Equity in Health. 2004;3:3–14. doi: 10.1186/1475-9276-3-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Wild TC, Cunningham J, Adlaf E. Nonresponse in a follow-up to a representative telephone survey of adult drinkers. Journal of Studies on Alcohol. 2001;62:257–261. doi: 10.15288/jsa.2001.62.257. [DOI] [PubMed] [Google Scholar]
  31. Zhao J, Stockwell T, Macdonald S. Non-response bias in alcohol and drug population surveys. Drug and Alcohol Review. 2009;28:648–657. doi: 10.1111/j.1465-3362.2009.00077.x. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Studies on Alcohol and Drugs are provided here courtesy of Rutgers University. Center of Alcohol Studies

RESOURCES