Reasons for Nonresponse in a Web-Based Survey of Alcohol Involvement Among First-Year College Students

Abstract

This study conducted a follow-up telephone survey of a probability sample of college students who did not respond to a Web survey to determine correlates of and reasons for nonresponse. A stratified random sample of 2 502 full-time first-year undergraduate students was invited to participate in a Web-based survey. A random sample of 221 students who did not respond to the original Web survey completed an abbreviated version of the original survey by telephone. Nonresponse did not vary by gender, but nonresponse was higher among Blacks and Hispanics compared to Whites, and Blacks compared to Asians. Nonresponders reported lower frequency of past 28 days drinking, lower levels of past-year and past 28-days heavy episodic drinking, and more time spent preparing for classes than responders. The most common reasons for nonresponse were “too busy” (45.7%), “not interested” (18.1%), and “forgot to complete survey” (18.1%). Reasons for nonresponse to Web surveys among college students are similar to reasons for nonresponse to mail and telephone surveys, and some nonresponse reasons vary as a function of alcohol involvement.

1. Introduction

What are the reasons for and correlates of nonresponse in Web-based studies of alcohol involvement among college students? With increasing Internet access, a growing number of researchers have conducted Web-based surveys on a variety of topics, including substance use and abuse (Kypri et al., 2004; McCabe et al., 2002). In populations where Internet access is high (e.g., college students), Web surveys can be more cost-effective and convenient than mail and phone surveys (Parks et al., 2006). Despite their advantages, nonresponse is a major concern in Web surveys because it is one of the factors, along with differences between responders and nonresponders, that may lead to nonresponse bias. However, “there is currently little information available on nonresponse in Web surveys” (Couper, 2000).

The current study was designed to address this gap in our knowledge by conducting a telephone follow-up survey of a random sample of nonrespondents in a larger study of Residential Learning Communities (RLCs) and alcohol misuse among college students (McCabe et al., in press). The present study (a) examined differences between responders and nonresponders in alcohol involvement and time allocation, (b) explored reasons for nonresponse, and (c) tested for associations between reasons for nonresponse and alcohol involvement in a Web survey of alcohol use and academic engagement among college students.

2. Method

A stratified random sample of 2 502 full-time first-year undergraduate students was selected to participate in a longitudinal study of college alcohol use, and a total of N=1 196 students participated at wave 1, for a response rate of 47.8%. A nonresponse follow-up survey (NRFU) was conducted to assess the extent of potential nonresponse bias and reasons for nonresponse. A random sample of 640 students who did not respond to the original Web survey and who were recorded as enrolled full-time at the beginning of the Fall 2005 semester was selected from the list of nonresponders. These nonresponders were contacted by telephone approximately one month after the Web survey was closed. Final dispositions for all cases were based on the guidelines for Final Disposition Codes for Surveys of Specifically Named Persons established by the American Association for Public Opinion Research Standard Definitions for Final Dispositions of Case Codes (AAPOR, 2004). Telephone interviews were completed with a total of n=221 initial nonrespondents for a response rate of 34.5%.

2.1 Measures

Reasons for nonresponse were assessed by asking participants “Why did you not complete the survey when you received the invitation?” Lifetime and past year alcohol use was measured using the following question: “On how many occasions (if any) have you had alcohol to drink (more than just a few sips) [in your lifetime or during the past 12 months]? Response options ranged from (1) never had alcohol to (7) 40 or more occasions. Past year heavy episodic drinking in a 2-hour period. Based on recent NIAAA guidelines (NIAAA, 2004), past year heavy episodic drinking (HED) was defined as the consumption of five or more drinks in a 2-hour period during the past year for men and four or more drinks in a 2-hour period during the past year for women. Past 28 days alcohol use was assessed with the question, “In the past four weeks (28 days) on how many days, if any, did you have at least one drink of beer, wine or liquor?” Heavy episodic drinking on one day in past 28 days was assessed by asking participants “What is the most number of drinks that you had on any one day in the past 28 days?” Heavy episodic drinking (HED) was defined as having five or more drinks for men (four or more for women) in a row in the past 28 days (Wechsler et al., 1995). Primary alcohol-related consequences were assessed by asking participants how often in the past 28 days they had (1) been hurt or injured after drinking and (2) experienced a blackout or memory loss as a result of drinking. Both variables were dichotomized into yes/no variables for purposes of analyses.

3. Results

We compared responders (n=1,196) with nonresponders who participated in the NRFU (n=221). Results showed that there was no association between gender and response status, or between RLC status and response status. However, there was a statistically significant association between race/ethnicity and response status, χ² (4) = 15.1, p < .01, and post-hoc comparisons showed that, compared to the original responders, (a) a lower percentage of NRFU participants were White (54.8% vs. 66.7%, z= -3.4, p < .01); and (b) a higher percentage of NRFU participants were Black (9.0% vs. 5.1%, z= 2.3, p < .05).

Results (not shown) also indicated that, when race/ethnicity was controlled, there were no significant differences between responders and nonresponders on prevalence of lifetime, past year, or past 28-days alcohol consumption. As seen in Table 1, there were no differences on frequency of lifetime or past-year alcohol use, or on prevalence of alcohol-related consequences. However, nonresponders reported significantly lower rates of past-year and past 28 days HED, and lower frequency of past 28-days alcohol use, even when race/ethnicity was statistically controlled. In addition, nonresponders reported spending significantly more time on preparing for classes than the responders.

Table 1. Comparisons Between Responders (N=1,196) and Non-Responders (N=221).

	Responders % or M (SD)	Non-Responders % or M (SD)	χ2 or t (df)
Frequency of Lifetime Alcohol Use	4.3 (2.2)	4.0 (2.2)	1.8 (1376)
Frequency of Past Year Alcohol Use	3.8 (2.1)	3.6 (2.1)	1.4 (1370)
Prevalence of Past Year Heavy Episodic Drinking (2 hours)	50.9%	39.4%	9.4**
Frequency of Past 28 Days Alcohol Use	4.2 (4.8)	2.8 (3.5)	4.4 (1357)**
Prevalence of Past 28 Days Heavy Episodic Drinking	46.0%	37.2%	5.6*
Prevalence of Past 28 Days Hurt or Injured After Drinking	10.9%	10.9%	0.0
Prevalence of Past 28 Days Blackout or Memory Loss as a Result of Drinking	19.2%	25.6%	2.9
Average Number of Hours Spent Weekly on Class Preparation	5.0 (1.7)	5.4 (1.8)	-3.1 (1350)**
Average Number of Hours Spent Weekly on Extracurricular Activities	2.6 (1.4)	2.7 (1.4)	-0.9 (1342)
Average Number of Hours Spent Weekly on Socializing	4.0 (1.6)	4.3 (1.7)	-2.0 (1349)a
Average Number of Hours Spent Weekly on Partying	2.3 (1.2)	2.2 (1.0)	0.7 (1345)

^aDifference was statistically nonsignificant when race/ethnicity was statistically controlled.

^*p < .05.

^**p <.01.

For all χ² tests, df=1.

Percentages for each reason for nonresponse in the NRFU sample, overall and by gender, are presented in Table 2. The most common reason for nonresponse was “too busy.” Results from multiple logistic regression analyses showed that there were no associations between any of the alcohol involvement measures and “forgot to complete survey” or “not interested.” However, past-year and past 28-days HED (adjusted ORs = 0.43 and 0.52, ps < .05, respectively) were associated with significantly lower odds of reporting “too busy” as a reason for nonresponse. Results from multiple logistic regression analyses controlling for gender and race/ethnicity showed that past-year HED was associated with higher odds of not recalling the e-mail invitation (adjusted OR=1.82, p<.05).

Table 2. Reasons for Nonresponse (N=221).

Reason	Total Sample %	Females (n=99) %	Males (n=122) %	χ2
1. Thought I already participated	1.4	2.0	0.8	0.6
2. Forgot	18.1	19.2	17.2	0.1
3. Lost e-mail and/or letter	9.0	11.1	7.4	1.0
4. Not interested	18.1	16.2	19.7	0.4
5. Too busy	45.7	42.4	48.4	0.8
6. Technical difficulties with the Website	0.9	1.0	0.8	0.02
7. Didn't think the survey was legitimate	2.7	4.0	1.6	1.2
8. Didn't think the survey really applied to me	0.5	0.0	0.8	0.8
9. Don't know	1.8	3.0	0.8	1.5
10. Confidentiality concerns	0.0	0.0	0.0	--
11. Don't have Web access	0.0	0.0	0.0	--
12. Refused	0.9	0.0	1.6	1.6

For all χ² tests, df=1.

4. Discussion

The finding of lower response rates among Blacks and Hispanics replicates previous work showing higher levels of nonresponse to mail surveys among Black and Hispanic college students (Szelenyi et al., 2005), and higher nonresponse to Web surveys among Black college students (McCabe et al., 2002). These differences may in part be attributable to lower levels of experience with the Web among entering first-year minority college students (Sax et al., 2001).

Differences between responders and nonresponders were observed for past-year HED, past 28 days HED and past 28 days frequency of consumption, but not for past-year frequency or prevalence of consumption. The lower rates of HED and past 28-days frequency of drinking observed among nonrespondents are consistent with some previous studies (Caetano, 2001). The results suggest that estimates of the prevalence of HED may be upwardly biased due to nonresponse. Further, nonresponders may be more involved in academics than responders, given that they spend more time on class preparation.

Results also indicated that reasons for nonresponse to Web surveys are similar to, but perhaps more circumscribed than reasons for nonresponse to mail and telephone surveys. Nonresponse due to noncontact did not emerge as an important reason for nonresponse in this sample. Also, nonresponse due to inability to participate in the form of technical difficulties with the Website was limited to less than 1% of the sample, and all nonrespondents reported accessing their university e-mail account subsequent to the start of the semester. Results also demonstrated that HED in the past year or past 28 days was associated with lower odds of reporting “too busy” as a reason for nonresponse. This is consistent with our earlier discussion about differences in time allocation between responders and nonresponders, and the finding that nonresponders appear to allocate more time to academics than responders.

The present study has several limitations. First, the response rate for the NRFU was relatively low, and thus may itself be subject to the same type of nonresponse error that we explored here. Also, the NRFU study was a telephone survey, and this may have influenced the results. However, studies of college students using Web, mail, paper and pencil, and telephone surveys have yielded few substantive differences by survey mode (McCabe et al., 2002; Parks et al., 2006). Further examination of mode effects is an important direction for future work.

These limitations notwithstanding, the present study makes several contributions to the literature on nonresponse. Our use of a probability sample allowed for rigorous assessment of nonresponse bias. Also, to our knowledge, this is the first study to document reasons for nonresponse to a Web survey on alcohol involvement among first-year undergraduate students, and results indicate that Web studies might achieve higher response rates by emphasizing the survey's brevity. Finally, to our knowledge, this is the first study to document associations between heavy episodic drinking and some reasons for nonresponse, and findings suggest that additional reminders might increase response rates among high-risk participants.