Reliability and validity of self-reported smoking in an anonymous online survey with young adults

Abstract

Objective

The Internet offers many potential benefits to conducting smoking and other health behavior research with young adults. Questions, however, remain regarding the psychometric properties of online self-reported smoking behaviors. The purpose of this study was to examine the reliability and validity of self-reported smoking and smoking-related cognitions obtained from an online survey.

Methods

Young adults (N = 248) age 18 to 25 who had smoked at least 1 cigarette in the past 30 days were recruited online and completed a survey of tobacco and other substance use.

Results

Measures of smoking behavior (quantity and frequency) and smoking-related expectancies demonstrated high internal consistency reliability. Measures of smoking behavior and smoking stage of change demonstrated strong concurrent criterion and divergent validity. Results for convergent validity varied by specific constructs measured. Estimates of smoking quantity, but not frequency, were comparable to those obtained from a nationally representative household interview among young adults.

Conclusions

These findings generally support the reliability and validity of online surveys of young adult smokers. Identified limitations may reflect issues specific to the measures rather than the online data collection methodology. Strategies to maximize the psychometric properties of online surveys with young adult smokers are discussed.

The Internet, increasingly used in survey research of health behaviors (Horvath, Bowen, & Williams, 2006), offers a number of benefits over face-to-face interviews including broader reach; greater inclusion of low-incidence or “hidden” populations; rapid, convenient input by respondents; and reduced bias in response to sensitive, potentially stigmatizing topics (Cantrell & Lupinacci, 2007; Rhodes, Bowie, & Hergenrather, 2003; Schonlau et al., 2004). These benefits are highly relevant for the assessment of health risk behavior in young adults, almost all of whom use the Internet (93% in a recent survey; Lenhart, Purcell, Smith, & Zickuhr, 2010). Young adults remain the age group most likely to go online and are less likely to present to traditional research settings for studies of health behavior (Bost, 2005; Davies et al., 2000).

Given that the Internet is so broadly used for survey research on health behaviors, and that anonymity presents important questions about validity of online data collection, there is a need for research demonstrating that it is a valid and reliable tool. Specifically, as with traditional paper and pencil measures, formal study is needed to evaluate whether online measures are psychometrically sound. Internet-based surveys of college students have yielded valid and reliable estimates of alcohol and other drug use (Kypri, Gallagher, & Cashell-Smith, 2004; McCabe, 2008; McCabe, Boyd, Couper, Crawford, & D’Arcy, 2002) and evaluations of attrition bias have further examined validity of alcohol use reports online (Kypri, Stephenson, & Langley, 2004; McCoy et al., 2009). However, although the Internet has been used to assess tobacco use (e.g., Hughes, 2010), the validity and reliability of young adults’ reports of smoking behaviors in Internet-delivered surveys have not been evaluated.

The purpose of the current study was to examine the psychometric properties of established measures commonly administered in face-to-face interviews to determine whether they perform similarly when administered anonymously online to a national, representative sample of young adult tobacco users. While previous work has examined prevalence estimates of smoking obtained through an online survey (Klein, Thomas, & Sutter, 2007), no work to date has examined smoking characteristics by daily and non-daily smoking status, which is particularly important for young adult smokers who may not smoke daily (Schane, Glantz, & Ling, 2009). Further, the current study adds to the literature by providing a comprehensive analysis of associations across multiple smoking constructs (e.g., stage of change, thoughts about smoking, expectancies) in young adults that goes beyond measures of quantity and frequency of smoking. As thoughts about smoking have been shown to vary by development stage (Myers, MacPherson, McCarthy, & Brown, 2003), and by daily and non-daily smoking status in young adults (Fagan et al., 2007), an examination of psychometric properties of multiple constructs was warranted.

We evaluated internal-consistency reliability of subject-reported smoking quantity and frequency on two different question formats (the Smoking Questionnaire and Timeline Followback method), with the expectation that similar items would be highly correlated. Internal consistency of responses on the Smoking Expectancy Questionnaire-Short form (S-SCQ) was examined in comparison to data from the original validation studies with adolescents and young adults (Myers et al., 2003).

We examined the construct validity of smoking behavior (quantity, dependence symptoms) and stage of change. Construct validity is the extent to which a construct can be operationalized through demonstrating relations with constructs that should be similar (convergent validity) and non-relations with constructs that should be dissimilar (divergent validity). Based on previous findings in the literature, in tests of convergent validity of smoking behavior, we hypothesized that cigarettes smoked per day would be associated with dependence symptoms and that both variables would be associated with years of smoking, making a past-year quit attempt, abstinence duration, desire to quit, efficacy for quitting, abstinence goal, and smoking-related expectancies. In tests of divergent validity, we expected that cigarettes per day and nicotine dependence would be unrelated to subjective social status and motivations to abstain from alcohol and marijuana. For tests of construct validity of stage of change, we hypothesized that smoking stage of change would be related to desire to quit, expected success with quitting, anticipated difficulty with staying quit, abstinence goal, and smoking-related expectancies. For tests of divergent validity, we hypothesized that smoking stage of change would be unrelated to respondents’ demographic characteristics.

We also evaluated concurrent criterion validity for smoking behavior and stage of change, defined as a construct’s ability to distinguish between groups that it should be able to distinguish. We hypothesized that reports of daily smoking would be more likely among less educated individuals, those with less steady employment, those with annual incomes < $20,000, men, and Caucasians and that heavy smoking would be more likely among Caucasians relative to other racial groups. We further hypothesized that stage of change would distinguish among daily and non-daily smokers and those with a past year quit attempt versus not.

Finally, we compared the smoking prevalence and dependence symptoms of the study sample to the household-based National Survey on Drug Use and Health (NSDUH; Substance Abuse and Mental Health Services Administration, 2009). Quantity and frequency of past month smoking and prevalence of nicotine dependence from our study and the NSDUH were expected to be comparable.

Methods

Participants and Recruitment

This cross-sectional survey study used three Internet-based recruitment methods described in detail previously [reference removed for blind review]. Participants were 18 to 25 years of age, English literate, and reported smoking at least 1 cigarette in the past 30 days. Internet-based advertisements invited young adults to participate in a 20-minute online survey with a chance to win a prize in a drawing (worth either $25 or $400). The campaign ran for 6 consecutive months, between 4/1/09 and 10/1/09. Advertisements contained a hyperlink that directed potential participants to the study’s IRB-approved consent form and secure online survey with data encryption for added security protection.

Participants consented and deemed eligible were asked to complete a demographic questionnaire and measures of smoking and other substance use behaviors and thoughts about use. Participants were required to answer all questions before they could continue to the next page of the survey, but could quit the survey at any time. Computer IP addresses were tracked to determine whether multiple entries were made from the same computer. Eligibility checks excluded respondents who: a) made multiple entries from the same computer within a short period of time, with the first entry not being eligible for participation (n = 8); or b) had a discrepancy in data from duplicate demographic questions (e.g., date of birth and age; n = 9).

Measures

Sociodemographics

Gender, age, race/ethnicity, years of completed education, employment status, and annual family income were assessed. Residential zip codes were assessed and categorized according to the four U.S. Census Regions: Northeast, Midwest, South, and West (U. S. Census Bureau, 2010). The MacArthur Scale of Subjective Social Status (SSS; Adler, Epel, Castellazzo, & Ickovics, 2000) presented a “social ladder” and asked individuals to place an “X” on the rung on which they feel they stand in terms of occupation, income, and social standing.

Smoking Behavior

A Smoking Questionnaire assessed participants’ years of smoking, prior quit attempts (lifetime and past year), and longest period of abstinence in a prior quit attempt (Hall et al., 2006). Participants also indicated the number of cigarettes smoked in the past 24 hours, past 7 days, and the average number of cigarettes smoked per day and number of days smoked per week. Daily smokers were participants who reported smoking 7 days a week on average. All other participants were categorized as nondaily smokers. The Fagerström Test of Nicotine Dependence (FTND; Heatherton, Kozlowski, Frecker, & Fagerström, 1991) assessed level of nicotine dependence. Timeline Followback procedures (Brown et al., 1998) assessed the number of cigarettes smoked each day in the past 30. The number of cigarettes smoked in the past 24 hours and seven days, as well as the average number of cigarettes smoked per day and days per week in the last 30 days were calculated.

Thoughts about Smoking

The Smoking Stages of Change Questionnaire assessed motivation to quit (Prochaska & DiClemente, 1983), categorizing smokers into one of three pre-action stages of change (Precontemplation: no intention to quit within the next 6 months; Contemplation: intention to quit within the next 6 months but no 24-hr quit attempt in the past year; Preparation: intention to quit within the next month and a 24-hr quit attempt in the past year). The Thoughts about Abstinence form (Hall, Havassy, & Wasserman, 1990) assessed desire to quit, abstinence self-efficacy, and perceived difficulty of quitting (scored on a scale of 1 to 10). Abstinence goals for cigarettes were categorized as no goal, total abstinence, or something in between. The 21-item Smoking Consequences Questionnaire-Short form (S-SCQ; Myers et al., 2003) measured smoking-related outcome expectancies on four factors (Negative Consequences 4-items, Positive Reinforcement 5-items, Negative Reinforcement 7-items, Appetite/Weight control 5-items) and a total score (range: 0–189). Responses are scored on a 10-point Likert scale (0=completely unlikely to 9=completely likely) to rate the likelihood of occurrence of each smoking consequence.

Thoughts about Alcohol/Marijuana Use

We used the Alcohol Stages of Change-Short Form Questionnaire to assess motivation to quit drinking alcohol (Laforge, Maddock, & Rossi, 1998), and adapted the 3-item Stage of Change – Short Form (DiClemente, Prochaska et al., 1991) to assess stage of change for use of marijuana. We adapted the Thoughts about Abstinence form to assess cognitions about marijuana.

Comparison data

The NSDUH is a yearly face-to-face interview survey conducted with a representative sample of US adolescent and adults age 12 and older in their homes (Substance Abuse and Mental Health Services Administration, 2009). The 2008 NSDUH employed a state-based design with an independent, multistage area probability sample and oversampled youths and young adults with a weighted response rate of 74.4%. Data for the present study were taken from respondents age 18 to 25 years with 23,248 completed interviews and a weighted response rate of 81.7% (Substance Abuse and Mental Health Services Administration, 2009). Items used from the NSDUH included the number of days smoked among past month smokers, the number of cigarettes smoked per day for daily smokers, and measured nicotine dependence using criteria derived from the Nicotine Dependence Syndrome Scale (NDSS) and the FTND.

Results

Sample characteristics

During the 6-month recruitment period, our online survey received more than 4606 hits; 940 people gave online consent to determine eligibility to complete the survey, of which 306 (32.6%) were eligible. Of the 306 eligible cases, 248 provided smoking data (81%), and 181 (59.2%) completed the entire survey. Demographic and smoking characteristics of daily and non-daily smokers for the individuals who provided smoking data (N = 248) are provided in Table 1. The sample was 56% male, 69% Caucasian, and had a mean of 21.5 years of age (SD=2.3). The sample averaged 9.3 cigarettes per day (SD=7.7), 4.7 years of smoking (SD=3.3), with a mean of 7 lifetime quit attempts; 42% were in the Precontemplation stage of change. There were no significant differences in demographic characteristics (gender, age, ethnicity, employment status, income, subjective social status, or region) between those who provided smoking data (n = 248) and those who left the survey early (n = 58). Compared to those who completed the survey (n = 181), those who just provided smoking data (n = 68) were slightly younger (21.7 vs. 21.0 years; t = −2.14, p = .03). Given the many differences in smoking patterns between daily and nondaily smokers, tests of validity were run separately by daily and nondaily smoking status.

Table 1. Sample characteristics of daily and non-daily young adult smokers (N = 248).

	Daily smokers (n = 167)	Non-daily smokers (n = 81)	Total sample (N = 248)
Gender (%)
Female	45.8	39.5	43.7
Male	53.6	59.3	55.5
Transgender	0.6	1.2	0.8
Age (M[SD])	21.7 (2.2)	21.3 (2.3)	21.5 (2.3)
Race/ethnicity (%)
African-American/Black	4.2	4.9	4.4
Asian/Pacific Islander	4.8	11.1	6.9
Caucasian/White	70.7	65.4	69.0
Hispanic/Latino	13.2	13.6	13.3
Other	7.2	4.9	6.5
Employment status (%)**
Full-time	34.7	33.3	34.3
Part-time	9.0	13.6	10.5
Unemployed/Homemaker	35.9	14.8	29.0
Student	20.4	38.3	26.2
Education (M years [SD])**	13.5 (2.0)	14.5 (2.0)	13.8 (2.1)
Annual family income (%)*
Less than $20,000	31.1	21.0	27.8
$21,000 – $40,000	19.2	23.5	20.6
$41,000 – $60,000	15.6	16.0	15.7
$61,000 – $80,000	15.6	6.2	12.5
$81,000 – $100,000	7.2	11.1	8.5
$100,000 – $200,000	9.6	14.8	11.3
Over $200,000	1.8	7.4	3.6
Subjective social status (M[SD])**	5.0 (1.8)	5.9 (1.8)	5.3 (1.9)
Region (%)
Northeast	19.2	19.8	19.4
Midwest	23.4	18.5	21.8
South	32.9	25.9	30.6
West	24.6	35.8	28.2
Smoking Variables
Cigarettes smoked per day (M[SD])**	12.3 (7.3)	3.2 (4.0)	9.3 (7.7)
Fagerström Test of Nicotine Dependence (M[SD])	3.2 (1.2)	3.2 (0.8)	3.2 (1.1)
Smoking duration (M years[SD])**	5.4 (3.4)	3.1 (2.5)	4.7 (3.3)
Lifetime prior quit attempts (M[SD])	7.0 (13.3)	8.2 (17.2)	7.4 (14.7)
Past year quit attempts (Median [interquartile range])	1 (3)	1 (3)	1 (3)
Stage of change (%):
Precontemplation	43.7	39.5	42.3
Contemplation	34.1	24.7	27.0
Preparation	22.2	35.8	30.6
Desire to quit (M[SD])	5.1 (2.9)	5.5 (3.1)	5.2 (3.0)
Expected success (M[SD})**	4.9 (2.9)	7.3 (2.7)	5.7 (3.1)
Expected difficulty (M[SD])**	6.7 (2.7)	5.0 (2.9)	6.2 (2.9)

^*p < .05,

^**p < .01 for t-test or χ² test of differences between daily and non-daily smokers.

Reliability

Smoking behavior

Reliability of smoking behavior reported on the Smoking Questionnaire and Timeline Followback method are presented in Table 2. All correlations between smoking variables from the two methods were significant, with items measuring the same construct (in bold) having the highest correlations (r between .80 and .90).

Table 2. Reliability of smoking quantity/frequency: Correlations between smoking items from the smoking questionnaire and TLFB questionnaire.

	Timeline Followback
Smoking Questionnaire	Average number of cigarettes smoked between today and yesterday	Number of cigarettes smoked in past 7 days	Average number of cigarettes smoked per day	Average number of smoking days per week
“How many cigarettes did you smoke in the past 24 hours?”	.83**	.83**	.82**	.51**
“How many cigarettes did you smoke in the past 7 days?”	.82**	.86**	.87**	.49**
“What is the usual number of cigarettes you smoke in a day?”	.85**	.89**	.90**	.48**
“On average, how many days in a week do you smoke cigarettes (0–7)?”	.47**	.49**	.50**	.80**

^**.p < .01 (2-tailed).

Bold indicates items measuring the same construct.

Smoking expectancies

Descriptive statistics and internal consistency reliability of the S-SCQ were examined for each of the four scales (range .90 to .94) and the full score (.93) (Table 3). Cronbach alpha values were comparable to data from the original validation studies. Mean scores for the Negative Consequences scale were similar across the three samples; Positive Reinforcement from smoking was slightly lower in our sample; and Negative Reinforcement and Appetite-Weight Control were higher in our sample than Myers et al.’s (2003) samples. The mean total scale score in our sample was comparable to Myers et al.’s adolescent sample and greater than the young adult sample.

Table 3. Short-Form Smoking Consequences Questionnaire Subscale Means, Standard Deviations, and Cronbach’s alpha in the study sample and from Myers et al (2003).

	Present study (n = 248)			Myers et al. young adult sample* (n = 107)			Myers et al. adolescent sample** (n = 125)
Subscale	M	SD	A	M	SD	α	M	SD	α
Negative Consequences	29.80	7.92	.90	31.31	6.87	.79	30.57	7.37	.84
Positive Reinforcement	25.21	12.37	.92	31.76	18.39	.94	42.36	18.40	.92
Negative Reinforcement	42.38	16.08	.93	20.37	13.36	.95	27.34	13.48	.95
Appetite-Weight Control	22.45	13.63	.94	16.44	12.29	.93	16.06	14.17	.93
Total	119.85	36.52	.93	99.88	37.36	.93	116.33	38.83	.92

^*Mean (SD) age = 19.9 (1.3); range = 18–24.

^**Mean (SD) age = 15.9 (1.2); range = 13–19.

Note. Myers et al data from “Constructing a short form of the Smoking Consequences Questionnaire with Adolescents and Young Adults,” by M.G. Myers et al., 2003, Psychological Assessment, 15: 163–172. Copyright, American Psychological Association, reprinted with permission.

Construct validity

Smoking behavior

For daily smokers, a greater number of cigarettes smoked per day was significantly correlated with a higher FTND score (r = .21, p < .01), longer length of time smoking (r = .44, p < .01), a greater number of lifetime quit attempts (r = .23, p < .01), lower efficacy for quitting smoking (r = −.23, p < .05), greater perceived difficulty of quitting (r = .25, p < .01), and more positive smoking-related effect expectancies (r = .23, p < .05; Table 4). Cigarettes per day was not significantly associated with making a past-year quit attempt, longest length of abstinence, desire to quit, or abstinence goal. FTND scores were not significantly associated with any measures of smoking or thoughts about smoking or quitting. For nondaily smokers, greater cigarettes per day was significantly associated only with years smoked (r = .35, p < .01) and more positive smoking-related expectancies (r = .27, p < .05), but not FTND scores, number of lifetime quit attempts, making a past-year quit attempt, longest length of abstinence, desire, efficacy or difficulty of quitting, or smoking goal. Higher FTND scores were associated with lower efficacy for quitting smoking (r = −.36, p < .01), but not other measures of smoking behavior or thoughts about smoking or quitting.

Table 4. Construct validity of smoking severity (cigarettes per day/dependence symptoms) among daily and nondaily smokers: Associations with smoking behaviors and thoughts.

	Daily Smokers (n = 166)		Nondaily smokers (n = 79)
	CPD	FTND	CPD	FTND
CONVERGENT VALIDITY
FTND	.21**	--	−.01	--
Length of smoking	.44**	.15	.35**	.07
Lifetime quit attempts	.23**	.14	−.09	.10
Past year quit attempt (y/n)\	−.06	−.01	−.01	−.21
Longest length of abstinence (days)	−.01	.06	−.14	.08
Desire to quit (1–10)	.03	.02	−.06	−.10
Quitting efficacy (1–10)	−.22*	−.13	−.10	−.36**
Difficulty of quitting (1–10)	.25**	.03	.22	.02
Goal (1–3)	−.05	−.11	.06	−.05
Smoking expectancies (S-SCQ)	.23*	.04	.27*	.10
DIVERGENT VALIDITY
Subjective Social Status	−.09	.01	.10	−.18
Stage of Change: Alcohol§	.18	.63	.68	1.66
Stage of Change: Marijuana§	.65	.19	.21	.88
Desire to quit marijuana	.10	.13	−.21	−.15
Expect success with quitting marijuana	.13	.09	−.27	−.19
Expect difficulty with quitting marijuana	.02	.00	.12	−.01

Note. All values in Table are Pearson product-moment correlations unless otherwise specified. CPD = Cigarettes per day on days smoked; FTND = Fagerström Test of Nicotine Dependence; PC: Precontemplation, C: Contemplation, P: Preparation; S-SCQ: Smoking Consequences Questionnaire – Short Form.

^*p<.05;

^**p<.01,

^§F-values reported.

In tests of divergent validity, as hypothesized, for both daily and nondaily smokers, there were no significant relationships between cigarettes per day and dependence symptoms with subjective social status, stage of change for alcohol or marijuana use, or thoughts about quitting marijuana (Table 4).

Smoking stage of change

Significant associations were observed between smoking stage of change and desire to quit (F(2, 246) = 73.44, p < .01), expectancy of abstinence success (F(2, 246) = 10.47, p<.01), having a goal of abstinence (χ²(4, N = 248) = 22.96, p < .01) and expecting negative consequences from smoking (F(2, 246) = 5.17, p < .01), with lowest values among precontemplators and highest values for those in preparation (Table 5). For expectation of difficulty of staying quit, contemplators had the highest value and precontemplators had the lowest value (F(2, 246) = 4.14, p < .05). There were no differences between stage of change groups on total smoking-related expectancies or expectations of positive reinforcement, negative reinforcement, or appetite/weight control.

Table 5.

Validity of smoking stage of change

	PC (n = 105)	C (n = 77)	P (n = 66)	F/χ2	P-value	Tukey HSD Effect size
CONSTRUCT VALIDITY
Desire to quit: M (SD)**	3.09 (2.37)	5.98 (2.10)	7.82 (2.24)	73.44	.00	PC < C < P
Expected success: M (SD)**	4.74 (3.51)	5.72 (2.42)	7.14 (2.36)	10.47	.00	PC, C < P
Expected difficulty: M (SD)*	5.48 (3.10)	6.80 (2.38)	6.44 (2.84)	4.14	.02	PC < C
Abstinence goal (%)**				22.96	.00
No change	49.4	20.0	18.0
Middle	44.4	66.7	60.0
Abstinence	6.2	13.3	22.0			PC < C < P
S-SCQ	114.92 (38.97)	120.86 (36.71)	126.42 (31.50)	1.56	.21
Negative Consequences*	27.8 (8.1)	30.4 (7.9)	32.2 (5.4)	5.17	.01	PC < P
Positive Reinforcement	25.8 (12.0)	25.2 (12.7)	24.3 (12.7))	0.23	.79
Negative Reinforcement	40.1 (18.8)	43.7 (13.8)	44.5 (13.4)	1.47	.23
Appetite/Weight Control	21.2 (14.2)	21.7 (13.4)	25.3 (12.7)	1.56	.21
CRITERION VALIDITY
% Daily smokers**	69.5	74.0	56.1	11.7	.00
%≥1 Past year quit attempt**	30.5	29.9	39.5	49.03	.00

PC: precontemplation, C: contemplation, P: preparation; S-SCQ: Smoking Consequences Questionnaire – Short Form.

^*p < .05;

^**p < .01

In tests of divergent validity, as hypothesized, stage of change did not vary as a function of participants’ age (F(2, 246) = 0.43, p = .96), ethnicity (χ²(8, N = 248) = 4.79, p = .78), years of education (F(2, 246) = 0.74, p = .48, or subjective social status (F(2, 246) = 1.19, p = .31).

Criterion validity

Smoking behavior

As hypothesized, daily smokers were significantly more likely to be unemployed and less likely to be full-time students (χ²(3, N = 248) = 16.18, p<.01) and had significantly fewer years of education (t(247) = 3.52, p < .01) than non-daily smokers (Table 1). Daily smokers also had lower annual family income (χ²(6, N = 248) = 13.42, p = .04) and perceived themselves to be of significantly lower social status (t(247) = 3.65, p < .01) than nondaily smokers. Contrary to hypotheses there were no significant differences in proportion of daily smokers by gender or ethnicity.

For daily but not nondaily smokers, there were significant differences in cigarettes smoked per day by ethnicity such that African-Americans, M = 7.7 (4.6), and Asian/Pacific Islanders, M = 7.6 (5.7), smoked less than Whites, M = 13.3 (7.3), Hispanics, M = 10.0 (7.7), and members of other ethnic groups, M = 11.7 (4.6); F(4, 161) = 2.72, p = .03. There were no significant ethnic differences in FTND for daily smokers. For nondaily smokers, Hispanic, M = 3.8 (0.9), and “other,” M = 3.5 (1.0), participants had more dependence symptoms than African-American, M = 3.3 (1.5), Asian/Pacific Islander, M = 3.3 (0.9), or Caucasian, M = 3.0 (0.6), participants (F = 3.2; p < .05).

Smoking stage of change

Stage significantly distinguished daily and nondaily smokers such that those in preparation were the least likely to be daily smokers compared to the other two stages (χ²(2, N = 248) = 11.7, p < .01). Stage also distinguished between those who had made a past year quit attempt, such that those in preparation were more likely to have made a quit attempt than the other two groups (χ²(4, N = 248) = 49.03, p < .01).

Comparison to NSDUH

Confidence intervals for reported cigarettes smoked per day overlapped for our study and the NSDUH (Table 8). The proportion of daily smokers was greater in the current study relative to national data (63.0% vs 48.1%), while fewer participants in the current study were 1 to 2 days/week smokers (3.8% vs. 13.4%). In the present study, there was a smaller proportion of participants with likely nicotine dependence based on FTND scores compared to the NSDUH.

Discussion

Overall, smoking-related information as reported by young adults in an online survey demonstrated strong reliability and validity, and estimates of smoking quantity, but not frequency, were comparable to those obtained from a nationally representative household interview among young adults.

Reliability

We found strong relationships between smoking estimates obtained with the Smoking Questionnaire and the Timeline Followback method. The very small amount of variance observed indicates that while estimates of smoking can vary with different item response formats, there may not be additional utility in using multi-item response formats for assessing tobacco use with young adults. This contrasts somewhat with previous findings with adults indicating that multi-item formats add important additional information (Klein et al., 2007).

Internal consistency reliability of the S-SCQ was almost identical to that found by Myers and colleagues (2003) in a similar population, indicating that smoking expectancies measured online with young adults are as reliable as paper-and-pencil measures.

Construct validity

Measures of cigarettes per day demonstrated stronger convergent validity for daily smokers than nondaily smokers. For cigarettes per day among daily smokers, findings were in line with previous work for associations with dependence symptoms (Heatherton et al., 1991), length of time smoking (Hu, Davies, & Kandel, 2006; Patton, Coffey, Carlin, Sawyer, & Wakefield, 2006), lifetime quit attempts (Zvolensky, Johnson, Leyro, Hogan, & Tursi, 2009), thoughts about abstinence (Hall et al., 1990), and smoking-related expectancies (Brandon & Baker, 1991; Myers et al., 2003). Fewer significant associations were found among nondaily smokers.

For FTND among daily and nondaily smokers, few associations were found with other smoking characteristics, suggesting an absence of convergent validity for this construct. Previous studies have demonstrated the validity of the FTND administered in a face-to-face format when compared with other measures of nicotine dependence which was not possible in the present study (Sledjeski et al., 2007; Van De Ven, Greenwood, Engels, Olsson, & Patton, 2010). For young adult non-daily smokers who are likely to be “social smokers” who smoke a high quantity infrequently, FTND may not be as clear a measure of smoking severity for these individuals (Moran, Wechsler, & Rigotti, 2004). Specifically, in our sample, the FTND did not distinguish between daily and non-daily smokers, nor was it related to smoking expectancies in daily or nondaily smokers. Future validation studies should include multiple measures of nicotine dependence symptoms to ideally evaluate the utility of this measure in young adults smokers online.

Convergent validity for smoking stage of change was strong overall. Consistent with the transtheoretical model of change and prior research with adults, our findings demonstrated that the perceived consequences of smoking are more negative at later stages of change (DiClemente, Fairhurst et al., 1991; Fava, Velicer, & Prochaska, 1995; Prochaska et al., 1994).

Concurrent validity

Associations between demographic variables and daily/nondaily smoking status were consistent with prior epidemiological studies of young adult smokers, including education, employment status, and income differences (Hu et al., 2006; Lawrence, Fagan, Backinger, Gibson, & Hartman, 2007), with the exception of gender. In the literature, men are more likely to be daily smokers than women (Hu et al., 2006; Lawrence et al., 2007).

We found no differences in daily/nondaily smoking status by ethnicity. Among daily smokers, however, Caucasians smoked more cigarettes per day than those of other ethnicities, consistent with prior studies (Hu et al., 2006; Lawrence et al., 2007; White, Nagin, Replogle, & Stouthamer-Loeber, 2004). Studies of young adult smoking behavior should consider daily and non-daily smokers as unique groups.

For criterion validity of stage of change, while stage did not distinguish among daily and non-daily smokers (Correia, Ballard, Henslee, & Irons, 2006), results were generally consistent with previous work showing that those in Preparation had more past year quit attempts (Prochaska et al., 2004) and smoked fewer cigarettes per smoking day (Johnson, Fava, Velicer, Monroe, & Emmons, 2002) compared to those in Contemplation and Precontemplation. This is consistent with the Transtheoretical Model tenet that those who are preparing to make behavioral changes take steps to do so before they take action.

Comparison to NSDUH

Compared to a national household survey, our sample demonstrated greater frequency of smoking, with higher reported rates of daily smoking and lower reported rates of smoking 1 or 2 days per month. The national survey data from NSDUH used a different method of gathering data, and although the weighting procedures used are designed to yield census-equivalent results, there are biases with regard to the population subset that was included and sampled for the study. Our findings suggest that the relative anonymity of online or electronic questionnaires may lessen social desirability bias (Bowling, 2005; Rhodes et al., 2003), as there are no interviewer effects.

Limitations

Relying on self-report, a study limitation is that respondents may not recall their behaviors accurately. However, this is true for other survey modes too. While we were unable to validate our recent smoking data with biological data due to concerns for anonymity, the comparisons to NSDUH data and comparisons between multiple measures of smoking information in our study suggest validity in reports. In addition, attrition was fairly high in that only 52% of the entire eligible sample completed the survey. However, this is consistent with other online survey studies with young adults (e.g., McCabe et al., 2002), and methods of tracking participants beyond what were employed here would have compromised a goal of the research to maintain participant anonymity. Finally, we made every effort to conduct formal statistical tests when possible, however some analyses did not allow for this (e.g., comparison of data here to national data), and less formal tests were used (e.g., comparison of confidence intervals). We acknowledge that this is not as precise a measure of data comparison as formal test would have been.

Conclusions

To our knowledge, the present study was the first to evaluate the validity and reliability of tobacco use and associated cognitions reported anonymously online in a representative sample of young adult smokers. Importantly, the study examined associations across multiple constructs related to tobacco use and examined differences by daily and nondaily smoking status. Given that the Internet is so broadly used for surveys of health behaviors, it is important to know that reliable and valid data can be collected online from young adult smokers. The Internet offers some important advantages over traditional methods of data collection (e.g., ease of access, anonymity) that are likely to be advantageous in research with young adults. Investigations should continue to evaluate the psychometric properties of health risk behaviors assessed online.

Table 6. Comparison of smoking characteristics of young adult smokers between online survey data and National Survey on Drug Use and Health (NSDUH).

	Present study % (95% CI)	2008 NHDUH, Age 18–25 % (95% CI)
“Number of days used in the past month among past month users”
1–2	3.8 (1.9, 7.6)	13.4 (12.9, 13.9)
3–5	5.4 (3.0, 9.7)	10.4 (9.9, 10.9)
6–19	10.3 (6.7, 15.6)	15.4 (14.8, 16.0)
20–29	17.4 (12.6, 23.5)	12.7 (12.2, 13.2)
30	63.0 (55.8, 69.7)	48.1 (47.3, 48.9)
# of cigarettes smoked per day among daily smokers
< 6	27.5 (21.3, 34.8)	24.7 (23.8, 25.6)
6–15	42.5 (35.3, 51.1)	43.1 (42.2, 44.0)
16–25	25.7 (19.7, 32.9)	26.1 (25.2, 27.0)
≥26	3.6 (1.7, 7.6)	6.0 (5.6, 6.5)
Not reported	0.6 (0.1, 3.3)	0.1 (.07, .13)
% Nicotine dependence among past month smokers	31.8 (26.3, 37.9)	45.3 (44.9, 45.7)

Note. In the present study, Nicotine Dependence (ND) was measured as percentage of the sample that had Fagerström Test for Nicotine Dependence ≥ 4; in the NSDUH, ND was measured using criteria derived from the Nicotine Dependence Syndrome Scale (NDSS) and the FTND.