Why Are National Estimates So Different? A Comparison of Youth E-Cigarette Use and Cigarette Smoking in the MTF and PATH Surveys

Abstract

Objective:

We compared estimates of adolescents’ nicotine product use and perceptions of harm from two national surveys: Monitoring the Future (MTF) and Population Assessment of Tobacco and Health (PATH). We explored one explanation for the different estimates for nicotine product use and adolescents’ perceptions of harm.

Method:

We used data source triangulation examining 30-day e-cigarette use and cigarette smoking, beliefs about harm, and friends’ use of these products in two samples of adolescents from the 2015–2016 MTF and PATH samples.

Results:

Differences were found, with MTF reporting higher prevalence rates in both past-30-day e-cigarette use (12.4% vs. 6.7%) and cigarette smoking (8.6% vs. 5.1%) when compared with PATH. Differences were significant at the .001 alpha level. MTF respondents were less likely than PATH respondents to view both e-cigarettes (17.7% vs. 48.6%) and cigarettes (75.6% vs. 82.4%) as harmful. The unadjusted odds ratio (OR) shows that PATH respondents had significantly lower odds of indicating either e-cigarette (OR = 0.509, 95% confidence interval [CI] [0.400, 0.648]) or cigarette smoking (OR = 0.571, 95% CI [0.433, 0.753]) when compared with MTF respondents. However, these differences in e-cigarette use (adjusted odds ratio [AOR] = 0.849, 95% CI [0.630, 1.144]) and cigarette smoking (AOR = 0.829, 95% CI = [0.578, 1.189]) were mediated when additional predictors were included in the model (i.e., friends use, risk of harm).

Conclusions:

Substantial differences were found between national surveys estimating population rates of e-cigarette use and cigarette smoking. Data source triangulation allowed for new explanations for several of the disparate nicotine use estimates between MTF and PATH.

The scientific community often observes disparate survey estimates of adolescents’ e-cigarette use and cigarette smoking even when the national surveys are conducted in the same year. Survey scientists have demonstrated that the location of survey administration or the mode of survey delivery may affect respondents’ answers, thus accounting for some of the differences in survey estimates (Beebe et al., 2006; Brener et al., 2006; Fendrich & Johnson, 2001; Gnambs & Kaspar, 2015; Vereecken & Maes, 2006; Wyrick & Bond, 2011). Moreover, it is generally assumed that higher prevalence survey estimates are more accurate indicators of respondents’ behaviors; however, this “more-is-more accurate” assumption is largely untested (Gnambs & Kaspar, 2015).

Brener et al. (2006) examined whether the prevalence of self-reported health risk behaviors among 9th–11th graders varied by survey venue (school vs. home) and mode of administration: either paper/pencil or computer-assisted self-interviewing (CASI). In 2004, students (N = 4,506) were randomly assigned to one of four conditions: school-based in classroom with paper/pencil survey; school-based in computer lab with web survey; home-based in quiet room with paper/pencil survey; and home-based in quiet room with CASI. All questionnaires were identically worded and drawn from the Youth Risk Behavior Survey (YRBS). Mode effects were stronger than setting effects, and the CASI survey was significantly associated with the reporting of more risky behaviors and the odds were highest for students in school. Several study characteristics should be considered when interpreting the findings: (a) school-based surveys, regardless of mode, were administered in groups, and home-based surveys were administered in a quiet room with the interviewer present but unable to see responses on the computer; (b) the behaviors were more strongly affected by setting (individual); (c) the CASI produced higher prevalence for the most socially unacceptable behaviors (suicide, drug use, etc.); (d) paper/pencil surveys, whether in the classroom or at home, generally had the lowest number of behavior endorsed; and (e) researchers did not determine whether parents were in the room at the time of home-based surveys. A major limitation of their study was that the comparisons between settings were confounded with group versus individual administration. Brener et al. (2006) presumed that the higher estimates seen with school-based surveys were more accurate because they were higher. This is an assumption, albeit not proven, that is almost universal in the survey world (Gnambs & Kaspar, 2015).

In their meta-analysis, Gnambs and Kaspar (2015) considered item sensitivity, interviewer presence, group administration, standardized setting, and computer audioenhancement as moderators of the relationship between sensitive disclosures (e.g., drug use, sexual identity) and mode of survey (paper/pencil or computer-based). They examined 48 independent samples of primarily adolescents and young adults, with a median age of 19 years. Two thirds of the studies were conducted in the United States and the sensitivity ranks came from the YRBS. Computer surveys provided the highest estimates for very controversial behaviors (e.g., cocaine use), and lesser estimates for behaviors such as smoking cigarettes. They concluded that computerized surveys led to significantly more reporting of socially undesirable behaviors compared with paper/pencil surveys (Gnambs & Kaspar, 2015).

The Monitoring the Future (MTF) survey (Johnston et al., 2017; Miech et al., 2017) and Population Assessment of Tobacco and Health (PATH) Study (National Addiction & HIV Data Archive Program, 2016) are two national studies that provide very different estimates of youth e-cigarette use and cigarette smoking, despite using similar questions. Survey environment (i.e., school-based group setting vs. home-based individual setting) and mode (i.e., paper/pencil vs. audio CASI) may account for some differences.

To gain a better understanding of why the MTF and PATH estimates of adolescents’ e-cigarette use and cigarette smoking differ so dramatically, we conducted data source triangulation and compared 30-day e-cigarette and cigarette use in the 2015–2016 MTF and PATH samples of 17-year-old adolescents. We examined variables guided by theory that may be important predictors of e-cigarette use and were similar between the surveys; we analyzed whether these variables explained the different MTF and PATH estimates. Our variable selection was guided by the Theory of Planned Behavior (TPB; Ajzen, 1991). The TPB considers substance use in the context of adolescents’ attitudes (i.e., beliefs of risk of harm), access (i.e., ability to use), and social norms (e.g., friends’ use) (Conrad et al., 1992; de Vries et al., 1988; DiClemente et al., 2013; Flay et al., 1994; Harakeh et al., 2004).

To align with TPB, we selected the following variables: beliefs about harm; use of nicotine/tobacco products; and friends’ use of e-cigarettes and cigarettes. We hypothesized that in the survey conducted in a group setting with peers (MTF), friends’ use of e-cigarettes and cigarettes would account for the higher prevalence estimates when compared with the survey conducted in a home setting (PATH). We chose the MTF and PATH databases because these surveys use variables that align with key constructs consistent with behavioral theory, allowing us to examine possibly why the venue of administration (in-school and in group settings vs. in-home and in individual settings) and survey mode (paper/pencil vs. computer assisted) might affect answers to questions about e-cigarette and cigarette use and friends.

Method

The MTF uses a cross-sectional, nationally representative sample of 12th-grade students attending U.S. secondary schools. A self-administered paper/pencil questionnaire was conducted in respondents’ classrooms or, in some cases, large rooms (e.g., cafeteria). The MTF used a multistage sampling procedure: Stage 1 was the selection of geographic areas within the four regions of the country (e.g., West, Midwest); Stage 2 was the random selection of public and private high schools with replacement (schools that declined were replaced with schools similar in geographic location, size, and urbanicity) (Miech et al., 2019); and Stage 3 was the selection of students within each school. Sample weights were used in the present analyses to account for the unequal probabilities of selection that occurred. The samples analyzed in this study consisted of only 17-year-old adolescents from the 12th-grade sample, with home-schooled students excluded. The response rate for the 2015 and 2016 12th-grade sample in the MTF was 83% and 80%, respectively.

The PATH collected the first wave of data in 2013 (National Addiction & HIV Data Archive Program, 2016) and has a nationally representative panel of adolescents between ages 12 and 17 years. Household interviews were conducted annually using audio computer-assisted self-interview (ACASI) in a quiet space (Hyland et al., 2017). A four-stage stratified area probability sample design was used for Wave 1. In Stage 1, a stratified sample of primary sampling units was selected (county or group of counties). In Stage 2, in the selected primary sampling unit, smaller geographical segments were created and then a sample of smaller segments was drawn. In Stage 3, the sampling frame included residential addresses located in these smaller segments. In Stage 4, selected respondents from the sampled households were identified. The response rate for the 2015–2016 sample was 83%.

Sample

We used data from respondents 17 years of age who were in the PATH sample in 2015–2016 (n = 1,788) or in the MTF 12th-grade sample in 2015–2016 (n = 1,619). It should be noted that some of the respondents in both the MTF and PATH might have been 16 years old given that the public use files do not provide specific age identifiers (i.e., the PATH provides age bands and grade level of respondent, whereas the MTF provides an identifier if the respondent is younger than 18 years of age). We excluded home-schooled youth from the PATH sample; therefore, the combined sample included only adolescents who attended school outside their homes.

Measures

We used questions from the MTF and PATH studies, noting that the wording of questions was only slightly different. The following questions were asked of all respondents who endorsed lifetime e-cigarette and cigarette use.

Past-30-day e-cigarette use

MTF.

“During the LAST 30 DAYS, on how many days (if any) have you used an electronic vaporizer such as an e-cigarette?” Six response choices ranged from none to 20–30 days. All variables were coded as binary (i.e., no use vs. any use).

PATH.

“In the past 30 days, on how many days did you use . . . ?” (respondent identified the type of e-cigarette used). Responses ranged from 1 to 30 days.

Past-30-day cigarette use

MTF.

“How frequently have you smoked cigarettes during the past 30 days?” Response choices were the following: 1 = not at all; 2 = less than one cigarette per day; 3 = one to five cigarettes per day; 4 = about one half pack per day; 5 = about one pack per day; 6 = about one and one half packs per day; 7 = two packs or more per day. All variables were coded as binary (i.e., no use vs. any use).

PATH.

“In the past 30 days, on how many days did you smoke cigarettes?” Responses ranged from 1 to 30 days.

Risk of harm with e-cigarette use

MTF.

“How much do you think people risk harming themselves (physically or in other ways) if they use electronic cigarettes (e-cigarettes) regularly?” Response choices were 1 = no risk; 2 = slight risk; 3 = moderate risk; 4 = great risk; 5 = can’t say, drug unfamiliar.

PATH.

“How much do you think people harm themselves when they use e-cigarettes or other electronic nicotine products?” Response choices were 1 = no harm; 2 = a little harm; 3 = some harm; 4 = a lot of harm.

Both MTF and PATH measures were coded as binary, with great risk/a lot of harm coded as 1 versus all other responses coded as 0. We also included questions about health warnings for cigarettes since both the MTF and PATH studies asked about the warnings.

MTF.

“In recent months, have you noticed the health warnings on cigarette packs?” Response choices were 1 = yes and 2 = no.

PATH.

“In the past 30 days, how often, if at all, have you noticed the health warnings on cigarette packs?” Response choices were 1 = never; 2 = rarely; 3 = sometimes; 4 = often; 5 = very often. Both questions were recoded into binary variables that reflected never versus rarely, sometimes, often, very often.

Friends’ use of e-cigarettes and cigarettes

MTF.

“How many of your friends would you estimate use an e-cigarette, e-pen, etc.?” Response choices were 1 = none; 2 = a few; 3 = some; 4 = most; 5 = all. “How many of your friends would you estimate smoke cigarettes?” Response choices were 1 = none; 2 = a few; 3 = some; 4 = most; 5 = all.

PATH.

“How many of your best friends use e-cigarettes or other electronic nicotine products?” Response choices were 1 = none; 2 = a few; 3 = some; 4 = most; 5 = all. How many of your best friends smoke cigarettes? Response choices were 1 = none; 2 = a few; 3 = some; 4 = most; 5 = all. Measures were coded as binary (i.e., none coded as 1 vs. all other categories coded as 0).

Control variables

We controlled for sex (male vs. female), race (White, Black, other), highest level of parental education (at least a bachelor’s degree or higher vs. less than a college degree), average grade in school (C or lower vs. C+/B or higher), and U.S. region (Northeast, Midwest, South, West).

Analysis

To assess differences between the MTF and PATH, we first ran chi-square analyses to assess bivariate differences (Table 1). Second, we used logistic regression to determine which factors could account for the differences in e-cigarette and cigarette use across surveys (i.e., Is the effect of participating in the MTF versus PATH seen when we account for variables aligned with TPB and other sociodemographic variables?). Because two samples were combined for the multivariable analysis, we were unable to use survey weights or account for the complex survey design in either study. These initial sets of analyses were performed with Stata 15.0 software (StataCorp LP, College Station, TX) and used listwise deletion to handle missing data on any of the items. Third, to formally test the potential mediators of survey type (i.e., MTF vs. PATH) on e-cigarette and cigarette use, multiple mediator models were fit using structural equation models (SEM). In particular, survey type was treated as the exogenous variable predicting both the potential mediators (e.g., attitudes toward e-cigarette use) and outcomes (e.g., e-cigarette use), whereas the potential mediators were treated as endogenous to survey type and exogenous to e-cigarette and cigarette use (the outcome variables).

Table 1.

Bivariate differences between the MTF and PATH samples

	MTF (2015–2016) (n = 1,619)		PATH (2015–2016) (n = 1,788)
Variable	n (%) [%]	%Missing	n (%) [%]	%Missing	χ2
Key measures
Behavior (any use in past 30 days)
Past-30-day e-cigarette use	193 (12.4) [11.9]	3.9	117 (6.7) [7.0]	2.7	31.09***
Past-30-day cigarette use	135 (8.6) [8.6]	3.3	89 (5.1) [5.1]	2.6	16.07***
Attitudes
Great risk/A lot of harm of using e-cigarettes	278 (17.7) [17.6]	2.8	865 (48.6) [48.0]	0.5	356.28***
Great risk/A lot of harm of using cigarettes	1,197 (75.6) [74.4]	2.2	1,471 (82.4) [82.4]	0.2	23.51***
Peer group
No friends use e-cigarettes	681 (43.1) [43.6]	2.5	1,202 (67.4) [66.5]	0.2	199.87***
No friends use cigarettes	602 (37.9) [37.3]	2.0	1,244 (69.7) [70.0]	0.2	342.78***
Awareness
Noticed warnings on cigarette packs	690 (43.6) [43.6]	2.2	901 (50.8) [51.2]	0.9	17.81***
Sociodemographics
Sex
Male	747 (47.4) [54.0]	2.7	886 (49.6) [48.9]	0.1	1.592
Female	828 (52.6) [46.0]		900 (50.4) [51.1]
Race
White	868 (53.6) [52.7]	0.0	1,123 (66.9) [68.9]	6.2	124.85***
Black	201 (12.4) [12.9]		269 (16.0) [16.5]
Other	550 (34.0) [34.4]		286 (17.0) [14.6]
Parental education
At least one parent has a college degree or higher	856 (55.9) [54.4]	5.4	714 (39.9) [44.4]	0.0	84.12***
Both parents have less than a college degree	676 (41.1) [45.6]		1,074 (60.1) [55.6]
Average grade in school
C+/B or higher	222 (85.6) [83.6]	5.1	257 (85.6) [86.6]	0.3	0.0003
C or lower	1,315 (14.4) [16.4]		1,525 (14.4) [13.4]
U.S. region
Northeast	371 (22.9) [19.4]	0.0	276 (15.4) [17.6]	0.0	43.01***
Midwest	305 (18.8) [18.0]		377 (21.1) [20.8]
South	584 (36.1) [36.0]		617 (34.5) [35.4]
West	359 (22.2) [26.7]		518 (29.0) [26.2]

Notes: Estimates in () represent unweighted percentages. Estimates in [] represent weighted percentages. All samples sizes are unweighted. All differences among the key measures and sociodemographics between the two surveys were statistically significant at the .001 alpha level (based on chisquare tests of independence). Only sex and “Average grade in school” found nonsignificant differences between the two surveys. MTF = Monitoring the Future study; PATH = Population Assessment of Tobacco and Health Study.

^***p < .001.

Mplus Version 8 (Muthén & Muthén, Los Angeles, CA) was used to estimate two models (i.e., one model assessing e-cigarette use and another assessing cigarette use) in order to evaluate both the direct effect of survey type on the outcomes and the indirect effects (i.e., mediational path) on the outcomes (MacKinnon, 2008). Mplus’s default option was used to estimate the indirect effects (i.e., Sobel’s method [Sobel, 1987]). Given that e-cigarette and cigarette use were binary, logistic regression was used on the main outcomes. Full information maximum likelihood estimation was used for these analyses to handle missing data across items.

Results

Table 1 provides the bivariate assessment of differences between the MTF and PATH studies. With respect to differences in prevalence rates between MTF and PATH in past-30-day e-cigarette and cigarette use, we found significantly higher prevalence rates in the MTF for both past-30-day e-cigarette and cigarette use. Respondents in the MTF were less likely to view both e-cigarettes and cigarettes as harmful when compared with respondents in the PATH. A smaller percentage of respondents in the MTF, contrasted with PATH respondents, indicated that none of their friends used e-cigarettes or cigarettes. In addition, a smaller percentage of respondents in the MTF noticed warnings on cigarette packs when compared with respondents in the PATH.

Table 2 provides the unadjusted and adjusted odds ratios among the predictors of both past-30-day e-cigarette and cigarette use. We see that the unadjusted odds show the PATH respondents had significantly lower odds of indicating either e-cigarette use or cigarette smoking when compared with respondents in the MTF. However, these differences between the MTF and PATH respondents in their e-cigarette use and cigarette smoking were mediated when the additional predictors are included in the model.

Table 2.

Unadjusted and adjusted odds ratios among the predictors of both past 30-day e-cigarette and cigarette use

Variable	E-cigarettes (past 30 days) OR [95% CI]	E-cigarettes (past 30 days) AOR [95% CI]	Cigarettes (past 30 days) OR [95% CI]	Cigarettes (past 30 days) AOR [95% CI]
Key measures
Survey
MTF	Reference	Reference	Reference	Reference
PATH	0.509*** [0.400, 0.648]	0.849 [0.630, 1.144]	0.571*** [0.433, 0.753]	0.829 [0.578, 1.189]
Attitudes
No risk/No harm, Slight risk/A little harm, Moderate risk/Some harm (e-cigarettes)	Reference	Reference	Reference	Reference
Great risk/A lot of harm of using e-cigarettes	0.243*** [0.171, 0.345]	0.470*** [0.316, 0.697]	0.531*** [0.383, 0.735]	1.512 [.998, 2.291]
No risk/No harm, Slight risk/A little harm, Moderate risk/Some harm (cigarettes)	Reference	Reference	Reference	Reference
Great risk/A lot of harm of using cigarettes	0.463*** [0.359, 0.597]	0.692* [0.514, 0.932]	0.203*** [0.153, 0.268]	0.227*** [0.160, 0.322]
Peer group
Some, most, all friends use e-cigarettes	Reference	Reference	Reference	Reference
No friends use e-cigarettes	0.073*** [0.050, 0.107]	0.118*** [0.078, 0.181]	0.209*** [0.151, 0.290]	0.689 [0.464, 1.021]
Some, most, all friends use cigarettes	Reference	Reference	Reference	Reference
No friends use cigarettes	0.274*** [0.211, 0.357]	0.891 [0.645, 1.23]	0.082*** [0.053, 0.128]	0.138*** [0.082, 0.232]
Awareness
Did not notice warnings on cigarette packs	Reference	Reference	Reference	Reference
Noticed warnings on cigarette packs	1.915*** [1.505, 2.438]	1.827*** [1.387, 2.407]	2.506*** [1.873, 3.352]	2.746*** [1.946, 3.874]
Sociodemographics
Sex
Male	Reference	Reference	Reference	Reference
Female	0.747* [0.589, 0.947]	0.913 [0.697, 1.195]	0.989 [0.752, 1.302]	1.338 [0.967, 1.853]
Race
White	Reference	Reference	Reference	Reference
Black	0.384*** [0.237, 0.622]	0.653 [0.383, 1.114]	0.415*** [0.241, 0.713]	0.410*** [0.219, 0.771]
Other	0.983 [0.751, 0.622]	0.999 [0.723, 1.380]	0.888 [0.646, 1.221]	0.586** [0.390, 0.879]
Parental education
Both parents have less than a college degree	Reference	Reference	Reference	Reference
At least one parent has a college degree or higher	0.905 [0.713, 1.149]	0.829 [0.628, 1.095]	0.644** [0.484, 0.856]	0.795 [0.565, 1.118]
Average grade in school
C+/B or higher	Reference	Reference	Reference	Reference
C or lower	1.719*** [1.276, 2.315]	1.471* [1.044, 2.072]	2.887*** [2.117, 3.938]	2.327*** [1.602, 3.381]
U.S. region
Northeast	Reference	Reference	Reference	Reference
Midwest	0.850 [0.598, 1.207]	0.933 [0.623, 1.396]	0.945 [0.623, 1.431]	0.969 [0.591, 1.588]
South	0.665* [0.482, 0.918]	0.816 [0.563, 1.181]	0.882 [0.608, 1.281]	1.045 [0.669, 1.636]
West	0.737 [0.525, 1.034]	0.863 [0.576, 1.293]	0.734 [0.486, 1.107]	0.933 [0.564, 1.545]

Notes: Analyses do not use weights given that the two data sets were merged. All analyses do not use weights or account for the complex sampling design in either the PATH or MTF studies. All analyses use listwise deletion to handle missing data across the items.

OR = odds ratio; CI = confidence interval; AOR = adjusted odds ratio; MTF = Monitoring the Future Study; PATH = Population Assessment of Tobacco and Health Study.

^*p < .05;

^**p < .01;

^***p < .001.

To determine which theoretically informed predictors were mediating the difference between surveys, multiple mediation analyses (Table 3) were estimated separately for e-cigarette use and cigarette smoking. According to the models, the direct effect of survey type on both e-cigarettes and cigarettes was fully mediated based on the set of predictors included in the model (e.g., friend’s use). To determine which predictors either fully or partially mediated the association between survey type and e-cigarette/cigarette use, we assessed the specific mediational paths that were found to be statistically significant.

Table 3.

Multiple mediator analysis among the predictors of both past 30-day e-cigarette and cigarette use

	E-cigarettes (past 30 days)			Cigarettes (past 30 days)
Variable	b	(SE)	β	b	(SE)	β
Total effect from survey type to past-30-day e-cigarette/cigarette use
Survey
MTF	Reference			Reference
PATH	-0.970***	(0.150)	-.216	-0.859***	(0.169)	-.188
Direct from survey type to past-30-day e-cigarette/cigarette use
Survey
MTF	Reference			Reference
PATH	-0.155	(0.140)	-.035	-0.250	(0.169)	-.055
Total and specific indirect effect from survey type to specific mediator to past-30-day e-cigarette/cigarette use
Total indirect effect	-0.814***	(1.02)	-.182	-0.609***	(0.117)	-.133
Attitudes
No risk/No harm, Slight risk/A little harm,
Moderate risk/Some harm (e-cigarettes)	Reference			Reference
Great risk/A lot of harm of using e-cigarettes	-0.247***	(0.061)	-.055	-0.107	(0.060)	.023
No risk/No harm, Slight risk/A little harm,
Moderate risk/Some harm (cigarettes)	Reference			Reference
Great risk/A lot of harm using cigarettes	-0.026*	(0.011)	-.006	-0.100***	(0.024)	-.022
Peer group
Some, most, all friends use e-cigarettes	Reference			Reference
No friends use e-cigarettes	-0.553***	(0.061)	-.123	-0.117*	(0.047)	-.026
Some, most, all friends use cigarettes	Reference			Reference
No friends use cigarettes	-0.036	(0.049)	-.008	-0.633***	(0.087)	-.139
Awareness
Did not notice warnings on cigarette packs	Reference			Reference
Noticed warnings on cigarette packs	0.038**	(0.013)	.008	0.064***	(0.019)	.014
Sociodemographics
Sex
Male	Reference			Reference
Female	0.002	(0.003)	.001	-0.005	(0.005)	-.001
Race
White	Reference			Reference
Black	-0.018	(0.011)	-.004	-0.030*	(0.015)	-.007
Other	0.006	(0.026)	.001	0.064*	(0.032)	.014
Parental education
Both parents have less than a college degree	Reference			Reference
At least one parent has a college degree or higher	0.028	(0.021)	.006	0.055*	(0.027)	.012
Average grade in school
C+/B or higher	Reference			Reference
C or lower	0.001	(0.004)	.001	0.001	(0.009)	.001
U.S. region
Northeast	Reference			Reference
Midwest	-0.001	(0.004)	-.001	-0.002	(0.005)	-.001
South	0.004	(0.005)	.001	0.001	(0.003)	.001
West	-0.011	(0.013)	-.002	-0.011	(0.016)	-.003

Notes: Analyses do not use weights given that the two data sets were merged. All analyses do not use weights or account for the complex sampling design in either the PATH or MTF studies. All analyses used full information maximum likelihood estimation listwise to handle missing data across the items. B = unstandardized estimate; SE = standard error (unstandardized); β = standardized estimate; MTF = Monitoring the Future Study; PATH = Population Assessment of Tobacco and Health Study.

^*p < .05;

^**p < .01;

^***p < .001.

With respect to past-30-day e-cigarette use, we found that the mediational paths from survey type to risk of harm (both e-cigarette and cigarette), survey type to peer’s e-cigarette use, and survey type to awareness were all statistically significant. Among the significant mediators of survey type on past-30-day e-cigarette use, 69.8% of the mediation effect was explained by peer groups’ use of e-cigarettes (i.e., -0.123/-0.176 = 0.694). Post hoc analyses assessing each mediational path separately found that peer groups’ e-cigarette use was the only variable in the model that fully mediated the association between survey type and past-30-day e-cigarette use (total effect, b = -0.825, p < .001; mediational path, b = -0.622, p < .001; direct effect, b = -0.204, N.S.).

When assessing past-30-day cigarette use, we found that the mediational paths from survey type to attitudes toward cigarettes, survey type to peer’s e-cigarette and cigarette use, survey type to awareness, survey type to race, and survey type to parental education were all statistically significant. Among the significant mediators of survey type on past-30-day cigarette use, 83.7% of the mediation effect was explained by peer groups’ use of cigarettes (i.e., -0.139/-0.166 = 0.837). Post hoc analyses assessing each mediational path separately found that peer groups’ cigarette use (total effect, b = -0.735, p < .001; mediational path, b = -0.809, p < .001; direct effect, b = 0.075, N.S.) and peer groups’ e-cigarette use (total effect, b = -0.596, p < .001; mediational path, b = -0.370, p < .001; direct effect, b = -0.226, N.S.) were the only two variables in the model that fully mediated the association between survey type and past-30-day cigarette use.

Discussion

The 2015–2016 MTF and PATH studies provide very different estimates of e-cigarette use and cigarette smoking by adolescents. Guided by behavioral theory, the current study advanced a possible explanation for the differences. We found that the difference between the MTF and PATH estimates for e-cigarette use and cigarette smoking disappeared when friends’ consumption of e-cigarette and cigarette products were used to adjust the prevalence between the two surveys. It is possible that the MTF experiences a greater peer effect. The MTF data were collected in a school setting with respondents completing a pencil/paper survey in a group setting in which responses could be observed (i.e., sitting next to their peers and possibly friends.) Adolescence is a developmental period in which peers influence an individual’s attitudes and behaviors. Thus, we may expect that youth are more likely to report certain behaviors in the company of their peers. The MTF setting contrasts with the PATH setting where the individually administered ACASI allows for greater privacy.

Of note, national surveys conducted in U.S. high schools in group settings provide remarkably similar prevalence estimates (Centers for Disease Control and Prevention, 2017; Jamal et al., 2017; Johnston et al., 2017; Miech et al., 2017; Singh et al., 2016). Thirty-day e-cigarette use was 11.3% from the 2016 National Youth Tobacco Survey (Jamal et al., 2017); 11% from 12th graders in the 2017 MTF; and 11.4% from the national 2017 YRBS. All these surveys were given in classrooms or computer labs and were usually in a paper/pencil format, although some schools administering the YRBS could use computer labs.

The PATH provided different estimates from MTF, and we found that one explanation for the difference might be friends’ influence within the data collection setting. This is consistent with critical influences on adolescent behavior informed by TPB. Unfortunately, the National Survey on Drug Use and Health (NSDUH; Center for Behavioral Health Statistics and Quality, 2018) did not collect e-cigarette or vaping data during a period (2015–2016) that would allow for direct comparisons to the PATH; however, the NSDUH prevalence estimates for youth cigarette use were more similar to the PATH data and less than MTF estimates. Like the PATH, the NSDUH uses home-based interviews with ACASI.

The interpretation of many of the mode studies using adolescent samples is constrained by the confounding of venue of administration (Eaton et al., 2010; Vereecken & Maes, 2006; Wyrick & Bond, 2011). Beebe et al. (2006) found that adolescents (N = 610) in a health setting differed in their responses based on if they received paper/pencil surveys compared with web surveys. The paper/pencil surveys largely reflected a higher prevalence of risky health behaviors. In contrast, Eaton et al. (2010) found that paper/pencil versus web-based surveys with the YRBS did not generally affect responses. They concluded that risk behavior estimates obtained by web or paper/pencil in a classroom setting were basically equivalent. Eaton et al. (2010) concluded that mode did not have a robust effect.

Historically, school-based surveys produce higher estimates (Brener et al., 2006; Fendrich & Johnson, 2001). Brener et al. (2006) using the YRBS found that for all risk behaviors there was a significant setting main effect; the odds of reporting the risk behavior were greater for students taking the survey in school. Again, the assumption was that school-based surveys are better because estimates are higher. However, setting can go two ways in contributing to motivated misinformation. A lack of privacy in the home may cause adolescents to underreport a behavior, but alternatively, adolescents in the presence of peers may unconsciously activate perceived norms and this may result in overreporting, another form of motivated misinformation. Simply put, it remains unclear whether more reporting is more accurate, and the field would benefit greatly from additional well-designed methods studies to determine why school-based versus home-based venues produce such different survey estimates.

Limitations

There are some limitations with using data triangulation with the MTF and PATH databases, although these limitations need not dampen enthusiasm for data source triangulation. Causal conclusions are limited by the crosssectional nature of the high school data from the MTF and the panel data from the PATH. Respondents in the PATH were interviewed before the 2015–2016 survey, and thus, were familiar with the survey protocol. The MTF data were collected during the school year (winter/spring of 2015 and 2016) and the PATH data obtained over a 12-month period (2015–2016), and this might contribute to prevalence differences. Moreover, we were limited to the measures provided by the MTF and PATH studies, and the questions relevant to variables of interest differed, albeit slightly. Different data collection modes may account for some of the differences we found (paper/pencil vs. ACASI). Although our use of the TPB is innovative when combined with data triangulation, we were constrained by the number of MTF and PATH items consistent with TPB. Further, the exclusion of some institutionalized subpopulations with higher rates of tobacco use, including youth not attending school (MTF) or in juvenile correctional facilities (PATH), may lead to underestimations. And last, although we provided weighted and unweighted estimates (Table 1), we were unable to use weighted estimates for the multivariate analysis. The analyses could not incorporate the complex sampling design (i.e., primary sampling unit and stratum) due to the exclusion of these variables in the public use MTF data files. Although this may provide more liberal standard errors and increase the chance of a Type I error, the results are well within an acceptable threshold of statistical significance (i.e., p < .001) and substantive importance.

Conclusions

Do school-based surveys (conducted in peer-friendly environments) overestimate adolescent e-cigarette and cigarette use, or do home-based surveys (conducted in family-friendly environments) underestimate? This exploratory study suggests that future research is needed to determine a reason for the dramatically different estimates in school-based group settings versus household-based individual settings.

One novel aspect of this data source triangulation study is that we used the TPB to guide variable selection rather than running the data to see what “emerged to report.” Thus, we believe that data source triangulation such as described here, particularly when grounded in a behavioral theory, is one analytic method that provides for new insights and, in this case, reveals an interesting aspect about the differences between MTF and PATH data.

Acknowledgments

The authors thank Kathryn Lundquist, Research Assistant, University of Michigan School of Nursing, for assistance with editing and formatting.

Conflict-of-Interest Statement

The authors have no financial conflicts to disclose.