Does Telephone Audio Computer-Assisted Self-Interviewing Improve the Accuracy of Prevalence Estimates of Youth Smoking? Evidence from the UMass Tobacco Study

Abstract

Despite their advantage for obtaining representative samples of adolescents, telephone surveys have been regarded as inferior for collecting data on youth tobacco use because they yield lower estimates than school-based self-administered surveys. Although no gold standard for smoking estimates exists, the lower estimates in telephone surveys have been attributed to underreporting due to youths’ concern about parents or others overhearing their responses. Telephone audio computer-assisted self-interviewing (T-ACASI) is a cost-effective method for obtaining a representative sample of youths and provides increased privacy for the respondent. We hypothesized that using T-ACASI would encourage youths to more fully report smoking behavior compared to traditional interviewer-administered telephone methods. Our analysis further assessed whether respondent age, gender, race/ethnicity, and parental attitude toward smoking moderated the relationship between survey mode and smoking reports. Using data from a statewide tobacco use survey that randomly assigned youth respondents to either T-ACASI or interviewer-administered mode, we found youths were more likely to report smoking behaviors in T-ACASI mode and that this was especially true for girls, particularly those who believed their parents would disapprove strongly of their smoking. Findings suggest that traditional telephone surveys may underestimate smoking prevalence in most girls by a factor of two, and that a technique for insuring privacy for these respondents is an important component of effective telephone survey methodology.

There has long been debate about the most accurate and efficient means of assessing smoking behaviors among adolescents (Fowler and Stringfellow, 2001; Gfroerer, Wright, and Kopstein, 1997). In-person household surveys are slow and costly, but they can effectively reach the entire youth population, not just those who regularly attend school. In-person surveys also permit the use of private modes of questioning, such as paper self-administered questionnaires (SAQs) and audio-computer-assisted self-interviewing (A-CASI), which have been shown to elicit more frequent, and presumably less biased, reports of substance use behaviors (Torangeau and Smith, 1996; Turner, Forsyth, et al., 1998; Turner, Lessler, and Devore, 1992). Even when these more private modes of interviewing are used, the possibility that the respondents’ parents might see their responses may inhibit full reporting of smoking in household surveys. In-school surveys, on the other hand, eliminate concerns about parents, but they entirely miss adolescents who have dropped out of school and underrepresent those with poor attendance records.

It is therefore not surprising that estimates of youth smoking prevalence from various household and school-based surveys differ significantly. For example, in 1997 the in-home National Household Survey on Drug Use (NHSDA) found 24% of tenth and twelfth grade students smoked cigarettes, whereas two school-based self-administered surveys, Monitoring the Future (MTF) and the Youth Risk Behavior Survey (YRBS), estimated smoking at 30% and 35%, respectively (Fowler and Stringfellow, 2001). The gap between estimates from telephone surveys and in-school surveys is even greater. For example a national telephone survey of youth tobacco use (TAPS) estimated past-month smoking for youth between 12 and 18 years of age to be 15.7% in 1989. The two school-based studies cited above put the rate at over 27% in 1991 and 1992 (USDHHS, 1994). Even allowing for the difference in years and minor differences in age groups covered by the surveys, the 68% increase over the telephone survey estimate is striking.

Traditional telephone surveys have generally been regarded as inferior for the collection of data on substance use among both adolescents and adults. Two major studies have reported comparisons of prevalence estimates of drug use obtained from interviewer-administered telephone surveys versus those obtained using in-home SAQs (Gfroerer and Hughes, 1992; Aquilino, 1994). Both studies found reports of illicit drug use to be substantially higher when respondents completed SAQs in the home rather than verbally answering questions posed by telephone interviewers.

A new advance in survey technology, telephone audio computer-assisted self-interviewing (T-ACASI), eliminates the requirement that respondents in a telephone survey answer aloud to questions posed by a human interviewer (Turner, Miller, Smith, Cooley, and Rogers, 1996; Cooley, Miller, Gribble, and Turner, 2000). In a T-ACASI survey, a human interviewer is used to screen and recruit eligible subjects. Then, after the subject has completed non-sensitive portions of the survey, the phone call is transferred to the T-ACASI system and computer-controlled, pre-recorded questions are read to the subject. The subject provides responses by pressing keys on a touch-tone telephone.

In addition to providing greater privacy for respondents, T-ACASI methodology has the potential to address additional sources of error in smoking studies that employ school-based methods or other traditional survey methods. First, compared to in-home or in-school surveys, T-ACASI is a more cost-effective method for obtaining a representative sample. Telephone surveys cost considerably less than in-person interviews, and are much more likely to include dropouts and others who never or rarely attend school. Second, compared to school-based surveys, telephone surveys may have higher response rates. Nonresponse in school-based surveys includes refusals to participate at both the school and the individual level, which often produce lower overall response rates than household surveys (Fowler and Stringfellow, 2001). Third, compared to self-administered paper-and-pencil surveys, telephone surveys do not require respondents to have a literacy level that allows them to comprehend all written questions and instructions. Fourth, compared to traditional telephone surveys, T-ACASI respondents are all read the questions and response alternatives in the exact same way, providing high standardization in the data collection protocol.

All of these considerations reinforce the expectation that the T-ACASI mode has considerable potential to provide the most accurate estimates of youth smoking behaviors. Early results with T-ACASI in the 1999–2000 National STD and Behavior Measurement Experiment (NSBME) indicate that this technology has a pronounced effect on reporting of sexual behaviors among adults. The NSMBE study compared survey reports across adult respondents (age 18 to 45) who were randomly assigned to receive either a standard interviewer-administered telephone survey or a private computer-administered survey using T-ACASI. NSBME respondents interviewed by T-ACASI were 1.9 times more likely to report multiple concurrent sexual partnerships (p<.001), 1.7 times more likely to report engaging in anal sex (p<.001), and 1.5 to 1.6 times more likely to report same-gender genital contact in their lifetime, in the past 5 years, and in the past 12 months (p<.05) (Al-Tayyib, Turner, Rogers, and Villarroel, 2002; Villarroell, Turner, Eggleston, Al-Tayyib, and Rogers, 2002).

In this study, we report estimates of youth smoking behavior from a household telephone survey that experimentally varied the privacy of youth’s responses by randomly assigning respondents to either T-ACASI or the traditional interviewer-administered method.

Hypotheses

The literature on the measurement of youth smoking behavior and the impact of privacy in survey interviews raises the general expectation that we would observe higher reporting of youth smoking behavior in the T-ACASI mode than in the interviewer-administered mode. Although both modes involve interviewing in the household setting, the T-ACASI mode makes it impossible for others to see or hear the respondent’s answers. Also, the T-ACASI mode does not require youths to report smoking to an adult interviewer whom they might expect would disapprove.

We also consider several additional factors that could moderate the impact of survey mode on youths’ reports of smoking. The first factor is the sensitivity of the behavior being reported. Mode effects should be seen more strongly for behaviors deemed more sensitive or unacceptable (Aquilino, 1994). We examine three distinct smoking behaviors—ever smoking a cigarette in one’s lifetime, smoking cigarettes in the past year, and smoking cigarettes in the past month. The sensitivity of these behaviors should vary among adolescents. We expect that having smoked once (even a single “puff”) would be seen as more acceptable than having smoked in the past year, which, in turn, would be more acceptable than having smoked in the past month.¹ Given these distinctions, we would expect the mode effect to be greatest for reports of past-month smoking and least for reports of lifetime smoking.

Second, we consider three demographic characteristics that might interact with survey mode: age, gender, and race/ethnicity. Smoking prevalence increases with age among adolescents. For example, in 2001, the Monitoring the Future Study reported the prevalence of past-month smoking to be 12.2% among eighth graders, 21.5% among tenth graders, and 29.5% among twelfth graders (Johnston, O’Malley, and Bachman, 2002). Since smoking is more common among older adolescents, youths are likely to consider smoking more acceptable as they grow older. This suggests that younger respondents (those age 12 to 14) would have a greater motivation than older teens to conceal smoking behavior from their parents, and consequently be less likely to report smoking in interviewer-administered mode than in T-ACASI mode. With regard to gender, the fact that girls are lower risk-takers than boys (Byrnes, Miller, and Shafer, 1999) leads us to expect that the mode effect may be stronger for girls. Supporting this expectation is the finding that girls are more likely to report alcohol and marijuana use in CASI rather than SAQ surveys (Webb, Zimet, Fortenberry, and Blythe, 1999). With regard to race/ethnicity, Aquilino (1994) found stronger mode effects among Black and Hispanic respondents than among White respondents in a study of substance use among adults, with higher levels reported in the more private mode. We might therefore expect the mode effect on smoking reports to be stronger among non-White adolescents than among White adolescents.

In addition to age, gender, and ethnicity, another factor that might influence youth reports of smoking behavior across survey modes is the degree to which respondents perceive their parents as disapproving of smoking. Youth who feel their parents would strongly disapprove of smoking have a stronger motivation to downplay or hide such behavior than those who believe their parents would not object as strongly. As a result, we would expect adolescents who report strong parental disapproval of their smoking to also be less likely to report smoking behavior in interviewer-administered mode.

We test these hypotheses using data from a statewide telephone survey of tobacco use and related issues that includes interviews in both T-ACASI and interviewer-administered modes.

Methods

Sample

The 2001/2002 baseline survey of the UMASS Tobacco Study used a statewide, list-assisted, random-digit-dialed (RDD) sample generated using the Genesys Sampling System. The GENESYS system identifies all residential clusters of 100 telephone numbers (area code + exchange + first two digits of phone number) that have at least one published residential number.² These clusters are updated quarterly. The clusters then form the sample frame for selection of final sample telephone numbers. The advantage of the GENESYS system is that since all possible clusters are used to create a sample frame, the final sample is not clustered as in traditional Mitofsky-Waksberg RDD samples. Sample telephone numbers were dialed and screened to determine if they were residential. Ninety percent of all sample telephone numbers were successfully screened in this manner. For those whose residential status could not be determined after at least 14 call attempts over a minimum two-week period and calls on different days and different times of the day, 4% were assumed to be residential. This rate has been consistently verified in RDD studies conducted by the Center for Survey Research by taking random subsamples of such numbers and verifying their residential status with telephone business offices.³

The population of interest for this study was youth between the ages of 12 and 17, inclusive, who lived within households in Massachusetts. Since the study was being conducted by telephone, a practical limitation of the sample was that households needed to have a working telephone. Since over 98% of Massachusetts households have working telephones, this was not a restrictive limitation.

Once a telephone number was determined to be residential, a screening interview with an adult resident was attempted. Information collected in the screening interview included demographic information about all household members and the smoking status of all adult household members. For households with someone 12 to 17 years old, parental permission was sought to interview all resident youth in this age range. Regardless of the assigned survey mode, parents were provided identical information about the nature and purpose of the youth survey.

Using an AAPOR Method 3 calculation, a response rate of 66% was obtained for the screening interviews (AAPOR, 2000). Through these interviews 6,006 youth aged 12 to 17 were found. Parental permission was obtained for 76% of these youth and interviews were then successfully conducted with 84% of those for whom permission was obtained.⁴ This produced an AAPOR Method 3 response rate of 42% for all youth interviews.

Approximately two-thirds of the youth were randomly assigned to be interviewed about their smoking experiences using standard, interviewer-administered, computer assisted telephone interviewing (CATI). The remaining one-third were assigned to the T-ACASI mode. These assignments were made in advance of telephone numbers being dialed to assure a totally random allocation. Advance assignment to interviewing mode ensured that the interviewer, respondent, and the interviewer-respondent interaction had no impact on the mode of completion. Since T-ACASI administration requires a touch-tone telephone, any T-ACASI assigned case that did not have such a telephone was done using CATI. This amounted to only 2% of all T-ACASI cases. These cases were marked and eliminated from all data analyses. Data collection activities lasted from January 2001 through June 2002.

Data Collection Protocol

All interviews were completed in English. Interviewers introduced the study to all youth respondents and pointed out that they should complete the interview in a situation where they had some privacy, agreeing to wait if necessary if the youth preferred to move to another phone. The youth assigned to the T-ACASI condition were read the following script:

“I will be asking some of the questions, while others have been pre-recorded and will be asked by a computer. You can answer the questions the computer will ask by pushing the number buttons on your phone. To do this, you will need a touch-tone phone. Do you have a touch-tone phone?”

Interviewers then conducted the first two sections of the interview (18 questions) that dealt with details of school attendance, after-school activities, and television viewing patterns. At this point, respondents assigned to T-ACASI were told:

“The next set of questions will be asked by the computer. When you have finished answering the next set of questions, the computer will reconnect you to me so we can finish up the survey. I’m going to put you on hold now for just a minute while I connect us to the computer.”

Respondents assigned to T-ACASI were then connected to computers at RTI in North Carolina using technology developed by Cooley (Cooley, Miller, Gribble, and Turner, 2000). Once the connection was made, the interviewer stayed on the line for three nonsensitive training questions and to answer any questions the respondent had about the procedure. Then the recorded voice said that the interviewer was going to hang up so that the respondent could answer privately.⁵ The interviewer then hung up the phone, leaving the respondent connected to the computer.

After completing the section on smoking behavior and parental knowledge and attitudes about smoking (mean time: 7.5 minutes), respondents were automatically returned to the interviewer and completed the remaining six sections of the interview on topics such as exposure to antismoking advertising, attitudes about secondhand smoke, and smoking policy in their homes. The interviewer was blind to the smoking status of the respondent for the entire interview.

Measures

Smoking status

The primary outcome variables included lifetime, past-year, and past-month smoking status. The specific wording for each of these survey items appears in Appendix A. Respondents who reported either having smoked 100 cigarettes in their lifetime or having “ever smoked a cigarette, even a puff” were considered to have smoked in their lifetime. To be considered a past-year smoker, a respondent needed to have smoked at least one whole cigarette and to report that they had last smoked within the past year. To be considered a past-month smoker, a respondent needed to have smoked at least one whole cigarette and to report that they had last smoked within the past 30 days.

Appendix A.

Survey Items Used to Construct Variables for All Analyses

Variables	Survey Items
Smoking Status
Lifetime Smoking	1. Have you smoked at least 100 cigarettes in your life? (yes/no) If 1 = no: 2. Have you ever experimented with cigarette smoking, even a few puffs? (yes/no)
Past-Year Smoking	1. Have you ever smoked a whole cigarette?” If 1 = yes: 2. When did you last smoke or puff on a cigarette?” (within past 12 months)
Past-Month Smoking	1. Have you ever smoked a whole cigarette?” (yes/no) If 1 = yes: 2. When did you last smoke or puff on a cigarette?” (within past 30 days)
Parental Attitude Toward Youth Smoking
Parental Disapproval of Smoking	If past-month smoking = yes: 1. Do your parents know that you smoke? (yes/no) If 1 = yes: 2. How do your parents feel about your smoking? Do you think they don’t mind, disapprove a little, or disapprove a lot? If past-month smoking = no or 1 = no: 3. How would your parents feel if they found out that you smoked/if you did smoke – do you think they wouldn’t mind, would disapprove a little, or would disapprove a lot?
Respondent Characteristics
Gender	Are you male or female? (male/female)
Age	Could you tell me your birth date please?
Race/Ethnicity	1. Are you of Latino or Hispanic origin? (yes/no) If 1 = no: 2. Which of the following best describes your racial background – White, Black, Asian/Pacific Islander, Native American, Other or Multiracial?
Household Characteristics
Education of Adult Informant	What is the highest grade or year of regular school or college that you completed?
Household Income	In studies like this, households are sometimes grouped according to income. I’m going to read you some categories. Please stop me when I get to the one that best describes your total household income last year. That means income before taxes from all sources, such as salaries, interest, retirement or any other source for all household members.

Parental attitude toward youth smoking

In order to measure perception of parental disapproval, the survey included two different forms of the question: one for youth who were not past-month smokers and one for those who were past-month smokers. We asked those who did not smoke in the past month how their parents would feel if they found out they had smoked—whether their parents wouldn’t mind, would disapprove a little, or would disapprove a lot. For past-month smokers, we first asked respondents whether their parents knew that they smoke. Based on their answer to the first questions, they were asked either, “How do your parents feel about your smoking?” or “How would your parents feel if they found out that you smoked?” The same three response alternatives were used for both forms of the question. We then coded parental attitude as a dichotomous variable that categorized respondents into those whose parents would (or do) disapprove “a lot” versus those who parents would (or do) “not mind” or “disapprove a little.”

Respondent and household characteristics

The survey also gathered demographic information about the respondent (age, gender, and race/ethnicity) and, during the screening interview with the adult informant, about the youth’s household (education of adult informant and household income). Appendix A details the survey items used to construct the measures of respondent and household characteristics. For ethnic and racial background, respondents were first asked if they were of Latino or Hispanic origin. If so, the respondent was classified as such. If not of Latino or Hispanic origin, the respondent was asked whether their racial background was best described as “White, Black, Asian or Pacific Islander, Native American, other or multiracial?” We then coded the variable Ethnicity as “minority” for all respondents who identified themselves as being of Latino or Hispanic origin in the first question or selected a racial background other than White in the second question. We computed the age variable by calculating the difference between the respondent’s interview date and their self-reported birth date. To measure the adult informant’s education level, we asked informants to report the highest grade or year of regular school or college they had completed. To capture household income, interviewers read respondents categories in $10,000 increments and asked respondents to stop them when they reached the level that best describes the total household income for the prior year, either 2000 or 2001.

Analysis Plan

For all analyses, we used list-wise deletion of missing data. In both T-ACASI and interviewer-administered modes, missing data were quite minimal and very similar across modes. For example, for past-month smoking, one interviewer-administered case and zero T-ACASI cases had missing data. Missing data for variables included in our analyses ranged from 0.0% to 2.4% of all cases in both modes, with the exception of household income, which was about 20% in both modes. Appendix B provides the missing data percentages for each variable included in the analysis by survey mode.

Appendix B.

Missing Data Percentages for All Analytic Variables by Mode

	Interviewer Administered % (n = 2797)	T-ACASI % (n = 1023)
Age Group	0.6	0.2
12–14
15–17
Gender	0.0	0.0
Male
Female
Ethnicity	2.4	1.1
White
Minority
Education of adult informant	1.9	2.2
12 years or less
More than 12 years
Household Income	20.0	18.6
$50,000 or less
More than $50,000
Smoked in Lifetime	0.1	0.1
Yes
No
Smoked in Past Year	0.1	0.1
Yes
No
Smoked in Past Month	0.1	0.1
Yes
No

Note: All percentages are based on unweighted data.

The sample for this study was a simple random sample of households in Massachusetts with telephone service. There was no stratification or clustering of households in the sample design. Furthermore, since all household residents 12–17 were asked to participate in the survey, there was no random selection of respondents within households. This implies that the probability of selection for eligible sample members was equal by design. The only factor that affected this probability of selection was the distinct number of telephone lines that connected to each sample household. This factor was appropriately incorporated into the calculation of the probabilities of selection and by extension into the calculation of survey weights.

Since weighting by the inverse of the number of telephone lines is the sole deviation from simple random sampling—and since this factor is not a large adjustment—it had a negligible impact on key statistical analyses.⁶ To insure that our published estimates project to the population, we used the survey weights in all analyses reported in this paper. To simplify analysis, we used SPSS (Release 11.0.1, 2001) for all procedures and we did not conduct further analyses of the impact of weighting beyond those cited in endnote 6.

Initially, we used contingency tables to explore potential differences in respondent characteristics across survey mode (interview-administered versus T-ACASI) that might affect smoking behavior. We then used multiple logistic regression analyses to examine the effect of mode while controlling for age, gender, race/ethnicity, and parental attitude toward smoking.⁷ Next, we sequentially added a term for the interaction between each of the control variables and survey mode, one interaction term at a time, to test whether any of these factors moderated the relationship between survey mode and smoking reports. Finally, in order to see whether prevalence estimates derived from using T-ACASI reduced the usual discrepancy between telephone and in-school self-administered surveys, we retabulated our results by grade and compared smoking prevalence estimates with those derived from the Massachusetts 2001 YRBS survey.

Results

Initial comparisons of the demographic characteristics of the two experimental groups indicated there were no statistically significant differences in the age, gender, race/ethnicity, parental education, or household income of the youth interviewed in T-ACASI and interviewer-administered conditions. Respondents in the T-ACASI condition were significantly more likely to report smoking in the past year than those in the interviewer-administered condition (18% versus 14.7%, p<.01). Differences in reporting lifetime and past-month smoking were not significant in these bivariate comparisons (see Table 1).

Table 1.

Sample Characteristics by Mode

	Interviewer-Administered % (n = 2797)	T-ACASI % (n = 1023)
Age Group
12–14	49.3	51.6
15–17	50.7	48.4
Gender
Male	51.6	51.8
Female	48.4	48.2
Ethnicity
White	77.4	79.0
Minority	22.6	21.0
Education of adult informant
12 years or less	34.4	31.5
More than 12 years	65.6	68.5
Household Income
$50,000 or less	34.9	31.9
More than $50,000	65.1	68.1
Smoked in Lifetime
Yes	29.9	31.2
No	70.1	68.8
Smoked in Past Year **
Yes	14.7	18.0
No	85.3	82.0
Smoked in Past Month
Yes	9.0	10.4
No	91.0	89.6

Note: Associations tested by chi-square.

^***p<.001,

^**p<.01,

^*p<.05

The logistic regression analyses presented in Table 2 control for demographic characteristics and parental attitudes, and they generally support the expectation that youths assigned to the T-ACASI mode would report more smoking than those assigned to interview-administered mode. Controlling for demographic characteristics and parental attitudes, adolescents were 1.5 times more likely to report smoking in the past year (p<.01) and 1.4 times more likely to report smoking in the past month (p<.05) in T-ACASI mode. The mode coefficient for reporting of “ever smoking” was not statistically significant at conventional levels (p=.073).

Table 2.

Predictors of Smoking among Massachusetts Adolescents in 2000/2001 (N = 3844)

Independent Variables	Ever Smoked in Lifetime		Smoked in Past Year		Smoked in Past Month
	Adj. OR	95% CI	Adj. OR	95% CI	Adj. OR	95% CI
Mode (T-ACASI)	1.17	0.99, 1.39	1.51***	1.23, 1.86	1.38*	1.07, 1.79
Age Group (15–17)	3.76***	3.20, 4.42	4.43***	3.52, 5.57	4.11***	3.05, 5.53
Sex (Male)	1.09	0.93, 1.27	1.20	0.98, 1.45	1.39**	1.10, 1.77
Race/Ethnicity (White)	0.95	0.79, 1.14	1.52**	1.18, 1.95	1.47*	1.08, 2.01
Parental Attitude (Doesn’t strongly disapprove)	5.35***	4.26, 6.73	5.59***	4.44, 7.03	6.03***	4.66, 7.81
Constant	.15	----	.03	----	.02	----

Note: Each predictor adjusted for all others in table. Reference categories for predictors: Mode (Interviewer-administered); Age Group (12–14); Sex (Female); Race/Ethnicity (Non-White); Parental Attitude (Strongly disapproves).

^***p<.001,

^**p<.01,

^*p<.05

These results provide mixed support for our expectation that the strength of the relationship between survey mode and smoking reports would generally increase as the social acceptability of the particular behaviors decreased, although somewhat stronger effects were predicted for past-month rather than past-year smoking. In light of the absence of a mode effect for lifetime smoking, no further analyses will be reported for that variable.

In addition to the mode effect, age and race/ethnicity had significant associations with reports of past-year and past-month smoking. Controlling for other variables, older respondents were more likely than younger ones to report both past-year and past-month smoking and Whites were more likely than non-Whites to report smoking. Reports of past-month smoking differed by gender: boys were more likely than girls to report smoking in the past 30 days. Perceived parental attitude had the greatest association with smoking reports among all independent variables, with youth who believed their parents either wouldn’t mind or would disapprove only “a little” being 5 to 6 times as likely to report smoking than those who believed their parents would disapprove “a lot”.

Since each of the covariates had significant effects on reports of both past-year and past-month smoking, we added the interaction of each one with mode separately to the baseline models. Results are not shown for models where the added interaction term was not statistically significant. The age by mode and parental attitude by mode interactions did not have a significant impact on either of the smoking indicators; however, both gender and race/ethnicity (i.e., non-White versus White) did interact significantly with mode on one of the smoking behaviors. These results led us to further explore the relationship between gender, race/ethnicity, and mode in youth smoking reports. Since the samples of males and females were sufficiently large, we re-ran the logistic regressions for each gender separately, examining the effect of mode while controlling for age, race/ethnicity, and parental attitude toward smoking. Tables 3a and 3b present the odds ratios from logistic regression analysis of past-year and past-month smoking separately for male and female respondents. In addition to models examining the main effects, we also ran additional models with age by mode, race/ethnicity by mode, and parent attitude by mode interactions entered separately. For male respondents, models including only main effects showed no significant effect of mode (see Table 3a, Model 1). Subsequent models showed no significant interaction of age by mode or parent attitude by mode (not shown); however, models that included an interaction of race/ethnicity by mode demonstrated that the impact of mode was significantly different for non-White and White males (see Table 3a, Model 2).

Table 3a.

Predictors of Past-Year and Past-Month Smoking among Male Adolescents in Massachusetts in 2000/2001

Independent Variables	Males (N =1957)
	Smoked in Past Year				Smoked in Past Month
	Model 1		Model 2		Model 1		Model 2
	Adj. OR	95% CI	Adj. OR	95% CI	Adj. OR	95% CI	Adj. OR	95% CI
Mode (T-ACASI)	1.13	0.83, 1.53	0.36*	0.15, 0.87	1.06	0.72, 1.56	0.22*	0.05, 0.86
Age Group (15–17)	5.61***	3.93, 8.01	5.71***	3.99, 8.16	5.00***	3.09, 8.09	5.09***	3.14, 8.24
Race/Ethnicity (White)	1.47*	1.02, 2.11	1.06	0.70, 1.59	1.38	0.87, 2.18	0.95	0.58, 1.57
Parent Attitude (Doesn’t strongly disapprove)	5.71***	4.13, 7.91	5.73***	4.13, 7.96	5.74***	3.95, 8.34	5.75***	3.94, 8.37
Race/Ethnicity X Mode	---		3.83**	1.50, 9.78	---		6.29*	1.48, 26.79
Constant	.03	---	.04	---	.02	---	.02	---

Note: Each predictor adjusted for all others in table. Reference categories for predictors: Mode (Interviewer-administered); Age Group (12–14); Sex (Female); Race/Ethnicity (Non-White); Parental Attitude (Strongly disapproves).

^***p<.001,

^**p<.01,

^*p<.05

Table 3b.

Predictors of Past Year and Past Month Smoking among Female Adolescents in Massachusetts in 2000/2001

Independent Variables	Females (N =1887)
	Smoked in Past Year				Smoked in Past Month
	Model 1		Model 2		Model 1		Model 2
	Adj. OR	95% CI	Adj. OR	95% CI	Adj. OR	95% CI	Adj. OR	95% CI
Mode (T-ACASI)	1.96***	1.47, 2.61	2.19***	1.60, 3.01	1.72**	1.21, 2.43	2.26***	1.52, 3.37
Age Group (15–17)	3.72***	2.75, 5.04	3.74***	2.76, 5.06	3.63***	2.48, 5.32	3.68***	2.51, 5.39
Race/Ethnicity (White)	1.56**	1.09, 2.22	1.55*	1.09, 2.21	1.56*	1.02, 2.39	1.55*	1.01, 2.38
Parent Attitude (Doesn’t strongly disapprove)	5.43***	3.91, 7.53	6.36***	4.36, 9.29	6.30***	4.40, 9.02	8.48***	5.56, 12.95
Parent Attitude X Mode	---		0.54	0.26, 1.13	---		0.34**	0.15, 0.77
Constant	.04	---	.03	---	.02	---	.02	---

Note: Each predictor adjusted for all others in table. Reference categories for predictors: Mode (Interviewer-administered); Age Group (12–14); Sex (Female); Race/Ethnicity (Non-White); Parental Attitude (Strongly disapproves).

^***p<.001,

^**p<.01,

^*p<.05

Among female respondents, the mode effect was significant in the initial model, indicating that girls were from 1.7 to 2 times more likely to report past-year and past-month smoking in T-ACASI than in interviewer-administered mode. Subsequent models showed no significant interaction between age and mode or race/ethnicity and mode (results not shown), but the parental attitude by mode interaction did have significant effect on reporting of smoking in the past month (see Table 3b, Model 2).

In examining models with interactions, the odds ratio for mode can be interpreted as the effect of mode for the referent group. For example, in the analysis of females’ reports of past-month smoking, the odds ratio of 2.26 for mode means that girls who reported that their parents would disapprove strongly of their smoking (the referent group) were 2.26 times more likely to report past-month smoking in the T-ACASI than in the interviewer-administered mode.

The odds ratio for the remaining group (i.e., girls whose parents would not disapprove strongly) can be obtained by setting this group as the referent (Jaccard, 2001). Table 4 shows the resulting adjusted odds ratios from the models where a significant interaction by mode had been found for one of the outcomes. This table clarifies that the effect of survey mode on youth reporting of smoking was greatest for adolescent girls who believed that their parents would strongly disapprove of their smoking. These girls were more than twice as likely to report smoking in the past year and in the past month in T-ACASI mode than in interviewer-administered mode. The reports of smoking by girls who believed that their parents would not disapprove strongly was not significantly affected by survey mode.

Table 4.

Adjusted Odds Ratios for Impact of T-ACASI Mode Relative to Interviewer-Administered Mode on Adolescents’ Reports of Past-Year and Past-Month Smoking for Selected Subgroups

Subgroup	Past-Year Smoking		Past-Month Smoking
	Adj. OR	95% CI	Adj. OR	95% CI
Male and White	1.39	1.00, 1.95	1.35	0.89, 2.05
Male and Non-White	0.36*	0.15, 0.87	0.22*	0.05, 0.86
Female and Parents Disapprove Strongly	2.19***	1.60, 3.01	2.26***	1.52, 3.37
Female and Parents Don’t Disapprove Strongly	1.19	0.61, 2.30	0.77	0.38, 1.56

Note: Models for males adjusted for age group and parental attitude. Models for females adjusted for age group and race/ethnicity.

^***p<.001,

^*p<.05

Survey mode did not have a significant impact on White boys’ reporting of smoking. Among non-White boys, however, mode had a paradoxical effect: They were more likely to report both past-year and past-month smoking to a human interviewer than in T-ACASI mode. The bivariate cross-tabulations presented in Tables 5a and 5b illustrate, respectively, the interaction between race/ethnicity and mode among male adolescents and the interaction between parental attitude and mode among female adolescents.

Table 5a.

Smoking Behavior by Race/Ethnicity and Survey Mode (Males Only)

Smoking Behavior	White Males		Non-White Males
	T-ACASI (n=413)	Interviewer-Admin. (n=1089)	T-ACASI (n=107)	Interviewer-Admin. (n=299)
Smoked in Lifetime	31.7	28.0	21.6	32.5*
Smoked in Past Year	17.3	14.5	6.0	13.6*
Smoked in Past Month	9.6	8.0	1.7	8.4*

Note: Associations tested by Chi Square.

^***p<.001,

^**p<.01,

^*p<.05

Table 5b.

Smoking Behavior by Parental Attitude and Survey Mode (Females Only)

Smoking Behavior	Parent Disapproves Strongly		Parent Doesn’t Disapprove Strongly
	T-ACASI (n=445)	Interviewer-Admin. (n=1209)	T-ACASI (n=49)	Interviewer-Admin. (n=178)
Smoked in Lifetime	28.0	24.4	68.0	68.8
Smoked in Past Year	17.8	9.9***	49.0	45.6
Smoked in Past Month	10.6	5.3***	31.4	37.1

Note: Associations tested by chi-square.

^***p<.001

^**p<.01

^*p<.05

In order to see whether the use of T-ACASI resolves some of the discrepancy between telephone and school-based self-administered surveys, we retabulated reports of past-month smoking for a subset of youth who matched, as closely as possible, those in the 2001 YRBS Massachusetts survey that is, youth currently enrolled in grades 9 through 12. Table 6 compares the estimates derived from the YRBS survey (USDHHS, 2003) and from the two experimental conditions in the present telephone survey. This comparison shows that for males, the school-based YRBS estimate (25%) is approximately twice that of the two telephone estimates (11.5% and 12.3%). For females, the YRBS estimate (27%) is about twice that of the interviewer-administered estimate (12.5%), but only 1.5 times that of the T-ACASI estimate (17.4%).

Table 6.

Past Month Smoking among Massachusetts Youth Grades 9–12 by Mode

Survey Mode	Males	Females
Self-administered in school (YRBS, 2001)	25.0 %	27.0 %
T-ACASI	12.3 %	17.4 %
Telephone Interviewer-Administered	11.5 %	12.5 %

Discussion

The primary expectation of this study was that T-ACASI would elicit higher, and presumably more accurate, estimates of smoking prevalence than standard interviewer-administered telephone interviews by reducing youths’ concerns about reporting a socially unacceptable behavior in a setting where parents might overhear their responses. These findings confirmed our expectations in general, but demonstrated quite strikingly that the impact of the greater privacy afforded by T-ACASI is not the same for all adolescents. The impact of privacy afforded by T-ACASI appears to vary in importance as a function of gender, and within gender as a function of race/ethnicity (for boys) and of perceived parental attitude (for girls). Girls, regardless of race/ethnicity, seem more likely to report smoking if they can do so by pushing a button on their touch-tone phone rather than by providing answers aloud to a human interviewer. Indeed, our results suggest they are twice as likely to do so if they are concerned that their parents would disapprove strongly should they learn about the smoking. Most adolescents (over 85% of our respondents) do believe that their parents would strongly disapprove if they learned that their son or daughter smoked. This means that among the vast majority of adolescent girls, traditional telephone interviewing methods might underestimate smoking prevalence by as much as 50%. Other research has shown that girls report more parental monitoring than boys (Webb, Bray, Getz, and Adams, 2002). We suspect that girls’ sensitivity to this monitoring may underlie the stronger impact of T-ACASI on their reporting of smoking.

Overall, White adolescent boys do not report their smoking differently based on interview mode. Non-White boys, however, were more likely to report past-year and past-month smoking to a human interviewer than in T-ACASI mode. This finding is surprising, given that previous findings indicated that non-White respondents demonstrated greater underreporting of substance use in less private survey modes (Aquilino, 1994). In light of the relatively weak effect (p<.05) in the context of multiple statistical comparisons in this study, we think this finding should be viewed with caution until it is replicated in future research. Given this unexpected result, researchers should further investigate the role of survey mode and race/ethnicity in adolescent boys’ reports of smoking and other sensitive behaviors.

These findings advance our understanding of both the accuracy of youth reports of smoking in household surveys and the role of computer-assisted survey techniques in improving reporting of socially unacceptable behaviors. On the first issue, our results support research indicating that standard household surveys administered by interviewers seem likely to suppress girls’ reports of their smoking behavior (Fowler and Stringfellow, 2001; Gfroerer, Wright, and Kopstein, 1997). More generally, these findings are also consistent with research demonstrating that adolescents generally report more substance use in survey modes that offer greater privacy (Supple, Aquilino, and Wright, 1999). This study also expands our knowledge of why estimates of adolescent smoking on telephone surveys tend to be lower than those on school-based self-administered surveys. Among girls, our findings suggest that concern about parental monitoring suppresses reports of smoking in traditional telephone surveys. If so, either this concern was not totally alleviated in the T-ACASI mode, or other factors are at work that promote greater reporting of smoking by both girls and boys in school-based surveys.

The comparison of estimates derived from our telephone survey with those obtained by the 2001 Massachusetts school-based YRBS survey indicates that T-ACASI mode reconciles some, but not all of the difference for girls between the YRBS and interviewer-administered telephone surveys. As Table 6 shows, the school-based estimates are still substantially higher.⁸ Our data do not allow us to determine whether T-ACASI or school-based estimates are more accurate. Youth reports of smoking behaviors in T-ACASI mode might still include some degree of underreporting, while reports in school-based surveys might include some overreporting of smoking behaviors. Differences in item wording, nonresponse, and the context in which questions on smoking are placed may also account for some of the discrepancy. Differences in sample composition (i.e., the noncoverage in school surveys of youths who do not attend school) would tend to exacerbate the discrepancy between telephone and school-based surveys because these populations have been shown to have substantially higher smoking rates than school attenders (Pirie, Murray, and Leupker, 1988).

Future research should include designs that allow investigators to further disentangle how different survey protocols affect youth reports of smoking behavior. Nevertheless, this research demonstrates that T-ACASI methodology is acceptable to youth, and may provide more accurate estimates of smoking in telephone surveys, particularly among adolescent girls.