Response Distortion in Adolescents Who Smoke: a Pilot Study

Abstract

A wide range of data are obtained with self-report. Information obtained from persons using substances is generally reliable and valid, however, many studies show that some proportion of self-reports regarding substance use are inaccurate. This study examines self-reported response distortion in adolescents who received a brief intervention to reduce their smoking. Findings indicate that age and ethnicity of respondent may influence reported response distortion. Factors that appear to influence under- and over-reporting include social desirability and fear of repercussion. Response distortion does not appear to affect rated usefulness of the intervention, nor does intervention type appear to influence whether respondents thought the researcher wanted them to report less use. Results point to the need for further research regarding adolescents and response distortion.

While the information obtained from persons using substances is generally reliable and valid, many studies show that some proportion of self-reports regarding substance use are inaccurate [1, 2]. Similarly, while self-reports of smoking are generally accurate, research indicates that some individuals do misreport their smoking behaviors [3]. Patrick, Cheadle, Thompson, et al. conducted a meta-analysis of 26 studies containing comparisons between self-reported smoking and biochemical assessments [3]. Results suggested that interviewer-administered questionnaires yield higher overall correct classification accuracy than self-administered questionnaires [3]. In addition, participants in interventions studies are more likely to under-report their smoking as compared to participants in observational studies and, as compared to respondents from the general population, minors appear more likely to deny smoking when biochemical tests classify them as smokers [3].

Other research also suggests that sample characteristics play a role in misreported smoking. When adults in a community intervention trial were surveyed by telephone regarding smoking cessation, the extent of false-reported quitting, assessed using saliva cotinine, ranged from 5 to 9 percent [4]. Similarly, when adults were given a low intensity smoking intervention, 4 to 5 percent who reported giving up smoking failed biochemical confirmation [5]. In one study [6], rates of under-reported smoking ranged from 0.6 to 8 percent for persons engaged in community, occupational, or health screening surveys; 13 to 28 percent among smokers seeing a physician for smoking related illness; and 15 to 36 percent among participants completing treatment for smoking. Velicer, Prochaska, Rossi, et al. point out that accuracy of self-report is affected by type of population studied (volunteers vs. high risk medical patients), type of intervention (minimal vs. intensive), and demand characteristics (whether participants know self-report is subject to biochemical validation) [7].

Palmer, Dwyer, and Semmer found that elevated cigarette use among high school students who scored high on a scale of rebelliousness and deviance was due in part to an over-reporting bias [8]. Although over-reporting does occur, it probably occurs less frequently than under-reporting. In one study, students were surveyed at school without a participating family member and at home regarding cigarette, marijuana, and alcohol use [9]. Answers were generally consistent across settings. However, differences that were found for each setting suggested that adolescents under-reported their use of liquor, cigarettes, and beer for the last 30 days when the survey was conducted at home. Similarly, when substance-abusing adolescents deny their use to a physician, they often report doing so because a parent was present [10]. These studies suggest adolescents might under-report to avoid reprimand.

In a 1998 survey in the United Kingdom, 7 percent of adolescents who initially indicated they never smoked, later in the survey admitted to having done so [11]. In addition, 68 percent of those who said they usually smoked one to six cigarettes per week had recorded in a diary that they smoked seven or more cigarettes over the past week [11]. Among adolescents who said they smoked less than one cigarette per week, 21 percent smoked on average one or more cigarettes per day in the previous week according to diary reports [11]. These authors indicated that such inconsistencies are unlikely a result of deliberate distortion, but rather may result from the erratic and experimental smoking behavior of adolescents.

Smoking may also be underestimated among young adult African Americans as compared to young adult Whites [12]. Bauman and Ennett surveyed adolescents, ages 12 through 14, whose self-reported tobacco use was validated with expired CO or saliva cotinine [13]. More African-American than White teens said they did not use tobacco when a biochemical test indicated they did [13]. In a similar study, Wills and Cleary also found that more minority than White teens said they did not smoke when a biochemical test indicated they did [14]. Reasons as to why ethnicity is related to under-reported smoking are unclear, and may be related to distrust of research in general or perceived differences between the respondent and interviewer.

Although adolescent self-reports of smoking are generally valid [15], studying variables that affect response distortion specifically in adolescents is important because of the unique role that developmental effects may have for adolescents. Adolescents pose special challenges for assessment due to immature cognitive development and identity formation [16]. Also, adolescents may be particularly vulnerable to the power differential between themselves and adult professionals requesting information. One potential result of such a situation is for an adolescent to respond in a way that she or he feels would please the examiner [17]. Misreports of smoking may be more frequent and differently motivated in adolescents as compared to adults [3, 7]. In addition, the strategies utilized by adolescents to purposely under- or over-report smoking may differ between adults and adolescents. Factors that may contribute to such differences between adolescents and adults include fear of repercussions for smoking (since it is illegal for minors) and adolescents having less experience in effective deception. Although Velicer et al. minimize the impact of under-reporting on research conclusions in general, these authors specifically point out that a greater tendency to misreport in no- or low-intensity interventions studies might occur with adolescents [7, 18].

As indicated in the studies noted above, misreporting may affect prevalence estimates in general population surveys and in the evaluation of treatment outcome trials. In addition, misreported smoking may also directly impact upon effectiveness of treatment. No studies to date have examined the effects of misreporting on treatment effectiveness in adults or adolescents. Effects of misreporting on treatment effectiveness may be particularly germane to interventions that utilize assessment feedback (for example, Motivational Interviewing developed by Miller and Rollnick [19]; the Expert System intervention developed by Velicer, Prochaska, Bellis, et al. [20]).

Given the concerns raised above, this pilot study examined variables associated with self-reported response distortion in adolescents. Participants were part of a larger study evaluating the effectiveness of a brief intervention for adolescent smoking. One purpose of this pilot study was to determine rates of under- and over-reporting. A second purpose was to examine reasons why adolescents under- and over-report behaviors. Third, we examined characteristics of adolescents who misreport. The final purpose was to determine the impact of misreporting on adolescents' perceptions of the usefulness of treatment. Upon direct inquiry, adolescents do admit to over- and under-reporting [10, 21, 22]; however, few studies have asked adolescents directly about the truthfulness of their reports. This technique was utilized in an effort to examine conscious and willful response distortion. While some factors associated with untruthfulness are not changeable, others might be modifiable by interviewers and this could lead to methods to increase truthful reporting by teens.

Methods

Participants

Eighty-five participants were recruited from a hospital outpatient clinic and emergency department. Twenty-one of these participants completed the study before we began tracking misreported responses. Of the 64 remaining participants, two withdrew from the study, 10 adolescents could not be contacted at follow-up, and data for one participant were lost due to researcher error. The final study sample was comprised of 51 adolescents. All received medical care prior to being approached by research staff. Reasons for visiting the hospital included routine checkup or physical, vaccination, illness (such as a cold or asthma), and treatment of an eating disorder. None of the patients were seeking smoking treatment. Patient smoking status was screened by medical or research staff in a confidential setting. Patients were eligible if they were 12 to 19 years old and if they reported smoking one or more cigarettes per day for the last 30 days. The study was introduced as a project that would help us learn more about adolescent smokers. Adolescents were told that they did not have to be interested in quitting to participate. Procedures were briefly described; if an adolescent was interested, then informed consent was obtained from adolescent and parent (parent consent not obtained for adolescents over 18 years old). Adolescents and parents received monetary compensation for their participation. Table 1 summarizes the demographic and smoking characteristics of the final sample (N = 51).

Table 1.

Sample Characteristics (N = 51)

Characteristic	M	SD	Percent
Agea	16.39	1.63	—
Number of cigarettes/day	13.20	9.82	—
FTQ total	5.82	1.74	—
Number of years smoked	3.28	2.03	—
School gradesb	4.24	2.17	—
Ethnicity
White	—	—	52.9
African American	—	—	9.8
Hispanic	—	—	25.5
Other	—	—	11.7
Gender
Female	—	—	70.6
Male	—	—	29.4

^aAge is in years.

^bSchool grades are on a 1 (A's) to 9 (F's) scale.

Measures

Demographics Questionnaire

Parents completed this self-report questionnaire at baseline. It included information such as age, gender, average school grades, and ethnic origin of the adolescent. Parents completed this questionnaire without the adolescent present.

Fagerstrom Tolerance Questionnaire (FTQ)

This questionnaire measures nicotine dependence, and was administered to adolescents at baseline and at 1-, 3-, and 6-month follow-ups [23]. It provides information about the number of cigarettes smoked per day, nicotine level, and minutes to first cigarette upon waking. Information from this questionnaire was used to provide feedback to adolescents (see Procedures). Questions were interviewer-administered.

Biochemical Measures

Saliva cotinine and expired carbon monoxide levels were collected to encourage honest reports of smoking status at baseline and at 1-, 3-, and 6-month follow-ups. Expired CO was also used to provide feedback to adolescents. Adolescents were informed at the outset of the study that biochemical tests would be used as another indicator of cigarette use.

Prevalence of Cigarette and Other Substance Use

At baseline, Timeline Followback (TLFB) was used to assess cigarette, alcohol and drug use during the previous 30 days [24]. At baseline, 1-, 3-, and 6-month follow-ups, adolescents were also asked the number of days during the past month they had used cocaine, opiates, and hallucinogens. At 1-, 3-, and 6-months, adolescents were asked the number of cigarettes smoked each day during the past week. These questionnaires were interviewer-administered.

Therapist Adherence Questionnaire

After the intervention at baseline, adolescents in the Motivational Intervention condition (described below) filled out this questionnaire that assessed the delivery and perceived usefulness of various components of the intervention. Adolescents rated, for example, the usefulness of receiving information on the amount of money spent in purchasing cigarettes, how their use compares to peers, their dependence symptoms, and their levels of exhaled CO. This paper-pencil questionnaire was self-administered by the adolescent. Participants were assured that their answers would not be seen by the interviewer, and adolescents placed the completed questionnaire in a sealed envelope.

Exit Questionnaire

This questionnaire was self-administered at the final 6-month follow-up. Again, participants were assured that their answers would not be seen by the interviewers, and adolescents placed the completed questionnaire in a sealed envelope. The Exit Questionnaire made a variety of inquiries including how important it was to the adolescent that the interviewer like him or her, how frequently adolescents under- or over-reported their behaviors in general during the study, and open-ended questions regarding why adolescents under- or over-report smoking.

Procedures

Face-to-face interviews and telephone contacts were utilized to gather data and administer treatment. Interviews were conducted in private without parent present. Confidentiality of information was emphasized during all sessions. The 1-, 3-, and 6-month follow-ups were primarily conducted in the hospital setting.

Following the 45-60 minute baseline assessment that was conducted in the hospital, patients were randomly assigned to either Motivational Intervention (MI) or Standard Care (SC). The MI was consistent with the central principles of MI as described by Miller and Rollnick [19]. Essential elements of the MI session included: an empathic, non-confrontational style; an emphasis on client responsibility and choice; individualized feedback; a menu of goal alternatives; advice to change; acceptance of the individual's level of readiness to change; and an emphasis on increasing self-efficacy [25]. For more details regarding the intervention, see Colby, Monti, Barnett, et al. [26]. One week after the initial session, adolescents received a 15-minute phone booster session from the interventionist. During the booster, progress toward goals for changing smoking behaviors that were set during the MI were assessed.

After completing the baseline assessment, adolescents randomized to the SC intervention received brief advice to quit smoking and encouragement to get assistance as needed. The brief advice was consistent with clinical practice guidelines for smoking cessation intervention [27]. At the 1-week booster session, interventionists called adolescents to remind them of their next follow-up appointment.

Baseline interviews, interventions, and phone booster sessions were conducted by interventionists, whereas follow-up contacts were conducted by research assistants blind to treatment condition. Research staff were predominately White (one was Hispanic and one was Asian). Four men and six women had direct contact with adolescents and parents. All research staff received at least one month of training and weekly supervision throughout the study.

Coding Free Responses

In a free response format (Exit Questionnaire), adolescents were asked why they or other adolescents might under- or over-report smoking. Free responses were then sorted separately by two independent raters into categories. One rater received her Psy.D. in clinical psychology, and had 10 years of research and four years of clinical experience in substance abuse. The second rater received his Ph.D. in clinical psychology, and had five years of research and three years of clinical experience in substance abuse. Disagreements between raters were resolved by the first author after examination of the content area of the items. The purpose of the first set of raters was to obtain categories and membership of items into categories.

A second set of independent raters was asked to use the response categories previously generated to classify responses participants gave when asked why adolescents under- or over-report smoking. One rater received her Ph.D. in clinical psychology, and had five years of research and six years of clinical experience in substance abuse. Another rater received his Ph.D. in anthropology, and had four years of research and one year of clinical experience in substance abuse. The purpose of the second set of independent raters was to provide information on level of agreement when sorting items into given categories.

Results

Table 2 indicates the reporting patterns of adolescents. Although no adolescent indicated consistent over-reporting, about 4 percent of participants indicated that they consistently under-reported their behaviors. A majority of adolescents reported being truthful during the entire study; however, almost 16 percent under-reported part of the time, nearly 4 percent over-reported sometimes, and about 4 percent both under- and over-reported their behaviors during the study. Although about 27 percent misreported overall, nearly 33 percent of all participants considered misreporting on their responses to inquiries (see Table 3). About 41 percent of adolescents thought the interviewer wanted them to under-report cigarette use and about 25 percent of adolescents were concerned that information would be shared with parents.

Table 2.

Reporting Patterns of Participants (N = 51)

Pattern	Percent
Always over-reported my behaviors	0.0
Sometimes over-reported my behaviors	3.9
Both under- and over-reported my behaviors	3.9
Sometimes under-reported my behaviors	15.7
Always under-reported my behaviors	3.9
Was honest throughout study	70.6
No answer	2.0

Table 3.

Percent of Participants Endorsing Exit Interview Items (N = 51)

Item	Yes (%)	No (%)	No answer (%)
Important that interviewer like me.	19.6	78.4	2.0
Interviewer wanted me to report less use.	41.2	54.9	3.9
Thought there were times when I shouldn't be honest.	33.3	64.7	2.0
Concerned information would be shared with parents.	25.4	66.7	7.8
Concerned information would be shared with others (not parents).	23.5	74.5	2.0
Parents wanted to know about the project.	41.2	51.0	7.8

Table 4 presents the categories of under- and over-reporting that resulted from the free response ratings described earlier. In developing categories of over-reporting, the first pair of independent raters generated identical categories except in one case. One rater developed a Miscellaneous category comprised of three free response items. The first author examined the content area of these items and agreed with the other rater in the placement of these items. The first pair of raters agreed in the placement of 74 percent of items. In developing categories of under-reporting, the first pair of independent raters again generated identical categories except in one case. These raters agreed in the placement of 62 percent of items. The relatively poor item agreement may be attributable in part to the rather odd answers some participants provided when asked why adolescents might under-report smoking (“get a head rush”). After determining categories and determining item membership into categories, a second pair of independent raters sorted the items. As shown in Table 4, this second set of raters had an overall agreement of 82 percent in sorting responses for under-reporting. These raters also sorted responses for over-reporting and achieved 97 percent agreement (see Table 4).

Table 4.

Misreport Categories^a

Category	Number of items		Participants endorsing (%)	Agreement (%)
– Item example
Under-reporting
Fear of addiction	2		6.5	100.0
– Don't want to get addicted.
Manage feelings	5		13.0	72.7
– Feel guilty.
– Feel better about self.
Desire to quit	3		13.0	100.0
– Want to feel like making progress.
– Trying to quit.
Miscellaneous	3		6.5	0.0
– Get a head rush.
Obstinate	2		6.5	100.0
– Just to lie.
Social desirability	16		65.2	76.0
– To fit in.
– To look good.
– Someone might think I'm a bad person.
Little smoking	3		4.3	42.9
– Hardly get any cigarettes.
– Only smoke when smoke weed.
Fear of repercussion	10		45.7	66.7
– Scared to get in trouble.
– Thought project would tell parents.
Overall	44		—	82.1
Over-reporting
Addiction		3	13.0	100.0
– They're addicted.
– Their body needs a cigarette.
Attention		3	13.0	75.0
– To get attention.
– To get help.
Obstinate		4	13.0	100.0
– Adolescents exaggerate any way.
– Just to say it.
Social desirability		13	80.4	100.0
– To impress people.
– Peer pressure.
– To seem tough.
Misappraisal		1	4.3	66.7
– May actually think they smoke a lot when they really don't.
Manage feelings		2	8.7	100.0
– Calms them down.
– Low self-esteem.
Overall		26	—	97.4

^aN = 46 to generate these data. Categories created by two independent raters; agreement rates generated on a second set of independent raters.

Next, we examined demographic differences between adolescents who reported being honest during the study and those who reported being dishonest. Under- and over-reporting were collapsed due to small sample size. No significant effects were found for age, grades in school, or gender. More non-Whites than Whites reported upon direct inquiry that they misreported during the study, with χ² = 5.06, df = 1, N = 50, p < .03, and Effect Size (ES) = 0.32 (medium; in terms of w). Differences between honest and not-honest groups were compared on rated usefulness of treatment components and results were non-significant.

A χ² test was performed to determine if treatment condition influenced whether adolescents thought the researcher wanted them to report less smoking and results were non-significant. Finally, χ² and t-tests were performed to determine whether demographic characteristics were related to adolescents' concerns that information would be shared with parents. Results were non-significant for gender and ethnicity. As compared to older adolescents (M = 16.62, SD = 1.60), younger adolescents (M = 15.38, SD = 1.45 were more likely to be concerned information would be shared with parents, with t = 2.43, df = 45, N = 47, p <.02, ES = .79 (large; in terms of d).

Discussion

Results indicate that in the context of an intervention for adolescent smoking, purposeful misreporting does occur. Self-reported under-reporting was more common than over-reporting, but most common was a claim of truthful reporting. Among factors that may influence under-reporting, pleasing the interviewer and concerns about confidentiality appear to play a substantial role. Such reasons were endorsed by 25 percent to 40 percent of participants. This suggests that in order to improve rates of honest reporting, one method might be to increase participants' beliefs that no one else will be told their responses.

Results also indicate that major reasons adolescents under-report are related to impression management and fears of being reprimanded. In examining Table 4, the miscellaneous response category is rather ambiguous and raters were unable to agree on any of the three items. Similarly, poor agreement was found for the little smoking category, which would suggest removal of this response category. However, it is noteworthy that some respondents are aware that adolescents may honestly underestimate their smoking (because they actually smoke very little). This reflects a type of response distortion known as misappraisal [28], and does not reflect purposeful and willful distortion. This is in contrast to an obstinate response set in which respondents willfully misrepresent themselves simply “to lie.” It is also noteworthy that participants believe adolescents may under-report due to fears of being addicted, to manage feelings (to make themselves feel better), and because they have a desire to quit. Adolescents may be positing different types of “self-deception” [29] or alternate forms of impression management (e.g., “I under-report because I want you to think I'm trying to quit”).

Many of the same types of reasons for under-reporting were also given for over-reporting. Respondents seem to be aware that adolescents may be motivated to seek attention or over-report in an effort to get help for smoking. Items in the addiction response category seem to suggest that adolescents may over-report smoking because they believe they are dependent. This may be another form of misappraisal in that participants believe adolescents may overestimate how much they smoke based on how addicted they perceive themselves to be.

In examining characteristics of adolescents who reported being honest versus those who reported not being honest during the study, results indicate that ethnicity may be a factor. This finding is consistent with results presented by Wagenknecht et al. [12], Bauman and Ennett [13], and Wills and Cleary [14] that suggest smoking is under-reported in young African Americans. In the current study, a medium effect size was found for ethnicity, indicating that Whites were more likely to report being honest than non-Whites during the study. This may be due to cultural effects such as a general distrust for research or undervaluing research. Alternatively, it may be that non-Whites are more honest than Whites about their behaviors upon direct inquiry. It may also be due to ethnic differences between researchers and non-White participants. The role of ethnicity in response distortion deserves further research.

The role of age in misreporting also merits further research, since younger adolescents appear to be more concerned that information may be shared with parents. In the present study, since most respondents who were not honest under-reported (see Table 2), this trend may reflect younger persons being more fearful of repercussions. Believing the researcher wanted adolescents to report less smoking does not appear to be related to type of intervention (MI or SC) received. In addition, misreporting does not appear to affect perceived usefulness of treatment.

Researchers need to address the possibility that self-report data are subject to response distortion. An effort should be made in studies to assess the degree to which such distortion might influence results. Base rates of misreporting need to be established so that assessment instruments can be properly evaluated. Information on characteristics of people who misreports and reasons for misreporting can lead to methods of increasing the validity of self-reports and thereby further scientific endeavors. Clearly, reassuring participants of confidentiality and using biochemical validation does not deter all individuals from distorting responses.

This study was limited by a small sample size. Future studies should attempt to recruit larger sample sizes. This will enhance power and it will also allow examination of effects for specific types of response distortion. For some analyses, the current study dichotomized responses into Honest and Not-Honest. It is important to examine effects for under-reporting separately from over-reporting. Similarly, future studies need to examine effects of random responding, and this response set may have been operating in the present study. Perhaps this explains why some of the free responses appeared irrelevant (“to get a head rush”). Future studies may also seek to determine whether results presented here generalize to non-treatment samples.

Another limitation of the present study is that patterns of over- and under-reporting as shown in Table 2 are not necessarily specific to smoking, since alcohol and other drug use were also queried during assessments. Future studies need to address response patterns for specific behaviors. Also, it is important to establish how participants under- and over-report. For example, they may distort the amount of substance used, the frequency of use, or the problems associated with use. Such questions must be addressed separately for adolescents and adults due to potential effects of cognitive development in ability to effectively distort. Assessment of response distortion applies to many areas of research but it is especially relevant to substance use research. Rates of under-reporting were as high as 20 percent in this smoking study. We might assume that rates of distortion increase with illicit substances such as cocaine. In addition, attention should be given to assessing more “unconscious” or covert styles of response distortion. For example, what factors are involved in “honest” misappraisals? Although the current study did not address this directly, it appears that adolescents believe this is a factor that may operate in inaccurate reports.

Finally, the under- and over-reporting categories developed in examining the free responses of adolescents should be considered tentative at this time. These serve to generate further research in the assessment of misreporting in adolescents. More stringent research is needed to determine whether categories will be replicated in other samples and the exact meaning of such categories.

Despite their believed objectivity, biochemical measures do not provide a gold standard, nor are they perfect measures of accuracy for use in assessing criterion validity [3, 13]. Variations in smoking patterns, environment, and the manner in which cigarettes are smoked can produce erratic and unreliable biochemical results [3, 30]. Biochemical tests may increase drop-out rates, are intrusive, more difficult to obtain than self-report, are expensive, and validate smoking status only near the time of specimen collection [3]. Asking adolescents at the end of a study about conscious and willful response distortion may circumvent some of the difficulties in using biochemical tests. This may be especially true if adolescents believe in the confidentiality of their responses regarding misreporting. Neither biochemical tests, nor making direct inquiries regarding misreporting, is without flaw. Future studies should attempt to access larger samples so that analyses can be conducted to determine the concordance between self-reports and biochemical tests versus self-reported misreports. We view both techniques as useful and complementary.