Concordance Between Cigarette Smoking and the Modified Fagerström Tolerance Questionnaire in Controlled Studies of ADHD

Abstract

Objective

To compare scores on a smoking questionnaire to a diagnosis of cigarette smoking.

Methods

As part of follow-ups in studies of ADHD, we assessed for cigarette smoking using structured interviews and the modified Fagerström Tolerance Questionnaire (mFTQ).

Results

Data were obtained from 162 subjects (mean=19.2 yrs). ROC analysis and kappa coefficients revealed that a cutoff score of three on the mFTQ showed the strongest agreement with a full diagnosis of cigarette smoking (kappa=0.68).

Conclusion

Clinicians and researchers using the mFTQ in adolescents and young adults should consider a cutoff score of 3 to be indicative of cigarette smoking.

Introduction

Cigarette smoking among adolescents and young adults has long been recognized as a serious public health problem¹. In addition to the direct health effects, cigarette smoking may represent an early stage in the developmental sequence into illicit drug use ^{2, 3}. For example, we recently found adolescent smoking to be associated with a significantly higher risk for the subsequent abuse of alcohol and drugs - particularly in youth with Attention-Deficit/Hyperactivity Disorder ⁴.

Currently, common measures for cigarette smoking are the modified versions of the original 8-item Fagerström Tolerance Questionnaire (FTQ)⁵, including the 7-item modified FTQ (mFTQ)⁶ and the Fagerstrom Test for Nicotine Dependence (FTND)⁷. These instruments provide dimensional methods to measure the degree of physical dependence to nicotine. The self-report mFTQ is an appealing measure because of its widespread use, quantitative determination of nicotine dependence, and validation with objective ancillary information (ie. salivary cotinine levels)⁸. By generating a score, the mFTQ reports the level of smoking severity, which quantifies dependence and can be used to compare individual differences or study samples. In contrast, a clinical diagnosis provides categorical evidence of cigarette smoking. While both methods overlap in some areas, they also appear to measure unique aspects of smoking dependence. For example, our diagnosis of cigarette smoking and the mFTQ both measure the number of cigarettes smoked, but the mFTQ provides evidence mainly for physical dependence.

To date, a cutoff score on the mFTQ for identifying nicotine dependence in adolescents and young adults has not been established. A prior study showed little agreement between the FTND (the 10-point version of the FTQ) and the Diagnostic and Statistical Manual of Mental Disorders, third-revised (DSM-III-R)⁹ at any cutoff score, with highest concordance at a cutoff of 7 (kappa=0.21) ¹⁰. While scores of 3–5 on the mFTQ are considered indicative of moderate nicotine dependence ⁶, findings from Kandel et al. (2005)¹¹ showed that using a score of 4 or greater on the mFTQ identifies significantly fewer dependent smokers than the DSM. These findings lead us to speculate that using a cutoff of 4 may not be sensitive enough to detect cigarette smokers, especially among lighter, adolescent and young adult smokers.

In this paper, we compare the mFTQ scores as predictors of cigarette smoking in a well-characterized group of adolescents and young adults with and without ADHD. We sought to expand on the existing literature by evaluating each individual score on the mFTQ, rather than using a predetermined cutoff. Based on the literature, we hypothesized that a cutoff score less than 4 on the mFTQ would show the strongest concordance with our diagnosis of full cigarette smoking.

Methods

Subjects

Subjects were derived from two longitudinal case-control family studies of ADHD described in detail elsewhere^12–14. At baseline, we studied female and male subjects aged 6–18 years with ADHD (Boys study: N=140; Girls study: N=140) and without ADHD (Boys study: N=120; Girls study: N=122) ascertained from pediatric and psychiatric sources. Potential subjects were excluded if they had been adopted, if their nuclear family was not available for study, if they had major sensorimotor handicaps (e.g., paralysis, deafness, blindness), psychosis, autism, inadequate command of the English language, or a Full Scale IQ less than 80.

The present study reports on the 5-year follow-up of the female subjects and the 10-year follow-up of the male subjects, which occurred simultaneously during the same five-year time period (Boys study: N=217, Girls study: N=235). Since our subjects spanned two gender-based studies with different mean ages, we restricted our sample to subjects within overlapping age ranges, in order to minimize collinearity of age and gender. We truncated the sample age distribution by eliminating 65 female subjects aged 10–14 and 46 male subjects aged 26–31. From these samples, only subjects participating after the first year of the five-follow-up period received the mFTQ (it was amended into the studies after the first year). Parents and adult offspring provided written informed consent to participate, and parents also provided consent for children under the age of 18. Children and adolescents provided written assent to participate. The human research committee at Massachusetts General Hospital approved this study protocol.

Clinical Assessments

For the follow-up data presented here, subjects 18 years of age or older received the Structured Clinical Interview for DSM-IV (SCID)¹⁵; subjects under 18 years received the Schedule for Affective Disorders and Schizophrenia for School-Age Children (Epidemiologic Version; KSADS-E) ¹⁶. We conducted direct interviews with subjects and indirect interviews with their mothers and considered a disorder positive if diagnostic criteria were unequivocally met in either interview. Cigarette smoking was assessed by supplementing the module from the Diagnostic Interview for Children and Adolescents¹⁷. Cigarette smoking was defined by criteria described previously¹⁸ and classified into one of three categories of smokers (non-smokers, sub-threshold smokers and full smokers). For adults 18 and older, smoking one pack or more per day at least four times a week was defined as full smoking. Sub-threshold was coded for smoking any amount per day, more than three times a week. For subjects under 18 years of age, any amount of smoking every day for at least one month was defined as full smoking. Smoking was coded sub-threshold for children if the frequency was greater than three times per week. All structured clinical interviews were presented to a review committee of senior psychologists and psychiatrists for final diagnosis.

The raters were blind to the clinical status of the subject and all prior assessments. The raters had undergraduate degrees in psychology and were trained to high levels of inter-rater reliability. First, they underwent several weeks of classroom style training, learning interview mechanics, diagnostic criteria and coding algorithms. Then, they observed interviews conducted by experienced raters and clinicians. They subsequently conducted at least six practice (non-study) interviews and at least two study interviews while being observed by senior interviewers. A senior investigator supervised the interviewers throughout the study. We computed kappa coefficients of agreement by having experienced, board certified child and adult psychiatrists and licensed clinical psychologists diagnose subjects from audio taped interviews. Based on 500 assessments from interviews of children and adults, the median kappa coefficient was .98. Kappa coefficients for individual diagnoses included: ADHD (0.88) and nicotine dependence (1.0).

For demographic variables, socioeconomic status (SES) was measured using the 5-point Hollingshead scale ¹⁹, where one represents the most affluent, upper-class and five represents the least affluent, lower-class. Family intactness was recorded as a binary variable indicating if a subject’s parents had ever been divorced or separated.

Fagerstrom Tolerance Questionnaire

The 8-item questionnaires were mailed out in a packet of study forms as subjects completed the forms at home and returned them on the day of their assessment. The FTQ generates a score based on the sum of the 8 questionnaire items, which are variably weighted. The questionnaire was scored according to the 7-item mFTQ (does not include the question “What brand of cigarettes do you smoke?”). A higher mFTQ score indicates a greater degree of physical dependence to nicotine (range: 0–9)⁵.

Statistical Analysis

We modeled mFTQ score with linear regression using dummy variables for smoking categories (non-smokers, sub-threshold smokers, and full smokers). Wald’s test was used to evaluate differences between the smoking categories. Logistic regression was used to model current full cigarette smoking as a function of mFTQ score. A Receiver Operating Characteristic (ROC) analysis was also performed and curves plotted. All tests were two-tailed with an alpha-level of 0.05 to assert statistical significance. We calculated all statistics using STATA 9.0²⁰.

Results

Overall we assessed 341 subjects and obtained mFTQ data and structured interview data from 162 probands between the ages of 15 and 25 years. Subjects who did not complete and return the mFTQ (N=175) or who had missing cigarette smoking interview information (N=4) were dropped from analyses. There were no significant differences between included and excluded subjects in age, ethnicity, socioeconomic status, family intactness, lifetime GAF score or rates of smoking dependence (all p’s NS). The dropped subjects were significantly different based on sex (Included: 40% male, Excluded: 59% male, χ²₍₁₎=12.4, p<0.001). Demographic features of the sample are presented in Table 2.

Table 2.

Demographic features of sample (N=162)

	Mean ± (SD)
Age	19.2 ± 2.7
Socioeconomic Status	1.78 ± 1.0
Gender	N ( %)
Male	67 (40)
Female	99 (60)
Caucasian	155 (93)
Family Intactness	106 (64)

Mean mFTQ Scores

Mean mFTQ scores stratified by cigarette smoking diagnosis are depicted in Figure 1. There was an overall positive linear relationship between smoking diagnosis and mFTQ score (F_(2,159)=140.3, p<0.001) indicating mean scores were greater as the severity of the diagnosis increased. Post-estimation analyses revealed significant differences in mean mFTQ scores between sub-threshold and full smoking groups, non-smoking and sub-threshold groups and between full smoking and non-smoking groups.

Figure 1.

Mean mFTQ scores for Cigarette Smoking diagnoses. Differences between groups: non-smokers vs. sub-threshold smokers, wald test: F_{(1, 159)}=37.5, p<0.001; sub-threshold vs. full smokers: F_{(1, 159)}=15.7, p<0.001; non-smokers vs. full smokers: F_{(1, 159)}=261.8, p<0.001.

In reference to the extent of dependence, 8 of 26 (31%) subjects diagnosed as full smokers reported that they smoke when they so ill that they are in bed most of the day (none of the sub-threshold or non-smokers reported this). Also, 14 (54%) full smokers had at least one biological parent who smokes (20% of sub-threshold smokers, 22% of non-smokers).

ROC Curve

We then examined the operating properties of the mFTQ relative to a full smoking diagnosis. There were no significant effects on the association between the mFTQ and cigarette smoking due to ADHD (p=0.26) or sex (p=0.68). An ROC curve for full smoking predicted by mFTQ scores is illustrated in Figure 2. The area under the ROC curve was 0.93 indicating that there is a 93% chance that the mFTQ score of a randomly selected full smoker will be greater than that of a randomly selected non-full smoker. The mFTQ cutoff scores (identifying subjects as full smokers when greater than or equal to that score) are labeled as a reference.

Figure 2.

ROC curve comparing mFTQ cutoff scores to full cigarette smoking. mFTQ scores are given as a reference (1–7).

Cutoff mFTQ scores

Kappa coefficients, sensitivity, specificity, positive predictive values and negative predictive values are displayed at each mFTQ cutoff score assessing full cigarette smoking (Table 3). The sensitivity is the proportion of subjects meeting full smoking criterion that were correctly identified by an mFTQ score greater than or equal to the cutoff score. The specificity is the proportion of non-full smokers that were correctly identified by an mFTQ score less than the cutoff score. The positive predictive value indicates the proportion of subjects positively identified by the mFTQ who actually met for full smoking. The negative predictive value signifies the proportion of subjects negatively identified by the mFTQ that were actually not full smokers. By examining kappa coefficients for levels of concordance, full cigarette smoking shows the strongest agreement with the mFTQ at a score of 3 or greater.

Table 3.

Kappa coefficients, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of cutoff scores on the mFTQ used assess full cigarette smoking. The cutoff at ≥3 is bolded to indicate the greatest kappa value for concordance.

mFTQ Cutoff Score	Kappa	Sensitivity	Specificity	PPV	NPV
≥1	0.58	92%	84%	52%	98%
≥2	0.60	88%	86%	55%	98%
≥3	0.68	81%	93%	68%	96%
≥4	0.63	58%	98%	83%	92%
≥5	0.61	50%	99%	93%	91%
≥6	0.43	31%	100%	100%	88%
≥7	0.23	15%	100%	100%	86%

Comparison between adolescents and adults

Albeit limited by small sample sizes, we examined the effect of age groups (15–18 years and ≥18–25 years) on the mFTQ and cigarette smoking concordance. The area under the ROC curve was 0.99 for subjects under 18 and 0.89 for subjects over 18, indicative of a strong overall agreement in each age range. However, we found some notable differences in concordance. In the younger subjects (N=52; 10 full smokers, 1 sub-threshold), the strongest agreement was shown at an mFTQ cutoff score between 2 and 3 (2: kappa=0.89, 3: kappa=0.88); whereas among the older subjects in the sample (N=110; 16 full smokers, 9 sub-threshold), the strongest agreement was shown at a score between 3 and 4 (3: kappa=0.58, 4: kappa=0.59).

Discussion

The results of our analyses show that a good concordance exists between cigarette smoking criteria and the mFTQ in older adolescents and young adults. The findings from our data support our hypothesis that the mFTQ and cigarette smoking did not demonstrate the highest levels of concordance at a cutoff score of 4; rather, an mFTQ of 3 appears more optimal for determining full smoking. In comparison, the mFTQ selectively captures most full smokers and is even better at classifying subjects who do not meet the full smoking criterion (sensitivity and specificity, respectively). These findings suggest that the mFTQ is a helpful and a potentially cost-effective tool for identifying adolescents and young adults for consistent patterns of cigarette smoking.

The results of our study are similar to those of Kandel et al. ¹¹, in that when using cutoff score of 4, rates of dependence were higher on the DSM than on the mFTQ. However, the concordance between our smoking criteria at a cutoff score of 4 (kappa= 0.63) was considerably greater than the DSM diagnosis in the previous study (kappa= 0.34)¹¹. Our findings also contrasted with an earlier study by Hughes et al. (1987)²¹ who found the concordance identified by the original FTQ and DSM-III criteria for tobacco dependence to be low (kappa= 0.3). These findings add to the extant literature suggesting that cutoff score of 4 by the mFTQ is too conservative.

Our analysis of cutoff scores (Table 2) shows that the mFTQ was more effective in identifying subjects who did not meet for full smoking (specificity) than it is at identifying subjects who did meet for full smoking (sensitivity). Using a cutoff of 3 on the mFTQ (the greatest concordance) will correctly classify approximately four out of five (81%) subjects who were full smokers, while correctly classifying more than nine out of ten (93%) subjects who do not meet for full smoking. In a prior study on adolescent substance abusers, a cutoff score of 3 on the mFTQ resulted in the same sensitivity (81%) but had a much lower specificity (40%)²². The relatively high level of sensitivity with the mFTQ may also serve to predict the future severity of cigarette smoking. In support of this notion, the FTND was useful in predicting the quantity of cigarettes smoked at the 2-year follow-up in a longitudinal study of college students²³.

The properties of the mFTQ and cigarette smoking seem to line up fairly well in our sample. While the full smoking diagnosis depends on the frequency and quantity of cigarettes smoked, the mFTQ adds an extra dimension with the behavioral components of physical dependence to nicotine. This leads some researchers to conclude that the FTQ measures behavioral dependence (eg. behavioral intensity of use or compulsivity of use) rather than physical dependence on nicotine (signaled by tolerance or withdrawal) and should be related to environmental exposure to nicotine^{24, 25}. Our finding on age differences seems to show that the mFTQ score at a cutoff of 3 had a much greater concordance for adolescents than for adults (adolescents: kappa=0.88, adults: kappa=0.58). While the precise mechanism by which the mFTQ is more sensitive and specific to cigarette smoking in specific age groups is unclear, it may be that versions of the FTQ are not reliable for samples involving heavily smoking adults, since predictions of subsequent quitting could not be made in studies involving adults with moderate to high scores on these measures^{25, 26}.

The findings from this study must be considered in the context of their limitations. Our sample included both adolescents and adults in a combined sample. The difference in cigarette smoking criteria between the groups seemed to slightly affect the concordance between the two scales, however both groups shared an accurate mFTQ cutoff score of 3. A potential restriction of this study is our definition of full smoking. Our classification of smokers reflects repeated, frequent use and quantity of cigarettes smoked, but does not consider other physical symptoms of dependence. However, previous findings using this criteria has been consistent with the existing literature, indicating that our definition of full smoking is compatible with that used in other studies¹⁸. Our study is also limited by the lack of objective measures such as urine cotinine as such measures were not collected in this sample. Another limitation is the use of self-report on the questionnaire and in clinical interviews. When using self-report measures, there is the potential for underreporting. For this reason, we obtained direct and indirect interviews on subjects under the age of 18 and used the most severe diagnosis. There was also a low response rate on the mFTQ largely because the questionnaire was amended into the studies after the first year. Some subjects also refused to complete the studies’ questionnaires, however, there were no significant differences between included vs. dropped subjects other than a larger percentage of females.

Our findings suggest that the mFTQ is a selective tool for identifying adolescents and young adults with and without a diagnosis of full cigarette smoking. In context to the literature, these findings suggest that clinicians and researchers should consider a score of ≥3 on the mFTQ to be indicative of full smoking. Further research in this area is needed to evaluate the association of scores on this questionnaire with urine cotinine levels and other objective information for physical validation. In addition, instruments to capture nicotine dependence in non-cigarette tobacco use are needed. Longitudinal research would also be useful in evaluating the progression of smoking in order to determine if “sub-threshold” scores on the mFTQ are predictive of later, higher mFTQ scores and increasing levels of nicotine dependence. For future research, quantitative objective measures should be completed in tandem with structured clinical interviews and questionnaires.

Table 1.

The modified Fagerstrom Tolerance Questionnaire and scoring criteria.

Items	Scoring
How much do you smoke per day?	0: < half a pack, 1: 1/2 pack-1 pack, 2: > 1 pack
Do you inhale?	0: Never
	1: Sometimes, 2: Always
Do you find it difficult to refrain from smoking in places where it is forbidden?	0: No
	1: Yes
Do you smoke when you are so ill that you are in bed most of the day?	0: No
	1: Yes
Do you smoke more frequently in the morning than during the rest of the day?	0: No
	1: Yes
How soon after you wake up do you smoke your first cigarette? (exact # of minutes)	0: > 30 minutes
	1: ≤30 minutes
What cigarette would you hate to give up most?	0: Last, Other
	1: First