Concordance Between Timeline Follow-Back and Single Question Assessment of Self-Reported Smoking in a Clinical Trial

Abstract

Background

Smoking cessation clinical trials assess tobacco abstinence using self-report and biomarkers. Optimum methods for each are unclear; a common question assesses smoking in the prior 7 days. In contrast, timeline follow-back (TLFB) is another technique often used to assess use of alcohol in treatment trials; it is used less frequently in studies of smoking cessation. The goal of this study was to assess concordance between the 7-day smoking question and a 7-day TLFB.

Methods

Secondary analysis of data from a randomized clinical trial of smoking cessation conducted at a busy, urban hospital emergency department (ED) from October 2010-December 2012. At 1, 3, and 12 months, subjects were contacted by phone to assess smoking status. Those reporting abstinence at 3 months were asked to return for an in-person measurement of exhaled carbon monoxide. For this analysis, we compared smoking status at 1 month for subjects in response to 2 questions asked concurrently, addressing 7-day point prevalence tobacco use, and a 7-day TLFB.

Results

Of 780 subjects, 666 (85.4%) were available for 1-month follow-up. Of these, 99 (14.9%) reported no smoking in response to the 7-day question, and 96 (14.4%) reported no smoking in response to the 7-day TLFB. The overall proportionate agreement between the 2 methods was 98.6%, with a kappa of 0.95 (95% CI 0.91-0.98).

Conclusions

A single question that assesses smoking at 7 days provides excellent concordance with the more detailed TLFB. The single question appears adequate to assess self-reported tobacco use in clinical trials of smoking cessation.

Introduction

Clinical trials of smoking cessation assess abstinence through 2 basic mechanisms: self-report and biochemical verification. For each method, several techniques are possible. Self-report may be assessed in person, by telephone, by use of interactive voice response (IVR), or through cellphone-based texting. Tobacco use may be queried for the preceding 7 days, 14 days, 30 days, continuously since the intervention, or via number of cigarettes smoked per day.¹ Biochemical techniques include assays for exhaled carbon monoxide, thiocyanate, cotinine (a nicotine metabolite), and anabasine (a constituent of tobacco not found in nicotine replacement products).²

Smaller trials (approximately < 1000 subjects) of smoking cessation often perform biochemical confirmation of smoking status if subjects report tobacco abstinence. Larger, population-based trials, often of quitline services³ or wide-scale policy interventions,⁴ typically assess smoking status by self-report only.

The need for biochemical confirmation of tobacco abstinence has been debated in the literature, and there is no consensus on the question.^2,5,6 The methodologic rationale for biomarker verification is that social desirability bias might lead smoking subjects to self-report abstinence, but studies of this phenomenon have found only modest effects of inaccurate or disingenuous reporting. Biochemical confirmation adds significant cost and logistical challenges to trials of tobacco dependence treatment.

Timeline follow-back (TLFB) is a validated technique that has been used widely in trials of a variety of behavioral interventions, often in studies of alcohol consumption.^7-9 TLFB entails asking subjects whether they engaged in a specific behavior (e.g. substance use) on each day of a defined period of time, in a structured, prompted fashion. Periods of 7 and 30 days are commonly used.

TLFB has been used in studies of tobacco dependence treatment, ^10-14 but is less common. In an early study of tobacco use, TLFB was been shown to have good test-retest reliability.¹² A more recent study found fair agreement between TLFB and ecological momentary assessment (EMA) for the measurement of daily cigarette consumption, with greater disagreement found in smokers with greater daily consumption.¹¹

The concordance between TLFB and the single-question assessment of tobacco use is unknown. Hence, we endeavored to measure the concordance between the 2 methods, as a secondary analysis of data from a completed clinical trial of tobacco dependence treatment. Our hypothesis was that TLFB would yield a more conservative estimate of abstinence from smoking than a single question of cigarette use in smokers enrolled in a cessation trial, and that this technique would therefore mitigate some of the effects of social desirability bias.

Methods

Study design

These data represent a secondary analysis of a randomized clinical trial conducted from October 2010 to December 2012. Main results from the trial have been reported previously.¹⁵ The study was conducted in the adult emergency department of a university hospital in an urban area in the northeastern United States. Eligible subjects were aged 18 years or older, English-speaking, endorsing smoking every or some days with lifetime consumption of at least 100 cigarettes, presenting with any non-psychiatric problem, and either uninsured or insured by Medicaid. Consenting individuals were randomized to 1 of 2 arms: usual care, in which subjects were provided with a smoking cessation brochure, and enhanced care, in which subjects received the brochure, a motivational interview, 6 weeks of nicotine patches and gum tailored to daily consumption, a proactive referral to the state smokers' quitline, and a booster phone call 2-3 days after enrollment. Subjects were contacted by phone 1, 3, and 12 months after enrollment. Subjects endorsing tobacco abstinence at 3 months were asked to return to the hospital for a measurement of exhaled carbon monoxide.

Assessment of tobacco use

Self-reported tobacco abstinence was assessed at each time point by 2 questions, as shown in Table 1. The first is a standard question assessing point prevalence abstinence from smoking in the previous week: “Have you smoked, even a puff, in the last 7 days?” with responses of yes, no, don't know/not sure, or refused.¹

Table 1.

Two questions to assess self-report of tobacco use at follow-up.

7-Day Point Prevalence
Have you smoked, even a puff, in the last 7 days?	Yes No Don't Know/Not Sure Refused
7-Day Timeline Follow-Back
We want to get an idea of how many cigarettes you smoked on each day during the past 7 days. How many cigarettes did you smoke yesterday (Day 1)? How many cigarettes did you smoke the day before that (Day 2)? Etc.	Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7

Next, subjects were led through a scripted, calendar-aided timeline follow-back (TLFB) of their smoking behavior to assess their day-to-day tobacco use in the previous 7 days. Research assistants recorded subjects' responses to tobacco use on each of the 7 days, including an assessment of the number of cigarettes smoked on days subjects reported tobacco consumption.

Data analysis

Data were recorded on paper-and-pencil machine-readable forms and scanned into a Microsoft Excel database. Data were then exported into SPSS 10.0, which was used for all analyses. Concordance between the 2 measures of smoking was assessed with overall proportionate agreement, as well as the kappa statistic, using the parameters of Landis and Koch to define the qualitative level of agreement.¹⁶ Quantitative measures of numbers of cigarettes smoked daily were compared between TLFB and a single question using a Bland-Altman plot.¹⁷ We report data from complete cases ascertained at 1 month, when follow-up was maximal. No imputations were made, to allow direct comparison of both subject-reported measures. In the parent trial, an intention-to-treat (ITT) model imputed continued smoking to subjects lost to follow-up. In addition, in this analysis, we are not reporting the agreement between self-reported measures of tobacco use and carbon monoxide readings, because the latter were done at 3 months only for subjects endorsing tobacco abstinence.

As a sensitivity analysis, we also report concordance between TLFB and the single tobacco use question at the 3 and 12 month waves of follow-up. Again, we used complete-case analysis. The study was approved by the Human Investigation Committee of (blinded) University. The trial was registered with www.clinicaltrials.gov and has Identifier NCT01328431.

Results

Description of subjects

Between October 2010 and December 2012, 780 subjects were enrolled, of whom 666 (85.4%) were contacted by telephone for follow-up at 1 month. Baseline clinical and demographic characteristics of study subjects were comparable between treatment groups. In brief, subjects' mean age was 40.5 years; 371 (47.1%) were male; 305 (38.7%) were white, 308 (39.1%) African-American, 158 (20.0%) Hispanic, and 7 (0.9%) Asian or other race. Of all subjects, 656 (83.2%) were insured by Medicaid; the remaining 122 (16.8%) were uninsured. Median daily cigarette consumption, 11/day, was comparable to that found in other studies of ED smokers.^18,19

Smoking status concordance

Of the 666 subjects who were reached at 1 month, 99 (14.9%) reported no smoking in response to the 7-day point prevalence question; 96 (14.4%) reported no smoking in response to the 7-day TLFB. The proportionate agreement was 98.6%. Table 2 reports the results of the concordance analysis, which was near-perfect, using the grading system of Landis and Koch.¹⁶

Table 2.

Concordance between 2 questions of self-reported tobacco use.

	7-Day Timeline Follow-Back
7-Day Point Prevalence	Smoking	Abstinent
Smoking	564	3
Abstinent	6	93

Overall proportionate agreement 98.6%; kappa = 0.95 (95% CI 0.91-0.98).

At 3 months, 626 subjects (80.2% of the total cohort) were available for analysis. Of these, 97 reported no smoking to the 7-day point prevalence question, and 100 to the TLFB. The proportionate agreement was 98.6%, with a kappa of 0.43 (95% CI 0.41, 0.45).

At 12 months, 594 subjects (76.2% of the total cohort) were available for analysis. Of these, 107 reported no smoking to the 7-day point prevalence question, and 102 to the TLFB. The proportionate agreement was 97.8%, with a kappa of 0.42 (95% CI 0.40, 0.44).

Cigarette consumption concordance

The Figure provides a quantitative comparison of the measure of agreement between the 2 methods. The Bland-Altman plot's x-axis displays the number of cigarettes smoked daily, assessed as the mean of the values obtained by TLFB and single-question response at 1 month. The y-axis represents the difference between the 2 methods assessing the number of cigarettes smoked daily. The mean difference was 0.701 cigarettes, with a 95% confidence interval of 2.77. Again, the plot displays generally excellent agreement between the 2 methods in assessing daily cigarette consumption. At higher levels of consumption, e.g. more than 10 cigarettes/day, there are more outlier values, but these appear to be clinically unimportant. Critically, for the subjects self-reporting tobacco abstinence, there is near-perfect agreement in the quantitative measures.

FIGURE 1 Bland-Altman plot showing agreement between self-reported cigarettes smoked/day by single question and 7-day timeline follow-back at 1 month. Upper and lower lines represent 95% confidence limits for mean value of the difference; middle line represents the mean value.

Figure 1.

Bland-Altman plot showing agreement between self-reported cigarettes smoked/day by single question and 7-day timeline follow-back at 1 month. Upper and lower lines represent 95% confidence limits for mean value of the difference; middle line represents the mean value.

Discussion

In 2003, a task force appointed by the Society for Research on Nicotine and Tobacco (SRNT) published a white paper offering a number of validated measures of self-reported cigarette use.¹ These measures include use of tobacco within the preceding 7 days, 14 days, or 30 days, or use of tobacco since the intervention. Use of tobacco products other than cigarettes is counted as a failure of treatment, but use of FDA-approved nicotine replacement products is not.¹ In addition, the number of cigarettes smoked per day may be measured as an endpoint.

Timeline follow-back techniques were not included in the SRNT task force's report. They were initially developed to measure daily alcohol consumption,²⁰ and have become a standard means of self-reported assessment of patterns of alcohol use.⁸ Because we have also participated in trials of treatment for alcohol use disorders using the TLFB,^13,14 we thought it would be useful to compare agreement between TLFB and the more common point prevalence measure used in tobacco dependence treatment trials.

Timeline follow-back and the single 7-day question provide outstanding agreement in assessing a binary measure of self-reported tobacco use. We found no evidence that TLFB provides a more conservative or accurate estimate of 7-day tobacco abstinence than the single question. In fact, the results tend to validate the reliability of the single question as a reasonable measure of ongoing cigarette use. A sensitivity analysis comparing the two questions at the 3-and 12-month timepoints confirms the high levels of agreement between the two measures.

The Bland-Altman plot also suggests excellent agreement between single-question assessment and TLFB for a quantitative measure of daily cigarette consumption. There is some disagreement between the 2 measures at higher daily levels of smoking, but for the purposes of our study, these are clinically unimportant. The higher level of disagreement with greater daily smoking has been reported previously.¹¹

One limitation of the study is the possibility of order effects. The single question was asked first, with the TLFB conducted three questions later (after additional items about quantity smoked and last 30-day smoking). Research staff were instructed not to correct or prompt subjects whose TLFB and single-question responses were discordant. Given their proximity, it is possible that answering the question about 7-day smoking may have affected the responses to the TLFB. In any event, the proportionate agreement was excellent, and there were no directional effects. And, the consistency of the agreement at 1, 3, and 12 months suggest order and proximity may be less likely an explanation for the high concordance than reliability.

Conclusions

Comparison of TLFB and a single-question assessment of point prevalence tobacco use confirms the reliability of both. The single question is acceptable for use in assessing ongoing tobacco use, irrespective of investigators' desire for biochemical confirmation. Similarly, a single question asking about the number of cigarettes smoked daily yielded data that agree well with TLFB.