Withdrawal Symptom, Treatment Mechanism, and/or Side Effect? Developing an Explicit Measurement Model for Smoking Cessation Research

Abstract

Introduction

Assessment of withdrawal symptoms, treatment mechanisms, and side effects is central to understanding and improving smoking cessation interventions. Though each domain is typically assessed separately with widely used questionnaires to separately assess each domain (eg, Minnesota Nicotine Withdrawal Scale = withdrawal; Questionnaire of Smoking Urges-Brief = craving; Positive and Negative Affect Schedule = affect; symptom checklist = side effects), there are substantial problems with this implicit “one questionnaire equals one construct” measurement model, including item overlap across questionnaires. This study sought to clarify the number and nature of constructs assessed during smoking cessation by developing an explicit measurement model.

Methods

Two subsamples were randomly created from 1246 smokers in a clinical trial. Exploratory and confirmatory factor analyses were conducted to identify and select a model that best represented the data. Measurement invariance was assessed to determine if the factors and their content were consistent prior to and during the quit. Improvement in construct overlap within this model was compared against the implicit measurement model using correlational analyses.

Results

A 5-factor measurement model composed of negative affect, somatic symptoms, sleep problems, positive affect, and craving fits the data well prior to and during quitting. All factor content except somatic symptoms was consistent over time. Correlational analyses indicated that the 5-factor model attenuated construct overlap compared to the implicit model.

Conclusions

The models generated from data-driven approaches (eg, the 5-factor model) reduced overlap and better represented the constructs underlying these measures. This approach created distinct, stable constructs that span over measures of side effects and potential treatment mechanisms.

Implications

This study demonstrated that measures assessing treatment mechanisms, withdrawal symptoms, and side effects contain problematic overlap that reduces the clarity of these key constructs. The use of data-driven approaches showed that these measures do not map on to their posited latent constructs (eg, the Minnesota Nicotine Withdrawal Scale does not yield a withdrawal factor). Rather, these measures form distinct, basic processes that may represent more meaningful constructs for future research on cessation and treatment. Assessments designed to individually examine these processes may improve the study of treatment mechanisms.

Introduction

A better understanding of treatment mechanisms in clinical trials is critical for evaluating theory, advancing personalized medicine, and developing novel, targeted therapies (eg, Kazdin¹ and Kraemer et al.²). In the smoking literature, several candidates have been identified as plausible targets for treatment effects (eg, withdrawal, craving, negative affect [NA];^3,4). However, evidence supporting these constructs as mediators of treatment outcome is modest and mixed.^3,5 Although there is some evidence that withdrawal, craving, and NA each function as a partial mediator between pharmacotherapy (nicotine replacement therapy [NRT] and bupropion;^3,5) and smoking cessation, these effects are not consistent across studies and, when they do occur, appear to be overlapping rather than additive. For example, mediation by a withdrawal measure is attenuated when a separate measure of craving or NA is included in the statistical model.⁵ This overlap hampers the ability to evaluate and target putative treatment mechanisms with precision.

To advance research on treatment mechanisms, we must improve the assessment paradigm. Ideally, research employs a multi-trait, multi-method framework⁶ to establish a nomological net.⁷ In fact, most studies employ a mono-method, multi-trait framework with brief questionnaires individually assessing withdrawal (eg, Minnesota Nicotine Withdrawal Scale [MNWS]),⁸ craving (eg, Questionnaire of Smoking Urges-Brief [QSU-B]),⁹ and affect and/or mood (eg, Positive and Negative Affect Schedule subscales of negative and positive affect, PANAS-NA and PANAS-PA).¹⁰ An additional checklist or questionnaire is typically used to assess medication side effects (eg, Side Effect Checklist [SEC]).¹¹

This measurement model implicitly assumes a one-to-one relationship (Figure 1, designated as the “implicit measurement model”); treatment effects are reported separately for each questionnaire, reifying each measure as a construct (eg, MNWS scores are “withdrawal,” QSU-B scores are “craving”).^12,13 The problem is that these measures are not independent, empirically or conceptually. Figure 2 illustrates the degree of overlap (r²) observed among questionnaires (MNWS, PANAS-NA, PANAS-PA, QSU-B, and SEC) administered 1 week after the target quit day in a recent clinical trial.¹¹ The strength of these relationships (eg, MNWS and PANAS-NA r = .68) clearly shows that the questionnaires assess overlapping, rather than distinct, constructs.

Figure 1.

Implicit measurement model of treatment mechanisms and side effects, assessed by widely used measures in studies of smoking cessation, assuming a one-to-one relationship between the questionnaire and the construct. MNWS = Minnesota Nicotine Withdrawal Scale; PANAS = Positive and Negative Affect Schedule; QSU-B = Questionnaire of Smoking Urges-Brief.

Figure 2.

Visual depiction of the overlapping variance occurring among widely used measures based on the r² value between each measure at 1 week postquit from a large, multisite randomized clinical trial.¹¹ MNWS = Minnesota Nicotine Withdrawal Scale; PA = Positive Affect; PANAS = Positive and Negative Affect Schedule; QSU-B = Questionnaire of Smoking Urges-Brief; SEC= Side Effect Checklist; NA = Negative Affect.

Overlap is specified in leading conceptual models. For example, NA is hypothesized as the “motivational core” of the withdrawal syndrome and may serve as a critical mechanism in treatment.⁴ However, cessation research often treats NA, withdrawal, and side effects independently, without explicitly dealing with the problem that NA is well represented on measures of withdrawal and side effects.^8,11,14 The marked overlap among these measures precludes precise evaluation of independent candidate treatment mechanisms.

Such overlap partially occurs because the measures share identical or near identical items. For example, the terms “depressed,” “irritable,” and “anxiety”/“anxious” are common on widely used measures of smoking withdrawal, craving, NA, and side effects.^8,10,11 Other symptoms related to somatic concerns and sleep problems (eg, dizziness, nausea, insomnia) also overlap between the withdrawal and side effect measures.^8,11

The absence of explicit consideration of how to handle the overlap among NA, side effects, and withdrawal can create problems that extend into clinical practice and policy. In 2009, adverse psychological reactions (eg, suicidality, hostility) that were attributed to side effects of the smoking cessation medications varenicline and bupropion led to Food and Drug Administration black box warnings on both drugs,^15,16 resulting in a decline in prescriptions for varenicline.¹⁷ In the multisite Evaluating Adverse Events in a Global Smoking Cessation Study (EAGLES) trial, Anthenelli et al.¹⁸ reported that all treatment arms (varenicline, bupropion, NRT, placebo) showed similar rates of serious psychological reactions, suggesting these reactions resulted from quitting smoking, rather than medication side effects. Although these and related findings led to the removal of the black box warnings,¹⁹ some smokers developed concerns regarding the safety of these medications.²⁰ Thus, imprecision in our assessments can produce conceptual quandaries that impede science and practice.

A more explicit measurement approach could overcome some of the problems inherent in an implicit measurement model. Both theoretical and data-driven approaches using latent statistical modeling may be useful. Latent factor analysis has been conducted using common measures in the literature (see Figure 3). Most notably, factor analyses of various withdrawal scales suggest that several constructs are being assessed, with NA, somatic symptoms, craving, and sleep problems commonly emerging.^21–23 These more specific domains may be differentially related to relapse.²⁴ Similarly, factor analyses of craving measures (ie, the QSU and QSU-B) typically reveal two correlated factors,^9,25,26 and the PANAS has a large literature suggesting a robust two-factor structure (PA and NA).²⁷

Figure 3.

To date, the structure of side effect measures has been largely unexplored, with analyses examining each symptom independently²⁸ or focusing on a single composite measure.²⁹ These approaches do not typically consider the relations among side effects or their relations to other measures, resulting in ambiguity—and sometimes serious misidentification (see earlier)—of side effects.

Although research has examined the factor structure of individual questionnaires, the major problem of the implicit model, the tendency to overlook the substantial overlap across measures, has been unaddressed. To the best of our knowledge, no study has attempted to develop an explicit smoking cessation measurement model using multiple questionnaires to capture a range of hypothesized constructs. This study aims to take this initial step by identifying the constructs represented across common measures. We used a combination of exploratory analyses (exploratory factor analyses [EFAs]; exploratory structural equation modeling) and confirmatory factor analyses (CFAs) to assess the number and nature of latent constructs underlying measures of smoking withdrawal, craving, affect, and side effects (Aim 1).

Given the dynamic nature of smoking cessation symptoms,³⁰ we also examine the stability of the latent constructs at various points prior to and during quitting. Constructs may converge or emerge at different stages of quitting, such as NA and craving merging into a withdrawal factor during the quit attempt.^8,14 We tested measurement invariance of the model to determine if the timing of assessment influenced which constructs emerged during cessation (Aim 2).

Finally, this study began to examine the utility of the explicit measurement model compared to the implicit measurement model by evaluating the degree to which the explicit approach attenuated construct overlap as assessed by correlations (Aim 3).

Methods

Participants

Participants (N = 1246) were treatment-seeking smokers randomly assigned to treatment with either varenicline, NRT (ie, the nicotine patch), or placebo in a multisite smoking cessation trial. Primary inclusion criteria were typical of smoking cessation trials (see Lerman et al.¹¹ for details). All participants provided written informed consent. This study was approved by the institutional review boards at all sites (clinicaltrials.gov identifier: NCT01314001).

Measures

Participants completed self-administered questionnaires at each visit. This study focuses on the first three timepoints to minimize missing data: 1 week prequit, on the target quit date (TQD), and 1 week postquit. Most participants provided data at all three timepoints (100% at prequit, 93% at TQD, and 91% at postquit). The measurement models were composed of the total pool of individual items from all the questionnaires described below.

Minnesota Nicotine Withdrawal Scale-Revised

The revised MNWS is a widely used self-report questionnaire designed to assess smoking withdrawal symptoms during the past week. This study used the 15-item version.³¹ Item responses used a 5-point Likert scale (from “None” to “Severe”). The MNWS reflects a range of withdrawal symptoms including craving, NA, cognitive deficits, sleep disturbances, and physiological symptoms (eg, constipation).⁸

Questionnaire of Smoking Urges-Brief

The QSU-B is a common, brief (10 items) measure of cigarette craving in the current moment. The 7-point Likert scale ranges from “Strongly Disagree” to “Strongly Agree.”⁹

Positive and Negative Affect Schedule

The PANAS is a 20-item questionnaire designed to assess affect over the past week using a 5-point Likert scale ranging from “Very slightly or not at all” to “Extremely.” The PANAS is composed of two scales that assess PA and NA (PANAS-PA and PANAS-NA).¹⁰

Side Effect Checklist

The SEC is a compilation of 29 side effects previously implicated in studies of varenicline and the nicotine patch.¹¹ Participants used a 4-point Likert scale ranging from “None” to “Severe” to rate the severity of each side effect (eg, skin rash, nausea, depressed mood) over the past week.

Analyses

These data were randomly split into two subsamples, one for exploratory analyses (n = 622) and the other for confirmatory analyses (n = 624).

To examine the number and nature of constructs that emerged during the quit process (Aim 1), a series of exploratory and confirmatory factor models were estimated in Mplus 7.3.³² A robust weighted least squares estimator was used for all models because some measures only had four response options and numerous items were heavily skewed.³³

First, EFAs were conducted to uncover the number of possible factors and the content of these factors. We examined the eigenvalue Scree plot and conducted a parallel analysis at each of the three timepoints to determine the maximum number of factors likely occurring.^34,35 Separate EFAs were conducted at each timepoint to identify items that performed acceptably across each assessment and determine which solutions could plausibly replicate across timepoints. All models used an oblique rotation (geomin), which allowed factors to be correlated.³⁶

Next, candidate solutions were selected by examining the interpretability of the factors and considering the inter-factor correlations.³⁷ We compared two competing CFA models at each timepoint to replicate the exploratory results and establish model fit. Chi-square difference tests were used to directly compare changes in model fit with significance indicating worsening model fit and nonsignificance indicating no change to model fit. To assess overall fit of a single model, we used standard interpretive conventions³⁸ for the comparative fit index, Tucker–Lewis index (TLI), and the root-mean-square error of approximation (see Table 3 for cutoff values).

Table 3.

Fit Indices From Exploratory, Confirmatory, and Measurement Invariance Models

	Exploratory analysis
Fit index	Pre-Quit n = 621		TQD n = 577		Post-Quit n = 567
	4-Factor	5-Factor	4-Factor	5-Factor	4-Factor	5-Factor
Model χ2	3363.06	2834.44	3612.24	3040.18	4319.99	3538.58
CFI	0.95	0.96	0.93	0.95	0.92	0.94
TLI	0.94	0.96	0.92	0.94	0.91	0.93
RMSEA	0.04	0.03	0.04	0.04	0.05	0.04
	Confirmatory analyses
	Pre-Quit n = 624		TQD n = 579		Post-Quit n = 569
	4-Factor	5-Factor	4-Factor	5-Factor	4-Factor	5-Factor
Model χ2	3470.28	3272.23	3518.91	3264.95	4094.04	3668.83
CFI	0.95	0.95	0.93	0.94	0.93	0.94
TLI	0.94	0.95	0.93	0.94	0.93	0.94
RMSEA	0.04	0.04	0.04	0.04	0.05	0.04
χ2 Difference test	69.67, p < .001		80.10, p < .001		115.77, p < .001
	5-Factor model: Measurement invariance (n = 622)
		Configural		Full factor loading		Partial factor loading
Model χ2		19 812.36		20 074.87		19 800.30
CFI		0.94		0.94		0.94
TLI		0.94		0.94		0.94
RMSEA		0.02		0.02		0.02
χ2 Difference test		N/A		346.06, p < .0001		93.71, p = .18

Standard interpretive conventions for model fit: CFI (>.90 is acceptable, >.95 is good), TLI (>.90 is acceptable, >.95 is good), and RMSEA (<.06 is good, >.10 is poor).³⁹ CFI = comparative fit index; Post-Quit = 1 week postquit; Pre-Quit = 1 week prequit; RMSEA = root-mean-square error of approximation; TQD = target quit date.

All Model χ²ps < .0001.

After a final CFA model was chosen, we next examined longitudinal measurement invariance across the three timepoints (Aim 2) to determine if the same number of constructs were emerging across the quit period (configural invariance) and if the content of these constructs remained consistent (factor loading invariance). Factor loading invariance evaluates the extent to which the strength of factor loadings on a latent variable changes significantly over time.³⁹ Non-invariant factors contain a high proportion of items with variable factor loadings, suggesting the same construct is not emerging over time.⁴⁰ Factor loading invariance may initially be rejected due to a small number of problematic items. In such situations, examining partial factor loading invariance, through freeing a few factor loadings, is considered appropriate.⁴¹ Within Aim 2, each factor was allowed to correlate with its respective factors at each timepoint (eg, NA at prequit was correlated with NA at TQD and postquit, see Supplementary Material). Likewise, residual covariances of individual items were allowed to correlate with their counterparts at each timepoint.

To determine if the final CFA model improved overlap among constructs (Aim 3), we compared the inter-factor correlations of the CFA model (see Figure 4) to the Pearson bivariate correlations among the questionnaire total scores, and the inter-factor correlations of the implicit measurement model (see Figure 1). The implicit model was specified using CFAs to establish a one-to-one relationship between each scale and its hypothesized construct (eg, MNWS and withdrawal). Latent modeling of both the implicit model and the final CFA model from Aim 1 allowed for more direct comparisons.⁴²

Figure 4.

Results

The means and standard deviations for the summed scores from the self-report questionnaires among the full sample can be seen in Table 1.

Table 1.

Mean (Standard Deviation) of the Summed Scores From Each Self-administered Questionnaire

	Pre-Quit	TQD	Post-Quit
MNWS	7.09 (4.66)	8.10 (5.68)	8.91 (6.49)
PANAS: negative affect	12.94 (4.02)	13.55 (4.76)	14.11 (5.55)
PANAS: positive affect	32.46 (8.89)	33.86 (9.18)	33.12 (9.93)
QSU-B	29.79 (14.72)	26.08 (14.64)	20.49 (12.82)
SEC	3.52 (4.34)	4.25 (4.37)	5.57 (5.13)

Range of response options are as follows: MNWS = 0–4; PANAS = 1–5; QSU-B = 1–7; SEC = 0–3. Pre-Quit = 1 week prequit, TQD = target quit date, Post-Quit = 1 week postquit.; MNWS = Minnesota Nicotine Withdrawal Scale; PANAS = Positive and Negative Affect Schedule; QSU-B = Questionnaire of Smoking Urges-Brief; SEC= Side Effect Checklist.

Data Reduction

Both prior to and during the EFAs, problematic items were identified and deleted based on four criteria: (1) infrequently endorsed (<3%) items that produced heavy skew, (2) low-factor loading magnitudes on all factors (ie, <|.30|), (3) inconsistent loadings on multiple factors (ie, cross-loadings of |.30–.50|), or (4) inconsistent factor loadings across the three timepoints (see Table 2).⁴³ Redundant items were also addressed within the EFAs. Three items were worded identically on the MNWS and SEC (dizziness, nausea, and constipation), and each item pair was highly correlated at every assessment (rs = .60–.76). Each item pair was combined to form a composite (see Table 2).

Table 2.

Individual Item Information Including Original Measure and Factor Location

	NA		Somatic symptoms		Sleep problems		PA		Craving		Excluded
MNWS	Angry, irritation, frustration	Difficulty concentrating	Constipationc	Nauseae	Insomnia, sleep problems, awakening at night				Desire/craving to smoke		Increased app./hungry /weight gainb
	Anxious, nervous	Impatient	Dizzinessd	Sore throat	Restless
	Depressed mood, sad	Restless	Coughing		Dreaming or nightmares
QSU-B									Desire for a cigarette right now	Urge for a cigarette
									Nothing better than cigarette	Cigarette would taste good
									I would smoke right now	Do anything for a cigarette
									Control things if smoke	Smoking less depressed.
									All I want right now is a cigarette	Smoke ASAP
PANAS-NA	Distressed	Irritable
	Upset	Ashamed
	Guilty	Nervous
	Scared	Jittery
	Hostile	Afraid
PANAS-PA							Interested	Alert
							Excited	Inspired
							Strong	Determined
							Enthusiastic	Attentive
							Proud	Active
SEC	Irritability	Disturbance in attention	Nauseae	Headache	Sleep problems						Irregular heartbeata	Skin rednessa
	Depressed mood	Feeling of Weakness	Constipationc	Diarrhea	Insomnia						Suicidal thoughtsa	Chest painb
	Increased heart rate, palpitations	Hostility	Dry mouth	Abdominal pain	Abnormal dreams						Vomitinga	Weaknessa
	Anxiety		Gas	Flatulence							Skin swelling, rasha
	Agitation		Indigestion	Dizzinessd							Fatigueb

Bolded items were factor loading invariant in the 5-factor model. MNWS = Minnesota Nicotine Withdrawal Scale; NA = Negative Affect, PA = Positive Affect; PANAS = Positive and Negative Affect Schedule; QSU-B = Questionnaire of Smoking Urges-Brief; SEC= Side Effect Checklist.

^aDeleted for heavy skew.

^bDeleted for low or inconsistent factor loadings.

^c–eSingle indicator composites from items with identical content, composite repeated to show original measure locations.

Aim 1: Assessing the Number and Nature of Constructs for the Measurement Model

Exploratory Analyses

The eigenvalue Scree plot and parallel analysis indicated that up to 10 factors could be extracted at each timepoint (see Supplementary Material). Exploratory models with three or fewer factors appeared to include factors containing two or more distinct constructs, indicating under-extraction (see Figure 5). At each timepoint, 4- and 5-factor solutions were interpretable as indicated by the distinct, coherent factors (seen in Figure 5), with small to moderate inter-factor correlations (range = [−0.23 to 0.37] and [−0.23 to 0.40] for the 4- and 5-factor solutions, respectively). Models with six or more factors showed signs of overextraction, including factors containing unusual combinations of constructs (eg, NA split between a pure NA factor and an NA/gastrointestinal symptom factor). Consequently, only the 4- and 5-factor solutions were compared quantitatively to establish the best-fitting model.

Figure 5.

The conceptual labels applied to the factors emerging in exploratory solutions 1 through 6. Arrows indicate how factors from lower solutions split as more factors were extracted.

Table 3 shows the fit indices of the exploratory analyses. Both the 4- and 5- factor solutions obtained acceptable to good fit at all three timepoints suggesting the proposed NA, PA, somatic symptoms, craving, and sleep problems factors were adequately capturing the variance from these questionnaires over the quit process. Given the large sample size and variability in item timeframe and measurement scale, as well as uncontrolled variance due to treatment condition and smoking status, it is not surprising that all model chi-square were significant (p < .0001), indicating there was still room for improvement in fit between the observed and model implied data.⁴⁰

Confirmatory Factor Analyses

As both the 4- and 5-factor model were qualitatively and quantitatively supported based on the interpretability of the factors and the fit indices, these models were specified for CFAs. Items were assigned to specific factors for the CFAs based on where they loaded most consistently and most strongly during the EFAs. Decision rules for item factor assignment included (1) items with at least a weak loading (>|.30|) on a single factor for one or more timepoints were assigned to that factor and (2) items with inconsistent loadings on multiple factors were assigned based on the factor the item loaded on most frequently across the three timepoints. We attempted to establish a simple structure by minimizing cross-loading items to improve interpretability and reduce construct overlap.³⁷

As seen in Table 3, both the 4- and 5-factor models yielded acceptable fit at all three timepoints for all fit indices, with all model chi-square values remaining significant. Compared to the 4-factor solution, the 5-factor model typically showed superior fit at all three timepoints (see the chi-square difference tests in Table 3). These results suggested that the 5-factor model better represented the observed data. Therefore, the 5-factor model was chosen to examine measurement invariance.

Aim 2: Assessing the Model Throughout Quitting Using Measurement Invariance

Longitudinal measurement invariance was tested to determine if the 5-factor solution remained stable over the three timepoints. First, configural invariance was tested by applying the 5-factor structure to all three timepoints within a single model (see Supplementary Material). As seen in Table 3, the configural invariance model achieved acceptable fit based on the values of the comparative fit index, TLI, and root-mean-square error of approximation, indicating that the structure of the model remained consistent across timepoints.

Because acceptable fit was achieved, factor loading invariance was assessed to determine if the content of the five factors was stable. Full factor loading invariance was first tested by constraining each item’s factor loadings to be equal over the three timepoints. Constraining the factor loadings produced a worse fitting model, compared to the configural model as indicated by the significant chi-square difference test (see Table 3). This suggests that at least some items have varying factor loadings over the three timepoints and that the content of certain factors might change during cessation.

Next, we examined partial factor loading invariance by iteratively freeing items that showed the most variability in their factor loadings across the three timepoints; we freed one additional factor loading at a time until the partial factor loading invariance model did not differ significantly from the configural model (see Supplementary Materials for details).^40,41 The fit values for the final partial factor loading invariance model can be found in Table 3. NA, sleep problems, PA, and craving factors exhibited factor loading invariance across timepoints, indicating the same constructs were being assessed for these four factors. However, the somatic symptom factor was not invariant through the quit process (see Table 2), suggesting that the nature of the somatic symptoms construct shifts, with items become more or less related to one another at different points in the quit process.

Aim 3: Examining Construct Overlap and Fit of the Implicit Model

Reductions in construct overlap were assessed by examining correlations among the 5-factor model (see Figure 4), the implicit model (see Figure 1), and the total scores for each measure. As seen in Table 4, stronger correlations were observed within the implicit model and the total scores. In particular, withdrawal showed considerable overlap with NA and side effects, suggesting the MNWS, PANAS-NA, and SEC are not measuring three distinct processes. In contrast, inter-factor correlations were attenuated in the 5-factor model, suggesting these data yielded unique constructs. However, these constructs were not orthogonal as sizeable inter-factor correlations occurred (eg, sleep problems, NA, and somatic symptoms; NA and craving during the quit period).

Table 4.

Inter-factor Correlations Within the 5-Factor and Implicit Model, and Pearson’s Bivariate Correlations for the Summed Scores at Each Timepoint

		5-Factor model					Summed scores				Implicit model
		NA	Somatic	Sleep problems	PA		NA	Side effects	Withdrawal	PA	NA	Side effects	Withdrawal	PA
Pre-Quit	Somatic	0.45				Side effects	0.38				0.51
	Sleep problems	0.55	0.52			Withdrawal	0.65	0.60			0.85	0.87
	PA	−0.08	−0.05	−0.08		PA	0.00	−0.11	0.01		−0.03	−0.09	−0.10
	Craving	0.11	0.07	0.10	0.03	Craving	0.01	0.03	0.09	−0.01	0.10	0.09	0.16	0.02
TQD	Somatic	0.42				Side effects	0.47				0.65
	Sleep problems	0.42	0.41			Withdrawal	0.69	0.56			0.82	0.86
	PA	−0.13	−0.04	−0.11		PA	0.00	−0.09	−0.02		−0.04	−0.15	−0.15
	Craving	0.32	0.07	0.14	−0.09	Craving	0.23	0.09	0.32	−0.18	0.31	0.22	0.31	−0.10
Post-Quit	Somatic	0.39				Side effects	0.48				0.61
	Sleep problems	0.38	0.37			Withdrawal	0.70	0.61			0.81	0.80
	PA	−0.22	−0.05	−0.07		PA	−0.18	−0.21	−0.18		−0.20	−0.15	−0.18
	Craving	0.40	0.15	0.15	−0.15	Craving	0.31	0.22	0.40	−0.20	0.36	0.21	0.50	−0.15

High correlations (>|.50|) are bolded, whereas weak correlations (<|.20|) are shaded gray. Pre-Quit = 1 week prequit, TQD = target quit date, Post-Quit = 1 week postquit, NA = Negative Affect, PA = Positive Affect.

Discussion

Despite the importance of treatment mechanism research for advancing science and practice,^1,2 most smoking cessation trials have employed a suboptimal measurement model that implicitly assumes a one-to-one pairing between questionnaires and latent constructs—typically withdrawal, craving, NA, and side effects. The current results show that the implicit model is not the best representation of the data. The implicit measurement model is problematic due to the substantial conceptual overlap—and even item-level overlap—occurring among the typical questionnaires. Both the EFAs and CFAs conducted on the present data demonstrated that NA contributed strongly to three of the five questionnaires in the study (PANAS-NA, MNWS, and SEC). Thus, it would be difficult to justify treating each of the three measures as indices of separate constructs.

This study developed a data-driven model of commonly used self-report instruments to clarify the processes involved in quitting smoking. In general, the 5-factor model, including NA, somatic symptoms, sleep problems, PA, and craving, best fits the data. Moreover, the 5-factor model improved the interpretation of the underlying processes with substantially reduced overlap among the factors compared to the overlap in the implicit measurement model (see Table 4). Further, the evaluation of invariance suggested that the structure and the content of the factors (except somatic symptoms) were consistent prior to and during quitting. Moreover, the 5-factor model adequately fit the data even though variance associated with treatment condition (participants were randomized to NRT, varenicline, or placebo) and smoking status was not explicitly modeled. In future work, additional tests of invariance should examine the factor structure as a function of treatment and cessation status.

Unlike the other four factors, the content of the somatic symptoms factor was not stable over time. This pattern likely reflects variability associated with both treatment condition and cessation status, variability that will be important to consider as empirical work on the measurement model is taken forward. Indeed, good fit, modest factor intercorrelations, and factor stability do not address whether any of the constructs are mediators of treatment outcome. These characteristics provide a psychometrically sound foundation for more precise evaluations of candidate mechanisms in future work.

One of the most surprising results of this study was the absence of a clear withdrawal factor. Prior research and theory suggest that at TQD and postquit, several constructs including craving, NA, and sleep problems would form a “withdrawal syndrome.”⁴⁴ Although the individual factors emerged in the data-driven models, they were relatively independent and were evident in the data prior to quitting. The present data do not signify that aspects of withdrawal do not exist, rather they raise important questions about the utility of a global withdrawal construct. Consistent with the current perspective, some factor analytic work of the MNWS has suggested that a focus on separate facets of withdrawal would improve assessment,²⁴ and other work has focused on developing “multidimensional” measures of withdrawal.²³ Thus, it appears that a shift from focusing on a global withdrawal construct to basic processes that may be related to quitting smoking and treatment response is warranted. Such an approach would allow a range of processes to be assessed for their unique roles in quitting and smoking, with or without a given therapy. For example, cessation increases NA, sleep problems, and craving.⁴⁴ The introduction of treatment may attenuate one or more of these processes, revealing a candidate treatment mechanism, while exacerbating another, revealing a side effect.

The 5-factor model is an intermediate approach; although it represents progress compared to the implicit model, there is more to be done. First, it is important to replicate the current approach in other samples and over longer postquit periods. Second, its ability to predict treatment outcomes (eg, cessation status) is unclear; thus, future work must examine predictive validity. Third, it is not comprehensive, reflecting only the constructs that were well-represented in the item pool.⁴⁵ Other constructs may be quite important but were under-represented. For example, the single item related to appetite (increased appetite, hungry, or weight gain) was removed from the models due to low factor loadings; items related to other constructs, such as anhedonia,⁴⁶ were simply not included.

Two approaches could be taken to improve the assessment of under-represented processes. We could add items to the typical item pool to try to better capture a broader range of processes. This may include the development of a measure assessing the separate domains of cessation using a common response and time scale. Alternatively, to the extent that a construct is worthy of evaluation, it may be more reasonable to identify psychometrically strong, preexisting measures. In some cases, such as cognition or reinforcement, the best assessments are likely to be task-based rather than self-report. Ultimately, the selection of assessments will need to be tailored to the particular clinical trial, with trials ideally targeting a few key processes using a multi-trait, multi-method approach⁶ that rigorously evaluates hypothesized treatment mechanisms. Given that task-based paradigms are much more common in behavioral pharmacology studies, there is great potential for collaboration between laboratory and clinical scientists.⁴⁷

A process-oriented approach has been emphasized in other areas of psychopathology via the adoption of National Institute of Mental Health’s Research Domain Criteria,⁴⁸ which focuses on multiple levels of analysis in five domains: negative valence, positive valence, arousal/regulatory systems, cognitive systems, and social processes. The results of this study and prior research examining other domains relevant to cessation^49,50 suggest that adopting such a process-oriented approach may improve smoking cessation research.

In summary, this study demonstrates some of the pitfalls of widespread use of “implicit measurement models” in which the same small battery of measures can become the norm, often without explicit consideration of the alignment between measures and constructs. We believe that the systematic application of modern assessment approaches will provide a better foundation for understanding cessation, treatment mechanisms, and related processes such as side effects. Finally, we call for the integration of data- and theory-driven approaches in future work. In order to know what to assess, and how to interpret the data, we need strong theory; in turn, we need precise assessments in order to rigorously evaluate competing mechanistic models of quitting and relapse.

Funding

This work was supported in part by the Pharmacogenomics Research Network, Pharmacogenetics of Nicotine Addiction Treatment Research Group, funded primarily through the National Institutes of Health (National Institute on Drug Abuse, National Cancer Institute, National Institute of General Medical Sciences, and National Human Genome Research Institute; U01-DA20830) for the data collection portion of this study. The development of this article was supported in part by the National Institutes of Health (National Cancer Institute; R01CA206193).

Declaration of Interests

Pfizer provided the study medication and placebo for the randomized clinical trial that yielded the data for the current article but had no role in project design, analysis, or article preparation. MCM has served on the Speaker’s Bureau for Pfizer and as the medical director of the New York State Smokers Quit Line.