The Modified E-Cigarette Evaluation Questionnaire: Psychometric Evaluation of an Adapted Version of the Modified Cigarette Evaluation Questionnaire for Use With Adults Who Use Electronic Nicotine Delivery Systems

Abstract

Introduction

The subjective experience of positive and negative effects likely contributes to e-cigarette use, and the Modified Cigarette Evaluation Questionnaire (MCEQ) previously has been adapted to assess the reinforcing and aversive effects of vaping. However, the psychometric properties of the MCEQ for use with e-cigarettes have not been established.

Aims and Methods

We examined the psychometric properties of the Modified E-cigarette Evaluation Questionnaire (MECEQ) within a sample of 857 adults who recently used e-cigarettes in a smoking cessation attempt (52.4% male; 40.84 [12.25] years old; 62.8% non-Hispanic white; 22.4% daily e-cigarette users). Analyses included confirmatory factor analysis of the original structure, exploratory/confirmatory factor analyses to identify the alternate latent structure(s), internal consistency, measurement invariance, between-group differences, and test-criterion relationships with vaping-related outcomes.

Results

The original five-factor structure and a novel four-factor structure were supported. Each was scalar invariant across several participant subgroups (eg, current smoking status, daily vaping status). All multi-item subscales were internally consistent. Both versions detected several between-group differences. For example, current smokers reported stronger aversive effects than did exclusive e-cigarette users. Finally, adjusted relationships between both MECEQ versions and vaping-related outcomes provided evidence for concurrent validity.

Conclusions

The five-factor and four-factor versions of the MECEQ evidenced good-to-excellent internal consistency, scalar measurement invariance, and concurrent relationships with vaping-related outcomes. While both versions could be used to assess subjective vaping effects in adults with histories of cigarette smoking and vaping, additional research is needed to evaluate the applicability of these factor structures to other samples (eg, e-cigarette users with no smoking history, youth).

Implications

Although the MCEQ has been adapted in previously published studies to assess the subjective reinforcing and aversive effects of vaping, the psychometric foundation necessary for doing so had not been established. We showed that the MECEQ can be scored using the original five-factor MCEQ format or using a newly identified four-factor structure. Both versions evidenced construct validity, internal consistency, measurement invariance (permitting between-group comparisons), and concurrent validity with vaping-related outcomes. Results strengthen the interpretability of previously published work using the five-factor MCEQ structure and provide an alternative scoring approach for vaping-specific subjective effects.

Introduction

The Cigarette Evaluation Questionnaire¹ and its revised version, the Modified Cigarette Evaluation Questionnaire (MCEQ),² have been used for decades to assess the subjective reinforcing and aversive effects of smoking cigarettes. The MCEQ is scored as five subscales: four positive subscales (ie, Satisfaction, Psychological Reward, Enjoyment of Respiratory Tract Sensations, and Craving Relief) that relate to continued smoking and abuse liability and one negative subscale (ie, Aversion) that assesses the experience of acute negative effects of smoking that may deter use.² Prior research indicates that the MCEQ evidences good psychometric properties,² and both the constructs of reinforcing and aversive effects and the MCEQ’s items seem applicable to the use of e-cigarettes. As such, several studies have modified the five-factor MCEQ for use with e-cigarettes. To date, a modified version of the MCEQ assessing e-cigarette use has been used to assess the subjective reinforcing and aversive effects of vaping compared with smoking^3–5 and compared with using a nicotine inhaler.⁶ This modified scale also has been used to examine subjective effects of vaping among individuals who successfully switched from smoking cigarettes to exclusive e-cigarette use⁷ and adults with schizophrenia who smoke cigarettes.⁸ While these studies cite the psychometric properties of the original MCEQ and/or note that previous studies have modified the MCEQ for use with vaping, no published study has established the psychometric properties of MCEQ for use with e-cigarettes. As such, the interpretability of all prior findings obtained using this measure rest on the assumption that the e-cigarette version of the MCEQ has comparable psychometric properties to the original version for combustible cigarettes.

In the current study, we examined several key psychometric properties of the Modified E-Cigarette Evaluation Questionnaire (MECEQ) to formally evaluate its utility for assessing the subjective effects of vaping. First, we evaluated whether the original five-factor structure of the MCEQ fits the data for the MECEQ. If the model fits well, this would suggest that the same scoring applied to the MCEQ could be applied to the MECEQ. However, we also ran exploratory factor analyses to determine whether a unique factor solution may better reflect subjective vaping effects. For each latent factor structure that fits the data well, we then evaluated measurement invariance to determine if the MECEQ can be used to examine between-group differences in subgroups of interest (eg, adults who vape daily vs. non-daily) and, subsequently, examined between-group differences in subsamples for which support for doing so was observed. For each latent factor structure that fits the data well, we examined adjusted relationships between the MECEQ and vaping-related constructs (ie, past-month vaping frequency, total duration of e-cigarette use, e-cigarette dependence, and the experience of withdrawal when using e-cigarettes to quit smoking). If the five-factor structure fits the data well and was associated with other adequate psychometric properties, this would improve the interpretability of previous research findings derived from the e-cigarette version of the MCEQ and support scoring the MECEQ as a five-factor measure moving forward. If a superior, unique factor solution were observed, this would suggest that subjective vaping effects need to be conceptualized and scored in a different way.

Methods

Participants and Procedures

All procedures were approved by the Yale University Institutional Review Board. In Summer 2021, 857 adults completed a 20-minute, anonymous survey through Qualtrics Online Sample. Qualtrics recruited participants directly via emailing panelists who were likely to be eligible based on age and information such as smoking status. Eligibility for the parent study included living in the United States, being at least 21 years old, having a history of smoking cigarettes (≧1 year), and using e-cigarettes during a smoking cessation attempt in the past 2 years. Eligible participants who completed the survey were compensated directly by Qualtrics Inc. based on the terms of pre-established agreements with their panelists.

Measures

Demographics

Participants reported on sex (female/male), age, Hispanic ethnicity (no/yes), and race (White, Black, Asian, Native American, Pacific Islander/Native Hawaiian, Other). For all analyses, ethnicity/race was dichotomized as non-Hispanic White versus other because of insufficient sample sizes among other ethnic/racial groups.

Cigarette Smoking Status

Participants reported on their current smoking status (“I used to smoke cigarettes, but I quit;” “I currently smoke cigarettes”).

Successful Quit Smoking Attempt Using E-Cigarettes

Participants reported on the longest amount of time during the past 2 years that they went without smoking specifically because they were using e-cigarettes (less than a day, less than a week, 1–3 weeks, 1 month, 2–3 months, 4–6 months, 7–9 months, 10–12 months, and more than a year). For invariance analyses, participants were considered “successful,” if they reported abstinence from smoking for at least 1 month.

E-Cigarette Use

Participants reported on past-month vaping frequency (0–30 days). For invariance analyses, two dichotomous variables reflecting any past-month e-cigarette (no/yes) and daily e-cigarette use (no/yes) also were calculated.

Total Duration of E-Cigarette Use

Participants reported on the total duration of e-cigarette use in their lifetime (less than a day, less than a week, 1–3 weeks, 1 month, 2–3 months, 4–6 months, 7–9 months, 10–12 months, and more than a year).

Nicotine E-Liquid Use

Participants reported whether they typically use nicotine in their e-cigarette (nicotine-free [no nicotine] only, nicotine only, both nicotine-free and nicotine, I don’t know). For invariance analyses, a variable was created indicating any nicotine use (no/yes). People reporting “I don’t know” (n = 33) were omitted from this variable.

E-Cigarette Dependence

The 4-item E-cigarette Dependence Scale⁹ was used to assess e-cigarette dependence. Internal reliability in the current sample was 0.88.

Withdrawal Symptoms

Participants completed the Minnesota Tobacco Withdrawal Scale (MTWS),¹⁰ which assesses nine symptoms of withdrawal from tobacco/nicotine products. Prior research has demonstrated that MTWS has solid psychometric properties,¹¹ and it is the most commonly used measure of withdrawal.¹² Response options for each item ranged from 1 “none” to 5 “severe.” Given that the parent study aimed to examine factors that relate to a successful quit smoking attempt using e-cigarettes, the instructions for the measure were as follows: “Early on in a quit attempt when you were using e-cigarettes/vapes to quit smoking, to what extent did you experience or feel any of the following?” Internal reliability in the current sample was 0.94.

The Modified E-Cigarette Evaluation Questionnaire

The MCEQ² is a 12-item measure that captures the reinforcing and aversive effects of smoking cigarettes. Prior analyses support a five-factor solution for cigarettes: Smoking Satisfaction (satisfying, taste good, enjoy smoking), Psychological Reward (calm down, more awake, less irritable, help concentrate, reduce hunger), Enjoyment of Respiratory Tract Sensations (enjoy the sensations in your throat and chest), Craving Reduction (immediately relieves craving), and Aversion (dizzy, nauseous). In the current study, we revised item wording to reflect vaping instead of smoking (eg, “I enjoy vaping” instead of “I enjoy smoking”; see Table 2 for items). We retained the original 7-point response scale (“not at all” to “extremely”).

Table 2.

Confirmatory Factor Analysis for the Original Five-Factor Structure of the MECEQ and the Novel Four-Factor Model

Original five-factor model			New four-factor model
Model fit indices	Full sample		Model fit indices	Random sample 2
	N = 857			n = 428
Root Mean Square Error of Approximation	0.048 (0.039 to 0.059)		Root Mean Square Error of Approximation	0.049 (0.033 to 0.066)
Bentler’s Comparative Fit Index	0.987		Bentler’s Comparative Fit Index	0.985
Standardized Root Mean Square Residual	0.023		Standardized Root Mean Square Residual	0.027
χ2 (61) = 6345.89***			χ2(55) = 2810.76***

MECEQ items by subscale	Loading	S.E.	MECEQ items by subscale	Loading	S.E.
Vaping Satisfaction	𝛼 = 0.90		Stimulant Effects	𝛼 = 0.89
1. E-cigarettes/vapes are satisfying	0.91	0.01	5. Vaping makes me feel more awake	0.85	0.02
2. Vaping tastes good	0.82	0.02	7. Vaping helps me concentrate	0.88	0.02
12. I enjoy vaping	0.89	0.01	8. Vaping reduces my hunger for food	0.79	0.03
Psychological Reward	𝛼 = 0.91		Positive Reinforcement	𝛼 = 0.88
4. Vaping calms me down	0.89	0.01	2. Vaping tastes good	0.80	0.03
5. Vaping makes me feel more awake	0.91	0.02	3. I enjoy the sensations of vaping in my throat and chest	0.86	0.02
6. Vaping makes me feel less irritable	0.86	0.02	12. I enjoy vaping	0.86	0.02
7. Vaping helps me concentrate	0.86	0.01	Negative Reinforcement	𝛼 = 0.89
8. Vaping reduces my hunger for food	0.71	0.02	4. Vaping calms me down	0.86	0.02
Enjoyment of Respiratory Tract Sensations	—		6. Vaping makes me feel less irritable	0.83	0.03
3. I enjoy the sensations of vaping in my throat and chest	1.00	0.00	11. Vaping immediately relieves my craving for a cigarette	0.8	0.03
Craving Reduction	—		Aversion	𝛼 = 0.87
11. Vaping immediately relieves my craving for a cigarette	1.00	0.00	9. Vaping makes me dizzy	0.90	0.03
Aversion	𝛼 = 0.86		10. Vaping makes me nauseous	0.86	0.03
9. Vaping makes me dizzy	0.89	0.03
10. Vaping makes me nauseous	0.86	0.03

𝛼 = Cronbach’s alpha; S.E. = standard error.

Factor loadings are standardized. The values in parentheses represent the 95% confidence interval for RMSEA.

***p < .001.

Data Analytic Plan

Mplus 8.6 was used for conducting analyses assessing the latent structure and measurement invariance of the MECEQ. SPSS 27 was used to conduct all remaining analyses.

Confirming the Five-Factor Latent Structure of the MECEQ

We fit the originally proposed five-factor model using confirmatory factor analysis to determine whether the original MCEQ structure fits our data for vaping. There were no missing data for the MECEQ. Full-information maximum likelihood with robust standard error was used to handle any non-normally distributed data and to produce fit indices. Bentler’s Comparative Fit Index (CFI) > 0.95,¹³ Root Mean Square Error of Approximation (RMSEA) < 0.07,¹⁴ and Standardized Root Mean Square Residual (SRMR) < 0.08¹⁵ were used as indices of adequate fit. Given that large samples produce significant chi-square statistics that are unreliable, we did not interpret chi-square statistics for any models although the values are included in Table 2.¹⁶

Exploratory/Confirmatory Factor Analysis (EFA/CFA) to Identify an Alternative Factor Solution for the MECEQ

To identify potentially viable factor solutions, we randomly split the full sample into two datasets. Within the first random sample, we ran an EFA with an oblique rotation (ie, goemin). We permitted 1–6 factors to be extracted. A combination of model fit indices, model comparisons generated within Mplus, solution interpretability, and number of items per factor was used to identify potentially suitable latent factor solutions. We then used confirmatory factor analysis to fit the optimal latent factor solution to the second half of the sample.

Evaluating the Internal Reliability of the MECEQ Subscales

Cronbach’s α values were calculated for all multi-item subscales.

Evaluating Relationships Between the Two Versions of the MECEQ Subscales

Bivariate correlations were run to examine relationships between the original five-factor MECEQ and the novel factor solution identified via EFA/CFA.

Evaluating Measurement Invariance of the Latent Structure of the MECEQ

We used Multigroup CFA to examine invariance of the five-factor and four-factor latent structure by sex (female/male), ethnicity/race (Non-Hispanic White/other), current cigarette smoking status (no/yes), having a successful quit smoking attempt using e-cigarettes (no/yes), past-month vaping (no/yes), daily vaping (no/yes), and nicotine e-liquid use (no/yes). Three levels of invariance were evaluated for each subgroup: configural (ie, invariance of the number of latent factors and items per factor), metric (ie, invariance of item factor loadings), and scalar (ie, invariance of item factor loadings and intercepts). If the model fits the data and all items loaded significantly onto their respective factors, configural invariance was met. When factor loadings were constrained to equality, metric invariance was achieved if changes in fit indices from the configurally invariant model did not exceed RMSEA ≥ 0.015, CFI ≥ 0.01, or SRMR ≥ 0.03.¹⁶ When factor loadings and intercepts were constrained to equality, if changes in fit indices from the metrically invariant model did not exceed CFI ≥ 0.01 (plus a decrement in SRMR ≥ 0.01 or RMSEA ≥ 0.015), scalar variance was achieved.¹⁶

Evaluating Between-Group Differences in MECEQ Scores

For all subgroups evidencing scalar invariance, which is a requirement for evaluating between-group differences,^17,18 we ran independent-samples t-tests to examine mean-level differences in subscale scores for the five-factor and four-factor versions of the MECEQ. To account for familywise error, we adjusted the threshold for statistical significance to α <0.001. While we only discuss findings that are significant at this level within the paper, we show findings that were significant at values of α < 0.05 and α < 0.01 within Table 3 because type II error may have been inflated by our conservative approach.

Table 3.

Between-Group Differences in Subscale Scores of the Five-Factor and Four-Factor Versions of the Modified E-Cigarette Evaluation Questionnaire

			Original five-factor structure
			Vaping Satisfaction	Psychological Reward	Respiratory Tract	Craving Reduction	Aversion
Sex	Male	449	4.99 (1.53)***	4.52 (1.63)***	4.58 (1.84)***	4.81 (1.71)***	2.80 (2.06)***
	Female	408	4.44 (2.69)	3.80 (1.68)	3.69 (1.99)	4.08 (1.89)	2.18 (1.55)
Non-Hispanic White vs Other Ethnicity/Race	Other	319	4.83 (1.49)	4.29 (1.56)	4.27 (1.82)	4.63 (1.75)*	2.34 (1.70)
	Non-Hispanic White	538	4.66 (1.71)	4.11 (1.77)	4.09 (2.04)	4.36 (1.87)	2.60 (1.94)*
Smoking Status	No	93	5.15 (1.40)**	4.54 (1.39)*	4.43 (1.81)*	5.22 (1.78)***	1.78 (1.24)
	Yes	764	4.67 (1.65)	4.13 (1.72)	4.14 (1.98)	4.37 (1.82)	2.59 (1.90)***
Successful Quit Smoking Attempt Using E-cigarettes	No	454	4.41 (1.72)	3.85 (1.76)	3.87 (2.05)	4.10 (1.90)	2.38 (1.73)
	Yes	403	5.08 (1.45)***	4.55 (1.54)***	4.48 (1.82)***	4.86 (1.66)***	2.65 (1.99)
Past-Month Vaping	No	125	3.30 (1.68)	2.87 (1.59)	2.59 (1.85)	3.44 (1.77)	2.23 (1.65)
	Yes	732	4.97 (1.49)***	4.40 (1.61)***	4.43 (1.86)***	4.63 (1.79)***	2.55 (1.89)
Daily Vaping	No	665	4.49 (1.63)	3.99 (1.71)	3.92 (1.97)	4.23 (1.80)	2.51 (1.82)
	Yes	192	5.56 (1.34)***	4.82 (1.48)***	4.99 (1.71)***	5.27 (1.70)***	2.48 (1.99)
Nicotine E-liquid Use	No	140	4.80 (1.63)	4.45 (1.64)	4.35 (1.89)	4.63 (1.61)	2.55 (1.98)
	Yes	684	4.76 (1.61)	4.17 (1.70)	4.16 (1.98)	4.49 (1.86)	2.52 (1.85)

			New four-factor structure
			Stimulant Effects	Positive Reinforcement	Negative Reinforcement	Aversion
Sex	Male	449	4.37 (1.73)***	4.86 (1.56)***	4.77 (1.60)***	2.80 (2.06)***
	Female	408	3.54 (1.77)	4.22 (1.71)	4.15 (1.72)	2.18 (1.55)
Non-Hispanic White vs Other Ethnicity/Race	Other	319	4.10 (1.68)	4.64 (1.52)	4.59 (1.55)	2.34 (1.70)
	Non-Hispanic White	538	3.90 (1.86)	4.51 (1.75)	4.40 (1.76)	2.60 (1.94)*
Smoking Status	No	93	4.13 (1.55)	4.84 (1.44)*	5.18 (1.46)***	1.78 (1.24)
	Yes	764	3.96 (1.82)	4.53 (1.69)	4.39 (1.69)	2.59(1.90)***
Successful Quit Smoking Attempt using E-cigarettes	No	454	3.67 (1.84)	4.29 (1.75)	4.11 (1.76)	2.38 (1.73)
	Yes	403	4.32 (1.68)***	4.87 (1.51)***	4.89 (1.49)***	2.65 (1.99)
Past-Month Vaping	No	125	2.65 (1.66)	3.06 (1.69)	3.28 (1.60)	2.23 (1.65)
	Yes	732	4.20 (1.72)***	4.82 (1.52)***	4.68 (1.61)***	2.55 (1.89)
Daily Vaping	No	665	3.91 (1.82)	4.40 (1.68)	4.32 (1.68)	2.51 (1.82)
	Yes	192	4.29 (1.66)*	5.31 (1.40)***	5.18 (1.51)***	2.48 (1.99)
Nicotine E-liquid Use	No	140	3.94 (1.80)	4.62 (1.63)	4.62 (1.56)	2.55 (1.98)
	Yes	684	4.34 (1.73)*	4.59 (1.66)	4.49 (1.70)	2.52 (1.85)

Findings that were significant at p < .001 are reported in the manuscript to account for familywise error and are bolded in the current table. However, findings that were significant at p < .05 and p < .01 are noted in the table because future research may be conducted using this measure to address more targeted research questions. Data are presented as Mean (standard deviation). Sex (female/male), Smoking Status (current smoking no/yes), Successful Quit Smoking Attempt Using E-cigarettes (no [< 1 month]/yes [≧ 1 month]), Past-Month Vaping (no/yes), Daily Vaping (no/yes), and Nicotine E-liquid Use (no/yes).

*p < .05; *p < .01; ***p < .001.

Examining Adjusted Relationships Between MECEQ Scores and Vaping Outcomes

We ran univariate general linear models (GLM) to evaluate adjusted relationships between the five-factor and four-factor MECEQ subscales, respectively, and vaping frequency, vaping duration, e-cigarette dependence, and withdrawal. Covariates for all models included sex, age, ethnicity/race, cigarette smoking status, and nicotine e-liquid use. Past-month vaping frequency also was included for the models predicting dependence and withdrawal.

Results

Descriptive Statistics

Descriptive statistics for all study variables are presented in Table 1.

Table 1.

Participant Demographics

Participant characteristics	% or Mean (SD)
Male sex	52.4%
Age	40.84 (12.25)
Non-Hispanic White	62.8%
Currently smoking cigarettes	89.1%
Successful quit smoking attempt using E-cigarettes	47.0%
Nicotine E-liquid use	79.8%
Past-month vaping frequency (# days out of 30)	15.98 (10.69)
Any past-month vaping	85.4%
Daily vaping	22.4%
Total vaping duration	6.03 (2.45)
E-cigarette dependence	1.91 (10.8)
Withdrawal	24.52 (9.77)
MECEQ (original five-factor model)
Satisfaction	4.73 (1.63)
Psychological Reward	4.18 (1.69)
Respiratory Tract	4.16 (1.97)
Craving Reduction	4.46 (1.83)
Aversion	2.51 (1.86)
MECEQ (revised four-factor model)
Stimulant Effects	3.98 (1.80)
Positive Reinforcement	4.56 (1.67)
Negative Reinforcement	4.47 (1.68)
Aversion	2.51 (1.86)

Confirming the Five-Factor Latent Structure of the MECEQ

The five-factor latent structure fits the data well, with each item loading significantly onto its respective factor (p-values < .001; see Table 2, the section titled “Original five-factor model”).

Exploratory/Confirmatory Factor Analysis (EFA/CFA) to Identify an Alternative Factor Solution for the MECEQ

The EFA generated one-, two-, three-, and four-factor solutions. Model fit was poor for the one- and two-factor solutions, and the three-factor solution contained a factor upon which no items loaded above 0.40. Fit indices suggested that the four-factor model fits the data well (RMSEA = 0.056, CFI = 0.990, SRMR = 0.009), and model comparisons indicated that the four-factor solution was superior to the one-, two-, and three-factor solutions (p < .001). Furthermore, the four-factor latent structure was interpretable, and all latent factors comprised multiple items with primary loadings of at least 0.52 and cross-loadings less than 0.30 (see Supplemental Table 1 for item factor loadings by subscale). The four factors reflected Stimulant Effects, Positive Reinforcement, Negative Reinforcement, and Aversion (which was the only subscale that mirrored the five-factor structure). Note that 1 of the 12 original items (“E-cigarettes/vape are satisfying”) was marked for deletion because it loaded onto Negative Reinforcement at 0.53 with a cross-loading on Positive Reinforcement of 0.40.

The CFA indicated that the 11-item, four-factor model fits the data well, with each item loading significantly onto its respective factor (p-values < .001; see Table 2, the section titled “New four-factor model”).

Evaluating the Internal Reliability of the MECEQ Subscales

For the original five-factor solution, each of the multi-item subscales was internally consistent (Vaping Satisfaction [α = 0.90]; Psychological Reward [α = 0.91]; Aversion [α = 0.86]). For the four-factor solution, each subscale also evidenced good internal consistency (Stimulant Effects [α = 0.89]; Positive Reinforcement [α = 0.88]; Negative Reinforcement [α = 0.89]; Aversion [α = 0.87]).

Evaluating Relationships Between the Two Versions of the MECEQ Subscales

Significant correlations were observed among the subscales of the original five-factor version of the MECEQ and the four-factor version of the MECEQ (Supplemental Table 2).

Evaluating Measurement Invariance of the Latent Structures of the MECEQ

The latent structures of the five-factor and four-factor versions of the MECEQ were scalar invariant within each subgroup tested (Supplemental Table 3), permitting between-group differences to be examined.

Evaluating Between-Group Differences in MECEQ Scores

The Original Five-Factor Model

Individuals, who identified as male, had a successful quit smoking attempt using e-cigarettes, used e-cigarettes in past-month, and used e-cigarettes daily reported higher average ratings of vaping satisfaction, psychological reward, enjoyment of respiratory tract effects, and craving reduction than did their respective counterparts (Table 3). Participants who had quit smoking reported greater craving reduction than did those who continued to smoke. Finally, males and current cigarette smokers reported higher aversion than did females and those who had quit smoking. No differences in MECEQ subscale scores were observed based on nicotine e-liquid use.

The New Four-Factor Model

Individuals, who identified as male, had a successful quit smoking attempt using e-cigarettes, and used e-cigarettes in the past month reported higher average ratings for stimulant effects, positive reinforcement, and negative reinforcement (Table 3). Individuals who used e-cigarettes daily reported higher average ratings for positive and negative reinforcement than did non-daily vapers. Individuals who had quit smoking cigarettes reported greater negative reinforcement from vaping than did current smokers. Finally, males and current cigarette smokers reported higher aversion than did females and those who had quit smoking. Again, no differences in MECEQ subscale scores were observed based on nicotine e-liquid use.

Examining Adjusted Relationships Between MECEQ Scores and Vaping Outcomes

The Original Five-Factor Model

After accounting for covariates, reporting greater vaping satisfaction, greater enjoyment of respiratory tract sensations, and weaker aversive effects was associated with increased past-month vaping frequency and longer total vaping duration (Table 4). Stronger psychologically rewarding effects and aversion were associated with increased e-cigarette dependence and withdrawal. Finally, experiencing more craving reduction was associated with reduced withdrawal symptoms.

Table 4.

Adjusted Relationships Between the Modified E-Cigarette Evaluation Questionnaire Subscales and Vaping-Related Outcomes

	Original five-factor model
	Past-month vaping frequency						Total vaping duration						Dependence						Total Withdrawal Symptoms
	B	SE	t	95% CI		ηp2	B	SE	t	95% CI		ηp2	B	SE	t	95% CI		ηp2	B	SE	t	95% CI		ηp2
Intercept	4.93	1.88	2.62	1.23	8.62	0.01***	2.65	0.44	6.02	1.79	3.51	0.04***	0.49	0.18	2.74	0.14	0.84	0.01**	23.36	1.81	14.59	22.81	29.90	0.21***
Female sex	0.48	0.67	0.72	−0.82	1.79	0.00	−0.13	0.16	−0.84	−0.44	0.18	0.00	−0.13	0.06	−2.12	−0.26	−0.01	0.01	−1.44	0.64	−2.27	−2.69	−0.19	0.01*
Age	−0.07	0.03	−2.49	−0.13	−0.02	0.01*	0.03	0.01	4.41	0.02	0.04	0.02***	0.00	0.00	−0.07	−0.01	0.01	0.00	−0.10	0.03	−3.45	−0.15	−0.04	0.01**
Race other than NH White	−0.04	0.67	−0.05	−1.36	1.29	0.00	0.15	0.16	0.96	−0.16	0.46	0.00	−0.23	0.06	−3.55	−0.35	−0.10	0.02***	−0.90	0.64	−1.39	−2.16	0.37	0.00
Smoking Status (no)	4.07	1.05	3.87	2.00	6.14	0.02***	1.12	0.25	4.56	0.64	1.61	0.03***	−0.09	0.10	−0.93	−0.29	0.10	0.00	−0.16	1.02	−0.16	−2.16	1.83	0.00
Nicotine Concentration	1.42	0.86	1.66	−0.26	3.10	0.00	−0.04	0.20	−0.19	−0.43	0.35	0.00	−0.27	0.08	−3.29	−0.43	−0.11	0.01**	−2.71	0.82	−3.31	−4.32	−1.10	0.01**
Past-Month Vaping Frequency	—	—	—	—	—	—	—	—	—	—	—	—	0.00	0.00	0.92	0.00	0.01	0.00	−0.09	0.03	−2.78	−0.16	−0.03	0.01*
Satisfaction	2.09	0.42	4.99	1.27	2.92	0.02***	0.51	0.10	5.19	0.32	0.70	0.03***	−0.03	0.04	−0.83	−0.11	0.05	0.00	−0.45	0.41	−1.10	−1.25	0.35	0.00
Psychological Reward	0.37	0.40	0.91	−0.43	1.16	0.00	−0.12	0.09	−1.29	−0.31	0.06	0.00	0.29	0.04	7.49	0.21	0.36	0.07***	1.48	0.39	3.84	0.72	2.24	0.02***
Respiratory Tract	0.68	0.30	2.25	0.09	1.27	0.01*	0.18	0.07	2.52	0.04	0.32	0.01**	0.03	0.03	0.87	−0.03	0.08	0.00	−0.11	0.29	−0.37	−0.67	0.46	0.00
Craving Reduction	0.09	0.29	0.30	−0.48	0.66	0.00	0.07	0.07	1.09	−0.06	0.21	0.00	0.02	0.03	0.79	−0.03	0.08	0.00	−0.68	0.28	−2.46	−1.23	−0.14	0.01*
Aversion	−0.60	0.20	−3.09	−0.99	−0.22	0.01**	−0.35	0.05	−7.62	−0.44	−0.26	0.07***	0.14	0.02	7.53	0.11	0.18	0.07***	1.86	0.19	9.91	1.49	2.23	0.11***
	New four-factor model
Intercept	6.21	1.82	3.42	2.64	9.78	0.02***	1.05	0.19	5.55	0.68	1.42	0.04***	0.44	0.18	2.47	0.09	0.79	0.01*	26.47	1.79	14.77	22.95	29.98	0.21***
Female Sex	−0.10	0.63	−0.16	−1.34	1.14	0.00	−0.02	0.07	−0.36	−0.16	0.11	0.00	−0.13	0.06	−2.06	−0.25	−0.01	0.01*	−1.56	0.64	−2.45	−2.81	−0.31	0.01*
Age	0.01	0.03	0.46	−0.05	0.07	0.00	0.01	0.00	4.91	0.01	0.02	0.03***	0.00	0.00	−0.20	−0.01	0.01	0.00	−0.10	0.03	−3.75	−0.16	−0.05	0.02***
Race other than NH White	−0.20	0.64	−0.32	−1.46	1.05	0.00	0.06	0.07	0.94	−0.07	0.20	0.00	−0.23	0.06	−3.53	−0.35	−0.10	0.02***	−1.00	0.65	−1.54	−2.27	0.27	0.00
Smoking Status (no)	6.25	1.03	6.08	4.23	8.27	0.05***	0.47	0.11	4.39	0.26	0.68	0.02***	−0.13	0.10	−1.26	−0.32	0.07	0.00	−0.47	1.03	−0.46	−2.49	1.55	0.00
Nicotine Concentration	1.05	0.81	1.31	−0.53	2.64	0.00	0.01	0.09	0.11	−0.16	0.18	0.00	−0.26	0.08	−3.14	−0.42	−0.10	0.01**	−2.68	0.83	−3.25	−4.30	−1.06	0.01*
Past-Month Vaping Frequency	—	—	—	—	—	—	—	—	—	—	—	—	0.00	0.00	0.84	0.00	0.01	0.00	−0.09	0.03	−2.81	−0.16	−0.03	0.01**
Stimulant Effects	−0.09	0.32	−0.27	−0.72	0.55	0.00	−0.06	0.04	−1.71	−0.13	0.01	0.00	0.14	0.03	4.21	0.07	0.20	0.02***	0.70	0.33	2.13	0.05	1.34	0.01*
Positive Reinforcement	1.74	0.38	4.54	0.99	2.49	0.03***	0.24	0.04	6.13	0.16	0.32	0.04***	−0.02	0.04	−0.47	−0.09	0.06	0.00	−0.64	0.38	−1.67	−1.39	0.11	0.00
Negative Reinforcement	0.88	0.39	2.26	0.12	1.64	0.01*	0.07	0.04	1.84	−0.01	0.15	0.04*	0.19	0.04	4.88	0.11	0.26	0.03***	0.23	0.38	0.61	−0.52	0.99	0.00
Aversion	−0.33	0.19	−1.78	−0.70	0.04	0.00	−0.16	0.02	−7.90	−0.20	−0.12	0.07***	0.15	0.02	7.79	0.11	0.19	0.07***	1.88	0.19	9.86	1.50	2.25	0.11***

B = beta; CI = confidence interval; NH = Non-Hispanic; OR = adjusted odds ratio; SE = standard error; η_p² = partial eta squared.

*p < .05; **p < .01; ***p < .001.

The New Four-Factor Model

Reporting experiencing stronger positive and negative reinforcement was positively associated with past-month vaping frequency and total vaping duration (Table 4). Experiencing less aversion also was associated positively with total vaping duration. Experiencing stronger stimulant effects and more aversion was associated with both withdrawal and dependence. Finally, experiencing stronger negative reinforcement was also positively associated with dependence.

Discussion

Although studies have been published in which the five-factor MCEQ was revised to apply to e-cigarettes, the current study is the first of which we are aware to examine the psychometric properties of the measure for use with e-cigarettes. Results suggest that the MECEQ can be used to assess vaping effects in a reliable and valid manner moving forward. Given observed support for the original five-factor structure of the MCEQ, our findings strengthen the interpretability of prior research using the previously unvalidated five-factor measure. However, results also suggest that the latent structure of the MECEQ may reflect four subscales: stimulant effects, positive reinforcement, negative reinforcement, and aversion. Of note, only the aversion subscale overlaps with the original five-factor structure.

With regard to specific findings, both the original five-factor latent structure of the MCEQ and the novel, four-factor solution that was identified via EFA were confirmed in our data. Consistent with strong item factor loadings, internal reliability for all multi-item subscales was good to excellent across both versions of the measure. In addition, measurement invariance testing indicated support for using both versions of the measure to examine between-group differences by sex, non-Hispanic white versus other ethnicity/race, current cigarette smoking status, success of a previous quit smoking attempt using e-cigarettes, past-month vaping status, daily vaping status, and the use of nicotine-free versus nicotine e-liquids.

Both the five-factor and four-factor versions of the MECEQ were sensitive to detecting between-group differences by sex, smoking status, success of a quit smoking attempt using e-cigarettes, past-month vaping status, and daily vaping status. Although the findings regarding sex need further investigation, other effects related to e-cigarette use align with expectations. For instance, we would expect individuals who currently use e-cigarettes daily to experience more reinforcing effects than those who use them less frequently, supporting their continued use.

When considering adjusted relationships between the MECEQ subscales and vaping-related outcomes, the observed pattern of findings for the original five-factor version was both expected and somewhat unexpected. With regard to expected effects, experiencing immediate craving reduction would be expected to be inversely related to experiencing withdrawal.¹⁹ Furthermore, the fact that stronger experiences of satisfaction, stronger enjoyment of respiratory tract sensations, and weaker experiences of aversion were associated with vaping frequency and duration was expected because experiencing more positive effects and less aversive effects should contribute to continued use. However, psychological reward, which is considered a subscale comprising positive effects, unexpectedly was not associated with vaping frequency or duration. Rather, psychological reward was positively associated with the experience of dependence and withdrawal. While this appears counterintuitive at first, a closer examination of some of the items in the psychological reward subscale may help to explain the findings. For example, “Vaping makes me feel less irritable,” Vaping calms me down,” and “Vaping helps me concentrate” could be conceptualized as positive effects of vaping. However, these positive effects may actually reflect the relief of negative effects associated with dependence and withdrawal (consistent with the grouping of these three items onto the Negative Reinforcement subscale of the four-factor MECEQ).¹⁹ For instance, when someone who is dependent on nicotine enters a state of withdrawal, they often feel irritable. Thus, vaping may make someone feel less irritable because it relieves the irritability associated with being in withdrawal.¹⁹ The same rationale may explain the positive relationships between aversion (feeling dizzy, feeling nauseous) and dependence and withdrawal; the aversive effects a person experiences and reports may be due to withdrawal rather than a direct result of vaping.

Finally, the observed pattern of findings for the new four-factor version of the MECEQ was also both expected and somewhat unexpected. The fact that experiencing stronger positive and negative reinforcement was associated with vaping frequency and duration of use was expected, as experiencing more positive effects understandably should contribute to continued use. Although possibly counterintuitive (as was described above for five-factor MECEQ), the observed positive relationship between negative reinforcement and dependence may be linked to the need to continue using e-cigarettes to avoid the experience of anxiety, irritability, and cravings (however, it is not clear why a positive relationship between negative reinforcement and withdrawal symptoms, specifically, was not observed). Again, the same rationale may be applied to interpreting the positive relationships between aversion and both dependence and withdrawal; aversive effects may be due to withdrawal rather than a direct result of vaping. Finally, given the novelty of the four-factor structure, additional research is needed to explain the observed pattern of findings for stimulant effects. If stimulant effects like feeling more awake and having better concentration are conceptualized as positive effects, we may have expected to observe relationships with e-cigarette use (ie, frequency and duration). However, stimulant effects only were associated with dependence and withdrawal. Similar to what was observed for the negative reinforcement subscale, it may be that people who experience more increased wakefulness and concentration when vaping may be combating withdrawal symptoms. However, alternative explanations must be explored.

While the findings provide psychometric support for scoring the MECEQ as a five-factor and four-factor measure, several limitations merit note. Data were collected online from Qualtrics panelists who had tried to quit smoking using e-cigarettes in the past 2 years, which could limit generalizability. However, this concern is mitigated by the demonstration of measurement invariance for both versions of the measure by current smoking and vaping status, suggesting that both variants can be used reliably regardless of current smoking or vaping status. That said, future research is needed to evaluate the psychometric properties of the five-factor and four-factor versions of the measure in other samples (eg, exclusive e-cigarette users with no history of smoking cigarettes, youth). In addition, the five-factor MECEQ comprises two single-item subscales (respiratory tract effects and craving reduction). While not a fatal flaw, it is not possible to estimate model parameters (eg, factor loadings) for single-item subscales. Future research might consider testing the utility of additional items for these subscales. Furthermore, the measure of withdrawal used in the current study was specific to withdrawal experienced during a quit smoking attempt using e-cigarettes. Future research is needed to examine the relationship between MECEQ subscales and withdrawal experienced exclusively as a result of refraining from vaping. Finally, while there was racial diversity within the sample, there was insufficient representation of individual ethnic/racial minority groups to permit a thorough examination of the utility of the five-factor and four-factor versions of the MECEQ for use in diverse subsamples. Thus, future research is needed in more racially diverse populations.

In sum, both the five-factor and four-factor versions of the MECEQ evidenced construct validity, scalar measurement invariance for the several subgroups assessed, internal reliability, sensitivity to detecting between-group differences, and concurrent validity with several vaping-related outcomes. The five-factor scoring may facilitate direct comparisons of subjective effects across vaping and smoking in future studies. However, additional psychometric analyses are needed to ensure that the five-factor structure is invariant by product type (ie, cigarettes and e-cigarettes) before direct comparisons can be made. The four-factor scoring of the MECEQ may have additional utility for addressing research questions that are specific to e-cigarettes. In addition, given that published psychometric work on the original MCEQ is limited to a single study,² future research is needed to examine the applicability of the newly identified four-factor structure to cigarettes and other tobacco/nicotine products. That said, the current study supports the use of the MECEQ in its either five-factor or four-factor form for assessing adults’ subjective experience of reinforcing and aversive vaping effects.

Supplementary Material

A Contributorship Form detailing each author’s specific involvement with this content, as well as any supplementary data, are available online at https://academic.oup.com/ntr.

Funding

Efforts by MEM and KWB were supported by grant number U54DA036151 from the National Institute on Drug Abuse of the National Institutes of Health and the Center for Tobacco Products of the US Food and Drug Administration. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the US Food and Drug Administration.

Declaration of Interests

MEM and KWB have no conflicts of interest to declare related to the current study.

Data Availability

Data will be made available upon reasonable request to the corresponding author.