Methods of the ITC Four Country Smoking and Vaping Survey, Wave 1 (2016)

Abstract

Background

This paper describes the methods of the 2016 International Tobacco Control (ITC) Four Country Smoking and Vaping (4CV) Survey, conducted in 2016 in Australia (AU), Canada (CA), England (EN), and the United States (US).

Methods

The respondents were cigarette smokers, former smokers (quit within the previous two years), and at-least-weekly vapers, aged 18 and older. Eligible cohort members from the ITC Four Country Survey (4C) were retained. New respondents were sampled by commercial firms from their panels. Where possible, ages 18–24 and vapers were over-sampled. Data were collected online, and respondents were remunerated. Survey weights were calibrated to benchmarks from nationally representative surveys.

Results

Response rates by country for new recruits once invited ranged from 15.2% to 49.6%; cooperation rates were above 90%. Retention rates from the 4C cohort ranged from 35.7% to 44.2%. Sample sizes for smokers/former smokers were 1504 in AU, 3006 in CA, 3773 in EN, and 2239 in the US. Sample sizes for additional vapers were 727 in CA, 551 in EN, and 494 in the US.

Conclusion

The ITC 4CV Survey design and data collection methods allow analyses to examine prospectively the use of cigarettes and NVPs in jurisdictions with different regulatory policies. The effects on the sampling designs and response quality of recruiting the respondents from commercial panels are mitigated by the use of demographic and geographic quotas in sampling; by quality control measures; and by the construction of survey weights taking into account smoking/vaping status, sex, age, education, and geography.

Introduction

This paper describes Wave 1 of the ITC Four Country Smoking and Vaping (4CV) Survey conducted from July to November 2016 in Australia (AU), Canada (CA), England (EN), and the United States (US). The 4CV Survey is an expansion of the original ITC Four Country (4C) Survey^{[1],[2],[3],[4]}, which had focused on evaluating the impact of tobacco control policies on smoking-related beliefs, attitudes and behaviors. During the 13-year period 2002–2015, between 9 and 11 waves were conducted in each of the four countries.

This paper briefly outlines the objectives of the 4CV Survey, the sampling design for the survey, data collection, survey outcome rates such as response and completion rates, and the approach used for weighting data in an effort to generate a sample broadly representative of each country’s population of adult smokers, recent former smokers and vapers at the time of the survey. The paper also summarizes the strengths and weaknesses of the 4CV Survey methodology. A full description of the methods of the 4CV Survey is available in the ITC 4CV Wave 1 (4CV1) Technical Report^[11].

Objectives of the ITC 4CV Survey

Responding to the rapid evolution of the nicotine delivery market, the ITC Project team designed the 4CV Survey to address important issues regarding the use of this broad array of products, with special attention to the use of nicotine vaping products (NVPs). An important specific objective was to examine how policies on smoked tobacco, NVPs and other alternative nicotine products may influence the use of these classes of products and transitions among them by present and former smokers. The behavioral effects of these policies on known psychosocial precursors (mediators) of behaviors such as quit attempts and quit success can serve as inputs to simulation modeling efforts to estimate the effects on population health^[3],[4],[5].

The 4CV Survey provides an opportunity to study these questions in jurisdictions with different regulatory policies^{[7],[8],[9],[10]}. The policy environments for smoking and NVPs in the four countries and the conceptual model that describes how policies on cigarettes (and other smoked tobacco products), NVPs and other alternative nicotine products are hypothesized to affect use of these products and the transitions among them are described in greater detail by Fong et al. (2018)^[6].

4CV1 Survey content

The 4CV Wave 1 (4CV1) questionnaire addressed vaping patterns, both current and past; brand choices and purchasing; reasons for use; knowledge, beliefs, and perceptions of harmfulness of NVPs; and attitudes toward regulation. At the same time, all sections of the 4C questionnaire on cigarette policies were retained, although the number of specific measures was reduced to allow for the addition of questions about NVPs. Table 1 provides a list of measures in the 4CV1 questionnaire on vaping and on cigarette smoking. More detailed information on the topics is available in the 4CV1 Technical Report^[11] and the questionnaire^[12].

Table 1.

Measures in the 4CV1 Survey questionnaire

Demographic Variables: Gender, age, ethnicity, education, income, state of health

Other personal moderators: Quitting history, nicotine dependence, levels of stress including financial stress and depressed mood, use of intoxicants (e.g., alcohol, cannabis), and experiences of use

Environmental moderators: Number of smokers/NVP users in the household, and in social network

Policy-specific (proximal) variables (same measures for cigarettes/smoking and NVP use unless indicated):

1) Price paid per unit of product, total weekly cost, product type/variant, purchasing unit, price perceptions

2) Use of cessation services, recall of advice, NVP and/or other medicines use in conjunction with professional assistance, advice on appropriateness of NVP use

3) Advertising/ marketing: noticing advertising and frequency in key channels (TV, print, internet), susceptibility to advertising, reports about whether NVP advertising makes respondent think about cigarettes

4) Health warnings and packaging: salience and noticing of health warnings (if any), brand usage, perceived risks, perceived impact on product use; forgoing cigarettes/NVPs because of the warnings

5) Vapor-free laws: exposure to vaping across a range of settings, perceived impact on product use, reports on restrictions

6) Restrictions on access: perceived availability

7) Nicotine content, flavor and other product characteristics: nicotine content and flavors of vaping brands used, perceived addictiveness of NVPs and cigarettes, and NVP appeal

8) Media campaigns: awareness and recall of media campaigns on NVPs

Non-policy-specific (distal) psychosocial mediator variables: Social norms for both vaping and smoking, outcome expectancies, intentions for NVP use, reasons for NVP use, self-efficacy and intentions to quit smoking; relative harmfulness, health concerns, functions of smoking, substitutability of functions to NVP.

NVP and tobacco use behaviors: Key outcomes along with some of the distal variables for intermediary analyses. Use of NVPs and other nicotine products: frequency of use, duration, and intensity of use (e.g., cigarettes per day); usual brand/type of product; quit attempts (smoking), duration of abstinence (smoking), product switching.

Focus on population subgroups

There are some important differences between the designs of the 4CV Survey and the 4C Survey because of the new research directions and the need for focus on certain subgroups. Because of our interest in NVP use and how it relates to smoking cessation, we included people who were former smokers of cigarettes (within two years) at the time of recruitment; we were interested in the stability of cessation vs. relapse over this period and the fact that NVPs had only recently gained popularity as a cessation method. We expanded the recruitment criteria to include current (at least) weekly vapers, and oversampled these in CA, EN, and the US to ensure sufficient numbers were captured in our sample for analysis purposes; it was judged that vaping at least weekly would be necessary for NVP use as an aid to smoking cessation^[13],[14]. In AU, a supplementary dedicated vaper sample was recruited via referral sampling from online vaper forums and vape stores. Data from this group are not included in the core data set used for most analyses; given the sources and sampling method these respondents are not expected to be representative of the population of all Australian vapers. Since other studies had noted that younger adults were most likely to take up vaping (e.g. Kasza et al.^[15]), smokers aged 18–24 were oversampled in CA, EN and the US. Unfortunately, limited resources precluded over-sampling younger adults in AU.

Inclusion criteria for 4C respondents

Cohort members in the final 4C wave in each of the four countries were invited to the 4CV Survey. In AU, members of the existing cohort were excluded if they had not smoked cigarettes for more than two years and did not vape at least weekly, at the time of the 4CV1 Survey. In the other three countries, former smokers who had not smoked cigarettes for more than two years were retained, to provide supplementary samples of longer term quitters.

Inclusion criteria for new respondent subgroups

New respondents in each country were recruited from the panels of commercial survey firms. A new respondent (aged 18 or older) could be recruited as someone who:

• had smoked at least 100 cigarettes in lifetime and was currently smoking at least monthly, or “less than monthly but occasionally”); or

• had smoked at least 100 cigarettes in lifetime and had quit within the previous two years; or

• was currently vaping at least weekly.

Note that a respondent eligible because of smoking or cessation status might vape only monthly, occasionally, or not at all; likewise, a respondent eligible because of vaping status might be a long-term quitter or a never smoker. The recruitment of less-than-monthly smokers was motivated by the wish to study transitions of occasional smokers into smoking, vaping or cessation over time.

The use of commercial firms for recruitment

The gold standard of probability sampling, such as address-based sampling with face-to-face recruitment, or random-digit dialing (RDD) with telephone recruitment, is becoming increasingly difficult to attain, due to the high costs of maintaining strong contact and participation rates. The ITC Project has witnessed this dramatic increase in the difficulty in conducting national surveys employing rigorous probability sampling methods. The 4C Wave 1 Survey in 2002 was a telephone survey with recruitment by RDD. This method was maintained for the first six waves, with ever-increasing costs. From Wave 7 (2008–09) onwards, we began to move our survey administration over to the web, while continuing to recruit respondents through RDD. The shift was gradual, to allow modeling to take account of measurement differences between the two survey modes, and to prevent the loss of participants who were without web-access or who otherwise preferred telephone administration. The RDD method was supplemented in Waves 9 and 10 (2013–2014) by recruitment from commercial panels, as an increasing proportion of the sample members (including telephone recruits) were responding online.

At the final 4C wave (ending in 2015), the data collection costs had tripled and become unattainable. From considerations of practicality and cost, it was decided that new recruits to the 4CV Survey would be entirely sourced from web panels in each of the four countries. We chose panels that (1) had high standards regarding the recruitment of their panel members¹; (2) offered strong collaborative relationships with our survey management team in survey development, programming, testing, and validation; and (3) quoted higher expected rates of retention over time. All survey response data were to be collected through an online questionnaire, hosted by the Survey Research Centre (SRC) at the University of Waterloo².

Below we describe the sampling designs and further details on data collection for the ITC 4CV1 Survey; survey outcome rates; and the construction of survey weights.

Sample Sizes, Sampling Designs, and Data Collection

In CA, EN, and the US, the new recruit sample for the 4CV1 Survey was to consist mainly of cigarette smokers and former smokers (within two years), aged 18 and older. Because of the importance of longitudinal analysis over waves, the 4CV1 sample sizes for the smoker/former smoker respondents aged 25+ were chosen so as to provide at least 1400 who would be present in both Wave 1 and Wave 2 (to be conducted 18 months later), using retention rate estimates that the survey firms provided at the design stage. Table 2 shows the original 4CV1 target sample sizes and realized sample sizes, for this group and the over-sampled ones.

Table 2:

Original target sample sizes (realized sample sizes)

Component sample	AU	CA	EN	US
Smokers/former smokers aged 25+	1500 (1504)	2350 (2271)	2960 (2886)	1590 (1725)
Smokers/former smokers aged 18–24		750 (735)	1100(887)	500 (514)
Additional (at least) weekly vapers	N/A	715 (727)	500 (551)	500 (494)

Note: Realized sample sizes in “Smokers/former smokers” rows include also some long term former smokers in the ITC cohort retained by virtue of vaping at least weekly; the vapers in the final row are all newly recruited in 4CV1.

In AU, the sampling design was targeted at 1500 respondents satisfying the eligibility criteria, without any over-sampling.

To mitigate potential non-representativeness of the samples from the survey firms, geographic, age, and sex quotas were applied to sample sizes as indicated in Table 3. Further details on the determination of quotas are given in the 4CV1 Technical Report^[11].

Table 3:

Types of Quotas Applied to Sample Composition

Sample component	Quota cells
AU smokers/former smokers age 18+	Geographic regions crossed with sex (realized approximately)
CA, US smokers/former smokers age 18–24	Geographic regions; language in Canada
CA smokers/former smokers age 25+	Geographic regions; language
US smokers/former smokers age 25+	None; most of the sample was from the 4C cohort, expanded in 4C Wave 9
CA, US, EN additional vapers age 18+	Geographic regions
EN smokers/former smokers age 18–24	Geographic regions crossed with sex
EN smokers/former smokers age 25+	Geographic regions crossed with sex and age group

Sample sources

Table 4 shows the sources and their sample sizes in each of the countries.

Table 4:

Sources and sample sizes for 4CV1 respondents

Country, subsample	Sources	Sample size
AU, smokers/former smokers 18+	ITC 4C cohort (retention rate 43.1%)	515
	Roy Morgan Single Source (probability-based)	504
	Survey Sampling International	485
CA, smokers/former smokers 18+	ITC 4C cohort (retention rate 41.6%)	567
	Léger Marketing, where possible from their probability-based panel	2439
CA, additional vapers 18+	Léger Marketing	727
EN, smokers/former smokers 18+	ITC 4C cohort (retention rate 35.7%)	254
	Ipsos	3519
EN, additional vapers 18+	Ipsos	551
US, smokers/former smokers aged 25+	ITC 4C cohort	1372
	GfK Knowledge Panel (probability based)	127
	Ipsos	226
US, smokers/former smokers aged 18–24	ITC 4C cohort (retention rate 44.2%)	6
	Lucid (GfK partner panel, opt-in)	496
	Ipsos	12
US, additional vapers 18+	Ipsos	494
Total		12294

Remuneration of respondents

The 4CV1 questionnaire was lengthy; median survey lengths by country ranged from 39 to 48 minutes (Table 5). The survey tended to be longer for respondents who both smoke and vape, and respondents who had made failed quit attempts. It was shorter for users of cigarettes only or vaping products only, and for respondents who had quit completely. Because of the significant time commitment, respondents received remuneration for their time upon completion of the questionnaire. The remuneration took various forms (cheque, gift card, reward points, entry into a prize draw), depending on the source and country of respondents. Remuneration by cheque ranged between $20 and $30 (US). The remuneration not only aided recruitment, but also is expected to impact retention rates at future waves favorably. Remuneration details are available in the 4CV1 Technical Report^[11].

Table 5:

Outcome rates by country, for new recruits

Country	AU	CA	EN	US-GfK	US-all***
1. Total interviewed	1575	3182	4129	131	1377
2. Refusals/breakoffs	116	309	328	12	335
3. Not known if eligible*	6338	67194	171438	434	1422
4. Estimated rate of eligibility and quota-not -full	49.3%	19.7%	13.3%	28.0%	36.6%
5. Estimated number of eligible and quota-not-full non-respondents in 3.	3123	13204	22784	122	580
6. Response rate****	32.7%	19.1%	15.2%	49.6%	60.1%
7. Cooperation rate****	93.1%	91.1%	92.6%	91.6%	80.4%
8. Median survey length**	48 min	44 min	39 min	NA	45 min

^*Sent to Survey Research Centre (SRC), did not respond, unknown if eligible. Roy Morgan (and their partner SSI) in AU and IPSOS in EN pre-screened respondents before sending them to SRC’s website. Léger in CA and GfK in the US pre-identified individuals that would likely be eligible before sending them to SRC’s website.

^**The questionnaire differed somewhat from country to country.

^***Rates in the US-all column should be viewed with caution and are likely to be over-estimates; Lucid (for respondents aged 18–24) and IPSOS (for respondents recruited as VNP users) were not able to provide the total number of respondents they invited to the survey.

^****The denominator of the response rate is the sum of 1., 2. and 5. The denominator of the cooperation rate is the sum of 1. and 2. The numerator for each of those rates is 1.

Quality control

The commercial survey firms applied their own quality control checks to completed questionnaires before assigning them a “complete” status. Once the data were received by the ITC Data Management Core, they were further examined for identification and removal of poor quality responses^[16. Three indicators of possible poor quality were used: seconds per question, percentage of responses that were either Refused or Don’t Know (RDK), and percentage of questions which had the first response checked (topbox). Some extreme values occurred for both of these variables, e.g. times of less than 1.7 seconds per question, which by published estimates do not allow time for reading the questions (reading speeds quoted by Zhang and Conrad (2013)^[16] are 200 – 300 milliseconds per word), and RDK responses for more than 70% of the questions completed.

The basic strategy for removal was to create a group of normal respondents by dropping all “suspicious” respondents, namely those with very low seconds per question and/or high percentage RDK or topbox, and to use this main group to calculate normal behavior ranges. Then points were assigned for the degree of departure of each of the three indicators from normal behavior, and the questionnaires of those for whom the points totals were too high were discarded. In this manner, 87 records (0.7 % of the sample) with low seconds per question and/or high percentage RDK or topbox (26 from the US, 13 from CA, 48 from EN, and 0 from AU) were removed from the initial data set.

Survey outcome rates

Response rates and cooperation rates are provided in Table 5.

Detailed information about the disposition codes and the computation of rates can be found in the 4CV1 Technical Report^[11]. The cooperation rate is defined as the number of completed interviews as a percentage of the number of those who entered the survey and proceeded as far as confirming their eligibility. The response rate is defined as the number of completed interviews as a percentage of an estimated number who were invited to the survey at a time when their quota was “open” and who were eligible to participate. The estimated number in the denominator of the response rate was obtained from observed eligibility rates for those whose eligibility status is known, and the observed quota open rate for the survey as a whole.

Survey weights

As with most survey weights, the 4CV1 weights are constructed to correct and adjust for sample misrepresentation caused by unequal sampling probabilities, frame error (i.e., under-coverage and multiplicity), and non-response, as well as to improve precision of estimates through the use of auxiliary information (e.g. smoking prevalences).³

Except in the case of the cross-sectional inflation weights, all weights were rescaled to sum to overall sample size within each country. The cross-sectional weights are intended to be used for analyses using 4CV1 data alone; the longitudinal weights are constructed for analyses using both 4CV1 data and 4C data of respondents who were present in later waves of the 4C Survey.

Cross-sectional sampling weights

We constructed 4CV1 cross-sectional inflation weights for the data set. The inflation weight of a respondent can be interpreted as the number of people in the population represented by that respondent. Inflation weights were computed by dividing all respondents into four broad user groups:

• A: users of cigarettes but not vaping products

• B: users of both cigarettes and vaping products (individuals who smoke at least occasionally and vape at least monthly)

• C: exclusive vapers (individuals who vape at least monthly, but are not current or former cigarette users)

• D: former cigarette users, who may or may not be vapers

Former cigarette users were further categorized into four subgroups determined by their quit duration (< 1 year and 1–5 years in the US, CA and EN; <1 year and 1–2 years in AU⁴) crossed with whether or not they vape at least monthly.

Calibration or benchmark figures were then obtained from national population surveys for each of groups A to D and the four subgroups of D. The benchmark surveys used for our purposes are described in Table A.1 (Online Material). See Tables A.2 and A.3 for more details of the sources, and see the references for the online documentation of these surveys^{[18],[19],[20],[21],[22]}.

In addition to an overall estimated number of individuals in each of user groups A to D and the four subgroups of D, estimates were obtained from the same sources for the following cross-tabulations: user group by sex, user group by age group, user group by geographic region, user group by ethnicity (US only), user group by education (except in CA where education was not collected in the benchmark survey) and user group by language (CA only). A raking procedure (Battaglia et al. 2009^[23]) was then applied to calibrate the weights using the above-mentioned cross-tabulations; this was done separately for each country. Some cells from the above-mentioned cross-tabulations were collapsed in order to reduce the instability of the sampling weights that would result due to small cell sizes.

The cross-sectional inflation weights are designed to make respondents in each of groups A to D and subgroups of D representative of the corresponding population at the time of 4CV1 data collection. For example, the cross-sectional inflation weight of someone in the CA group B sample (individuals who smoke cigarettes and also vape) is the estimated number of people in the CA group B population (in the same age-sex group) represented by that individual. Rescaled cross-sectional weights are also provided, which are simply the 4CV1 cross-sectional inflation weights rescaled to sum to the sample size of each country. As a consequence, they have an average value of 1 in each country.

The inflation weights are needed for estimating population totals and numbers; the rescaled cross-sectional weights can be used for most other purposes, such as estimation of means and proportions and regression analyses.

The cross-sectional inflation weights should not normally be used in analyses involving two or more countries. This is due to the fact that the numbers of smokers (and vapers) differ greatly by country. From the data used to calibrate the weights, there were about 39.8 million cigarette smokers in the US at the time of 4CV1 data collection, compared to only 3.6 million such individuals in CA, 7.4 million in EN, and 2.7 million in AU. Hence, any joint analysis using data from all four countries will be dominated by the US if the inflation weights are used. If the rescaled weights are used, EN and CA will have a slightly greater impact on the results (since the sample sizes in EN and CA are larger than those of AU and the US; see Table A.1, Online Material), but no country will dominate the analysis. Various other rescaled weights have been created for specific user groups for multi-country analyses. More information about those weights is available in the 4CV1 Technical Report^[11].

Longitudinal sampling weights

The rescaled Waves w-to-4CV1 longitudinal weights (where w = 8, 8.5, 9 or 10) are computed for the respondents from the relevant countries who completed the 4C Wave w survey, and were successfully retained and interviewed at 4CV1. These weights are designed to make these smokers (and former smokers) representative of their country’s population of smokers at the time of 4C Wave w data collection. They would be used in analyses of associations between outcomes such as cessation at the time 4CV1 and predictors for the same individuals in 4C data.

Discussion

This paper summarizes the objectives and methods of the ITC 4CV Survey. The strengths of the 4CV design are its deployment in four countries that are culturally and economically similar, but have different regulatory environments that affect the marketing and sale of NVPs; its longitudinal structure, allowing for the observations of transitions in product use and non-use; its relatively large sample sizes; and its focus on subpopulations of particular interest for examining the relationship between smoking and NVP use and transitions between them.

There are challenges and limitations connected with the implementation of the sampling design that are important to note. The 4CV1 samples were not selected as probability samples from the combined population of smokers, recent ex-smokers, and vapers of each country. With a limited budget, we recruited specified numbers of adult cigarette users, former smokers (within two years), and vapers from commercial databases, only some of which were put together from members of probability samples from one or several sources; respondents had to have accepted the invitation from the firm to be in the database, and to accept the invitation to participate in the 4CV Survey. Accordingly, in the 4CV1 Survey, it is generally not feasible to compute inclusion probabilities for sample members, or to adjust these for differential non-response. The effects of not using direct probability sampling, and of using respondents from commercial panels, have been to some extent mitigated by the use of sex, age, and geographic quotas in sampling; by efforts to eliminate speeders from the data set; and by the construction of survey weights taking into account cigarette/vaping status, sex, age, education (where possible), and geography. Well-established national general population surveys employing probability sampling methods were used to weight the 4CV1 data so that estimated prevalence rates for smoking and vaping status are approximately in line with what would be expected in each country.

For many analyses, such as those examining the relationships among attitude and behavior variables, the 4CV1 data are expected to be trustworthy. Given the approximate nature of the benchmarks, the usual advice for analyses with weighted data is especially important: in linear and logistic regression analyses, it is important to include among the explanatory variables or controls the main weighting variables, namely the smoker/vaper/former smoker user groups, sex, age group, and education.

However, 4CV1 “descriptive” estimates of population means and proportions, such as prevalences of certain behaviors, must be interpreted with caution. For one reason, the benchmark surveys are contemporary with the 4CV1 Survey for AU and the US, whereas national surveys conducted in 2015 were used to calibrate the sampling weights for CA and EN. It follows that if the 4CV1 CA and EN data are used for prevalence estimates, these prevalence estimates would be valid for an earlier time. More importantly, for all the countries, without a direct probability sampling design as a basis, prevalence estimates from 4CV1 for categories not used in the calibration could well be biased. For example:

1. The subsample of former smokers cannot be expected to be representative of former smokers in their populations. A sizable number of former smokers in the 4CV1 sample were initially recruited in the 4C cohort as smokers and cannot be said to have been sampled at random from the population of former smokers at the time of the 4CV1 data collection. Moreover, dropout from the 4C sample has likely been associated with cessation, and the cessation experience of the longitudinal respondents could well have been affected by being in the sample. For these reasons, among others, former smokers in the 4CV1 sample should not be regarded as representative of former smokers in their populations. Calibration has been used to make the recent former smokers in the sample more representative with respect to demographics (sex, age group, and education), quit duration, and vaping, but there is no assertion that the recent former smokers are representative with respect to these last two variables crossed with demographic variables.

2. Among vapers in the sample who are smokers or former smokers, monthly or occasional vaping is quite common. At the same time, the inclusion criteria for the additional vaper samples excluded those who ONLY vape, but do so less than weekly. For this and other reasons, it is useful to include vaping frequency in models of the associations of outcomes of interest with vaping.

Summary

The ITC 4CV Survey uses a sampling design, data collection methods, together with an analytic strategy intended to account for departures from the ideal situation of data from respondents recruited by probability sampling in all populations of interest in all countries. The result is a data set allowing for sound analyses to address the objectives: to measure and understand the impact of policy and non-policy factors on the use of cigarettes and NVPs, and on transitions among these products or away from their use (i.e., cessation).