Skip to main content
NCBI home page
Preventive Medicine Reports logoLink to Preventive Medicine Reports
. 2023 Mar 31;33:102194. doi: 10.1016/j.pmedr.2023.102194

Differences in demographics and behaviors across two web-based survey platforms: Observations from a study of risk perceptions of heated tobacco products (HTPs)

Akshika Sharma a,⁎, Brian Fix b, Andrew Hyland b, Amanda J Quisenberry b, Maansi Bansal-Travers b, Richard J O'Connor b
PMCID: PMC10201846  PMID: 37223582

Highlights

  • •

    Survey carried out on two web-based platforms for assessing tobacco-use behaviors.

  • •

    More racially diverse sample obtained from Prime panels in comparison with Mturk.

  • •

    Significantly greater females in prime panel’s sample compared to Mturk.

  • •

    More current smokers and e-cigarette users in Prime panel’s sample than Mturk.

  • •

    Significant differences between groups for COVID-19 disease severity and self-risk.

Keywords: Online platforms, Recruitment, Mturk, Panels, Tobacco-use, Health risk, Perceptions

Abstract

With an escalating popularity of online surveys in behavioral research, it is critical to understand how different sources of participants can yield differing outcomes. While Mturk has been used for online surveys for almost two decades, a recent introduction of online panels allows researchers to choose participants from variety of pools. This study aims to contribute to existing knowledge of how participants from different online platforms differ in their characteristics and behavioral responses which might affect the outcomes. 300 participants were recruited each from Mturk and Prime panels for a 20 mins long survey assessing perceptions and intentions to use Heated tobacco products (HTPs). Participants answered demographic and tobacco-use related questions including their vaccination and masking for COVID-19. They were shown a picture and description of a recently launched HTP. Further, participants answered questions about their awareness of HTPs, risk perception of health conditions from use of different tobacco products (cigarettes, e-cigarettes and HTPs) and perceived severity of COVID-19 infection in smokers, vapers and HTP users. Results showed significant differences in Mturk and Prime panel participants’ demographics and tobacco-use. Prime panels showed more racially diverse population (chi-sq = 10.07, p < 0.02) and significantly more current smokers (chi-sq = 44.74, p < 0.01) and current e-cigarette users (chi-sq = 38.04, p < 0.01) compared with Mturk. Mean perception scores for COVID risk in tobacco users were significantly different between Prime panels and Mturk. Study highlights significant differences in sample composition and responses that might be helpful in choosing one online platform over another based on specific study requirements.

1. Introduction

As in-person and telephone survey recruitment methods have become increasingly challenging over the past two decades, researchers have turned to web-based strategies for study recruitment (Palamar and Acosta, 2020, Ali et al., 2020). Several companies offer a platform and services for web-based recruitment of participants, such as Amazon Mechanical Turk (MTurk), CloudResearch, Survey monkey, and Prolific. Mturk (https://www.mturk.com/) is a large crowdsourcing platform launched in 2005 and widely used since then for gathering information from many people in a short amount of time with minimal cost (Mortensen and Hughes, 2018). In past 5–7 years, there has been an exponential increase in the number of published academic and organizational research that has used Mturk for data collection; their numbers are expected to increase in the future (Keith et al., 2017). The participants on Mturk are referred to as Mturkers and (Zhang and Sherice, 2020) they perform online tasks called Human Intelligence Tasks (HITs), receiving financial compensation for their participation. CloudResearch (https://www.cloudresearch.com/), formerly known as TurkPrime (Litman et al., 2017), procures survey participants from different web-based sources and provides a virtual environment where researchers can run projects with specific participant eligibility requirements. Participant recruitment on CloudResearch is carried out either on Mturk or through a number of other participant pools created by commercial panel companies (Prime Panels, https://www.cloudresearch.com/products/prime-panels/). The panel companies create large participant pools and identify eligible participants based on predetermined selection criteria. The amount and mode of compensation (cash, gift cards, reward points, or donation to charity of choice) for participation are decided as an agreement between the individual and the panel company, who act as an intermediary between participants and researchers (Litman et al., 2017). They also verify responses at the back end to maintain data quality such as removing duplicates, verification of IP address, and removing low quality respondents. Recruiting participants from multiple platforms via a single online environment such as CloudResearch also helps in ensuring that there are no repeat survey takers. It provides an option of excluding participants by identifying IP addresses for those who have already participated in the alternate survey. Hence maintaining data integrity.

In principle, online methods are more time-efficient and cost-effective for completing surveys and can potentially reach a larger number of diverse individuals (Chandler et al., 2019, Stewart et al., 2022, Ibarra et al., 2018). Despite seeming advantages and widespread adoption, concerns have been raised about data quality and integrity. Mturk is often looked upon as an overly exploited method as repeat participants are often exposed to similar surveys and studies, potentially creating a sample of ‘educated’ study participants. Because psychological, behavioral, and social research is frequently carried out using Mturk (Summerville and Chartier, 2013, Woo et al., 2015, Gosling and Mason, 2015), there are greater chances of participants becoming more aware of the possibility of repetitive questions and methods. Similarly, self-selection of Mturkers for surveys may impact data quality. These factors collectively can introduce bias. Further, Mturk participants tend to differ from United States (US) population on age, education, and income (Keith et al., 2017, Behrend et al., 2011, Brandon et al., 2014). Some evidence suggests that Prime panels have a more diverse pool of participants which is more representative of the US population (Chandler et al., 2019). Prime Panels haven’t been used as much as Mturk for behavioral research and have a continuous influx and movement of participants, leading to less exposure to similar behavioral surveys.

Cigarette smoking remains the leading cause of preventable disease within the US, with 14% of adults reporting cigarette smoking and more than 18% reporting use of more than one nicotine product (Cornelius et al., 2020). Further, HTPs, designed to heat tobacco sticks below their combustion point to release vapors which can be inhaled by users, are a relatively new addition to the U.S. tobacco market. IQOS, a prototypical HTP, was authorized for sale by Food and Drug Administration (FDA) in April 2019 and permitted to be marketed as a ‘reduced exposure’ product in July 2020. However, they were only available in four states (Georgia, Virginia, North and South Carolina). There are concerns amongst public health professionals about the misinterpretation of the claims as reduced harm (East et al., 2021, Popova et al., 2018, Zou et al., 2021) and a potential increase in youth uptake (Jeong et al., 2019). This authorization occurred during the COVID-19 pandemic, when health risks from infection with COVID-19 in smokers were highlighted (Giovenco et al., 2021, Nargis et al., 2021). Some studies have reported a reduction in smoking as a result of greater risk perception due to COVID-19 (Rigotti et al., 2021, White, 2021, Gravely, 2021).

For emerging tobacco products such as IQOS, it’s required to quickly assess risk perceptions and adoption to inform regulatory science. The convenience samples such as MTurk allows for easy access to population groups (Ibarra et al., 2018) and rapidly provide results comparable to probability samples (Jeong et al., 2019). However, findings such as risk perception can vary greatly across participants, and so different samples may potentially lead to different conclusions. Patterns of response may also be influenced by the concurrence of the COVID-19 pandemic, given it seems to have a particular health impact in smokers (Clift et al., 2022, Patanavanich and Glantz, 2021). To the best of our knowledge, to date there is no study reporting variances in tobacco use patterns and health risk perceptions of tobacco use and COVID-19 between different web-based recruitment platforms. The aim of this paper is to highlight the demographic and tobacco-use characteristics of participants in two web-based recruitment platforms, Prime panels and Mturk, and whether participants’ risk perceptions and COVID-19- related compliance differed significantly.

2. Methods

A cross-sectional web-based survey to assess between-subjects’ responses to IQOS ad claims was conducted using both Prime panels and Mturk via CloudResearch. Basic information about the surveys is shown in Table 1. Identical surveys were administered on both platforms and fielded for approximately the same time (approximately 20 min). The compensation amount for Mturk was scaled from NY minimum wage ($15/hr) for the projected time involved, while Prime panel compensation was quoted by CloudResearch based on guidelines from the vendors providing participants from their respective pools. The study aimed to recruit 300 United States (US) residents aged 18–45 years old from each platform. The study was approved by Institutional review board at Roswell Park. The study met the guidelines for protection of human subjects concerning safety and privacy.

Table 1.

Survey timeline and recruitment information for Mturk and Prime panels. *IQR = Interquartile range.

MTurk Prime Panels
Field dates December 21–24; December 29–30, 2021 December 15–20, 2021
Number of days in field 6 6
Respondent compensation $5.00 $1.93
Final N 302 302
Median completion time (IQR*) - minutes 11.29 (7.13) 8.94 (5.83)
Open in a new tab

After obtaining consent to participate in the survey, each participant provided information about their demographics (i.e., age, sex, race/ethnicity, education, annual income, occupation type, and work from home status) and tobacco use (i.e., 100 lifetime cigarettes use, ever e-cigarette use, current smoking and e-cigarette use status). Given that there was no nation-wide marketing or educating for HTPs, participants were shown a picture of the IQOS device and heat sticks along with a description of product. Immediately afterward, participants were asked about their awareness of the HTPs. They were further asked questions about their risk perceptions of various disease conditions as well as COVID-19 illness severity as a result of different tobacco product use, and their compliance with COVID-19 protocols. The participants’ attention was measured by adding an attention test question mid-survey. All analysis was performed using IBM SPSS statistics version 25.

2.1. Tobacco use and related behaviors

Awareness of HTPs was assessed by asking participants to choose from the statement that best applies to them. The response options included ‘I have never heard of heated tobacco products, before today’, ‘I have heard of heated tobacco products but have never tried them’, ‘I have tried heated tobacco products but do not use them anymore’ and ‘I currently use heated tobacco products’. Cigarette use was determined by asking whether participants had smoked 100 cigarettes in their lifetime; those who answered ‘yes’ were further asked about their current use of cigarettes (Everyday, Somedays, Not at all). Similarly, participants were asked about ever and current use of e-cigarettes.

2.2. COVID-19 risk and related behaviors

Participants’ perception of COVID-19-related risk was measured by asking a series of questions about perceived severity of illness if they themselves got infected with COVID-19, as well as perceived severity for smokers, e-cigarette users, and non-users of tobacco products. The responses were measured on a seven-point scale ranging from ‘a lot more severe’ to ‘a lot less severe’. They were further asked about their current vaccination status, willingness to get vaccinated, and wearing masks in public places.

2.3. Risk perceptions

Risk perceptions for various disease conditions from use of tobacco products was recorded. Participants were asked ‘If you SMOKED CIGARETTES, what do you believe your risk would be for developing Lung Cancer compared to a person who does NOT smoke cigarettes?’. Participants answered similar questions for different conditions such as emphysema, bronchitis, mouth cancers, other cancers, and tooth loss. They were also asked about the risk for each condition from use of e-cigarette when compared to cigarette smoking, as well as use of HTPs when compared to cigarette smoking, for example, ‘If you were to VAPE / use E-CIGARETTES, what do you believe your risk would be for developing Lung Cancer compared to a person who smokes cigarettes?’ and ‘If you were to use HEATED TOBACCO PRODUCTS such as IQOS, what do you believe your risk would be for developing Lung Cancer compared to a person who smokes cigarettes?’. All responses were measured on a 5-point Likert scale, with response options ranging from ‘much less likely’ to ‘much more likely’. Higher scores were suggestive of greater risk perception.

3. Results

3.1. Survey completion and demographics

The study recruited a total of 613 participants, 306 from Mturk and 307 from Prime panels. Due to incomplete data, 9 participants were removed, reaching a total of 604 valid responses (302 from Mturk and 302 from Prime panels). The mean time for survey completion by Prime panelists and Mturkers was 11.26 (6.8) and 12.87 (6.0) minutes respectively; the difference in time to completion between the two panels was significant (F = 8.75, p < 0.01; Fig. 1). For analysis, we restricted to participants with completion times between 5 and 60 mins. Before exclusion, there were 39 outliers in Prime panels and 13 in Mturk. Since there were no differences between the participants with respect to other survey responses and attention questions, the outliers were included in main analysis (Fig. 1; color).

Fig. 1.

Fig. 1

Open in a new tab

Box plot showing time taken to complete online survey by Prime panel and Mturk participants.

Overall, 34% of participants were aged between 21 and 30 years and 49 % between 31 and 40 years of age. The Mturk sample included approximately 62% males whereas Prime panels differed significantly with 59% female participants. There were no significant differences based on age. There were significant differences in race and ethnicity between the two groups, Mturk had a slightly greater proportion of non-Hispanic whites (68.4%) compared to Prime panels (65%). However, 16% participants in Prime panels were non-Hispanic Blacks compared to only 8% in Mturk. We also observed significant differences based on annual income and education (Table 2).

Table 2.

Demographic distribution of participants from Mturk vs Prime panels surveys.

Variable name Overall N (%) Mturk N (%) Prime panels N (%) Chi square (p-value)
Age (yrs.) (N = 604) 18–20 12 (2.0) 4 (1.3) 8 (2.6) 6.95 (0.07)
21–30 204 (33.8) 106 (35.1) 98 (32.5)
31–40 298 (49.3) 157 (52.0) 141 (46.7)
41–45 90 (14.9) 35 (11.6) 55 (18.2)
Sex (N = 604) Male 306 (50.7) 186 (61.6) 120 (39.7) 30.69 (<0.01)
Female 295 (48.8) 116 (38.4) 179 (59.3)
Non-Binary and Transgender 3 (5) 0 (0.0) 3 (1)
Race & ethnicity (N = 604) Non-Hispanic white 402 (66.6) 206 (68.4) 196 (64.9) 10.07 (0.02)
Non-Hispanic black 73 (12.1) 25 (8.3) 48 (15.9)
Hispanic 72 (11.9) 36 (12.0) 36 (11.9)
Others 56 (9.3) 34 (11.3) 22 (7.3)
Annual income (US dollars) (N = 604) <$20,000 112 (18.5) 30 (9.9) 82 (27.2) 64.63 (<0.01)
$20,000 to $34,999 72 (11.9) 41 (13.6) 31 (10.3)
$35,000 to $49,999 78 (12.9) 48 (15.9) 30 (9.9)
$50,000 to $74,999 130 (21.5) 84 (27.8) 46 (15.2)
$75,000 to $99,999 74 (12.3) 51 (16.9) 23 (7.6)
≥ $100,000 130 (21.5) 46 (15.2) 84 (27.8)
Prefer not to answer 5 (0.8) 1 (0.3) 4 (1.3)
I'm not sure 3 (0.5) 1 (0.3) 2 (0.7)
Education (N = 604) High school or less 126 (20.9) 45 (14.9) 81 (26.8) 20.90 (<0.01)
Post-High school 24 (4.0) 7 (2.3) 17 (5.6)
Some college 126 (20.9) 63 (20.9) 63 (20.9)
College graduate and post-graduate 328 (54.3) 187 (61.9) 141 (46.7)
Marital status Married 267 (44.2) 113 (37.4) 154 (51.0) 11.28 (<0.01)
Not Married 337 (55.8) 189 (62.6) 148 (49.0)
Open in a new tab

3.2. Tobacco use and related behaviors

When asked whether they have smoked 100 cigarettes in their lifetime, 68% Prime panelists and 64% Mturkers answered ‘yes’ (not statistically significant). When asked further, 21% Mturkers and 41% Prime panelists reported everyday smoking (Chi-sq = 30.98, p-value=<0.01). The ever use of e-cigarette was reported by 68% participants in overall sample. Current everyday and someday use of e-cigarette was reported by 8% and 28% of Mturk and 19% and 34% of Prime panel participants respectively (Chi-sq = 23.45, p-value < 0.01). Significant differences were seen for the type of e-products ever used by Mturk and Prime panels (Table 3).

Table 3.

Tobacco-use among participants from Mturk vs Prime panels in a web-based survey.

Tobacco-use Overall
N (%) 		Mturk N (%) Prime Panels
N (%) 		Chi-sq (p-value)
100 lifetime cigarettes
(N = 604)		 Yes 397 (65.7) 193 (63.9%) 204 (67.5%) 0.89 (0.35)
No 207 (34.3) 109 (36.1%) 98 (32.5%)
Current smoking status
(N = 397)		 Everyday 189 (31.3) 64 (33.2%) 125 (61.3%) 44.74 (<0.01)
Somedays 124 (20.5) 64 (33.2%) 60 (29.4%)
Not at all 84 (13.9) 65 (33.7%) 19 (9.3%)
e-cig ever use
(N = 604)
 Yes 408 (67.5) 206 (68.2%) 202 (66.9%) 2.06 (0.36)
No 194 (32.1) 96 (31.8%) 98 (32.5%)
Don't know 2 (0.3) 0 (0%) 2 (0.7%)
Do you now use electronic products?
(N = 408)		 Everyday 81 (13.4) 24 (11.7) 57(28.2) 38.04 (<0.01)
Somedays 189 (31.3) 85 (41.3) 104 (51.5)
Not at all 138 (22.8) 97 (47.1) 41 (20.3)
Type of e-product ever used
(N = 406)		 E-cig only 263 (43.5) 157 (76.6) 106 (52.7) 25.69 (<0.01)
Other e-products 20 (3.3) 8 (3.9) 12 (6.0)
More than one e-products 123 (20.4) 40 (19.5) 83 (41.3)
Awareness of HTPs

 Never heard of HTPs 337 (55.8) 187 (61.9) 150 (49.7) 9.19 (<0.01)
Aware of HTPs 267 (44.2) 115 (38.1) 152 (50.3)
Open in a new tab

3.3. Risk perceptions

For a standard comparison baseline, the risk perception scores were reverse coded for non-smokers vs smokers (Fig. 2a). Originally, the risk perception scores observed for non-smokers was opposite in direction but greater in magnitude, compared to risk perception in e-cigarette users and HTP users versus smokers. Hence, lower mean score in Fig. 2a represents higher perception of reduced risk in non-smokers as compared with smokers and vice versa. Whereas greater mean scores in Fig. 2b and c represent a greater risk in e-cigarette users and HTP users as compared to smokers. The Mturk users reported a significantly greater perception of risk in smokers compared to non-smokers for all disease conditions (F = 47.53, p < 0.01). Whereas Prime panels reported a significantly greater perception of risk in vapers (F = 46.17, p < 0.01) and HTP users (F = 27.28, p < 0.01) when compared to smokers (Fig.2(a,b,c); color).

Fig. 2.

Fig. 2

Open in a new tab

a. Mean scores for risk in non-smokers versus smokers, between Prime panels and Mturk. b. Mean scores for risk in vapers versus smokers, between Prime panels and Mturk. c. Mean scores for risk in HTP users versus smokers, between Prime panels and Mturk.

3.4. COVID-19 risk and related behaviors

For COVID-19, approximately 73% Mturkers reported being single or double vaccinated compared to 61% prime panel participants (Chi-square = 9.6, p-value < 0.01). Participants differed significantly between recruitment platforms for their beliefs and own risk perceptions about COVID-19 sickness (F = 49.3 (1), p < 0.01). Significant interaction was seen between recruitment platform and vaccination status (F = 6.2 (1), p-value = 0.01). Prime panel participants were significantly more confident of not getting coronavirus, however, they also showed a greater worry about contracting coronavirus (F = 4.00 (1), p = 0.05). There was a significant independent effect of vaccination status (F = 52.5 (1), p-value < 0.01) and recruitment platform on worry about contracting coronavirus as well as a significant interaction effect between these two variables (F = 4.0 (1), p-value = 0.05). Overall, current smokers were more confident of not getting coronavirus as compared to former smokers (Chi-square = 4.34, p-value = 0.02). In contrast, smokers expressed more worry about getting coronavirus compared to non-smokers (Chi-square = 3.78, p-value = 0.05) and current smokers were more worried as compared to former smokers (Chi-square = 6.45, p-value = 0.01). Prime panels reported a significantly reduced perception of COVID-19 induced disease severity for smokers, vapers, and vapers who also smoke cigarettes, when compared to non-smokers, non-vapers, and vapers who do not smoke respectively (Table 4).

Table 4.

Differences in means for risk perceptions of COVID-19 between Mturk and Prime panels.

Beliefs and perceptions about COVID risk Prime panel Mturk F (df) (p-value)
I am confident that I will not get COVID-19 novel coronavirus1 (Strongly disagree)- 5
(Strongly agree)		 3.63 2.90 49.30 (1) (<0.01)
How worried are you about getting COVID-19 novel coronavirus?1 (Not at all)-5
(Extremely)		 2.90 2.70 4.00 (1) (0.05)
If you got the coronavirus, how severe do you think the illness would be for you, COMPARED TO others your age who got it?1 (A lot less severe)-7
(A lot more severe)		 4.26 4.46 1.42 (1) (0.23)
Thinking about smokers in general -- when a smoker becomes ill with the coronavirus, how do you think their smoking will affect the severity of the illness?1 (A lot less severe)-7
(A lot more severe)		 3.17 2.50 20.19 (1) (>0.01)
Thinking about vapers, those who DO NOT ALSO SMOKE -- when a vaper becomes ill with the coronavirus, how do you think their vaping will affect the severity of the illness?1 (A lot less severe)-7
(A lot more severe)		 3.54 3.17 6.81 (1) (0.01)
Thinking about vapers, but those who ALSO SMOKE -- when a vaper who also smokes becomes ill with the coronavirus, how do you think their vaping will affect the severity of the illness?1 (A lot less severe)-7
(A lot more severe)		 3.23 2.60 18.55 (1) (<0.01)
Open in a new tab

4. Discussion

The purpose of this paper is to highlight differences between the two online recruitment methods by means of our findings. We observed significant differences in demographics, tobacco-use characteristics as well as perceived health risks from use of different tobacco products between two recruitment platforms. Demographics of Mturk sample was more in line with the previous Mturk based tobacco studies (Bauhoff et al., 2017, Leavens et al., 2019, Rass et al., 2015). Whereas Prime panels were more diverse in comparison with Mturk. This could be an effect of ‘pull in’ versus ‘push out’ method of recruitment by two types of online platforms as explained by Antoun et al. (2016) (Antoun et al., 2016), especially racial and ethnic as well as gender distribution. Tobacco-use for both cigarettes and e-cigarette was significantly higher amongst prime panelists as compared to Mturkers.

Additionally, there were differences based on risk perceptions about various tobacco products. The prime panelists reported lower mean scores for risk in smokers versus non-smokers, but a higher mean score for risk in Vapers and HTP users as compared to smokers, in comparison to responses of Mturkers. Overall, we observed greater values for mean scores for risk in smokers compared to risk in e-cigarette users and HTP users, which is somewhat in agreement with the results reported for risk perceptions in e-cigarette and cigarette smoking (Popova et al., 2018, Churchill et al., 2021, Czoli et al., 2017), while there is limited data on comparative risk perception of HTP use (Sugiyama and Tabuchi, 2020).

Our study observed greater confidence in current smokers as compared to former smokers about not contracting coronavirus however current smokers also showed greater worry about contracting coronavirus. A possible explanation for this rather contrasting observation could be a reason for quitting behavior in response to COVID related worry as observed by previous studies (White et al., 2021, Nyman et al., 2021). Additionally, we observed significant differences in risk perceptions between the two online platforms, which to best of our knowledge have not been reported previously.

Our study provides an important perspective on demographic and behavioral differences across online recruitment approaches. The variability in responses could be a result of differences in recruitment of subjects in each platform, composition of participants, and compensation amounts. While Mturk offers a relatively direct recruitment of participants, Prime panels are accessed via panel companies. The compensation in Mturk is often decided by researcher’s own will whereas, amounts in Prime panels are often calculated based on specific eligibility requirements, complexity of survey and sample size. Based on our results, Prime panels appear to be a preferred platform if study has a specific eligibility criterion or seeking special groups such as smokers. A limitation in using panels could be a relative inexperience in panel participants for completing the surveys or understanding behavioral questions. This might be a reason for observing greater variation in time to completion in our study amongst Prime panels as compared to Mturk. Mturk participants on the other hand, have a greater experience in completing behavioral surveys and may generate better data quality (Chandler et al., 2019). A limitation of our study was structural differences in compensation between Mturk and Prime panels. Another potential limitation is unavailability of location information for participants which could have been helpful in understanding geographical differences (if any) between the two recruitment platforms. Several studies have reported differences between gold standard probability samples and convenience samples from web-based platforms and have reported demographic and tobacco use differences (Jeong et al., 2019, Walters et al., 2018, Kraemer et al., 2017). However, little is known about how different web-based platforms compare in terms of demographics and tobacco use prevalence and beliefs.

5. Conclusion

Web-based recruitment platforms provide a fast and cost-efficient way of data collection in behavioral science. Different types of recruitment platforms may vary greatly in terms of demographic profiles and prevalence estimates, and the results might differ based on the sample size and eligibility criteria of a survey.

6. Role of funding sources

This research was supported by internal funding resources of Roswell Park Comprehensive Cancer Center. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Roswell Park Comprehensive Cancer Center.

Funding

The study was funded by Roswell Park Comprehensive Cancer Center’s internal funding resources.

CRediT authorship contribution statement

Akshika Sharma: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Writing – original draft, Writing – review & editing. Brian Fix: Data curation, Formal analysis, Software, Writing – review & editing. Andrew Hyland: Investigation, Methodology, Supervision, Writing – review & editing. Amanda J. Quisenberry: Investigation, Methodology, Supervision, Writing – review & editing. Maansi Bansal-Travers: Investigation, Methodology, Supervision, Writing – review & editing. Richard J. O'Connor: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.

References

  1. Ali S.H., Foreman J., Capasso A., Jones A.M., Tozan Y., DiClemente R.J. Social media as a recruitment platform for a nationwide online survey of COVID-19 knowledge, beliefs, and practices in the United States: methodology and feasibility analysis. BMC Med. Res. Methodol. 2020;20(1):116. doi: 10.1186/s12874-020-01011-0. Published 2020 May 13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Antoun C., Zhang C., Conrad F.G., Schober M.F. Comparisons of online recruitment strategies for convenience samples: Craigslist, Google AdWords, Facebook, and Amazon Mechanical Turk. Field Methods. 2016;28(3):231–246. doi: 10.1177/1525822X15603149. [DOI] [Google Scholar]
  3. Bauhoff S., Montero A., Scharf D. Perceptions of e-cigarettes: a comparison of adult smokers and non-smokers in a Mechanical Turk sample. Am. J. Drug Alcohol Abuse. 2017;43(3):311–323. doi: 10.1080/00952990.2016.1207654. [DOI] [PubMed] [Google Scholar]
  4. Behrend T.S., Sharek D.J., Meade A.W., Wiebe E.N. The viability of crowdsourcing for survey research. Behav. Res. Methods. 2011;43(3):800–813. doi: 10.3758/s13428-011-0081-0. [DOI] [PubMed] [Google Scholar]
  5. Brandon D.M., Long J.H., Loraas T.M., Mueller-Phillips J., Vansant B. Online instrument delivery and participant recruitment services: emerging opportunities for behavioral accounting research. Behav. Res. Account. 2014;26:1–23. doi: 10.2308/bria-50651. [DOI] [Google Scholar]
  6. Chandler J., Rosenzweig C., Moss A.J., Robinson J., Litman L. Online panels in social science research: expanding sampling methods beyond Mechanical Turk. Behav. Res. Methods. 2019;51(5):2022–2038. doi: 10.3758/s13428-019-01273-7. https://link.springer.com/article/10.3758/s13428-019-01273-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Churchill V., Nyman A.L., Weaver S.R., et al. Perceived risk of electronic cigarettes compared with combustible cigarettes: direct versus indirect questioning. Tobacco Control. 2021;30:443–445. doi: 10.1136/tobaccocontrol-2019-055404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Clift A.K., von Ende A., Tan P.S., et al. Smoking and COVID-19 outcomes: an observational and Mendelian randomization study using the UK Biobank cohort. Thorax. 2022;77:65–73. doi: 10.1136/thoraxjnl-2021-217080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cornelius M.E., Wang T.W., Jamal A., Loretan C.G., Neff L.J. Tobacco Product Use Among Adults - United States, 2019. MMWR Morbidity Mortality Weekly Rep. 2020;69(46):1736–1742. doi: 10.15585/mmwr.mm6946a4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Czoli C.D., Fong G.T., Mays D., Hammond D. How do consumers perceive differences in risk across nicotine products? A review of relative risk perceptions across smokeless tobacco, e-cigarettes, nicotine replacement therapy and combustible cigarettes. Tobacco Control. 2017;26(e1):e49–e58. doi: 10.1136/tobaccocontrol-2016-053060. [DOI] [PubMed] [Google Scholar]
  11. East K.A., Tompkins C.N.E., McNeill A., et al. ‘I perceive it to be less harmful, I have no idea if it is or not:’ a qualitative exploration of the harm perceptions of IQOS among adult users. Harm. Reduct. J. 2021;18:42. doi: 10.1186/s12954-021-00490-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Giovenco D.P., Spillane T.E., Maggi R.M., Lee E.Y., Philbin M.M. Multi-level drivers of tobacco use and purchasing behaviors during COVID-19 “lockdown”: a qualitative study in the United States. Int. J. Drug Policy. 2021;94:103175. doi: 10.1016/j.drugpo.2021.103175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Gosling S.D., Mason W. Internet research in psychology. Ann. Rev. Psychol. 2015;66:877–902. doi: 10.1146/annurev-psych-010814-015321. [DOI] [PubMed] [Google Scholar]
  14. Gravely S., et al. Smokers' cognitive and behavioral reactions during the early phase of the COVID-19 pandemic: Findings from the 2020 ITC Four Country Smoking and Vaping Survey. PloS One. 2021;16(6):e0252427. doi: 10.1371/journal.pone.0252427. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Ibarra J.L., Agas J.M., Lee M., Pan J.L., Buttenheim A.M. Comparison of online survey recruitment platforms for hard-to-reach pregnant smoking populations: feasibility study. JMIR Res. Protocols. 2018;7(4):e101. doi: 10.2196/resprot.8071. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Jeong M., Zhang D., Morgan J.C., Ross J.C., Osman A., Boynton M.H., Mendel J.R., Brewer N. Similarities and differences in tobacco control research findings from convenience and probability samples. Ann. Behav. Med. Publ. Soc. Behav. Med. 2019;53(5):476–485. doi: 10.1093/abm/kay059. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Keith M.G., Tay L., Harms P.D. Systems perspective of Amazon mechanical Turk for organizational research: review and recommendations. Front. Psychol. 2017;8:1359. doi: 10.3389/fpsyg.2017.01359. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Kraemer J.D., Strasser A.A., Lindblom E.N., Niaura R.S., Mays D. Crowdsourced data collection for public health: a comparison with nationally representative, population tobacco use data. Prev. Med. 2017;102:93–99. doi: 10.1016/j.ypmed.2017.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Leavens E., Stevens E.M., Brett E.I., Hébert E.T., Villanti A.C., Pearson J.L., Wagener T.L. JUUL electronic cigarette use patterns, other tobacco product use, and reasons for use among ever users: results from a convenience sample. Addict. Behav. 2019;95:178–183. doi: 10.1016/j.addbeh.2019.02.011. [DOI] [PubMed] [Google Scholar]
  20. Litman L., Robinson J., Abberbock T. TurkPrime.com: a versatile crowdsourcing data acquisition platform for the behavioral sciences. Behav. Res. Methods. 2017;49(2):433–442. doi: 10.3758/s13428-016-0727-z. https://link.springer.com/article/10.3758/s13428-016-0727-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Mortensen K., Hughes T.L. Comparing Amazon's Mechanical Turk Platform to Conventional Data Collection Methods in the Health and Medical Research Literature. J. Gen. Internal Med. 2018;33(4):533–538. doi: 10.1007/s11606-017-4246-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Nargis N., Li Q., Griffin L., Asare S., Bandi P., Majmundar A., Westmaas J.L., Jemal A. Association of teleworking and smoking behavior of U.S. wage and salary workers. J. Occup. Health. 2021;63(1):e12283. doi: 10.1002/1348-9585.12283. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Nyman A.L., Spears C.A., Churchill V., et al. Associations between COVID-19 risk perceptions and smoking and quitting behavior among U.S. adults. Addict. Behav. Rep. 2021;14:100394. doi: 10.1016/j.abrep.2021.100394. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Palamar J.J., Acosta P. On the efficacy of online drug surveys during the time of COVID-19. Substance Abuse. 2020;41(3):283–285. doi: 10.1080/08897077.2020.1784362. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Patanavanich R., Glantz S.A. Smoking is associated with worse outcomes of COVID-19 particularly among younger adults: a systematic review and meta-analysis. BMC Public Health. 2021;21:1554. doi: 10.1186/s12889-021-11579-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Popova L., Lempert L.K., Glantz S.A. Light and mild redux: heated tobacco products' reduced exposure claims are likely to be misunderstood as reduced risk claims. Tob. Control. 2018;27(Suppl 1):s87–s95. doi: 10.1136/tobaccocontrol-2018-054324. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Popova L., Owusu D., Weaver S.R., et al. Affect, risk perception, and the use of cigarettes and e-cigarettes: a population study of U.S. adults. BMC Public Health. 2018;18:395. doi: 10.1186/s12889-018-5306-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Rass O., Pacek L.R., Johnson P.S., Johnson M.W. Characterizing use patterns and perceptions of relative harm in dual users of electronic and tobacco cigarettes. Experiment. Clin. Psychopharmacol. 2015;23(6):494–503. doi: 10.1037/pha0000050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Rigotti N.A., Chang Y., Regan S., Lee S., Kelley J., Davis E., Levy D.E., Singer D.E., Tindle H.A. Cigarette Smoking and Risk Perceptions During the COVID-19 Pandemic Reported by Recently Hospitalized Participants in a Smoking Cessation Trial. J. Gen. Internal Med. 2021;36(12):3786–3793. doi: 10.1007/s11606-021-06913-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Stewart J., Krows M.L., Schaafsma T.T., et al. Comparison of racial, ethnic, and geographic location diversity of participants enrolled in clinic-based vs 2 remote COVID-19 clinical trials. JAMA Netw. Open. 2022;5(2):e2148325. doi: 10.1001/jamanetworkopen.2021.48325. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Sugiyama T., Tabuchi T. Use of multiple tobacco and tobacco-like products including heated tobacco and E-Cigarettes in Japan: a cross-sectional assessment of the 2017 JASTIS study. Int. J. Environ. Res. Public Health. 2020;17(6):2161. doi: 10.3390/ijerph17062161. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Summerville A., Chartier C.R. Pseudo-dyadic “interaction” on Amazon’s Mechanical Turk. Behav. Res. 2013;45:116–124. doi: 10.3758/s13428-012-0250-9. [DOI] [PubMed] [Google Scholar]
  33. Walters K., Christakis D.A., Wright D.R. Are Mechanical Turk worker samples representative of health status and health behaviors in the U.S.? PLOS ONE. 2018;13(6):e0198835. doi: 10.1371/journal.pone.0198835. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. White A.M., et al. Perceptions of tobacco product-specific COVID-19 risk and changes in tobacco use behaviors among smokers, E-Cigarette users, and dual users. Nicotine Tobacco Res. 2021;23(9):1617–1622. doi: 10.1093/ntr/ntab053. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. White A.M., Li D., Snell L.M., O’Connor R., Hoetger C., Croft D., Lester R.C., McIntosh S., Underwood M., Schneller L., Breland A., Barnes A.J., Cobb C.O., Ossip D.J. Perceptions of tobacco product-specific COVID-19 risk and changes in tobacco use behaviors among smokers, E-Cigarette users, and dual users. Nicotine Tobacco Res. 2021;23(9):1617–1622. doi: 10.1093/ntr/ntab053. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Woo S.E., Keith M., Thornton M.A. Amazon Mechanical Turk for industrial and organizational psychology: advantages, challenges, and practical recommendations. Ind. Organ. Psychol. 2015;8:171–179. doi: 10.1017/iop.2015.21. [DOI] [Google Scholar]
  37. Zhang B., Sherice G. Collecting online survey data: a comparison of data quality among a commercial panel & MTurk. Survey Pract. 2020;13(1) doi: 10.29115/SP-2020-0015. [DOI] [Google Scholar]
  38. Zou C., Wang X., Xie Z., Li D. Perceptions of the IQOS heated tobacco product on Twitter in the United States. Front. Commun. 2021;6:728604. doi: 10.3389/fcomm.2021.728604. [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Data will be made available on request.

Articles from Preventive Medicine Reports are provided here courtesy of Elsevier

RESOURCES