Facebook for recruiting Spanish- and English-speaking smokers

Abstract

Background

Recruitment for research is usually expensive and time consuming. Facebook (FB) recruitment has become widely utilized in recent years. The main aim of this study was to assess FB as a recruitment tool in a study for Spanish- and English-speaking smokers. Additionally, the study set out to compare performance of ads by language (Spanish vs. English), location (U.S. vs. San Francisco) and content (self-efficacy ad vs. fear appeal ad).

Methods

Participants of a one-condition smoking cessation webapp study were recruited utilizing FB ads and posts through two phases: a recruitment-focused phase and an experimental phase comparing language, location and content.

Results

During the recruitment phase 581 participants in total (U.S. = 540, San Francisco = 41) provided consent. Of the U.S. participants 275 were Spanish-speakers and 265 English-speakers. The cost-per-consent was $25.81 for Spanish-speakers, and $15.49 for English-speakers. During the experimental phase U.S. users performed better (i.e. more clicks, engagement and social reach) than San Francisco users, Spanish-speakers engaged more than English-speakers, and the self-efficacy ad performed better than the fear appeal ad.

Conclusions

This study showed that although there were differences in cost-per-consent for Spanish- and English-speakers, recruitment of Spanish-speakers through Facebook is feasible. Furthermore, comparing performance of ads by location, language, and ad content may contribute to developing more efficient campaigns.

Highlights

• •Recruitment of English-speakers and Spanish-speakers smokers through Facebook is feasible.
• •Cost-per-consent was higher for Spanish-speakers smokers than for English speakers
• •Ads for U.S. Facebook users performed better than for San Francisco users.
• •Spanish-speakers engaged more with the Ads than English-speakers.
• •A self-efficacy ad performed better than the fear-appeal ad.

1. Introduction

Recruitment for human subjects research is usually expensive and time consuming (Thornton et al., 2016). Online recruitment has been shown to be more efficient than offline methods in terms of total participants enrolled, enables accessibility to larger and more diverse participants, requires shorter recruitment periods, and reduces overall study recruitment costs (Christensen et al., 2017; Lane et al., 2015; Whitaker et al., 2017). However, there are concerns regarding sample representativeness (Choi et al., 2017) and minimal research exists on reach and enrollment of participants from diverse backgrounds (e.g., non-English speaking; Lane et al., 2015; Kayrouz et al., 2016; Bunge et al., 2018).

The most commonly utilized online recruitment sources include: Google Ads, Amazon Mechanical Turk (AMT), Facebook (FB), and Craigslist (Temple and Brown, 2011). Evidence suggests that these methods of recruitment result in similar samples and are as cost-effective as more traditional face-to-face recruitment methods (Thornton et al., 2016). A systematic review of 110 health and mental health studies utilizing FB recruitment concluded that, on average, recruitment costs were $6.79 per participant and few differences were noted when compared to samples recruited using face-to-face methods (Thornton et al., 2016). Furthermore, FB may also be more time-effective compared to traditional recruitment methods (e.g., emails, print advertisements and media releases), with one study finding participants were recruited up to 2.5 times faster (Kayrouz et al., 2016).

FB allows for various recruitment options (see Akers and Gordon, 2018), such as posting general ads, promoting public fan pages, and boosting posts. One study found that of these techniques, promoting FB pages and boosting posts demonstrated the largest impact on recruitment rates (Kayrouz et al., 2016). Furthermore, FB offers the promising advantage of potential respondent driven sampling, also known as snowball sampling, which capitalizes on FB's inherent peer networking structure (i.e., FB users recruit other FB users or non-FB peers; Pedersen and Kurz, 2016). FB maximizes the potential to access the aforementioned “hard-to-reach individuals,” given the omnipresent use of FB among various populations (Lane et al., 2015).

Several smoking cessation studies recruited participants through FB (Akers and Gordon, 2018; Carlini et al., 2014; Ramo and Prochaska, 2012; Ramo et al., 2014, Ramo et al., 2015a, Ramo et al., 2015b; Sadasivam et al., 2013, Sadasivam et al., 2016; Thornton et al., 2013). Ramo and Prochaska (2012) examined FB as a recruitment channel for surveying tobacco and other substance use for individuals 18–25 and reported a final cost per valid, completed survey of $4.28. FB estimated that 2.8% of accounts for people aged 18–25 were reached through tobacco and marijuana keywords (Ramo and Prochaska, 2012). Ramo et al. (2014) employed multiple ad types for a smoking cessation trial over a seven-week period, such as ads in news feeds, promoted posts, sponsored stories, and standard ads, averaging $8.80 per eligible, consented participant. They found that images of smoking and news feed ads yielded the greatest reach and clicks at the lowest cost, and posited that this may be partially due to news feed ads being viewable by mobile device. Ramo et al., 2015a, Ramo et al., 2015b conducted a mixed-methods study surveying young adults about tobacco and social media use to see if participants were interested in a smoking cessation intervention through FB and, if so, how FB should be used to help this population quit smoking. About one third of their sample (31%) reported that they would use FB to quit smoking, and interest in using FB was greater among those who were more motivated to quit, had made a quit attempt within the last year, and had previously used an online source to quit (Ramo et al., 2015a, Ramo et al., 2015b). Sadasivam et al. (2013) created a technology-assisted tobacco intervention to recruit people through peer marketing. The initial wave of their participants, or their “seeds,” (n = 190) were recruited through FB ads, the peer recruits (n = 569) were recruited through the seeds, and the overall cost per smoker recruited was $29.80 (Sadasivam et al., 2013). Interestingly, FB can target people who are not searching for smoking cessation information as well, a beneficial recruitment outcome that would not be accessible with other online recruitment methods such as Google Ads. For example, Akers and Gordon (2018) conducted a clinical trial to teach support skills to partners of smokeless users and recruited 1145 female partners of male smokeless tobacco users over a 15-month period.

While FB offers the benefit of broad reach, potential drawbacks include privacy concerns and a general “lack of clear guidance around human subjects issues” (Pedersen and Kurz, 2016, p. 5). Additionally, FB recruitment outcomes vary depending on how campaigns are designed and conducted, with some studies reporting that FB is more expensive than other similar online methods (Frandsen et al., 2014; Heffner et al., 2013). Heffner et al. (2013) compared the cost of various recruitment strategies (i.e., standard media, broadcast emails, word of mouth, Google AdWords, and social media) for a pilot study on web-based Acceptance and Commitment Therapy (ACT) for smoking cessation. Results demonstrated that FB ads and posts were the least cost effective method, costing $172.76 per participant vs. $46.98 per participant for standard media, for example, which included press releases on local television, radio, newspaper, and online news outlets (Heffner et al., 2013). Similarly, for an in-person smoking cessation clinical trial, Frandsen et al. (2014) compared FB ads to traditional methods (flyers, word of mouth, and newspaper advertisement) and found that the cost per participant using traditional methods was less than half of that for FB ads.

Other common limitations in the existing research that utilized FB recruitment strategies include: restriction of sample to individuals that have internet access and use FB regularly (Ramo et al., 2010), and ads that do not include diverse languages (Amon et al., 2014). To date, no FB recruitment studies have reported differences by language for smoking cessation studies. Although Carlini et al. (2014) conducted a study for low English proficiency Brazilians, with FB ads in Portuguese (the cost was $31.10 per smoker), it did not compare recruitment outcomes by language (English vs. Portuguese).

Few studies have reported on recruitment methods for Spanish-speaking populations for smoking cessation studies (Graham et al., 2012; Muñoz et al., 2006). Graham et al. (2012) focused on developing culturally-specific advertisements to recruit Spanish-speaking populations for a smoking cessation website. They rotated ads systematically across four popular Latino websites and found that loss-framed ads yielded a higher click-through rate than gain-framed ads, and surface-targeted ads outperformed deep-targeted ads for clicks, click-through rate, and number of registrants (Graham et al., 2012). A series of smoking cessation studies were also done by Muñoz and colleagues in both English and Spanish (Muñoz et al., 2006; Muñoz et al., 2009; Muñoz et al., 2012; Leykin et al., 2012). Most of the recruitment efforts were done through Google Ads. These studies examined the feasibility but not the costs of recruiting Spanish-speaking smokers worldwide. For example, in one study, 94,158 individuals from 152 countries and territories visited the site (Muñoz et al., 2012). A total of 9173 Spanish- and English-speaking smokers provided consent, however the recruitment costs were not reported (Muñoz et al., 2012).

The aim of the current study was to assess FB as a recruitment tool in a study for Spanish- and English-speaking smokers. Phase 1 examined ad lib recruitment of English- and Spanish-speaking adult smokers utilizing FB ads and posts for a smoking cessation webapp (recruitment-focused phase). Phase 2 systematically compared performance of recruitment methods by language (Spanish-speakers vs. English-speakers), location (San Francisco vs. U.S.) and content (self-efficacy ad vs. fear appeal ad) (experimental phase). Two main metrics were analyzed: number of clicks and engagement.

2. Method

2.1. Participants

Participants were recruited if they resided in the U.S. and reported being 18 years of age and older. The recruitment period was from January 9th, 2018 to May 16th, 2018. Participants were recruited for a one-condition smoking cessation study utilizing a webapp, where they completed questionnaires about their smoking habits and quit confidence, tracked their quit attempts, and received periodic follow up texts to track their quit progress. The responsive webapp was designed to work in a web browser on a computer or mobile device and did not require downloading a specific app or other software.

2.2. Assessments and measures

An extensive inventory of metrics exists to assess the success and connectivity of advertisement/recruitment strategies employed. The following metrics have been defined based on the Facebook Business (2018) stipulations, as FB is the recruitment channel of interest for this article. Metrics were divided in two: those based on FB algorithms and those based on participant behaviors. The metrics based on FB algorithms include: Click-Through Rate, Impression, Reach, Social Reach, Relevance, Unique Outbound Click. The metrics based on participant behaviors include: Clicks, Consents, Conversion rate, Cost-per-consent, Cost-per- click, Link clicks, Post comments, Post reactions, Post shares, Shares. See Table 1 for a description of each metric type.

Table 1.

Campaign type and metrics.

Campaign type
Ads	Paid advertisements that may appear in any given user's news feed or side-panel (run across FB, Instagram, Audience Network, and Messenger), can be targeted to specific populations.
Posts	FB Posts (text, images, or videos) are displayed within users' news feeds and can be self-posted or generated by their friends or followed pages. Boosted Posts. Users can pay to “boost” their posts' visibility.

Metrics based on Facebook
Click-Through Rate (CTR)	Percentage of times individuals see an ad and click on a link. The ratio of times that individuals click a link in an ad compared to the amount of times these individuals see the ad.
Engagement	The total number of actions that people take involving the ads.
Impressions	Number of times an ad is shown on screen for the targeted audience.
Reach	Number of individuals that see an ad at least once. (This differs from impressions which includes multiple views of an ad by the same individual.)
Relevance	Calculated based on an ad's positive feedback (e.g., app installs, clicks, video views) and negative feedback (e.g., User clicks “I don't want to see this” on the ad) and ranges from 1 (lowest) to 10 (highest). Higher relevancy can lower cost and improve delivery.
Social reach	Number of individuals that see an ad when displayed with social information, acting as word-of-mouth for advertisers (e.g., other FB friends who engaged with the ad or FB page).
Unique Outbound Clicks (UOC)	Number of clicks on links that redirect users off FB and to the targeted site.

Metrics based on participant behaviors
Clicks	Number of clicks on an ad. Counts multiple types of clicks, such as links to other destinations and links to expanded ad experiences.
Consents	Number of individuals consented to participate in the research study.
Conversion rate	Metric calculated based on the Unique Outbound Clicks (UOC; see definition above) and FB-attributed consents obtained.
Cost per consent	Cost per consenting individual obtained.
Cost per link click (CPC)	Average cost per each link click, benchmark for ad efficiency and performance. Calculated by dividing the total amount spent (U.S. $) per link click.
Link clicks	Number of clicks on ad links to specific destinations or experiences, both on and off FB-owned properties (e.g., websites, App stores, click-to-call, etc.).
Post comments	Number of comments per ad, which counts all comments made by individuals while the ad was running.
Post reactions	Number of times individual users reacted to a post with a click, which includes reactions such as “like” and “love.”
Post shares	Number of shares per ad while running (may also include Instagram shares sent to inboxes), but does not count engagement with the post after the share.

Notes. All definitions are based on Facebook Business' (2018) Glossary of Terms.

2.3. Procedures

The initial recruitment channels utilized in this study were Google AdWords, announcements in relevant list serves, local fliers, and distribution of 3 × 5 cards in Spanish and English with links to the website. The low recruitment rates led the researchers to add FB recruitment. This study only reports on participants recruited through FB.

Participants were recruited in two phases: a recruitment-focused phase and an experimental phase comparing language, location and content (self-efficacy ad vs. fear appeal ad). The recruitment-focused phase acted as a pilot test for recruiting participants with several different strategies that varied at a fast pace. Once the recruitment target was reached, the experimental phase focused on systematic comparisons between ads that performed well during the previous phase.

2.3.1. Recruitment-focused phase

The recruitment-focused phase primarily aimed to recruit as many participants as possible in a short period of time. Starting on January 9th 2018 up to April 30th 2018, two community managers (CM) were hired to recruit participants. One CM focused on recruitment of English-speaking smokers and the other CM focused on Spanish-speaking smokers. Each CM implemented her own recruitment strategy for her target audience. The English strategy mostly focused on Facebook Ads through the Ads Manager tool and the Spanish strategy mostly focused on boosted posts directly from Facebook. Across all campaigns between January 9th and May 16th, 2018 a total of 70 ads were run over 19 weeks; 50 ads in English and 20 ads in Spanish. Regarding posts, a total of 62 posts were run over 19 weeks; 15 posts in English and 47 posts in Spanish. During this phase, both ads and posts presented different text content, images, starting time and locations.

2.3.2. Experimental phase

The second phase systematically compared performance of recruitment methods by language (Spanish-speakers vs. English-speakers), location (San Francisco only vs. US) and content (self-efficacy ad vs. fear appeal ad). The experimental phase ran from May 3rd – May 16th 2018. For this phase two of the best performing ads from the recruitment-focused phase were selected based on CPC, CTR, and relevance. Campaigns were run by language (Spanish-speakers vs. English-speakers), location (San Francisco vs. US) and content (self-efficacy ad vs. fear appeal ad). For direct comparison, the ad images and content are shown in Fig. 1. Content for the self-efficacy ad included “How to Quit” text, while the fear appeal ad included “Just one cigarette a day can double your risk of heart disease” text.

Fig. 1.

Example of Spanish and English Facebook ads.

Notes. Ads (a) and (b) illustrate fear appeal, and ads (c) and (d) illustrate self-efficacy messages.

The complete study was approved by both the University of California San Francisco (15-17597) and Palo Alto University (15-042-H) Institutional Review Boards.

2.4. Analysis

Due to large variability and lack of controls during Phase 1, inferential statistics could not be completed for the recruitment-focused phase, so only descriptive analyses are provided. Phase 2, the experimental phase, was analyzed using a 2 × 2 × 2 binomial logistic regression with the three factors of language, location and content. The binomial logistic regression allows for direct comparison across language, location, and content despite the difference in Impressions or Reach because it treats each dependent variable as a yes/no outcome with a total N of the denominator variable (e.g., Impressions or Reach). Since Facebook does not reveal how their unique metrics (i.e., relevance) are calculated, only continuous outcomes with enough power to detect differences were included in the analysis. Because both Reach and impressions are determined by FB proprietary algorithms, there are likely differences in how different populations are targeted. Although impressions are often used as a base to calculate common industry metrics (e.g., CTR, CPC), Reach may be more appropriate for researchers looking to evaluate differences on an individual level. Both Impressions and Reach are determined by FB's proprietary algorithms, but once an individual is reached (sees the ad at least once), some metrics are based on the individual's behavior (clicks, shares, and engagement), not the FB algorithm. Therefore, clicks, total engagement, and social reach were run with both Impressions and Reach as denominators. UOC was only run with Reach as denominator because it measures unique clicks, and using impressions (which are not unique) as a denominator could cause confounded results.

3. Results

During the recruitment period, between January 9th and May 16th, 2018, the 70 ads and 62 posts received 2,357,181 impressions and yielded 30,169 unique outbound clicks for a total cost of $19,289.15. Total time spent managing the ads by the Spanish speaking and English speaking CMs was 164 h and 139 h, respectively.

3.1. Recruitment-focused phase

During the recruitment phase there were 581 participants in total: 540 were from elsewhere in the U.S. and 41 from San Francisco. Of the 540 U.S. participants that provided consent, 275 were Spanish-speakers and 265 English-speakers. A total budget of $11,203.40 was allocated for both languages; the final budget used was $7098.96 for Spanish language ads and $4104.44 for English language ads. Metrics generated for the Spanish and English language ads, respectively, were 1.83% vs. 1.77% for CTR, $0.43 vs. $0.44 for CPC, and a cost-per-consent of $25.81 vs. $15.49 (See Table 2).

Table 2.

Recruitment-focused phase (U.S.) n = 540: recruitment costs and performance metrics by language including posts and ads.

Metric type	Total	Spanish	English
Impressions	1,430,119	901,367	528,752
Reach	1,034,038	632,339	401,699
Frequency	1.38	1.45	1.32
Link clicks	25,835	16,486	9349
UOC	23,613	15,072	8541
Shares	1887	1513	374
Comments	491	359	132
Reactions	3828	3120	708
Social reach	8359	7106	1253
CTR, %	1.81	1.83	1.77
CPC, US$	$0.43	$0.43	$0.44
Consent to the study (FB)	540	275	265
Facebook mobile	475	266	209
Facebook website	65	9	56
Conversion rate (FB)	2.29%	1.82%	3.10%
Total spent	$11,203.40	$7098.96	$4104.44
Cost-per-consent (FB, US$)	$20.75	$25.81	$15.49

Note. CPC = cost-per-click; CTR = click-through-rate; FB = Facebook; UOC = Unique Outbound Clicks; all costs measured in U.S. dollars (US$).

3.2. Experimental phase

During the experimental phase a total of 43 participants were recruited (37 for U.S. and six for San Francisco). A total budget of $2600 was allocated for both languages and the final budget used was for $1299.87 Spanish and $1298.72 for English. Table 3 shows descriptive outcomes for ads only by location. Of the 37 U.S. participants that provided study consent, 18 were Spanish-speakers and 19 English-speakers. For Spanish-speakers, CTR was 1.41%, CPC was $0.41, and cost-per-consent was $27.78. For English-speakers, CTR was 1.44%, CPC was $0.49, and cost-per-consent was $26.31. Of the six SF participants that provided consent, four were Spanish-speakers and two English-speakers. For Spanish-speakers, CTR was 0.64%, CPC was $1.23, and cost-per-consent was $199.98. For English-speakers, CTR was 0.60%, CPC was $1.09, and cost-per-consent was $399.39 (See Table 3).

Table 3.

Descriptives for experimental phase (U.S. and San Francisco), recruitment costs and performance metrics by language for ads only.

Metric type	U.S.		SF
	Spanish	English	Spanish	English
Impressions	86,200	70,210	100,827	122,040
Reach	54,913	56,232	50,999	61,334
Frequency	1.57	1.25	1.98	1.99
Link clicks	1212	1010	649	730
Unique Outbound Clicks (UOC)	1117	951	621	696
Shares	102	31	6	3
Comments	44	13	1	1
Reactions	153	47	30	17
Social reach	616	135	937	80
Click-through-rate (CTR), %	1.41	1.44	0.64	0.60
Cost-per-click (CPC), US$	$0.41	$0.49	$1.23	$1.09
Consents (FB)	18	19	4	2
Facebook mobile	17	14	4	2
Facebook website	1	5	0	0
Conversion rate (FB)	1.61%	2.00%	0.62%	0.29%
Total spent	$499.96	$499.94	$799.91	$798.78
Cost-per-consent (FB,US$)	$27.78	$26.31	$199.98	$399.39

3.3. Impressions as denominator

Chi-square values and odd-ratios for the logistic binomial regressions calculated with impressions as the denominator can be found in Table 4. For clicks, there was a significant main effect observed for location (X² = 581.35, p < .005) and content (X² = 35.02, p < .005):U.S. participants were 2.28 times more likely to click an ad than San Francisco participants, individuals were 1.23 times more likely to click on the self-efficacy ad than the fear appeal ad. For Engagement, there was a significant main effect for location (X² = 832.88, p < .005), language (X² = 9.65, p < .005), and content (X² = 29.10, p < .005): U.S. participants were 2.56 times more likely to engage; Spanish-speaking participants were 1.11 times more likely to engage. The content variable showed that the self-efficacy ad was 1.20 times more likely to be engaged with compared to the fear appeal ad. Regarding Social Reach, there was a significant main effect for location (X² = 31.25, p < .005), language (X² = 1077.74, p < .005), and content (X² = 19.44, p < .005), as well as significant two-way interactions for location with language (X² = 82.63, p < .005) and content (X² = 6.35, p < .05). The odds-ratios indicate that, overall, U.S. participants were 1.52 times more likely to reach a new person via social means (i.e. shares, comments, etc.), Spanish-speaking participants were 7.32 times more likely to reach new users via social means, and the self-efficacy ad was 1.40 times more likely to reach people via social means. These main effects are qualified by the significant interactions of location by language and content. Specifically, Spanish-speakers within San Francisco (OR = 14.39) were far more likely to reach new users via social means than were Spanish-speakers in the U.S. (OR = 3.72). Similarly, the self-efficacy ad shown within San Francisco (OR = 1.70) was more likely to reach new users via social means than the same ad in the U.S. sample (OR = 1.15).

Table 4.

Binomial logistic regression results for clicks, engagement, and social reach, with impressions as denominator.

	Clicks		Engagement		Social reach
	X2	OR	X2	OR	X2	OR
Location US vs. SF Only	581.35⁎⁎⁎	2.28	832.88⁎⁎⁎	2.56	31.25⁎⁎⁎	1.52
Language Spanish vs. English	0.33	1.02	9.65⁎⁎⁎	1.11	1077.74⁎⁎⁎	7.32
Adset Self Eff. vs. Fear	35.02⁎⁎⁎	1.23	29.10⁎⁎⁎	1.20	19.44⁎⁎⁎	1.40

	Clicks			Engagement			Social reach
	X2	OR		X2	OR		X2	OR
		US	SF		US	SF		US	SF
Location ∗ Language Spanish vs. English	2.56	0.965	1.08	0.0	1.11	1.11	82.63⁎⁎⁎	3.72	14.39
Location ∗ Adset Self Eff. vs. Fear	2.20	1.29	1.17	2.68⁎⁎	1.26	1.13	6.35⁎	1.15	1.70

^⁎Significant at p < .05.

^⁎⁎Significant at p < .01.

^⁎⁎⁎Significant at p < .005.

3.4. Reach as denominator

Chi-square values and odd-ratios for the logistic binomial regressions run with Reach as the denominator can be found in Table 5. When run with Reach as the denominator, significant main effects of location (X² = 192.64, p < .005), language (X² = 14.43, p < .005), and content (X² = 20.13, p < .005) were found for clicks, as well as a significant interaction between location and content (X² = 6.89, p < .01). In general, users in the U.S. were 1.61 times more likely to click, Spanish-speakers were 1.14 times more likely to click, and the self-efficacy ad received 1.17 times the amount of clicks than did the fear appeal ad. Furthermore, the self-efficacy ad was far more effective at receiving clicks in the U.S. (OR = 1.28) than it was in San Francisco (OR = 1.07). For engagement, main effects were revealed for location (X² = 327.64, p < .005), language (X² = 41.96, p < .005), and content (X² = 15.25, p < .005), such that users in the U.S. were 1.81 times more likely to engage, Spanish-speakers were 1.24 times more likely to engage, while the self-efficacy ad was 1.14 times more likely to be engaged with. Significant interactions also emerged for engagement for location with language (X² = 13.45, p < .005) and content (X² = 8.01, p < .005), indicating that Spanish-speakers (OR = 1.40) or those who saw the self-efficacy ad (OR = 1.25) were most likely to engage if they were in the U.S. rather than San Francisco. Regarding social reach, main effects were found for language (X² = 1224.57, p < .005), and content (X² = 14.11, p < .005), which suggests that Spanish-speakers were 8.21 times more likely to be reached via social means, while the self-efficacy ad was 1.33 times more likely to reach users via social means. Significant interactions were also found for location with language (X² = 56.46, p < .005) and content (X² = 4.03, p < .05). Social reach was far more likely to be exhibited for Spanish-speakers within San Francisco (OR = 14.38) as well as the self-efficacy ad shown within San Francisco (OR = 1.55). Within UOC, significant main effects emerged for location (X² = 163.36, p < .005), language (X² = 12.15, p < .005), and content (X² = 20.87, p < .005), suggesting that users in the U.S. were 1.57 times more likely to click on an outbound link, Spanish-speakers were 1.13 times more likely to click on an outbound link, and the self-efficacy ad was 1.18 times more likely to have outbound links be clicked.

Table 5.

Binomial logistic regression results for clicks, engagement, social reach, and UOC, with reach as denominator.

	Clicks		Engagement		Social reach		UOC
	X2	OR	X2	OR	X2	OR	X2	OR
Location US vs. SF Only	192.64⁎⁎⁎	1.61	327.64⁎⁎⁎	1.81	0.82	1.07	163.36⁎⁎⁎	1.57
Language Spanish vs. English	14.43⁎⁎⁎	1.14	41.96⁎⁎⁎	1.24	1224.57⁎⁎⁎	8.21	12.15⁎⁎⁎	1.13
Adset Self Eff. vs. Fear	20.13⁎⁎⁎	1.17	15.25⁎⁎⁎	1.14	14.11⁎⁎⁎	1.33	20.87⁎⁎⁎	1.18

	Clicks			Engagement			Social reach			UOC
	X2	OR		X2	OR		X2	OR		X2	OR
		US	SF		US	SF		US	SF		US	SF
Location ∗ Language Spanish vs. English	3.56	1.22	1.07	13.45⁎⁎⁎	1.40	1.10	56.46⁎⁎⁎	4.69	14.38	2.36	1.20	1.07
Location ∗ Adset Self Eff. vs. Fear	6.89⁎⁎	1.28	1.07	8.01⁎⁎⁎	1.25	1.04	4.03⁎	1.14	1.55	2.68	1.28	1.09

^⁎Significant at p < .05.

^⁎⁎Significant at p < .01.

^⁎⁎⁎Significant at p < .005.

4. Discussion

Few articles have reported on recruitment methods for smoking cessation studies conducted among Spanish-speaking populations (Graham et al., 2012; Muñoz et al., 2006). The main aim of this study was to assess FB as a recruitment for a one-condition smoking cessation webapp study for Spanish- and English-speaking smokers. Recruitment was conducted in two phases, the first phase aimed to maximize recruitment outcomes during a specified period of time without implementing a predetermined or systematic approach to the ads. This phase served as a pilot study to identify the necessary recruitment procedures for an experimental design. Once the recruitment needs were met (i.e., as many participants as possible using a fast-paced ad lib, free-form approach), an experimental phase was implemented to systematically assess differences in recruitment campaigns by location, language, and content of the ads.

Overall results from the recruitment-focused phase indicate that Facebook ads and posts were effective in recruiting Spanish- and English-speaking smokers to an online smoking cessation webapp. A total of 581 participants provided consent (540 were from US and 41 from San Francisco), the number of U.S. participants that consented by language were 275 for Spanish-speakers and 265 for English-speakers; and the overall cost-per-consent was $20.75. Since the English-language and Spanish-language campaigns were different (English-language strategy focused mostly on ads and the Spanish-language strategy focused mostly on boosted posts), the outcomes of such campaigns are presented for informative purposes, but no generalizable assumptions can be made based on such outcomes. The cost-per-consent for the English campaign was $15.49, which is similar to the cost reported by Thornton et al. (2016) of $18.18 for English language studies that did not offer incentives. However, the cost-per-consent for the current study was higher than for studies that offered incentives (see Ramo et al., 2014 - $8.80 per-consent). A possible explanation for this difference may be that non-incentivized intervention studies that aim to change behavior may require extra motivation and commitment from individuals when compared to briefer or incentivized studies. Furthermore, comparisons between the current study and previous studies may be impacted by changes in FB's proprietary algorithm. Additionally, as FB becomes a more popular advertising platform, competition for ad space increases, which consequently affects both costs and click-through-rates. Finally, recruitment may vary depending on the types of ads that each study used, therefore, comparing outcomes of studies using different ads may be hard to evaluate.

Interestingly, the Spanish campaign had a higher cost-per-consent ($25.81) than the English campaigns. It is difficult to clearly ascertain the reason for this outcome as there is a dearth of studies that have examined these metrics among Spanish-speakers specifically and, more broadly, within smoking cessation web-based studies. Despite the fact that the Spanish language is the second most common native language spoken worldwide, it may be more expensive to recruit Spanish speakers in the U.S. given greater costs to target recruitment efforts. Additional external factors may further impede our ability to draw strong conclusions from this study. First, FB's proprietary algorithm determines how the different audiences are reached. For example, in our study Spanish-speakers were delivered ads more frequently than English-speakers. Second, the methods used by the community managers in the recruitment phase for each language were different. Together with previous studies using other online recruitment methods, such as Google ads (Graham et al., 2012; Muñoz et al., 2006; Muñoz et al., 2009; Leykin et al., 2012; Muñoz et al., 2012), this study demonstrates the feasibility of recruiting Spanish speaking smokers in the U.S. and contributes to the still limited literature on online recruitment methods for Spanish-speakers. For example, previous studies that have shown the feasibility of recruiting Spanish speaking smokers worldwide (Muñoz et al., 2006; Muñoz et al., 2009; Leykin et al., 2012; Muñoz et al., 2012), were done using Google Ads, and were carried out more than six years ago.

During the experimental phase a series of systematic comparisons between languages, locations, and ad content were concurrently deployed using the same ads. Direct comparisons were calculated using both Impressions and Reach as denominators. Although Impressions are often used as a standard to calculate common industry metrics (e.g., CTR, CPC), Reach may be more appropriate for researchers looking to evaluate differences on an individual level. Both Impressions and Reach are determined by FB's proprietary algorithm, but once an individual is reached (sees the ad at least once), some metrics are based on the individual's behavior (clicks, shares, and engagement), not the FB algorithm.

The results of the analysis using Impressions as the denominator indicated that U.S. participants had higher rates of clicks, engagements, and social reach than San Francisco participants. Spanish-speaking participants had higher rates of engagement and social reach than English-speakers. The self-efficacy ad had higher rates of clicks, engagements, and social reach than the fear ad. Additionally, Spanish-speakers within San Francisco had higher rates of social reach (OR = 14.39) than in the U.S.; and the self-efficacy ad when shown within San Francisco had higher rates of social reach than more broadly in the U.S.

When Reach was the denominator users in the US had higher rates of clicks, UOCs, and engagements than users in San Francisco. Spanish-speakers had higher rates of clicks, UOCs, engagement, and social reach than English-speakers. Additionally, Spanish-speakers who saw the self-efficacy adset had higher rates of engagement and social reach in the U.S. compared to San Francisco. The self-efficacy ad had higher rates of clicks, UOCs, engagements, and social reach than the fear ad. Additionally, the self-efficacy ad had higher rates of clicks and engagement in the U.S. compared to San Francisco.

Overall, results followed a similar pattern when evaluated based on Impressions or Reach. U.S. users performed better (i.e. more clicks, engagement, social reach, UOC) than San Francisco users, Spanish-speakers engaged more than English-speakers, and the self-efficacy ad performed better than the fear appeal ad. When using Reach as a denominator, there were significant interactions for clicks (location by language) and engagement (location by language, and location by type of ad) that were not evident when using impressions as the denominator. These results highlight the relevance of considering metrics other than industry standards (i.e., Impressions as a denominator) in order to capture a more accurate representation of users' behavior when recruiting for smoking cessation studies.

5. Limitations and future directions

There are several limitations to the current study. First, because during the recruitment phase each community manager had different strategies, generalizable conclusions are difficult to draw. The recruitment phase was successful in achieving higher conversion rates and lower cost-per-consent than the experimental phase. However, it is unclear if these outcomes were due to heterogeneity between campaign strategies, or continued recruitment from the same audience pool, or ads becoming less effective over time. Future studies could improve their recruitment efforts if direct comparisons between ad type and languages are implemented from the beginning.

Secondly, FB's proprietary algorithm made it challenging to determine what effects were due to the algorithm versus differences in individual behavior, keeping in mind that both can affect cost. Additionally, due to the types of data provided by FB, statistical analysis could only be performed on certain variables.

Third, it is difficult to evaluate differences between users residing in the broader U.S. and those within San Francisco only. The San Francisco campaign was less effective than the U.S. campaign, but that may be explained by differences in smoking prevalence rates and population size. Therefore, these results should not be generalized to other U.S. cities or states without further considerations of demographic and smoking differences within regions. Future studies could benefit from comparing ad performance between different states or cities. For example, it is possible that ads may have performed better in states with higher rates of smokers. Akers and Gordon (2018) suggested to analyze the tobacco use prevalence by state to conduct effective comparisons between states.

Fourth, ads in the experimental phase were previously run in the recruitment phase and originally designed for the English-speaking audience. Ads designed for an English-speaking audience then translated into Spanish may perform differently than ads designed for a Spanish-speaking audience and translated into English. Even though the text approximated a literal translation, that does not guarantee that they were equally perceived by both audiences. Thus, the design of the study did not allow for evaluation of language translation effects.

Fifth, while a variety of ad content were used during the recruitment-focused phase, only two types of content (fear appeal vs. self-efficacy) were used in the experimental phase. Therefore, more studies are needed in order to grasp a robust understanding of the effects of ads as results may not be generalizable to other types of content. Future studies should attempt to replicate the comparisons employed in the current study and expand them to other types of content.

Finally, although the experimental phase did have significant findings, it only yielded 43 participants (37 for US and 6 for San Francisco) over a two week period. As such, the impact of these findings in relation to Spanish speakers should be interpreted with caution and replicated with a larger sample.

6. Conclusion

To the best of our knowledge this is the first study to report on the use of Facebook ads to recruit Spanish-speaking smokers and to compare ad performance by location, language and ad content for the recruitment of smoking cessation webapp studies. This study demonstrates that FB is an effective platform for recruiting both Spanish- and English-smokers into clinical trials of smoking cessation webapps. Although there were differences in cost-per-consent for Spanish- and English-speakers, recruitment of Spanish-speakers through Facebook is feasible. Furthermore, comparing an ad's performance by location, language, and ad content may contribute to developing more efficient and effective campaigns. With increased awareness regarding the importance of including minorities and women in research, being able to effectively reach and recruit diverse audiences is critical. Therefore, Facebook's ability to target diverse audiences cost-effectively makes it an important recruitment tool.

Conflict of interest

Facebook for Recruiting Spanish- and English-Speaking Smokers.

The authors report no conflict of interest.