Using Facebook ads with traditional paper mailings to recruit adolescent girls for a clinical trial

Abstract

Introduction

Clinical trials require sufficient samples recruited within limited time and budget constraints. Trials with minors are additionally burdened by the requirement for youth assent and parental permission. This paper details the use of Facebook ads and traditional paper mailings to enroll 797 adolescent girls for a longitudinal, web-based, drug abuse prevention trial. Data on sample representativeness and retention are also provided.

Methods

Facebook ads appeared on the pages of females aged 13 or 14 years who reside in the U.S. Ads linked girls to a recruitment website. Girls who wanted more information submitted contact information and were mailed information packets to their homes containing, among other things, youth assent and parent permission forms. Returned forms were verified for accuracy and validity.

Results

The Facebook ad campaign reached 2,267,848 girls and had a unique click-through rate of 3.0%. The campaign cost $41,202.37 with an average cost of $51.70 per enrolled girl. Information packets were mailed to 1,873 girls. Approximately one-half of girls returned the forms, and 797 girls were enrolled. The Facebook campaign's success varied by ad type, month, and day of the week. Baseline data revealed comparability to national data on demographic and substance use variables.

Conclusions

Results suggest that Facebook ads provide a useful initial point of access to unparalleled numbers of adolescents. Clinical trials may benefit from a two-fold recruitment strategy that uses online ads to attract interested adolescents followed by traditional recruitment methods to communicate detailed information to adolescents and parents.

1. Introduction

Clinical trials require sufficient sample recruitment in a timely manner and within budget. Trials of drug abuse prevention programs rely heavily on school- and community-based settings for recruitment. Limiting the sampling frame to a small geographic location restricts the generalizability of study findings to broader populations. With 95% of U.S. teens aged 14–17 years online (Pew Internet Research, 2010), and 94% of U.S. teens aged 12–17 years on Facebook (Madden, 2013), new opportunities exist to recruit sufficiently large and representative samples of adolescents.

Internet recruitment is cost-effective and time-efficient (Miller & Sønderlund, 2010). Compared to other recruitment methods, internet approaches are fast and relatively inexpensive (Batterham, 2014). Internet recruitment methods have also successfully reached marginalized groups, such as participants with a genetic condition (Close, Smaldone, Fennoy, Reame, & Grey, 2013) or hidden populations of illicit drug users (Miller & Sønderlund, 2010). Among the available recruitment vehicles, the social networking site Facebook allows advertisers to target campaigns based on geographic location, gender, age, and keywords in user profiles.

Evidence of Facebook advertising's utility in clinical trial recruitment is growing. Facebook advertising can be more cost-effective and time-efficient than other internet recruitment methods (Loxton et al., 2015; Ramo & Prochaska, 2012; Ramo, Rodriguez, Chavez, Sommer, & Prochaska, 2014). Studies have used Facebook to successfully recruit adult participants (Batterham, 2014; Nelson, Hughes, Oakes, Pankow, & Kulasingam, 2014) as well as adolescent participants (Altshuler, Storey, & Prager, 2015; Chu & Snider, 2013; Ellis et al., 2012; Fenner et al., 2012; Gilligan, Kypri, & Bourke, 2014; Mustanski, Greene, Ryan, & Whitton, 2015). With the exception of Mustanski et al. (2015), however, these studies were cross-sectional and did not require parental permission for adolescent participation.

Using Facebook advertising to recruit participants, however, is not without difficulties, particularly when studies with adolescents require parental permission (for a review on recruiting adolescents through Facebook, see Amon, Campbell, Hawke, & Steinbeck, 2014). Regardless of the method, sample recruitment in such studies necessitates informed assent procedures for youths as well as procedures that secure informed parental permission. Therefore, the potential ease of using Facebook ads to reach youths may be offset by the additional effort required to identify and reach parents or guardians.

Furthermore, the potential cost and time advantages conferred by recruiting adolescents via online advertising, such as Facebook, diminish considerably if the recruited sample suffers high attrition. Indeed, Bajardi et al. (2014), found that participants recruited online had lower follow-up rates than participants recruited through traditional methods. Perhaps the impersonal context of online recruitment that favors cost and time constraints also hinders rapport and relationship building (Temple & Brown, 2012). To date, little evidence exists that an adolescent sample recruited through online procedures such as Facebook can be successfully retained in a longitudinal clinical trial (Amon et al., 2014).

To our knowledge, no published studies have examined Facebook's utility for reaching and retaining adolescents in a longitudinal study that requires parental permission. Accordingly, the present study reports procedures used to recruit a nationwide sample of 797 early adolescent girls using Facebook advertising and traditional mailings. Data on sample demographic and drug use variables as compared with national data sets are also provided.

2. Methods

2.1. Participants

As part of a larger study testing the efficacy of a web-based, gender-specific, drug abuse prevention program (Schwinn, Hopkins, & Schinke, 2016; Schwinn, Schinke, Hopkins, & Thom, 2016), participants were Facebook users who registered as female, were 13 or 14 years of age, and were U.S. residents. Participants had to read and understand English and have access to a private computer with internet access. Study procedures were approved by the Columbia University Institutional Review Board.

2.2. Facebook recruitment

Facebook offers either a cost per click (the advertiser is charged each time an ad is clicked) or cost per impression (the advertiser is charged each time an ad is displayed) pricing unit. The cost per click pricing model is more cost effective for advertisers who wish to direct people to a website rather than to boost brand name recognition. Because we wanted to direct girls to our study recruitment website, we chose the cost per click model. For each ad in a campaign, Facebook suggests a bid range (e.g., $0.45–$0.55 per click). The bid range suggested by Facebook varies depending on the frequency with which the ad will appear, the size of the target audience, and competition from others who are targeting the same audience. The maximum bid (the highest amount the advertiser is willing to pay per click) can be set lower, within, or higher than the suggested bid range. Ads set at the higher bid range should be more competitive with advertisers targeting the same audience. Further detail on ad campaign costs, setup, management, and definitions can be found at the Advertiser Help Center under Learn About Advertising (https://www.facebook.com/business/help/).

Adhering to Facebook Business advertising guidelines, we created 12 ads with the goal of identifying the ones that generated the most actions at the lowest cost. The text for all the ads emphasized either earning money to take surveys, or earning money to take surveys plus the chance to do online activities. Six ads were designated as study-focused because their headline read “Join RealTeen!” The images for these ads were our study logo, our intervention's narrator, or stock photos of a teenage girl on her computer. The other six ads were designated as incentive-focused because the headline read “Get Paid to Take Surveys!” The images for these ads were gift cards, our study logo, or stock photos of a teenage girl on her computer.

Using Facebook filters (e.g., age, gender, geographic location), we targeted our ads to appear on the pages of 13- and 14-year-old females living in the United States. Each time an ad was clicked, the girl was directed to the study recruitment webpage. The webpage contained a brief description of the study and inclusion criteria (aged 13 or 14 years, U.S. residence, English speaking, and a private computer with internet access). Girls who attested to meeting the criteria and wanted more information entered their name and address. When a girl clicked on an ad and provided her name and address, Facebook analytics recorded this as an “action.”

We initiated the ad campaign in April 2013 with a daily budget set at $750–$1000. We set the maximum bid for each ad slightly higher than Facebook's suggested bid range. For example, if Facebook suggested a bid range of $0.45–$0.55, we might set our maximum bid to $0.60 or $0.65. Ads did not run during school-day hours, but rather during hours and days that mirrored prime internet use time (i.e., weeknights, weekend days and nights, and holidays) for our sample demographic. After 1 week, we stopped running five ads that generated no action (four incentive-focused and one study-focused).

In early May, we paused the campaign (owing to a change in staff). Upon resuming the campaign in June, we decreased the daily budget to $100, set the maximum bid per ad at, or below, the amount suggested by Facebook, and only ran four ads (Study-focused 1, Study-focused 2, Incentive-focused 1, and Incentive-focused 2; see Table 1). The two incentive-focused ads achieved clicks but yielded few actions. Therefore, from July to September, we ran only the two study-focused ads and set the daily budget to $250–$350. Ads ran during the day and night from June through August; in September, ads ran nightly during the week and all day on weekends.

Table 1.

Average performance metrics for Facebook campaign.

Ad type	Ad image		Days	Impressions (per day)	Cost (per day)	uClicks (per day)	Actions (per day)	Cost Per Action	uCTR	Cost Per uClick
Study-focused 1		Take online surveys and earn up to $180 in gift cards. Enroll Now!	131	709,073	$159.01	296	8.73	$18.22	2.72%	$0.54
Study-focused 2		Earn up to $180 taking surveys. Have the chance to do fun, online activites. Enroll today!	130	275,913	$57.55	126	3.28	$17.56	1.52%	$0.46
Study-focused 3		Do you like to take surveys? Enroll in a Columbia Unversity study and earn up to $180!	34	388,921	$88.30	147	1.03	$85.77	0.46%	$0.60
Study-focused 4		Earn up to $180 taking surveys. Have the chance to do fun, online activites. Enroll today!	24	440,385	$97.84	176	3.13	$31.31	0.54%	$0.56
Study-focused 5		Earn up to $180 taking surveys. Have the chance to do fun, online activites. Enroll today!	17	291,940	$68.43	112	0.35	$193.89	0.38%	$0.61
Incentive-focused 1		Take online surveys and earn up to $180 in gift cards. Enroll Now!	50	386,523	$88.30	154	3.4	$25.97	0.90%	$0.58
Incentive-focused 2		Earn up to $180 taking surveys. Have the chance to do fun, online activites. Enroll today!	15	511,098	$88.42	182	1.2	$73.67	0.92%	$0.49
Other Ads	N/A		7	98,832–230,965	$10.72–$33.71	18–48	0	N/A	0.12%	$0.58–$0.71

Note. “Other Ads” were five ads (one study-focused and four incentive-focused).

2.3. Enrollment procedures

We used each action (receipt of a girl's name and mailing address) to mail an information packet home. The packet included: 1) a study overview for the girl that addressed potential questions and concerns, 2) a study overview for the parent(s) that addressed potential questions and concerns, 3) youth assent, parental permission, and contact information forms, and 4) a stamped, return envelope. If forms were not returned within 1 month, a second packet was sent.

Returned assent, permission, and contact information forms were verified for accuracy and validity. When parental and adolescent signatures were similar, we called parents to confirm permission. Once assent and permission were verified, we mailed copies of the forms to the parent, asking them to phone or email us if they had not provided permission for their daughter to participate or if their daughter was not 13 or 14 years old.

2.4. Statistical procedures

Data were analyzed using R (R Core Team, 2015). To test whether proportions in sample data differed from national data, Z scores were calculated with significance set at two-tailed, α = 0.05. When differences between sample and national data were significant, Cohen's d (Cohen, 1988) was calculated to estimate the effect size.

Data from the U.S. Census Bureau were used to compare race and ethnicity (U.S. Census Bureau, 2016), parental education (U.S. Census Bureau, 2014), and geographic region (U.S. Census Bureau, 2015). Data from the U.S. Department of Education were used to compare girls' type of school (Bitterman, Gray, & Goldring, 2013). City type (urban, large town, small town/rural) was estimated by comparing sample zip codes to Rural-Urban Commuting Area Codes (Rural Health Research Center, 2004). Comparative drug use data were from Monitoring the Future (Johnston, O'Malley, Bachman, Schulenberg, & Miech, 2014) and the Youth Risk Behavior Surveillance Survey (Centers for Disease Control and Prevention, 2013).

3. Results

3.1. Overall ad campaign

The campaign ran for 131 days between April 11, 2013 and September 30, 2013. The campaign had 187,018,812 impressions (the number of times the ads were displayed) and had a reach of 2,267,848 girls (the number of girls to whom the ads have been shown). The ads received 68,198 unique clicks (uClicks; a metric that removes the occurrence of repeat clicks by the same user) for a unique click-through rate (uCTR; unique clicks divided by reach) of 3.0%. Of the 68,198 clicks the ads received, 2.75% resulted in an action (receipt of a girl's name and mailing address).

Information packets were mailed to 1857 potential participants (16 actions did not meet enrollment criteria). We received completed enrollment packets from 936 potential participants; 797 potential participants met the verification requirements and were enrolled in the study. The Facebook ad campaign cost $41,202.37 and had an average cost of $0.60 per click, $22.00 per action, and $51.70 per enrolled participant.

3.2. Ad performance by calendar

Across the campaign, the average number of daily actions varied little by day of the week (Fig. 1) with two exceptions: actions were highest Sundays (19.14) and lowest on Mondays (11.45). From Tuesdays through Saturdays, the average number of daily actions ranged from 13.76–14.38. The average cost per action was similar for Monday (the lowest performing day), and Sunday (the highest performing day), at $20.79 and $20.51, respectively. The average cost per action was highest on Friday and Saturday, $27.35 and $24.87, respectively.

Fig. 1.

Average number of actions and cost per action by day of week.

The average number of daily actions varied greatly by month (Fig. 2). July had the highest number of actions per day (19.92); June had the lowest number of actions per day (2.47). Though April and August had the same number of actions per day (17.57), the average cost per action was $39.12 in April (the highest of all the months) and $19.60 in August. The month with the lowest cost per action was July ($12.70), which also had the highest number of actions. Ads ran for 19 days in April, 6 days in May, 24 days in June, 27 days in July, 28 days in August, and 27 days in September.

Fig. 2.

Average number of daily actions and cost per action by month.

3.3. Ad performance metrics

Each ad, the number of days the ad ran, and the performance metrics adjusted for days ran (impressions, cost, uClicks, actions, cost per action, uCTR, and cost per uClick) appear in Table 1. Three ads were responsible for generating 93% of the total actions: Study-focused 1 (61%) ran for 131 days, Study-focused 2 (23%) ran for 130 days, and Incentive-focused 1 (9%) ran for 50 days. All three ads outperformed the other nine ads on actions per day and cost per action. Study-focused 1 and Study-focused 2 also had high uCTR and cost per click, but performance on impressions, cost, and uClicks was inconsistent among these three ads. Poor performing ads were characterized by having high uClicks, high costs per uClick, few to zero actions, and having high costs per action (when generated).

3.4. Study sample and retention

Demographic and drug use data from the baseline survey (N = 788) were compared with national data sets to assess comparability (Table 2). The racial composition of the sample was 76% White, 4% Asian, 2% American Indian/Alaska Native, <1% Hawaiian Pacific Islander, and 2% more than one race; 15% of the sample was Hispanic. At 16%, more girls in the sample identified as Black, compared with national data (13%; Z = 2.39, p < 0.05, Cohen's d = 0.13).

Table 2.

Descriptive sample data (N = 788) and national data.

Variable			Sample data %			National data %			Z
Ethnic groupa
Latina			15			17			−1.15
Racial groupa
White			76			78			1.11
Black			16			13			2.39*
Asian			4			5			1.80
American Indian/Alaska Native			2			1			0.57
Hawaiian Pacific Islander			<1			<1			0.00
Two or more races			2			2			0.00
Parent level of educationb
High school or less			29			41			−6.84*
Some college			30			27			1.86
4-year degree			17			20			−2.10*
Graduate degree			18			12			5.17*
Unknown			6			—			—
Type of schoolc
Public			90			92			−2.07*
Private			10			8			1.96*
Geographic regiond
Northeast			18			18			0.00
Midwest			24			21			2.07*
South			34			37			−1.74
West			24			24			0.00
City typee
Urbanized area			81			79			1.38
Large town			9			10			−0.94
Small town/rural			10			10			0.00
		Grade			Grade			Grade
Past-month substance usef	8th	9th	10th	8th	9th	10th	8th	9th	10th
Alcohol	16	24	27	11	24	31	2.18*	0.00	0.90
Binge drinking	6	10	13	6	14	17	0.00	−2.16*	1.11
Cigarettes	10	11	14	5	10	13	3.12*	0.55	0.21
Marijuana	7	15	18	7	18	24	0.00	1.44	1.52

^*p < 0.05.

^aNational data (N = 316,427,395) from U.S. Census Bureau, 2016.

^bNational data (N = 242,248) from U.S. Census Bureau, 2014.

^cNational data (N = 53,988,330) from Bitterman et al., 2013.

^dSample data categorized according designations from U.S. Census Bureau, 2015 (N = 318,907,401).

^eZip codes from sample data categorized according to data from Rural Health Research Center, 2004 (N = 292,936,686).

^fSample data are 8th graders (n = 215), 9th graders (n = 412), and 10th graders (n = 139); National data for 8th graders (N = 6450) from Johnston et al., 2014; national data for 9th (N = 3500) and 10th graders (N = 3100) from Centers for Disease Control and Prevention, 2013.

Regarding parental education, and compared with national data, fewer girls in the sample reported their parents completed “high school or less” (29% versus 41%; Z = −6.84, p < 0.05, Cohen's d = 0.29) or “4-year degree” (17% versus 20%; Z = −2.10, p < 0.05, Cohen's d = 0.11) and more girls reported that their parents had a “graduate degree” (18% versus 12%; Z = 5.17, p < 0.05, Cohen's d = 0.26). The percentage of girls attending public school (90%) was less than national data (92%; Z = −2.07, p < 0.05, Cohen's d = 0.14), and the percentage of girls attending private school was more than national data (10% versus 8%; Z = 1.96, p < 0.05, Cohen's d = 0.14).

The percentages of girls' residing in the Northeast, South, and West were similar to national data; the percentage of girls living in the Midwest was higher (24% versus 21%; Z = 2.07, p < 0.05, Cohen's d = 0.09). Girls' resided in urban areas (81%), large towns (9%), and rural or small towns (10%) in similar percentages to national data.

Rates of past-month alcohol, cigarette, and marijuana use and binge drinking among the 8th, 9th, and 10th grade girls in the sample were similar to national data with the following exceptions: 8th grade girls in the sample reported more alcohol use (16% versus 11%; Z = 2.18, p < 0.05, Cohen's d = 0.24) and cigarette use (10% versus 5%; Z = 3.12, p < 0.05, Cohen's d = 0.41); 9th grade girls in the sample reported less binge drinking (10% versus 14%, Z = −2.16, p < 0.05 Cohen's d = 0.21).

Sample retention was 98% at posttest (approximately 3.5 months after pretest) and 97% at 1-year follow-up (from date of posttest); each survey required approximately 15 min to complete.

4. Discussion

This study suggests that using Facebook ads, in conjunction with traditional paper mailings, is a feasible strategy to enroll a large, representative, nationwide sample of early adolescent girls in under 5 months. Sample demographic and drug use data were largely comparable to national data sets. Statistical differences in demographic and drug use data were small. Furthermore, the high retention rates at post-test and 1-year follow-up suggest that the coupling of online ads and mailings potentially yields a sample that is motivated and sufficiently informed of study procedures, thus minimizing attrition.

The study's sample size and 4.5-month recruitment timeframe (131 days between April and September) compare favorably to a similar study in which we enrolled 513 youths in approximately 12 months using face-to-face recruitment at community-based agencies (Schinke, Schwinn, & Fang, 2010). Our sample and timeframe also compare favorably to a cross sectional study that used Facebook ads to recruit a subset of 38 minors in 4 months (Fenner et al., 2012). A meta-analysis of 85 drug abuse prevention program studies employing face-to-face recruitment found an average retention of 81% at 3-month follow-up and 73% at 1-year follow-up (Hansen, Tobler, & Graham, 1990). By comparison, our retention rates of 98% at posttest and 97% at 1-year follow up are high. These rates contrast with prior research that associated online recruitment with lower rates of retention compared to offline or face-to-face recruitment (Bajardi et al., 2014; Temple & Brown, 2012).

4.1. Interpreting Facebook campaign results

Success with Facebook advertising involves an element of risk and requires vigilant campaign monitoring. At the onset of a campaign, multiple ads must be created and tested to determine which ads will reach the target demographic. Facebook provides various metrics (e.g., impressions, clicks, cost per click, number of actions, cost per action) to monitor and assess the progress of a campaign. To accrue our target sample size on time and within budget, we focused on two metrics: 1) the number of actions (receipt of a girl's name and mailing address), and 2) the cost per action.

We terminated 9 of our 12 ads within a month because they generated few actions and the cost per action was high. We were surprised that among the poorest performing ads were the incentive-focused ads—those featuring the headline “Get Paid to Take Surveys” and images of such gift cards as iTunes and Visa. Indeed, within the first week of the campaign, we terminated four of the six incentive-focused ads because they were costly (a lot of girls clicked on them) and generated zero actions. Girls who clicked on these ads were likely motivated to earn money, but disillusioned to learn that the compensation would accrue over 3 years, and therefore chose not to provide their names and addresses. Regardless, costly ads that generate few or no actions must be terminated quickly to minimize depleting limited recruitment funds.

Through daily monitoring and terminating poorly performing ads, the July campaign generated more actions per day than April (19.90 in July versus 17.57), and at one-third of the cost per action ($39.12 in April versus $12.70 in July). The higher average cost per action in April was largely due to the inclusion of ads that averaged $73.67 – $193.89 per action. In July, we only ran two study-focused ads—those featuring the headline “Join RealTeen” and images of the study logo or a girl on her computer—that averaged $12.78 and $12.58 per action.

Low ad performance in June was likely attributable to our smaller daily budget of $100. Our experiences in August exemplify the unpredictability of Facebook advertising. August ads were identical to July ads and ran on the same schedule. Compared to July, however, August had 2.35 fewer actions per day, and each action cost an additional $6.90. The occurrence of fewer actions and higher costs per action—i.e., girls clicking on ads frequently but not providing their names and addresses—continued in September. Possibly, ads benefitted not only from increased visibility in the summer, but ultimately from adolescents' perceived amount of free time. In July, girls may have felt they had time to join and participate in a study.

4.2. Limitations

Study findings are limited to the nature of the sample: girls who are registered Facebook users, who clicked on at least one study ad, provided their name and address for more information, and then enrolled in the study. In addition, because clicking on an ad may be positively associated with the amount of time spent on Facebook, the sample may disproportionately include girls who spend significant time on Facebook. Findings are also limited by the study enrollment criteria: aged 13 or 14 years, U.S. resident, English speaking, and access to a private computer with internet access. Whether and how such girls differ from those not in our study is unknown.

4.3. Conclusion

The first of its kind, this study suggests that Facebook ads, together with a traditional paper mailing, can yield over a relatively brief timeframe, a large, nationwide sample of early adolescent females from the U.S. for a clinical trial. Procedures that involve online ads and traditional paper enrollment forms may serve to legitimize study procedures for adolescents and parents, reduce deceitful enrollment by adolescents who do not meet study requirements, and increase retention. Study data have positive implications for other efforts to employ the reach of social networking sites in the service of recruiting adolescents for clinical trials.

HIGHLIGHTS.

• Used Facebook ads and paper mailings to recruit adolescents for a clinical trial

• Recruitment procedures yielded a nationwide sample of 797 girls in 4.5 months.

• Facebook ad performance varied by ad type, month, and day of the week.

• Sample demographic and drug use data were largely comparable with national data.