Facebook ads to the rescue? Recruiting a hard to reach population into an Internet-based behavioral health intervention trial

Abstract

Objective

Facebook (FB) ads are touted as a way to facilitate recruitment of hard to reach participants into digital health research but the evidence has been mixed. This study aimed to empirically evaluate the impact and cost-effectiveness of paid ads for recruitment into a national trial testing an Internet-based, coached intervention for parents of children with Fetal Alcohol Spectrum Disorders.

Methods

Post hoc analysis of FB ad data and Google analytics on the online trial consent site (myStudies) were conducted on 11 campaigns employing static image/text ads. Standard metrics (e.g., click through rate, cost per 1000 impressions, cost per consent) were calculated and descriptive statistics comparing FB ad engagement and enrolled participants over time were conducted.

Results

Ad campaigns were active for a combined 115 days over 58 weeks resulting in 1533 links to the online recruitment site. During the ad campaigns, the mean rate of enrolment was 1 participant every 2 days. The first 3 ad campaigns were the most cost-effective. Mean cost per enrolment was $19.27 (Canadian dollars).

Conclusions

FB ads were efficient and cost-effective in broad dissemination of trial information, but more research is needed to explore the impact of saturation (how often ads are posted), design (what is in the ad), and individual determinants (who is likely to respond to an ad) on converting FB ad engagement into enrolment. Avoiding a reductionist approach to analytics will help ensure appropriate and targeted strategies remain the priority for digital health research recruitment through social media.

Highlights

• •The use of Facebook ad campaigns in recruiting from a low-prevalence population is discussed.
• •The cost-effectiveness and impact of ad campaigns for enrolling families for one Internet intervention study is reported.
• •Patterns and trends in how ads were responded to are highlighted.
• •Future directions for the use of social media in internet-intervention research are proposed.

1. Introduction

Suboptimal recruitment is a major obstacle to the successful and efficient completion of clinical trials. A recent survey of corresponding authors of randomized trials found that nearly 70% had either failed to meet their recruitment target or required an extended recruitment period (McDonald et al., 2006; Foy et al., 2003). Insufficient recruitment of study participants in digital health research may result in losing the statistical power of a predictive conclusion, as well as prolonging the time and increasing the cost associated with the study (Treweek et al., 2013). While sub-optimal treatment outcomes are increasingly communicated in the academic literature, inefficient recruitment methods often go unreported.

The issue of low recruitment becomes even more exaggerated when the target of an intervention study is not a common phenomenon, or involves those of low socioeconomic or minority status (Ejiogu et al., 2011). The persistent challenges of recruiting from these populations for clinical research have been well documented in the literature (Motzer et al., 1997). The implications of potentially biased samples or underpowered analysis threaten to undermine treatment advances for these populations (Caplan and Friesen, 2017). Moreover, digital health research and implementation continues to be plagued by low recruitment which threatens to further limit large-scale uptake within health systems (O'Connor et al., 2016; Sanders et al., 2012). Assessment of strategies to recruit broadly from hard to reach populations for technology-focused intervention trials is vital to ensure interventions are tested with truly representative samples and demonstrate feasibility across a broad population.

Online methods of recruitment have offered health researchers a new avenue of accessing the attention of a large percentage of the world's population. Facebook (FB) is the most popular social media platform in the Western world, with an estimated global membership of 2.07 billion monthly users (The Statistics Portal, 2018). Parents are especially avid users, with 75% of parents on FB logging in daily and 51% logging in multiple times a day (Duggan et al., 2015). FB offers researchers an opportunity to access potential participants with unprecedented precision via targeting of advertisements (ads) to user groups on factors like location, age, gender, ethnicity, and language. By virtue of its widespread use, relative anonymity, cost-effectiveness, and acceptability, FB is an attractive recruitment path (Thornton et al., 2016; Amon et al., 2014; Carter-Harris et al., 2016). The literature around FB use in recruitment is growing but there is limited empirical analysis of how effective these methods are across different study populations and the ethical implications of this form of recruitment for digital health trials. The limited research that has been reported shows results vary widely depending on study context. For example, ads have been found to be more effective at recruiting young females than more traditional means (Loxton et al., 2015), but studies in other populations show social media recruitment does not reduce disproportionate sampling (Pedersen et al., 2015). A 2017 review of FB in recruiting participants for health research found only 35 studies in the past 12 years (Whitaker et al., 2017). Most of the included studies were recruiting young people (<25 years of age), 63% were conducted in the United States and the median length of recruitment was 3 months (Reiter et al., 2017; Schwinn et al., 2017).

After experiencing slow recruitment for a longitudinal Canadian study on an Internet-based intervention for caregivers of children with Fetal Alcohol Spectrum Disorders (FASD) [Strongest Families FASD™] we implemented FB ad campaigns to improve recruitment reach. Canadians lead the world in Internet use, spending more hours online (36.7 per month) than anyone else. Fifty-nine percent of Canadians are active users of social media (Canadian Internet Registration Authority (CIRA), 2016) making the study of FB ads for clinical trial recruitment among this population particularly informative. Other studies on using FB to recruit for studies involving mental health have found that it is more cost effective (Teo et al., 2018; Batterham, 2014), has a faster rate of recruitment (Teo et al., 2018; Kayrouz et al., 2016), and better at targeting “hard to reach” populations such as ethnic minorities (Ünlü Ince et al., 2014) than traditional methods. However, it could lead to a disproportionate sample. Our post hoc analysis of recruitment data sought to answer two questions: 1) to what extent did FB ads impact on enrolment for the FASD digital health intervention trial over time?; and 2) what was the comparative cost-effective of FB ads across campaigns compared to standard marketing benchmarks?

2. Methods

2.1. Design and background information on the main study

This study is based on secondary, post hoc analysis emerging from a randomized controlled trial with the objective of comparing an Internet-based mental health intervention for Canadian parents of children with FASD (Strongest Families FASD: strongestfamilies.com) to standard care. Parents in the main study completed a baseline assessment as well as 5-month and 11-month follow-ups lasting 20–30 min each. The 11-module intervention combined tailored online psychoeducation and parenting skills-training (40 min to complete) via a web-platform (Wozney et al., 2015) with 45-minute phone sessions for each module with a paraprofessional coach. All informed consent processes took place via an online enrolment site (myStudies). On the myStudies site, parents were guided through 5 sections of an informed consent document, asked yes/no questions to confirm understanding, and completed a brief eligibility screening questionnaire. Inclusion criteria for the study included parents with children aged 4–12 years of age with a diagnosis under the umbrella term ‘Fetal Alcohol Spectrum Disorder’ (as reported by parents) who had been experiencing behavioral problems (as defined by the parent) for at least 6 months prior to study screening. Scores on a standardized questionnaire that assesses a child's adaptive functioning and behavior problems needed to reach clinical threshold for a parent to be enrolled in the trial. Further details of the inclusion/exclusion criteria and target population can be found in the published protocol (Turner et al., 2015). The design of this recruitment study was enacted as a response to the low enrollment rate arising from traditional strategies (posters, email campaigns, etc.) and emerging literature about the potential of FB as a recruitment strategy. The main study protocol was approved by the IWK Research Ethics Board and the Queen's University Health Sciences Research Ethics Board (REB). The REB was provided with sample Facebook posts including text and images. The protocol also advised the REB that social media posts would change frequently to maintain interest in the study, encourage “sharing” and extend recruitment reach.

2.2. Outcome measures (present study)

FB Analytics and Google Analytics (Google, 2015) were used to track actions related to the ads. Measures included: 1) impressions which is the number of times the ad was displayed; 2) cost per mile (CPM) is the most common metric for pricing marketing materials and measures the cost per 1000 impressions: CPM = (total cost of ad campaigns/# of impressions) × 1000; 3) click through rate (CTR) is a measure of how interested people were in the ad: CTR = # of clicks on the FB ad/# of impressions; 4) cost per link (CPL) is a measure of cost-effectiveness for participants being redirected from the Facebook ad to the myStudies site: CPL = cost for ad campaign/# of myStudies site hits; 5) cost per consent (CPC) is a measure of cost-effectiveness for individuals who completed enrollment: CPC = cost for ad campaign/# of completed consents; and 6) the number of myStudies visitors who completed the online consent process.

2.3. Procedures

Over the 58 week recruitment period in this study, typical community recruitment efforts such as posters, brochures, email circulation to health professionals, non-profit organization website promotion were maintained (Table 1). Some of these strategies and their effectiveness may have overlapped, making the unique comparison between more specific strategies unclear. However, since all study participants completed study enrollment screening and consent via an online consent platform (myStudies) we were able to reliably isolate traffic to the site from FB ads by examining Google Analytics output.

Table 1.

Selected examples of recruitment strategies for the main study.

	Method of recruitment	Details of strategy
1	Emailing	Direct via distribution lists and circulated through FASD clinics and organization members
2	Brochure/posters	Distributed to clinics, community centres, doctors' offices, mental health authorities etc. in (at least) 8 provinces and 1 territory
3	Other social media	Kijiji, IWK/CRFH Facebook page, CFRH Website, Google Ads
5	Community Group	Strongest Families Institute website, NeuroDevNet website
6	Conferences and Workshops	6th International Conference on FASD, BC; FASD Workshop, NS; Crisis and Trauma Resource Institute, Inc. workshop.
7	Facebook	Paid advertisements; regular posts

A FB page dedicated to our Research Centre was created at the start of the recruitment period. A series of 11 paid promotional ad campaigns were designed and run periodically for a total of 115 days over 58 weeks. A typical ad comprised a succinct description about the research with a hyperlink that directed the FB user to an eligibility screener and online consent form hosted on the myStudies (myStudies.ca) platform. All ads were picture/text combinations and presented in English. Images were selected to appeal to caregivers of children with FASD and vary demographically. Static images of parents, or parents with children, were chosen intentionally with imagery designed to invoke recall of first-hand experiences with challenging child behaviours and/or parent-child tension (Fig. 1). Audience targeting fields in the FB ad manager were selected to show ads to users with specific interests and demographics (e.g., Canada, parenting, Fetal Alcohol Spectrum Disorder, FASD, challenging behavior, parents, 18–60 years, and mental health professions). For the final ad campaign (# 11 Table 2) we selected a different ad objective option in the Ad Manager. Instead of ‘website clicks’ the ad objective was set to “post-engagement”. Despite these differences, the content of the posts were very similar.

Fig. 1.

Sample Facebook ads.

Table 2.

Descriptive statistics and cost-effectiveness of FB ad campaigns (2015–2016).

Campaign #	Campaign dates (M/D)	Duration (days)	Impressions	Link clicks	Total # consents	Total cost*	Cost/link click	Cost/consent
	2015
1	03/23–03/30	8	6322	178	11	$39.00	$0.22	$3.55
2	04/01–04/23	23	10,631	195	13	$61.00	$0.31	$4.69
3	06/12–06/19	8	5456	181	13	$30.00	$0.17	$2.31
4	07/03–07/13	11	17,119	293	6	$82.87	$0.28	$13.81
5	07/28–08/04	8	6192	95	2	$38.53	$0.41	$19.27
6	08/19–08/30	12	11,633	24	5	$54.87	$2.29	$10.97
7	09/09–09/23	15	10,098	180	4	$73.62	$0.41	$18.41
8	10/13–10/20	8	6711	94	3	$34.91	$0.37	$11.64
9	11/12–11/19	8	17,262	15	4	$36.00	$2.40	$9.00
	2016
10	01/28–02/04	8	8945	2	0	$36.00	$18.00	n/a
11	04/27–05/02	6	20,060	276	1	$99.02	$0.36	$99.02
Mean		10	10,948	139	6	$53.26	$2.29	$19.27
Total		115	120,429	1533	62	$585.82

*All costs in the table are reported in Canadian dollars (CAN$). n/a: there were no consents during this ad campaign. ⁺Means for days, clicks and consent values were rounded to the nearest whole number.

3. Results

Over the 58 weeks between March 2015 and May 2016, 11 ad campaigns ran for a total of 115 days. Four of the campaigns 1, 2, 5 and 7 (see Table 2) circulated the same exact ad (text and image), the remaining campaigns each had a unique ad. Ads ran for a mean length of 10 days [range: 6–23]. During the 115 days the ads were live, 62 parents were enrolled into the study (Fig. 2). Overall the ads resulted in 120,429 impressions (Table 2); the overall CPM score was $4.86. In approximately 1 in every 1000 ad views the ad was clicked on (CTR = 1.2). The cost of all the ads was CAN $585.82 with a mean CPL of CAN $2.29 and CPC of CAN $19.27. Our first 3 campaigns were the most cost effective and the most productive in terms of total number of consenting participants. After the 3rd campaign, CPC ratios doubled or tripled. The last ad campaign which used a slightly different ad format (described above) had a high number of FB impressions but only resulted in one additional consented participant (CPC = $99.02).

Fig. 2.

Comparison of FB link clicks and myStudies consents during each ad campaign time period.

Internet traffic analysis on the myStudies site showed a total of 2702 unique myStudies site views during the days when ads were running with 81% (n = 2201) of those views being redirects from Facebook. Seventy percent (n = 1533) of redirects were attributable to link clicks on one of the campaign ads. Although all ads were set to run for a week or more (M = 10.45, SD = 4.87), traffic originating from FB ads was typically highest within the first three days after the ad was originally posted. While different campaigns generated various numbers of impressions, the number of clicks and consents per impressions show a declining trend in the level of interest they generated (Fig. 3).

Fig. 3.

Clicks and consents per impression with trend lines.

4. Discussion

We have demonstrated that effective targeting of social media ads can substantially assist in broad engagement with the public around clinical trials at marginal cost. Over two thirds of the visits to the online consent site originated from FB during the ad campaigns. Our overall CTR was below the benchmark rate for health and medical industry standards (1.79%) (WordStream, n.d.) but consistent with the average global Facebook advertising CTR of 1.10%. While the FB ads clearly generated significant interest in learning more about the study only a small proportion of those potential participants went on to complete the online consent process. This is consistent with a recent study that found those who became aware of the study via social media were less likely to be eligible and enrol into the study when compared to those who were recruited through traditional methods (Frandsen et al., 2016). In our study the “conversion” of people seeing the FB ad to completing study enrolment involved several intermediary steps. Following ethical guidelines, all potential participants visiting the online consent site were invited to review the online informed consent document, complete a brief screening survey and respond to check-back questions aimed at verifying understanding of the trial. They were also advised that they could leave the site and contact the research team prior to completing consent if they had questions. These informed consent interactions may not map well onto consumer purchasing behaviour models that drive most marketing metrics and benchmarks. There are numerous reasons why participants choose not to participate in clinical trials (George et al., 2014) that, while beyond the scope of the present study, warrant further exploration. Understanding how social media ads shape and change knowledge about health research and how decisions are made between the point of clicking on the FB ad and providing informed consent remain ambiguous. Together the findings of this study raise an important question about whether the use of conventional sales-marketing benchmarks (e.g., CPL, CTR) are helpful for actually tracking health research participation decision-making.

Our CPM score of $4.86 per 1000 impressions demonstrates a more cost-effective approach than almost all other major media marketing channels, like direct mailing campaigns which typically have a CPM of $26–27 (Fou, 2014). Overall cost per enrolled participant ratio (CAN $19.27) was consistent with the average (US $14.41/ or CAN $17.90) (Schwinn et al., 2017) of other social media. With Facebook executive reports showing CPM rates went up 171% during the first half of 2017 the need for optimizing the cost-effectiveness and tailoring of ads for specific audiences is becoming more critical (Prater, 2017). The use of social media ads in trial recruitment requires purposeful use of tailoring parameters and interpretation of FB analytics to continually refine, modify and optimize ads for improved impact and reduced cost while remaining ethically sensitive (i.e., maintaining the voluntary nature of research) and transparent. There are very few evidence-based guidelines to inform health trialists in creating high-impact social media ads that also reflect the ethical principles of recruitment and consent (Gelinas et al., 2017). Trial and error remains the predominant learning mode. To our knowledge, our study is the first to report on the use of FB ads in recruiting for an internet-delivered health intervention for a low prevalence disorder and the first to report on recruitment of Canadian participants. A general weakness of this recruitment method is that it introduces selection bias by potentially over-representing those individuals who use Facebook frequently and are thus more likely to be exposed to the ads. However, this bias is similar to many other forms of population sampling conducted over the Internet or in other ways.

Finally, our results lend some credibility to the notion of a “saturation effect” or “ad fatigue” (Price, 2017). Cost-effectiveness and rate of consents declined steeply. Facebook executives have recently acknowledged that users may be reaching a saturation point with ads (Jowitt, 2017) which raises important future considerations for researchers planning to use Facebook for trial recruitment. Implications may be especially pronounced for longitudinal studies of low-prevalence health conditions where the pool of potential participants could be overexposed through targeted ads and develop negative associations or feel pestered (Hairong et al., 2013).

Methods to minimize ad fatigue have not been well documented, but may include using additional platforms to FB including Instagram (Burgess et al., 2017) or Craigslist (Staffileno et al., 2017). Although, our study made use of other social media platforms, such as Kijiji, while still seeing the symptoms of ad fatigue. Other studies have found that different types of ads offered by FB have different levels of success. Kayrouz et al. (2016) found that boosting posts and fan pages were more effective than ads, while Ramo et al. (2014) found that newsfeed ads were more effective then promoted posts, sponsored stories or standard ads. Our 11th campaign, which used the “post-engagement” ad metric generated many more impressions than the other campaigns and had a high rate of clicks per impression despite the ad fatigue. This did not however, lead to many more consents and we are left to speculate on whether it would have been more effective earlier on in the recruitment process. Further research is needed to determine whether there is an optimal mix or sequence of different dissemination strategies by platform, ad types, or metrics. A limitation of our study is that we did not conduct direct recruitment method cost comparisons between traditional methods (brochures, posters, etc.) and Facebook ads.

Analytics capabilities for FB and Google provide a rich opportunity to research more dynamic measures of reach (Juraschek et al., 2018). For example, investigating how ads are experienced by new potential participants versus those who are being recanvassed. Other factors impacting on ad success for digital health research are not well documented in the academic literature but could be informed by empirical evidence from the fields of media studies, human factors and marketing (Den Broeck et al., 2017). Comparative studies that explore aesthetic variety in ad design, personalizing of messages or use of different ad campaign goals could lead to improved strategies for reaching new and underrepresented participants (Lima Moraes de Oliveira et al., 2016). The messaging content itself has been shown to impact recruitment (Teo et al., 2018; Choi et al., 2017). Although, what messaging is most effective is different for each research population and context (Batterham, 2014; Burgess et al., 2017). A recent study has also shown that video ads were more effective than static images (Burgess et al., 2017). While the purpose of this secondary analysis was not to study the direct impact of different messaging our findings suggest future research could look at comparative impacts of the same add with different message wording or tone.

The use of social media advertising in engaging traditionally underrepresented individuals in health research is a promising but imprecise science. Our study contributes novel findings from a country with heavy internet users and for a health condition with low prevalence rates. In this rapidly developing area innovative analytic approaches are needed to mine the detailed data sets from FB and Google analytics to generate new knowledge around recruitment saturation (how often ads should be posted); design (what combination of text/video/image is optimal), and individual determinants (who is likely to respond to an ad) on enrolment. Social media use is increasing worldwide and analytics have significant potential for helping digital health researchers connect with the public in new ways and learn more about decision-making behaviour around participation.

Declaration of conflicting interest

The authors certify that they have NO affiliations with or involvement in any organization or entity with any financial or non-financial interest in the subject matter or materials discussed in this manuscript.

Funding

This work was supported by the Canadian Institutes of Health Research (Grant # PHE122190), the Public Health Agency of Canada, and NeuroDevNet, which is funded by the Networks of Centres of Excellence, a program of the Canadian federal government to advance science and technology. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Ethical approval

The main study protocol was approved by the IWK Research Ethics Board and the Queen's University Health Sciences Research Ethics Board.

Guarantor

LW.

Contributorship

LW and KT researched literature, conceived the study and conducted data extraction. BR-D was involved in manuscript development and data analysis. PM revised the manuscript for important intellectual content. All authors reviewed and edited the manuscript and approved the final version of the manuscript.