Recruiting young women of color into a pilot RCT targeting sexual health: Lessons learned and implications for applied health technology research

Abstract

Objective

To evaluate different recruitment methods to enroll participants into a mHealth pilot RCT: banner ads on Facebook and OkCupid, and targeted electronic outreach (e.g., emails to community-based organizations and to professors at local colleges).

Participants

Between October 2015 and May 2016, 114 college-aged Black and Latina women 18 to 24 participated in the study.

Methods

Recruitment methods compared online banner ads on social media to targeted electronic outreach. Individual banner ad images were compared by impressions, clicks, and cost by enrolled participants.

Results

More targeted electronic recruited participants enrolled than via banner advertisements. Banner ads with images of women yielded a higher click-through-rate and was more cost effective versus the logo alone.

Conclusions

Recruiting young women of color may be facilitated through known and trusted adults, such as college professors, rather than through anonymous banner advertisements on social media.

Background

In the United States, young Black and Latina women experience higher rates of sexually transmitted diseases (STDs), ¹ including HIV, compared to White women.² In fact, data from 2008–2010 show women at minority serving institutions (MSI) experienced increased rates of Chlamydia at almost twice the rate of non-minority serving colleges and universities, at 10.0% versus 5.4%.³ The ubiquity of web access and mobile phones have allowed HIV and STD researchers to recruit study participants using individuals (i.e., seeds) on social media^4–8 with a wide range of hidden populations including men who have sex with men (MSM),^9–12 drug-using populations,¹³ and homeless¹⁴ populations from various Internet websites, including Craigslist and Facebook.^5,15–17 Although nutrition and diet studies have successfully used social media to recruit adolescent women as participants,^6,18,19 these health issues tend to be less stigmatized than sexual and reproductive health. Another key difference between these studies and the one discussed here is the lack of comparison between social media and other approaches. In their study about recruitment and retention of young women into nutrition research studies Leonard and colleagues were unable to compare recruitment between their three studies due to differences in degree of burden and incentives across studies (study designs also differed across all three studies).¹⁹

Sexual health technology research among young women of color (YWOC) is an emerging area. Unlike technology-delivered interventions focused on other populations, such as MSM, a review of the use of technology for HIV prevention among adolescent and adult women found that the Internet or other social media platforms were infrequently utilized.²⁰ Rather, young women were typically recruited from in-person clinical or community sites with the exception of combining a Craigslist.org ad with other in-person approaches^20–22 and combining campus tabling with print and online advertisements.²³ In several recent reviews outlining the current state of the literature of technology-delivered HIV interventions (between 2011 and 2015),^20,24,25 ten focused on adult women of color, while 32 focused on other populations; out of the ten, four were identified as including young adult women of color between 18 and 29 years of age. Among those identified in which the focus was on YWOC, two recent sexual and reproductive health (SRH) computer-delivered HIV prevention studies targeting women for intervention included one using Craigslist ads amidst a variety of brick and mortar (i.e., clinical and community-based) sites to recruit participants; however, the researchers did not describe in detail which approach was more effective.^21,22 There is a dearth in understanding how banner ads might be utilized to recruit women and sub-populations among women (i.e., Black and Latina women).²⁶

Given the limited data on recruitment among Black and Latina women, this paper describes the recruitment process for a web-based pilot Randomized Controlled Trial (RCT) among Black and Latina women aged 18 to 25 years in New York City. This study examined demographic and behavioral characteristics of those who were eligible and who enrolled into the RCT compared to those who were eligible and began the screening survey but who did not enroll in the study. Effectiveness by recruitment source was also explored.

Methods

Participants and procedure

All study procedures took place between October 7, 2015 and May 15, 2016. Data for this study were taken from GURHL (Guide to Understanding Reproductive Health for Ladeez) Code, a randomized two-group pilot study. Participants were randomly assigned to receive a web-based application with a clinic and condom locator, sexual health resources, and text an expert function (intervention), or sexual health clinic listings without hyperlinks (control). The pilot explored the feasibility and preliminary efficacy of an online smartphone application tailored with and for Black and Latina women aged 18 to 25 in New York City. To help ensure cultural relevance, a community advisory committee was consulted to finalize the look, feel, and content of each form of recruitment (banner ads, recruitment emails, and fliers). This study first compared participant demographics by enrollment status, enrolled versus not enrolled and then we compared the effectiveness of online recruitment to targeted electronic outreach.

Eligible participants had to self-identify as women between 18 and 25 years old, and either Black and/or Latina; own a smartphone (i.e., a phone capable of accessing the Internet); live or work in NYC; and reported vaginal or anal intercourse with a male partner in their lifetime. Exclusion criteria were selected to eliminate potential subjects in circumstances that could increase a participant’s use of sexual or reproductive health services. These included being pregnant, or having children two years old or younger. Other exclusion criteria included only being sexually active with women, or being unable to read English. Among Spanish speaking Latinos in NYC, 71% reported speaking English well or very well, while 29% reported speaking English less than “very well” (n =5 35,798).²⁷ Regardless of recruitment source, all participants were directed to an initial online screening survey. All procedures were reviewed and approved by the City University of New York Institutional Review Board (protocol # 381039).

Recruitment strategy

Women were recruited via online banner ads on Facebook website and OkCupid, targeted electronic outreach (e.g., emails to community-based organizations and to professors at local colleges), and a free events website, Eventbrite, an online self-service ticketing platform where individuals, businesses, and organizations plan, manage, and promote events. However, given that only seven women began a screening survey who came via Eventbrite, all future discussion in this paper excluded this recruitment source. Recruitment fliers and banner ads included the study logo, study description (i.e., using a sexual health app on a smartphone for three months, post-test, optional follow-up focus group), possible $70 for participating in all research components, and directed participants to the online screening survey.

To complete enrollment in the pilot RCT, participants submitted a screenshot of the web-app saved as a bookmark on their phone to the study email address or study phone number. Research staff reached out via text or phone to participants who had completed the baseline survey but who failed to send a screenshot. For this pilot, participants were randomized after being screened, providing consent, and completing their baseline assessment. In the baseline survey, participants were asked how they heard about the study. In total, 114 Black and Latina women aged 18 to 25 years in New York City took part in the study.

Online recruitment procedures

We purchased Facebook and OkCupid banner ads and, for each ad, one of several paid stock photographs was used (see Figure 1). In addition, an electronic announcement was posted on Eventbrite. Utilizing the banner ad builder for both Facebook and OkCupid, we tailored to the population who would see the banner ads. Facebook selection allowed for a more refined selection for a potential reach of 99,000 people. Prior research shows that young people typically enter their full name, facial pictures, and hometown in their profiles,^28,29 and that they do so accurately.³⁰ As such, the selection criteria for both OkCupid and Facebook included gender and geography. Facebook additionally offered the ability to select the audience that would see the ad by relationship status, interest in men, age range, and language (see Supplemental Table 1 S1). Of note, the OkCupid ad ran for a total of ten weeks.

Figure 1.

Banner Advertisement Images

Targeted electronic outreach

An email was sent to faculty and staff at n = 24 CUNY campuses in NYC (n = 142 individuals). The content of the email introduced the primary investigator, included a description of the study, and an attached recruitment flier to forward to students and other faculty or staff. Both the email and attached flyer included a hyperlink to the online screening survey. Professors were asked to forward the email with the attached recruitment flier to students directly (i.e., to post it on an online course management system or on a class website), and/or to hand out in class. CUNY is the nation’s largest urban university and is a unique slice of the NYC population with a lower income status (two out of five students live in households that earn less than $20,000 per year), and represents a highly diverse population (in 2015 the enrollment was 30% Hispanic, 25% White, 25% Black, and 20% Asian/Pacific Islander).³¹ Two school LISTSERVs at the CUNY School of Public Health and the CUNY School of Social Work and seven CUNY-wide programs and clubs did a mass distribution to their students; thus, the number of emails sent to faculty and staff could be higher than our estimation.

Emails were also sent to n = 25 NYC community-based organizations (CBO) serving young women of color. These organizations were asked to forward the email with the attached recruitment flier and to print and post the flier in a visible place at their CBO.

Analytic plan

Sample characteristics

Among those eligible and after excluding those with missing data from analysis, we compared those who ultimately enrolled in the research study (n = 110) to those who were eligible but chose not to enroll (n = 46) on age, race, relationship status, education, individual income, employed status, insurance status, condomless sex acts in their lifetime, number of male sex partners in their lifetime, age of oldest male sex partner, and age of first sexual intercourse using t-tests, chi-square or Fisher’s exact tests as appropriate.

Recruitment approaches

Next, we report on web analytics, differentials of friend referrals, and cost-per-enrollee by recruitment source to evaluate recruitment approaches. To compare the different individual image banner ads, the following measures were used: impressions, clicks, and cost. An impression is defined as instances when a banner ad appeared on a user’s Facebook or OkCupid page regardless of whether the banner ad was clicked or not. A click was counted when a user clicked on the survey link and was then taken to the landing page for the online survey. The number of clicks per impression and those enrolled were compared between the online approaches (Facebook and OkCupid) to explore how many clicks yielded a single enrolled participant, how many impressions it took to generate a single enrolled participant, and the number of clicks on impressions by online recruitment source (Facebook and OkCupid), which were then analyzed using a Fisher’s exact test. The cost-per-enrollee was calculated by the number of clicks divided by the cost to display the ads. The cost for targeted electronic outreach recruitment was determined by the staff-hours spent at a rate of $20 per hour on recruitment efforts such as posting fliers in physical spaces, and sending out recruitment emails. The cost-per-enrollee was calculated by total staff hours divided by the number of enrolled participants. Chi-square or Fisher’s exact tests were used as appropriate to compare the number of screening surveys, number of eligible participants, and number of enrolled participants by the Facebook and targeted recruitment sources only as no respondents completed screen surveys who were enrolled by Eventbrite or OkCupid.

Banner advertisements

Ads were then compared based on analytic metrics including impressions and clicks. We then calculated a click-through rate, defined as the number of clicks on advertisements per impression. The percentage is the number of people who viewed the impression and then clicked on the ad where a higher percentage indicates a higher percentage of people who saw the ad and clicked on it.³² Chi-square was used to compare impressions between the three groups: logo only, an ad with an image of only a Black woman, and a third ad including an image of only a Latina woman. All statistical analyses were performed using SAS 9.4 software.³³

Results

Recruitment, enrollment, and retention

Recruitment, enrollment, and retention data are illustrated in Figure 2. In total, 583 participants visited the screening survey landing page, of which 492 consented to proceed with the baseline survey (84.4%). Forty-nine percent of those consenting (n = 243) did not complete the screening survey in full (i.e., closed their browser window, usually very early into the survey process) and 18.5% (n = 91) were ineligible for the study. Participants were deemed ineligible for any one of the following reasons: 89 (18.1%) were not between 18 and 25 years old, 76 (15.4%) were neither Black or Latina, 33 (6.7%) had participated in a peer sexual health education program of 10 weeks or longer, 34 (6.9%) neither lived nor worked in New York City, 20 (4.1%) had children under 2 years old, 18 (3.7%) were male, 6 (1.2%) were pregnant at the time they took the screening survey, and 2 (0.4%) were ineligible for the study due to not owning a smartphone.

Figure 2.

Consort Diagram

Of the 156 eligible participants, 28.5% (n =140) provided consent to complete the baseline survey, while 3.2% (n =16) were eligible for the study but did not provide consent to continue with the study. Of the 122 eligible women who completed the baseline survey, staff was unable to reach 8 women to complete all enrollment steps. The remaining 114 women completed all enrollment steps and were randomized into the controlled trial pilot (n = 61 intervention, and n = 53 control).

Demographic Characteristics

Those enrolled in the study reported significantly lower incomes and had significantly less education than those who chose not to enroll (see Table 1). Those enrolled in the study were more likely to report an individual income below $20,000 (78.2%), compared to those who did not enroll (37.0%) who had an even distribution of incomes between <$20,000 and $20,000 to $49,000 (32.6%) (χ² = 7.8, p < 0.01). No one reported an income above $50,000. Those enrolled in the study also tended to be working or to be a student compared to those who did not enroll in the study (90% versus 59%).

Table 1.

Demographic Characteristics and Health Risk Behaviors, Past 3 Months

	Enrollment Status
	Enrolled		Not Enrolled		Test Statistic
	n	%	n	%	χ2 (df)	p
	N = 110		N = 46 **
Race/ethnicity					χ2 = 1.47 (1)	p = 0.22
Latinas (including Black-Latinas)	66	60.0%	29	70.7%
Black	44	40.0%	12	29.3%
Married or Partnered					χ2 = 0.17 (1)	p = 0.68
Married or partnered	42	38.2%	12	26.1%
Single	68	61.8%	23	50.0%
Missing*	-	-	11	23.9%
Education					χ2 = 0.06 (1)	p = 0.80
Less than high school, high school, GED, some college	57	51.8%	19	41.3%
College degree or master’s degree completed	53	48.2%	16	34.8%
Missing*	-	-	11	23.9%
Currently enrolled in college					χ2 = 2.11 (1)	p = 0.15
Yes	96	87.3%	27	58.7%
No	14	12.7%	8	17.4%
Missing*	-	-	11	23.9%
Income					χ2 = 7.8 (1)	p = 0.0052
Up to $19,999	86	78.2%	17	37.0%
$20,000 to $49,999	24	21.8%	15	32.6%
Missing*	-	-	14	30.4%
Employed					χ2 = 3.9 (1)	p = 0.04
Working full or part-time or student	99	90.0%	27	58.7%
Not paid work (looking for work, unemployed, caretaker)	11	10.0%	8	17.4%
Missing*	-	-	11	23.9%
Condomless sex in lifetime					χ2 = 0.36 (1)	p = 0.55
Yes	97	88.2%	24	52.2%
No	13	11.8%	2	4.4%
Missing*	-	-	20	43.4%
Number of male partners					χ2 = 1.45 (1)	p = 0.23
1 to 5 partners	68	61.8%	12	26.1%
6 to15 partners	32	29.1%	10	21.7%
Missing*	10	9.1%	24	52.2%
	Enrolled		Not Enrolled
Continuous measures	n	M (SD) or %	n	M (SD) or %	t (df)	p
Mean age (SD)	110	22.1 (2.1)	39	22.3 (2.0)	0.65 (1)	p = 0.52
Mean age of first sex (SD)	110	17.1 (2.7)	24	16.2 (2.9)	–1.38 (31)	p = 0.17
Mean age of oldest male sex partner (SD)	110	25.9 (6.5)	26	27.1 (5.2)	0.96 (39.9)	p = 0.35
15 years old or younger	58	52.7%%	18	39.10%
16 to 18 years old	29	26.4%	8	17.4%%
19 to 21 years old	14	12.70%	—	—
22 to 23 years old	8	7.30%	—	—
50 years old	1	0.91%	—	—
Missing	—	—	20	43.50%

^*Missing values were excluded from significance testing.

^**This number varies due to missing values.

Table 1 displays demographic characteristics of women enrolled in the study (n = 110) compared with those who were eligible but did not enroll in the study (n = 46). Enrollment status did not significantly differ by age (Mean age = 22), race/ethnicity, or relationship status. Enrollment status did not significantly differ by mean age of first sex—enrolled M (SD) = 17.1 (2.7) versus not enrolled M (SD) = 16.2 (2.9), condomless sex in lifetime (88.2% among enrolled versus 52.2%), or number of male sex partners in lifetime (61.8% among enrolled and 26.1% among not enrolled reported between one and five partners, and 29.1% of those enrolled compared to 21.7% among those not enrolled reported between six and 15 partners).

Data by recruitment source

First, we compared the cost in dollars spent per participant enrolled by recruitment sources, as well as the number of impressions needed to yield an enrolled participant. Of note, 130 participants did not indicate their recruitment source and were thus excluded from these analyses; as a result, totals do not add up to the final number of participants recruited and enrolled. A total of $704.75 was spent on Facebook ads, generating 275,332 impressions and 1,986 clicks ($0.35 per click) with a 0.72% click-through rate. This resulted in 17 completed screening surveys (i.e., $41 spent per completed survey), five eligible participants (i.e., $141 spent per eligible participant), and two enrolled participants. Effectively, 137,666 impressions were needed to generate a single participant at a cost of $352 per participant. By comparison, a total of $287 was spent on OkCupid banner ads, generating 143,515 impressions. This resulted in 11 clicks ($26 per click), nine screening surveys started ($32 per survey), but no screening surveys were completed; thus, no participants were enrolled via OkCupid.

Next, we compared those sources that actually generated enrolled participants: Facebook and targeted electronic recruitment sources (i.e., emails to college professors and LISTSERVs). Participants recruited via targeted electronic sources were more likely to complete the survey after starting (63.9% vs. 34.0%), be eligible (45.1% vs. 29.4%), and (among those eligible) to enroll (99.1% vs. 40.0%) than were those recruited via Facebook. Targeted electronic recruitment was more cost-efficient than recruiting via Facebook ($1.59 was spent per enrolled participant versus $273.50 per enrolled recipient via Facebook). Examining cost by screening surveys started, completed, and cost per eligible and enrolled participant was also more cost effective by targeted electronic source compared to Facebook (see Table 2).

Table 2.

Analytics, Cost, Enrollment Data by Recruitment Source

	Targeted electronic (Professor)	Facebook	OkCupid	Totals	Fisher’s exact or χ2 (df)	p value
Web-analytics					—	—
Impressions on banner ads	NA	275,332	143,515		—	—
Total clicks	NA	1986	11		—	—
Click-through-rate (clicks/impressions)	NA	0.72%	0.00%		—	—
Screening Surveys					χ2 = 16.55 (1)**	p = < 0.0001
Screening surveys started	385	50	9	446	—	—
Screening surveys completed	246	17	0	263	—	—
Surveys completed/surveys started	63.9%	34.0%	0	0.589686099	—	—
Eligible Participants					χ2 = 7.32 (1)**	p = 0.0068
Ineligible	65	12	0	77	—	—
Eligible	111	5	0	116	—	—
# eligible/# completed	45.1%	29.4%	0	0.441064639	—	—
Enrolled Participants					Fisher’s exact**	p = < 0.0001
Enrolled	110	2	0	112	—	—
Not enrolled	1	3	0	4	—	—
# enrolled/# eligible	99.1%	40.0%	0		—	—
Cost					—	—
Total $ spent on recruitment	$175.00	$704.75	$287.02	$879.75	—	—
Amount $ spent per impression	—	$0.00	$0.00
Amount $ spent per click	—	$0.35	$26.09
Amount $ spent per started screened participant	$0.45	$14.10	$31.89	$1.97	—	—
Amount $ spent per completed screened participant	$0.71	$41.46	0	$3.35	—	—
Amount $ spent per eligible participant	$1.58	$140.95	0	$7.58	—	—
Amount $ per enrolled participant	$1.59	$352.38	0	$7.85	—	—

^*130 participants did not indicate their recruitment source and were excluded from analysis; thus, why totals do not add up to final recruitment number

^—no statistical test calculated.

^**Test statistic on Facebook and targeted electronic only.

Finally, Table 3 illustrates the comparison of three banner ads (logo-only, an image of a Black woman with the study logo, and another image depicting a Latina woman with the study logo) using web analytics (p < 0.001). Further paired chi-square tests revealed a significant difference when comparing all three banner ads (p < 0.001), where the banner ad including the image of a Black woman yielded the highest click-through-rate (1.66%). The images with women yielded a higher click-through rate (1.37% for the banner ad including the image of a Latina woman) in comparison to the logo-only banner ad (CTR = 1.07%). The cost per link click was $0.24 for the logo-only image, and $0.16 and $0.17 for the banners with images of the Black and Latina women, respectively. Thus, including images of women yielded a higher click-through-rate and was more cost effective.

Table 3.

Banner Analytics on Facebook Ads

				Test Statistic
	GURHL Code Logo Only	Banner ad including image of a Black woman	Banner ad including image of a Latina woman	χ2 (df)	p value
Amount Spent	$172.75	$310.10	$221.90	—	—
Impressions	66,500	115,821	93,011	—	—
Clicks	714	1925	1278	—	—
Cost-per-link-click	$0.24	$0.16	$0.17	—	—
Click-through-rate (clicks/impressions)	1.07%	1.66%	1.37%	χ2 =106.64 (2)	p = < 0.0001

Comment

This study compared efforts to recruit and enroll young Black and Latina women into a pilot sexual health RCT using online banner advertisements and targeted electronic outreach (i.e., emails to college professors and LISTSERVs). We additionally evaluated recruitment approaches using cost and analytics metrics. These included the amount of money spent (in terms of purchasing ads as well as staff-hours excluding incentives) per enrolled participant. Targeted electronic recruitment was more cost efficient than recruiting via Facebook. Despite a large number of impressions and clicks via the Facebook banner ads to the study survey, they generated more ineligible participants (n = 12) than eligible ones (n = 5) and only generated two enrolled participants. Unfortunately, the other attempted electronic approaches (OkCupid banner ads and a posting on Eventbrite) fared worse than Facebook and generated no completed screening surveys, and thus, yielded no eligible or enrolled participants. Eventbrite was initially selected because it was successfully used to recruit young women for formative focus groups, and may yet serve as a more useful tool to coordinate a focus group (i.e., sending out reminders, including details of where, when, how to get to venue, etc.) rather than as an effective recruitment approach for quantitative longitudinal research. Cross-sectional studies have been especially successful in recruiting participants online; whereas longitudinal studies have drawn their target sample size from a variety of recruitment approaches. Although online approaches were used to attract a sizeable sample in many cases, they were not the single source for recruits, especially for RCTs.

A review of 30 studies found that 32% of participants were recruited via social media (where the range was 0% [0/12] to 98.29% [1610/1638]),³⁴ and, had this study been better funded to run the ads for a longer period, it might have generated a greater proportion of enrolled participants as a result. A possible explanation of why MSM are more successfully recruited via online mediums versus what this study found could be that there are considerably more research opportunities for MSM given HIV disparities than there are for women of color (or just about any other sub-population). Perhaps the large number of MSM-focused studies has contributed to allowing researchers to make adjustments needed to successfully recruit MSM. The larger number of opportunities may also contribute to MSM expecting, understanding, and trusting research and, ultimately, be willing to participate in research in a way that YWOC are not as they do not have the same research opportunities.

Participants recruited via targeted electronic sources (i.e., email and LISTSERVs) were more likely to complete the baseline survey after starting, found to be eligible, and (among those eligible) to enroll in the study than were those recruited via Facebook. We acknowledge that professors may have shared the email we sent with other students, professors, and staff and thus, may have increased the reach beyond that reported here. It seems that recruiting young women of color might be facilitated through known and trusted adults, such as professors, who are connected to these young women rather than through an anonymous banner ad on social media. We believe this is akin to community based participatory research (CBPR) recruitment strategies that lends researchers additional credibility due to existing strong relationships to community members.^35,36 Our findings suggest that users may not want to engage with unknown entities without something or someone to validate the research for them. We recruited through professors and there is evidence suggesting that professors are a good recruitment source as students trust their professors.^37–39 We acknowledge that this is a select sample of college students who were primarily recruited from CUNY institutions, which is known to have 84.1% of senior and community colleges originate from a NYC public or private high school.⁴⁰ It is unknown whether online recruitment efforts supported off-line recruitment efforts, meaning, participants could have heard about the study on social media and not enrolled, but then may have been primed and ready to register when they heard about it through their professor or school.

Social networking websites and online dating websites have a broad reach with the target population, as evidenced by the 275,000 impressions and 1,986 clicks on banner ads to the study screening survey, however, the low consent rate to agree to participate in a longitudinal survey could be reflective of those who accidentally clicked on the survey link or of individuals who were not interested in the study. Perhaps an online approach should be combined with other methods, such as individual participants who are asked or incentivized to recruit other participants (i.e., seeds) as with the JustUs study⁵ and with a recent San Francisco-based study of transwomen.⁴¹ We initially set out to recruit using two methods that did not work. First, we sought to recruit on Tinder, but found banner ads were not allowed at the time of the study recruitment phase. Second, we attempted to recruit on Facebook by race/ethnicity, however, targeting ads in that way is not permitted since other advertisers were systematically excluding people of color for housing advertisements, a violation of the fair housing act. The actual image could state that we were seeking Black and Latina women, but we were disallowed to target the ad exclusively to Black and Latina women. This resulted in an approved Facebook ad running for 10 days before modifications were required. In retrospect, we agree that targeting banner ads by race/ethnicity is not allowed with good reason as this feature could be used to exclude people of color, so instead we used banner ads that included pictures of people of color.

Banner ads

Banner ads with images of women yielded a higher click-through rate (1.66% for the banner ad including an image of a Black woman and 1.37% for the banner ad including the image of a Latina woman, respectively) in comparison to the logo-only banner ad (CTR = 1.07%) and were statistically significant. The banner ad including the image of a Black woman was the most effective in terms of click-through rates and cost-per-link ($0.24 logo-only; $0.16 for the banner ad including a Black woman, and $0.17 for the banner including a Latina woman). Our findings aligned with other findings for social media and professor recruitment, which accounted for the majority of recruitment for this study.¹⁹ Potential enrollees responded more positively to the banner ads with people and the logo, rather than only the use of the logo. Future researchers and health providers should consider banner ads that include both a study logo and images reflecting the population of interest, and avoid limiting themselves to only recruiting via banner ads on social networks and online dating websites. The conversion rate was good in comparison to other health research click-through rates, which ranged from 0.02% to 0.06%.^6,8,17,42 thus, a future approach might be to expand funding to recruit women via banner ads for a longer period.

Kelly and colleagues developed a model suggesting that advertising on social networks is more likely to be avoided if: the user has expectations of a negative experience, the advertising is not relevant to the user, the user is skeptical toward the advertising message, or the consumer is skeptical toward the advertising medium.⁴³ Our findings align with the notion that college students on online social networks do not dislike advertisements, rather, they go unnoticed.⁴⁴. The logo-only ad might have been perceived as less relevant than ads that included images that might represent potential participants by ethnicity and gender. In addition, banner ads targeting MSM of color have been shown to increase the study survey click-through rate.⁴⁵

Limitations

There were several limitations to this study. Due to a technical error, the study relied on self-report recruitment source data, which is susceptible to recall bias and high missing values. For example, two participants indicated that they heard about the survey on Tinder, but this was not a recruitment site that was utilized. Using our best judgment, we recoded these data appropriately or as missing data. There were no recruits via OkCupid and this may be exacerbated by the fact that OkCupid users were able to pay $7 a month not to receive banner ads at the time of the study. Thus, it is possible that our intended banner ad target audience was reduced or that banner ads with the selected images were an ineffective recruitment tool. A better-funded study might draw a larger enrolled sample from Facebook banner ads for additional advertising. In the end, we recruited the majority of our sample within the CUNY system, which may impact our external validity. However, CUNY is a sub-population reflective of New York City with 83.7% of their total population reporting a high school background from within New York City.⁴⁰

Conclusions

Although we expected to recruit our sample capitalizing off the broad reach that social networks and online dating websites offered, we found that recruiting a sample of young Black and Latina women in New York City aged 18 to 25 was more easily achieved through CUNY professors and campus LISTSERVs. Targeted electronic recruitment (i.e., emails and LISTSERVs) generated a greater proportion of young Black and Latina women aged 18 to 25 who participated in a sexual health web-based app pilot RCT. Further research is needed to understand how social media banner ads might be used as an effective recruitment source for this specific population.