Going Social: Success in Online Recruitment of Men Who Have Sex with Men for Prevention HIV Vaccine Research

Abstract

Objective

To compare the use of four different social media sites to recruit men who have sex with men (MSM) and transgender women to a phase 2b HIV prevention vaccine trial, HVTN 505.

Design

Retrospective, observational study.

Methods

The University of Pennsylvania HIV Vaccine Trials Unit (Penn HVTU) employed street outreach and online recruitment methods to recruit participants for HVTN 505 using a combination of national recruitment images/messages with Philadelphia-specific language and imagery. We compared the efficiency (number of enrolled participants per number of completed phone screens) and effectiveness (number of enrolled participants per time interval employed) of each strategy, as well as the demographics and risk behaviors of the populations.

Results

Online recruitment strategies populated 37% (71/191) of trial participants at our site. Among the four social media strategies employed, 45.1% (32/71) were enrolled through Facebook, 16.9% (12/71) through Craigslist, 15.5% (11/71) through a web-based marketing company (WBMC), and 22.5% (16/71) via GRINDR. The number of participants enrolled per month of strategy and the months the strategy was employed were Facebook - 32(33 months), Craigslist - 12(33 months), WBMC -11(6 months), and GRINDR – 16(0.56 months). In-person and online recruitment strategies yielded participants of similar demographics and levels of risk behavior.

Conclusion

Use of several social media recruitment modalities produced large numbers of MSM engaging in high risk behavior and willing to participate in an HIV prevention vaccine trial. In comparison to other social media and online strategies, recruitment via GRINDR was the most effective.

Introduction

Recruitment strategies for large scale clinical trials require constant innovation and flexibility as successful strategies may wane over time. Recruitment for HIV prevention trials, compared to treatment studies, is often more labor intensive given the absence of a population of potential participants already under medical care. In-person outreach has been the primary strategy for recruitment of human subjects to HIV prevention trials.[1–5] This approach relies on recruiters being in the right place at the right time to engage with potential study participants, necessitating person hours and resources spent locating and interfacing with potential subjects, especially when recruiting within specific populations at increased risk for HIV acquisition. Outreach targeting MSM or trans persons can require engagement outside of traditional business hours, including late night locations. The internet has added a new dimension to recruitment for HIV prevention trials as it greatly reduces the virtual distance between recruiters and potential subjects. As the internet has grown, the ways people meet and socialize have evolved. Internet chat-rooms gave way to social networking and dating sites, which are now mobile in the form of smart phone applications (apps). In order to effectively recruit human subjects and keep pace with current technologies, clinical trials have looked to these alternative platforms for study recruitment.

Use of the internet and internet-based social media to reach large numbers of MSM, has been widely reported as a feasible recruitment strategy.[1–17] A 2006 meta-analysis reported that 64–99% of MSM find partners online.[18] It has been estimated that 48% of young MSM use the internet to find partners, and 50% of MSM spend 2–3 hours per week meeting partners online.[19–21] Banner advertisements on internet venues frequented by MSM have been successfully employed to recruit large numbers of MSM for screening and to collect behavioral data.[2, 13, 15] GRINDR is a smart phone based geosocial networking application targeted to MSM that is often used for finding sex partners. The app uses GPS technology to display the profiles of other users in the immediate area arranged in ascending order by distance. GRINDR has been used to recruit subjects for behavioral studies in Los Angeles and New York City,[10, 12, 14, 17] and for a trial evaluating the acceptability of a rectal microbicide.[1]

The University of Pennsylvania HIV Vaccine Trials Unit (Penn HVTU) utilized multiple forms of online social media to recruit large numbers of participants into the HIV Vaccine Trials Network (HVTN) 505, a phase 2b study evaluating the safety and efficacy of a multivalent HIV DNA prime/boost vaccine in MSM and transgender women. [22] Recruitment began in Philadelphia in June 2009 and continued through March 2013, when the study was halted. In this paper, we compare the effectiveness, the number of enrolled participants per time interval employed, and efficiency, the number of enrolled participants per number of completed phone screens, of four online methods used to recruit at-risk MSM, as well as the demographic characteristics and risk behaviors of potential research subjects identified through on-line and in-person community outreach.

Methods

Recruitment Materials

The Penn HVTU employed multiple images and slogans in our street outreach and online recruitment strategies. Images and slogans were disseminated via study fliers, in traditional newspaper advertisements, on posters at bus stops and train stations, on the Penn HVTU website (www.phillyvax.org), and on giveaway materials. Similar images were used during in person street outreach recruitment or in online advertising.

Online Recruitment Methods

Facebook

Facebook is a popular social networking site in which users create personal ‘profiles’ and share photos, news, and other information. Advertising via rotating side bar ads targeted to specific users is a widely utilized service offered on the site. The above described images were used to advertise on Facebook beginning in June 2009. Target users were between the ages of 18–50; they identified as male, interested in men, and had an address within a 25-mile radius of Philadelphia City Hall. Initially, individuals who clicked on an ad were redirected to www.hopetakesaction.org, the HVTN Core recruitment website. Individuals could find nearby trial sites and complete a contact form, which was directed to local site staff via email. In February 2011, the Penn HVTU instituted a Philadelphia-specific contact form on the website, www.phillyvax.org. A daily budget of $50 was set to pay for individuals who clicked on our ad (average cost per click of $1.37). From this time forward, individuals clicking on a Facebook ad were sent directly to this site where they could complete a contact form. Submitted forms were automatically emailed to study staff.

Craigslist

Craigslist is a free, city-based, online bulletin board for advertisement of goods and services. Both the HVTN Core and the Penn HVTU advertised on Craigslist during the recruitment phase of HVTN 505. The Penn HVTU began advertising on Philadelphia’s Craigslist under the ‘volunteering’ heading in July 2010. Ads initially relied on text describing the study, though later an image depicting a MSM was used along with the “I have a heart on” slogan. Posts were made every one to two weeks. For the duration of Craigslist advertising, interested parties were directed to the national www.hopetakesaction.org website, and completed contact forms.

Web Based Marketing Company (WBMC)

WBMC is a research company that specializes in contacting LGBT individuals based on expressed interests. The Penn HVTU contracted with WBMC on two occasions to recruit potential trial participants. Together, WBMC and the HVTU developed a recruitment message that was emailed to MSM ages 18–50 who participated in the company’s consumer marketing research panels and lived within a 50-mile radius of Philadelphia. Interested individuals answered two questions: 1) Are you a man or transwoman who has sex with men? and 2) Are you between 18 and 50 years old? Respondents who answered yes to both questions were asked to provide first name, telephone number, and email address that the company then provided to the HVTU.

GRINDR

In October 2012, recruitment images and slogans were re-purposed for GRINDR advertisements. Each ad included the local phillyvax.org website, and the site’s toll free telephone number. Advertising ‘blasts’ ran during predicted high-traffic weekends and holidays, and reached users within a 30-mile radius of the Penn HVTU. Advertisements appeared to GRINDR users for a total of 17 days. Four techniques were implemented to assure that potential subjects saw advertisements multiple times during a session of GRINDR use: 1) Multiple versions of HVTU banner ads appeared in rotation with other advertisements at the bottom of a user’s smartphone screen while the application was open. 2) Entire weekend periods of ‘road-blocking’ in which only HVTU banner ads appeared. 3) Dialogue box pop-up ads with the slogan “Got a Heart on…For an HIV vaccine? Call 866-HIV-PENN to find out how you can help. Compensation is available,” that appeared each time a user opened the application. Users were required to select “more” or “no thanks” before proceeding on to use the app. 4) ‘Interstitial ads’ or full-page advertisements that covered the screen when a user ‘blocked’ another user’s profile. Each blocking event triggered a full-page Penn HVTU ad, which appeared in rotation with other advertisements. As with other methods, clicking on any advertisement directed the user to the phillyvax.org website, and contact information forms.

In-person Outreach

Study staff traveled to areas in and around Philadelphia with known venues for reaching MSM and transgender women. Some recruitment efforts coincided with large LGBT pride events. Staff engaged potential participants, explained the study, answered questions, and solicited contact information.

Screening

Study staff attempted to reach individuals who provided contact information for a preliminary eligibility phone screen. Contact attempts were ceased after three phone calls and two emails if the individual did not respond. Eligible participants were invited to the HVTU clinic office for a face-to-face visit where they completed informed consent, and underwent further screening. All recruitment and screening materials were approved by the Fred Hutchinson Cancer Center Institutional Review Board, the IRB of record for this trial.

Data Analysis

Persons who completed a face-to-face screening were classified into one of five categories indicating the source of their recruitment: Facebook, Craigslist, WBMC, GRINDR or Outreach. Data collected during the initial phone screen interview and in-office screening visit was examined to determine whether various recruitment strategies yielded participants that differed by demographic characteristics or sex-risk behaviors. Demographic information included age (in years at time of interview), gender (male, female, or transgender: male-to -female or female-to-male), and race/ethnicity (White or Black/Hispanic). Information about sex-risk behaviors in the past six months included number of sex partners (of any gender), number of partners with whom condoms were not used, and number of HIV-positive partners. We calculated the efficiency of each recruitment strategy – the number of enrolled participants per number of completed phone screens, and effectiveness of each recruitment strategy – the number of enrolled participants per time interval deployed. Descriptive and comparative analyses included chi-square and t-tests and Wilcoxon Rank Sum tests to identify whether those screened in-office differed by demographic characteristics and sex-risk behaviors across recruitment strategies.

Results

Recruitment/Retention

Advertising HVTN 505 through online and social media sites disseminated notification of the trial widely in Philadelphia. Philadelphia-specific advertisements on Facebook produced 15,405 clicks from 5,397,717 impressions (viewings). Greater than 18,000 clicks were generated from GRINDR ads, and a total of 540 emails were opened by recipients of the WBMC recruitment communication. We were not able to quantitate the number of clicks resulting from Craigslist ads. Figures 1 and 2 demonstrate the flow of participants from first contact to enrollment. Between July 2010 and March 2013, HVTU staff phone screened a total of 1945 potential participants; 47% of these (n = 920) were initiated by contacts through one of the four online recruitment methods. The remaining 53% were recruited through street outreach, referrals from other participants, word of mouth, flyers, and traditional advertisements in print media posted at public transportation stops. Our data include only 1133 participants; the remaining 812 were excluded due to lack of documented source of recruitment and/or demographic data.

Figure 1.

Flow chart depicting movement of participants recruited via Facebook, Craigslist and Outreach through stages of screening.

*Ineligibility based on medical or psychiatric exclusion discovered on screening, participant reconsidered, or other.

Figure 2.

Flow chart depicting movement of participants recruited via GRINDR and WBMC through stages of screening.

*Ineligibility based on medical or psychiatric exclusion discovered on screening, participant reconsidered, or other.

Rates of ineligibility or decision to not participate at phone screen were similar among the groups, 45% (166/365) of Facebook recruits, 58% (124/214) of Craigslist recruits, 62% (67/108) of WBMC recruits, 48% (111/233) of GRINDR recruits, and 54% (116/213) of street outreach recruits. Reasons for ineligibility/non-participation included sexual behaviors of insufficient risk to meet enrollment criteria (48%), HIV seropositivity (7.9%), other medical or psychiatric exclusions (16%), inability to travel to the site (10%), or decision to not participate (17%).

Demographics

The demographics of participants who attended in-office screening are reported in Table 1. Mean age among recruitment methods was between 26 and 33 years old. Mean age differed significantly between outreach and all online methods combined. In pairwise comparison, however, only the mean age of those recruited by Facebook was significantly younger than those recruited by in person outreach (p = 0.01). The oldest participants were associated with recruitment via WBMC and in person outreach (mean = 32).

Table 1.

Demographic and Risk Variables by Recruitment Strategy, MSM Screened in Office, Philadelphia, 2010–2013.

	Outreach	Online	p-value	WBMC	Craigslist	Facebook	GRINDR
	N = 64	N = 232		N = 18	N = 46	N = 109	N = 59
Demographics
Age, mean1	32.2	29.4	0.03	32.0	29.8	28.4	30.3
Age, median1	30.5	27.0	0.02	31.5	26.0	26.0	27.0
Race, N (%)2,3
Black/Hispanic	17 (29%)	75 (34%)	0.46	2 (12%)	22 (50%)	34 (33%)	17 (30%)
White	41 (71%)	143 (66%)		14 (88%)	22 (50%)	68 (67%)	39 (70%)
Risk Behaviors
Number of Sex Partners in Last 6	5.95	4.21	0.09	4.88	4.11	3.30	5.73
Months, Mean Number of Sex Partners in Last 6 Months, Median4	3.0	3.0	0.24	3.0	3.0	2.0	4.0
Multiple Sex Partners in Last 6 Months5	55 (89%)	182 (80%)	0.11	15 (88%)	35 (78%)	77 (72%)	55 (93%)
Number of Sex Partners	2.79	1.46	0.06	1.12	1.13	1.22	2.29
Did Not Use Condoms, Mean Number of Sex Partners	1.0	1.0	0.54	1.0	1.0	1.0	1.0
Did not use Condoms, Median
Multiple Sex Partners Did not use Condoms	21 (33%)	66 (30%)	0.59	6 (35%)	11 (24%)	31 (30%)	18 (33%)

¹Mean and median age of screeners differed significantly between Facebook and Outreach (p = 0.01 for both mean and median).

²Note: Ns do not add up to recruitment strategy totals because “Other” category was not included in descriptive analyses.

³Minority (black and Hispanic) screeners were significantly more likely to be recruited from Craigslist than: WBMC (p = 0.01), GRINDR (p = 0.05) and Outreach (p = 0.03).

⁴Median number of sex partners in last 6 months differed significantly between: WBMC and Facebook (p = 0.05); Facebook and GRINDR (p = 0.0001); and Facebook and Outreach (p = 0.01).

⁵Report of multiple sex partners in last 6 months differed significantly between: Facebook and GRINDR (p = 0.001); Craigslist and GRINDR (p = 0.02); and Facebook and Outreach (p = 0.01).

NOTE: Percentages may not add up to 100% due to missing values.

For the purposes of our study, racial groups were categorized into Black/Hispanic or White. There were differences in the racial/ethnic categories of participants screened based on recruitment strategy. All methods except Craigslist produced predominantly White participants. Craigslist engaged equal proportions of Black/Hispanic and White participants, resulting in the highest proportion of Blacks/Hispanics by recruitment strategy.

Risk

Behavioral risk differed based upon recruitment method. Low risk behavior, defined by fewer than 2 anal sex partners in the preceding 6 months, was a common reason for ineligibility at phone screen (Figures 1, 2). Of all recruitment methods, Craigslist contacts reported the lowest risk behavior; 68% (84/124) screening out for low risk behavior. At the time of phone screen, the proportion of individuals who reported low risk sex behavior was similar for Facebook and GRINDR contacts, 51% (85/166) and 48% (53/111), respectively (Figures 1 and 2). The median number of partners was significantly fewer among Facebook recruits compared to GRINDR recruits (Table 1).

Approximately one third of participants from each recruitment modality reported sex without a condom in the past six months (Table 1). Across all modalities, the median number of unprotected sex partners was reported as 1 (p = 0.54). Although the actual number of partners differed significantly across recruitment modalities, there was no significant difference in reported condom use across groups.

Efficiency and Effectiveness

The enrollment efficiency (number of enrollees per completed phone screens) was similar among methods: 0.088 (32 enrollees per 365 completed phone screens) for Facebook, 0.056 (12/214) for Craigslist, 0.102 (11/108) for WBMC, 0.069 (16/233) for GRINDR, and 0.117 (25/213) for in person outreach. The number of enrollees per completed phone screens was higher for participants recruited through community outreach than for social media methods combined, though the difference was not statistically significant (0.117 vs. 0.077, p < 0.09).

We found GRINDR was the most effective recruitment method (number of enrolled participants per length of strategy utilization). Both Facebook and Craigslist ads were employed for all 33 months at our site. We utilized WBMC as a recruitment tool for 6 total months, and we ran GRINDR ads for seventeen 24-hour periods or 0.56 months. Figure 3 shows the number of phone screens and enrolls per month by recruitment method. GRINDR yielded 28.57 participants per month the strategy was employed. The next most effective strategy was WBMC with 1.8 participants enrolled per month employed. Craigslist was the least effective online strategy – 0.36 participants produced per month employed. In contrast, in person outreach yielded 0.76 participants per month employed. GRINDR was significantly more effective than all other strategies combined (p= 0.02). Figure 4 shows phone screen numbers over time since the introduction of online recruitment. There are multiple peaks in phone screens, some of which coincide with events or initiation of online strategies such as WBMC and GRINDR.

Figure 3.

Comparison of phone screens and enrolls generated per month of use by recruitment method.

Figure 4.

Recruitment trends including all methods of recruitment.

Discussion

Our results demonstrate that employment of a diverse array of online advertising strategies promotes successful recruitment into an HIV vaccine trial. We found the geosocial networking app GRINDR to be particularly effective at reaching a large number of potential participants in a short period of time with a level of risk behavior that exceeded that of participants recruited via other social media sites.

Online recruitment methods have been used for behavioral surveys, but few studies report their use for subsequent face-to-face office visits.[1, 4, 6, 9, 23] Of these, only one was an interventional trial.[1] For this two-year trial at our site, 37% of 191 participants were recruited via online methods. GRINDR recruits accounted for 22.5% of enrolled participants recruited from any online source, despite an on-line GRINDR presence that lasted less than one month. GRINDR was also used successfully to recruit MSM for participation in a trial of a rectal microbicide.[1]

Similar to other reported studies, we experienced a substantial loss of potential subjects during the period from initial contact to first face-to-face visit.[1, 2, 4, 5, 16] Among the 549 potential participants who met eligibility criteria following the initial phone screen, 46% (253) failed to continue in the screening process. In-person outreach produced the highest proportion of contacts who presented for a screening visit in the office 66% (64/97) followed by Facebook 55% (109/199), Craigslist 51% (46/90), and GRINDR 48% (59/122). WBMC recruits had the lowest proportion of screening visits as only 44% (18/41) of contacts appeared in the office.

The demographics of participants recruited online for HVTN 505 were similar to those of other studies using online recruitment of MSM.[1–6, 11, 13–18, 20, 21, 23, 24] Our participants largely fell between ages 25 and 35 although inclusion criteria allowed persons ages 18 to 50 years. Other groups recruited similar proportions of 18–30 year olds in field versus online methods.[2, 23] The majority of our participants were White. However, we were successful at recruiting substantial numbers of Black/Hispanic participants via online methods. The demographics of research participants recruited through on-line advertising have varied; however, engagement of racial minorities can be achieved through this strategy.[2, 15–18] We believe that part of our success may have been due to use of multiracial advertising, which has been seen previously.[25] The highest proportion of minority participants during our study came from Craigslist, with nearly 50% of recruited participants reporting race/ethnicity as Black or Hispanic. The use of the GRINDR chat feature to recruit young MSM subjects in Los Angeles resulted in mostly White and Latino participants.[17] Our GRINDR advertising engaged Black/Hispanic participants in numbers similar to other online sources, suggesting that the demographics of GRINDR users may differ geographically.

For the purposes of this study, risk behavior was defined by number of partners and number of unprotected sex encounters. There were five seroconversions among HVTN 505 participants enrolled at our site; two were recruited online, one via Facebook and the other GRINDR. While screening data suggested that GRINDR users may be at greater risk for HIV acquisition due to higher numbers of partners, true risk cannot be calculated in this analysis due to the short duration of follow up and few seroconversions.

When compared to other online methods, GRINDR was the most effective, yielding 16 to 79 times more participants enrolled per month of advertising compared to other online strategies. Unlike other online methods such as Facebook and Craigslist, GRINDR functions as a dynamic venue, reaching individuals while they are trying to meet potential sexual partners.

Although the majority of persons were recruited from street outreach and street outreach was the most efficient strategy, street outreach requires a substantial amount of resources in terms of time and money. In-person street outreach relies on recruiters being in the right place at the right time to engage with potential study participants. Outreach targeting MSM can require engagement outside of traditional business hours, at late night locations and events, resulting in increased overtime pay and increased security concerns/measures that may result in increased costs to ensure safety of the recruitment staff. On the other hand, the internet and smart phone apps reduce the virtual distance between recruiters and potential subjects and can be used for recruitment at any time of the day or night without incurring extra costs for overtime pay and security.

There are several limitations to this report. This work was geographically restricted to Philadelphia and the immediate surrounding areas, limiting its generalizability to other locations. At the time of phone screen, participants were asked to report where they heard about the study. There may have been reporting bias since all contact information was directed through the national Hopetakesaction.org website and staff relied on subject report. GRINDR and WBMC contact information, however, was labeled as such and could be referenced during phone screening. Also, penetration of our advertising to the community only increased with time, and it is possible our later recruitment strategies were more successful than earlier ones because participants had encountered our advertising previously. It is important to note that we may not have been able to maintain these levels of recruitment success through GRINDR if we employed the strategy for an extended period of time. When new recruitment strategies are initiated, the number of contacts may rise because the strategy reaches different populations willing to respond or there is greater appeal to the new strategy. Subsequently, the novelty of the effect may wane over time as the strategy continues to be employed. Hence, our measure of effectiveness may be biased because we utilized different online strategies for different periods of time. However, this was necessitated by time and cost. Placing ads on GRINDR is expensive so we utilized them as needed. The other online strategies were either free (e.g., Craigslist) or relatively inexpensive (e.g., Facebook) so we were able to continue advertising on these sites for a longer period of time. In addition, we added recruitment through GRINDR later in the trial to build on the success of other online strategies.

To recruit participants to any interventional trial, the right people must be engaged in the right context. Our multi-year experience with several online methods sheds important light on the utility of the internet in clinical trial recruitment. No single recruitment strategy will be appropriate for every single study, and some studies will often require multiple strategies. Our success illustrates the value of a combination of online and in-person recruitment strategies to reach MSM and transgender women for participation in biomedical intervention trials.