The CAN-DO-IT model: A process for developing and refining online recruitment in HIV/AIDS and sexual health research

Abstract

Purpose of review:

HIV/AIDS and sexual health research has increasingly relied on online recruitment in recent years. However, as potential online recruitment avenues (e.g., dating and sexual networking applications, websites, social media) have proliferated, navigating this process has become increasingly complex. This paper presents a practical model to guide researchers through online recruitment irrespective of platform.

Recent findings:

The CAN-DO-IT model reflects 7 iterative steps based on work by the authors and other investigators: conceptualize scope of recruitment campaign, acquire necessary expertise, navigate online platforms, develop advertisements, optimize recruitment-to-enrollment workflow, implement advertising campaign, and track performance of campaigns and respond accordingly.

Summary:

Online recruitment can accelerate HIV/AIDS research, yet relatively limited guidance exists to facilitate this process across platforms. The CAN-DO-IT model presents one approach to demystify online recruitment and reduce enrollment barriers.

Introduction

A brief history of online recruitment in HIV/AIDS research

In 2001, Mustanski published one of the first scientific papers articulating methodological considerations in conducting sexuality research online [1]. At the time, HIV/AIDS and sexual health researchers were skeptical about fundamental reliability and validity of data collected online. Off-the-shelf software did not exist to collect online survey data, appropriate methods for obtaining informed consent and providing participant compensation were new, and recruitment primarily occurred by building personal relationships with organizations and companies that would share recruitment materials online or via email lists. Despite these legitimate methodological, ethical, and practical hurdles, the potential for revolutionizing HIV/AIDS research was apparent: Gay and bisexual men who were disproportionately impacted by the HIV/AIDS epidemic were early adopters of using the internet for connecting for social and sexual purposes [2]. Reaching these communities online could dramatically improve the efficiency of HIV/AIDS behavioral research with sexual minority men relative to the groundbreaking but extremely challenging offline studies of the time, such as the Urban Men’s Health Study [3]. Participating in an online study was novel and of intrinsic interest to many gay and bisexual men, who had likely never had an opportunity to contribute to HIV/AIDS or LGBTQ health research.

Two decades later, online recruitment into HIV/AIDS research and interventions continues to have both great potential and also great need for methodological innovation. Recent reviews have articulated how online and sexual networking applications have overshadowed traditional in-person venues as the setting for meeting sex partners, connecting with the LGBTQ community, and learning facts and skills related to HIV prevention [2]. Thus, online recruitment and eHealth interventions can meet sexual minority men “where they are.”

As digital technologies have become more sophisticated and complex, so has the process of recruiting online. The internet is constantly changing, evidenced by the arms race among social media and sexual networking applications to deliver new features that drive market demand and produce revenue from advertising and monetizing user data. As such, platform-specific recruitment strategies that work today may not work two years from now because of evolving capabilities and user expectations of platforms. New questions about how to effectively reach populations disproportionately impacted by HIV/AIDS will similarly continue to arise. Given these challenges, the aim of this paper is to provide guidance to researchers around best practices in online recruitment for HIV/AIDS research. However, rather than focus on strategies that may be esoteric to contemporary media, we articulate a practical process model – CAN-DO-IT – for how to approach, implement, and refine strategies for different contexts.

Our experiences with online recruitment

Our model is based on nearly 20 years of lessons learned from conducting online recruitment in HIV and sexual health research, feedback from youth advisory councils and participants, and empirical data gathered on participants’ perspectives on advertising. Since the 2001 paper, our team has continuously used online approaches to recruit largely sexual and gender minority (SGM) adolescents and young adults [4–7], with increasing research on young couples [8] and parents of SGM youth [9]. We have advertised for our studies across a variety of platforms (e.g., search engines, social media, dating and sexual networking applications, online forums) as well as maintained a social media presence for longer-term studies. In addition, we continue to experiment with advertising on newer platforms used by our participant base. Despite diversity in our targets and methods, the general process by which we make decisions about the “where, what, why, when, and how” of online recruitment is consistent across studies.

The scope of this paper

We have synthesized our collective experiences with online recruitment and distilled them into a process that is irrespective of scope, budget, timeline, population, and platform. Although our steps and examples stem from HIV prevention and sexual health research with SGM youth, they likely generalize to other populations affected by HIV and other types of health research. Our hope is that by sharing our methods, we can help accelerate research for communities experiencing the greatest health disparities.

Several researchers have described and compared the effectiveness of various online recruitment approaches for HIV and sexual health research, and health research among SGM populations [e.g., 10, 11–14], so we do not intend to rehash those findings here. Other scholars have recently provided guidance about online recruitment across different types of studies and populations [15–20]. For instance, Kubicek et al. [16] discussed recruitment approaches for diverse populations in health research, and Howcutt et al. [19] proposed a marketing framework for recruitment that considers how social and psychological factors (e.g., motivation, perception, attitudes) impact decisions to participate in health research. Arigo et al. [20] identified and addressed common ethical (e.g., privacy risks, ethics review boards’ familiarity with emerging technologies) and methodological issues (e.g., reaching target audience) in using social media for health research recruitment, with a focus on Facebook and Twitter. Our work builds on these contributions by providing a process model with actionable tasks that can be applied across different online recruitment contexts.

The CAN-DO-IT model

Here, we describe the seven steps of the CAN-DO-IT model for online recruitment, with accompanying tasks and clarifying examples from our work. In Table 1, we apply CAN-DO-IT to LOOKING, a one-year study of experiences with sexual networking applications, sexual health, and HIV risk among SGM adolescents assigned male at birth [21, 22], to demonstrate the model’s utility and guide others using the model for their own online recruitment strategies. Additional case studies (in Supplementary Materials) illustrate different aspects of the CAN-DO-IT model for recruitment for a longitudinal in-person study and an online HIV prevention trial. Although the model is presented in a stepwise fashion, it is iterative in practice.

Table 1.

An application of the CAN-DO-IT model for online advertising to a 1-year, cross-sectional, online survey study

Step and tasks	Example: LOOKING study
Conceptualize scope  - Assess recruitment needs and goals  - Assess capacity (money, time, staffing)	• Recruitment needs/goals: To sample N = 300 racially/ethnically diverse, sexually active sexual and gender minority (SGM) adolescents assigned male at birth aged 15-18 and investigate their experiences using either online dating apps/websites or hookup/sexual networking applications for men who have sex with men (MSM). Goal was to recruit roughly equal sized groups of MSM app users, users of other apps not specific to MSM (e.g., Tinder), and those who used neither type to facilitate between group comparisons. • Capacity: TIME: 1 year study; goal to recruit all participants in 3 months or less. MONEY: $2500 budgeted for recruitment/advertising (includes paid social media ads, stock image fees) based on prior work with this population using similar methods. STAFFING: Team had one half-time research assistant (RA) in charge of developing ad campaigns; PI had 5% of dedicated effort toward project; data manger donated some time toward ensuring data quality and integrity for project.
Acquire expertise  - Identify technical expertise (e.g., social media marketing, analytics, public relations)  - Identify design expertise (e.g., video, graphics, photography)  - Identify expertise about the population/community  - Identify expertise about the population’s engagement with technology	• PI had 4+ years researching SGM adolescents and their use of technology for sexual health information, including some knowledge about their engagement with hookup/dating apps and recent experience asking Youth Advisory Council about perspectives on social media advertising for a previous study. RA had 1 year previous experience on research studies of SGM youth and themselves identified as SGM youth. • Both RA and PI had experience developing and monitoring Facebook/Instagram ads for similar population in another recent online cross-sectional survey study. • Neither PI nor RA had graphic/video design expertise or budget to outsource development of ads. Instead, RA consulted with staff from other studies who gave suggestions on low-budget ways to make engaging graphics (e.g., Canva, Adobe Spark)
Navigate platform and strategy  - Match population and capacity to platform(s)  - Read terms of service, advertising guidelines, and technical parameters of ads  - Identify platform culture and norms  - Identify key gatekeepers to engage as needed	• At the time of the study, Instagram and Snapchat were two platforms most widely adopted by adolescents. Facebook widely adopted but less popular with adolescents. Opted to rely on more affordable paid Facebook and Instagram ads, which share a self-service portal (Instagram is owned by Facebook). Snapchat’s minimum ad buy exceeded budget and required working with third party vendor, which did not match study timeline. • Reviewed Facebook ad guidelines to ensure ad copy and images (e.g., content, resolution) were compliant • Decided to rely on brief, animated videos and bright, eye-catching images given platforms’ reliance on visual content; ad copy written in a casual, fun tone aligned with platform, audience, and study topic • Did not work with gatekeepers in this study
Develop ad content  - Identify intrinsic and extrinsic motivations for the population  - Create ads that are NICE (noticeable, intriguing, credible, and engaging)  - Pilot ads with population	• Motivations: helping other teens like them (intrinsic); increasing representation of SGM adolescents in research (intrinsic); money (extrinsic) • NICE ads mentioned that study was online one-time survey (intriguing - brevity/convenience) that was paid (intriguing - motivation) through Northwestern University (credibility). Imagery included ads with static images of young attractive couples or GIFs/memes/brief videos relevant to study topic (noticeable). Ad featured link to online screener and call to action (e.g., join now; engaging). • Did not pilot ads with population given recent experience doing similar research with this population and recent experience consulting with Youth Advisory Council.
Optimize recruitment-to-enrollment workflow  - Delineate participant and staff behaviors  - Streamline user experience while designing purposeful barriers to participation  - Streamline staff procedures	• PARTICIPANT WORKFLOW: Youth sees online ad, clicks on online ad which redirects to screener, completes screener consisting of approximately 10-15 questions across multiple pages, notified of eligibility immediately (and if ineligible, not notified), eligible participants routed into online survey study. Disclosed length of survey in landing page (45-60 minutes) which may deter some participants. • STAFF WORKFLOW: RA developed ad copy and develop/source visual content, enter ads & refine ad targeting into self-service ad platform, monitor ad ROI and relevance scores using self-service ad dashboard, provides PI with daily or every other day reports on enrollment and ad performance. Data manager provided RA with information on suspicious/fake participants, numbers of individuals who answered screener and screener responses, participant eligibility & reasons for ineligibility. PI oversaw and provided high-level guidance on all of the above.
Implement campaign  - Target ads  - Pilot strategies	• Targeted ads based on age, gender, interests (15-18 years, male, interests aligned with LGBTQ youth) • Ad campaign began with ad spend of approximately $20/day and monitored performance for at least 3 days while ad set was in “learning” phase.
Track and respond  - Monitor return on investment  - Monitor where ads are being shared  - Address negative feedback  - Adjust platforms, content, workflow, or implementation as needed	• Throughout study, adjusted ad spend up or down depending on return on investment, weekly recruitment targets. • Monitored responses to item in the screening survey asking where youth heard about the study (e.g., Facebook, Instagram) to identify best-performing sources, and whether the screener link was being shared in other ways (e.g., among friends, posted on other platforms or sites). • RA enabled Facebook notifications on own devices in order to quickly address, hide, or delete homophobic/transphobic and inappropriate comments on ads. • RA adjusted ad content and targeting in attempt to increase number of participants naive to GSN app use. • RA identified low-performing ads using Facebook self-service dashboard and replaced with new ads weekly.

Conceptualize scope.

The internet is vast, with countless places and ways to find individuals. Many researchers make the mistake of pre-selecting a platform(s) and/or method(s) of recruitment without considering whether those strategies are appropriate or feasible. Thus, the first step is for your research team to establish the scope of online recruitment by (a) assessing your study’s recruitment needs and goals and (b) weighing them against your existing capacity. Specifically, whom you are trying to reach (i.e., inclusion/exclusion criteria, total sample size, recruitment quotas) and for what reason (e.g., to conduct observational/formative research, to test an HIV prevention intervention)? What are your recruitment timelines, advertising budget, and staff availability to manage development and maintenance of recruitment strategies? This scope will guide your decisions in subsequent steps.

Many of our studies recruit for minoritized populations with inclusion criteria that reflect our focus on HIV/STI risk. In our experience, narrower or more complex inclusion criteria are often, but not always, associated with larger advertising cost and longer recruitment periods, as are more intensive studies (e.g., longitudinal, intervention). Depending on the availability of resources, lead investigators on a small research team may need to be more selective about certain recruitment strategies (e.g., paid advertising, labor-intensive activities) and actively involved in the development and monitoring of those strategies. However, as the diversity of platforms and/or concurrent projects increases, the daily management of online recruitment will likely necessitate dedicated staffing.

Acquire expertise.

Having the right combination of expertise on your team is critical for online advertising success. Lattie et al. [15] describe how managing digital advertising campaigns requires skills such as social media marketing, analytics, design, and public relations. In our early forays into online advertising for HIV and sexual health research, our team often taught themselves these skills. As our research portfolio has grown, though, we have recognized the benefit of and prioritized hiring staff who have experience using a variety of online platforms – both as users and as advertisers – and creative skills or training outside of research that may be useful in crafting recruitment materials (e.g., photography, videography, graphic design, acting).

Possessing technical skills is necessary but insufficient, however. For online advertisements to resonate with the target population, your team must have knowledge about the culture of the community they hope to engage as well as about their motivations, priorities, and interests [19, 16]. Specific to online recruitment are the sociotechnical factors, or how your target population views and engages with certain technologies. For example, although many people have Facebook accounts, older adults are more likely to prefer and use Facebook than younger adults and adolescents, who are moving toward newer media [23]. In addition, different racial and ethnic groups may prefer different online spaces; for example, Black men who have sex with men may prefer Jack’d and Adam4Adam [24, 25] more than White or Latino men who may be more likely to use Grindr [25]. Community interests and technology use will likely shift over time, so periodic consultation with them will directly inform your choice of recruitment platform(s) and advertising content.

We have obtained expertise about our communities/populations of interest in ways aligned with community-based participatory research principles [26]. These include hiring members of the communities we study on our research teams [27], using online community advisory boards, some that are study-specific and others that provide feedback across studies [28]; partnering with experts and providers embedded in the community; and incorporating recruitment-related questions into ongoing projects to guide future work. For example, in a study with adolescents, we asked: “In your opinion, what websites, forums, social media, or apps would be the best places to advertise research studies for LGBT teens?” Responses to this question helped guide ongoing advertisement as well as inform future projects.

Navigate platform and strategy.

This step involves selecting and understanding the platform(s) you will use for recruitment, taking into consideration the study’s scope and drawing heavily on your sociotechnical expertise. We use “platforms” to refer to a variety of online recruitment sources ranging from static websites and participant registries to more participatory media (e.g., online forums and interest groups, social media, sexual networking applications).

This step becomes more complex as you incorporate more platforms into your recruitment strategy that are less familiar to the team. When selecting platforms, consider where the community that you are trying to recruit spends their time online. For example, for an HIV prevention study involving sexual minority adult men, sexual networking applications may be the most efficient source of recruitment, as their user base aligns closely with the desired study population [29]. In contrast, recruitment on multiple social media platforms with a younger user base [23] may be a better approach for adolescents. For example, in addition to paid advertisements on social media for studies with youth, we have explored recruitment through avenues newer to us, but widely used among our target populations, such as advertising on streaming media services and collaborating with social media influences. Others have described different considerations and strategies for online recruitment for HIV research with Black, cisgender women [30], transgender men and women [31–34], and transgender youth [14], and each has highlighted how knowing where their populations “are” online guided their strategies.

This step also involves familiarizing yourself with each platform’s terms of service, its advertising costs and guidelines, and being aware of changes in these over time. Different platforms vary in their stances towards research recruitment, and reviewing these guidelines can give investigators insight into whether and how they may be able to use that platform. Arigo et al. [20] and Gelinas et al. [35] provide recommendations on navigating ethical issues in using social media as a research recruitment tool (e.g., respect for privacy, investigator transparency) as well as ethical issues distinct from in-person recruitment (e.g., compliance with terms of service), which likely generalize to other types of websites and platforms outside of social media. Iribarren et al. [10] and Grov et al. [36] describe common issues in online advertising for HIV research, such as bans and moderation.

Provided the platform you choose permits (or does not explicitly discourage) research recruitment, consider the different ways you can use that platform to advertise. Following Kubicek and Robles [16], we diversify our online recruitment strategies to ensure recruitment targets are met and use a combination of free (not counting staff time) and paid advertising. Examples of free strategies might include moderator-approved posts in reddit forums or Facebook groups; posts on study-specific social media profiles designed to establish a following and credibility for long-term studies; posts on the study team’s personal social media accounts; and direct outreach to users of a platform. We rely predominantly on paid advertising using self-service ad managers and occasionally third-party advertisers, the latter of which is considerably more expensive. For paid ads, you should be familiar with options for ad targeting (e.g., demographics, interests) and technical parameters (e.g., what ads are permitted to say, word count, resolution and size of visual assets), which can guide decisions about ad formatting.

Next, you should understand the platform’s structure, culture, and norms, which will inform the content and tone of your advertisements. Regarding structure, consider if the media on the platform are primarily visual (e.g., videos, GIFs, photos, memes), text-based, a combination, or something else (e.g., audio). How do platform users engage with each other, and how is information spread (e.g., hashtags, mentions, shares)? Regarding culture and norms, is it professional (e.g., Linkedln), sexual (e.g., Grindr), or informal/playful (e.g., Snapchat)? Do users tend to be anonymous/pseudonymous, or do they use their real names, which may inform whether recruitment language might focus on confidentiality?

Although your team might directly advertise to prospective participants, also consider whether there are gatekeepers or influencers with whom you can engage to lend credibility to the recruitment materials and gain a wider audience [16]. In some cases, seeking approval from a moderator might be required, and collaborating with a popular opinion leader may dramatically increase visibility (e.g., if study materials are shared by individuals with a large following on Twitter or YouTube). One related approach is online respondent-driven sampling, which can leverage social networks of online-recruited participants and thus improve enrollment of minoritized populations [37]. For recruitment avenues with strong group norms (e.g., reddit, closed Facebook groups) researchers must be sensitive to them lest community members derail their advertising strategy. For example, in a study testing a couples-based HIV prevention intervention [38], an individual made negative comments about the project across several advertisements on a social media platform. This received a not insignificant amount of attention from other users, necessitating a quick response from our recruitment team. Similarly, closed Facebook groups may be protective of their members, and researchers should consider how their recruitment outreach to the moderator/group aligns with the interests, motivations, and values of its users. Planning ahead and training staff on how to respond to various community reactions to study marketing materials is essential to assure an appropriate and timely response.

Finally, depending on the scope and goals of the study, researchers may consider alternative online approaches. For example, research participant sites and internet panels may be useful for survey research [39]. Several HIV research studies have described use of mTurk, a crowdsourcing platform in which individuals are recruited to complete discrete tasks for a small fee [40, 41]. Research Match, a free service, includes a subregistry of HIV-positive individuals [42] but has not been widely adopted for HIV research to date. Large, probability-based online panels have also been utilized for online sexual health survey research with sexual minority men[43]. As a caveat, these approaches may have more limited representation of minority communities most at risk for or living with HIV relative to approaches that target online spaces frequented by those groups and may also yield samples with different sociodemographic and sexual behaviors [40, 41].

Develop ad content.

Recruitment materials include any text (e.g., recruitment email/brochure, ad copy, webpage) and media (e.g., photo, video, audio) used to advertise your study to prospective participants. This step may require several rounds of revision based on research team, community, and ethics board feedback as well as pushback from the platform’s monitors and terms of service. If using multiple online advertising channels, we advise not reposting identical ads across platforms given the differing norms and expectations across online spaces. We propose that all content should be NICE: noticeable, intriguing, credible, and engaging. Figure 1 provides examples of advertisements that were more and less successful and identifies features that follow the NICE heuristic.

Figure 1.

Examples of high and low performing advertisements.

a. High performing advertisement. This ad had both a high number of clicks as well as conversions into completed eligible screeners. The following features of this advertisement followed our “NICE” heuristic: a slideshow of photos of racially and ethnically diverse male couples, and language in describing men who have sex with men (i.e. “same-gender loving men”) that resonates with particular communities (i.e., Black MSM) along with the use of “gay, bi, and queer men” for broader appeal (Noticeable). In addition to what is noticeable, the ad mentions the incentive of free at-home test kits, coupons, and samples and implies the convenience of the program with the word “mobile-friendly” (Intriguing). The ad uses high-quality professional stock photos and is linked to the program’s Facebook page, which has more program information, and the URL for the website clearly indicates ties to an educational institution (Credible). Finally, the ad encourages viewers to “leam more” (Engaging).

b. Low performing advertisement.

This advertisement did not perform as well and cost three times as much per click as more successful advertisements, even though it appears to follow the “NICE” heuristic, and the advertisement was targeted to young sexual minority men who indicated they were “Interested in Men” on Facebook. The image features what appears to be a bright, painted rainbow flag and a heart, alluding to the study’s topic or population (Noticeable). The ad emphasizes payment (Intriguing), highlights the affiliation with a university (Credible), and encourages viewers to “learn more” and “share what matters to you” (Engaging). However, the advertisement likely suffered due to a lack of critical details (e.g., topic of study, target population, study location), as individuals who see the ad are given only a vague sense for what it is advertising, which may reduce how intriguing and credible it is, and thus render it less engaging.

Noticeable:

Online advertisements only work if you are able to sustain your target audience’s attention, so knowing what is appealing to your target population is key. For example, Nelson et al. [12] asked sexual minority adolescent boys about their preferences for advertising and recruitment. They rated bright colors and images, short phrases and bullet points, familiar language, and information about compensation as particularly important. In addition, reflecting your population of interest in your materials is critical. Several studies emphasize the need to use visual assets reflecting people of color if recruiting a racially and ethnically diverse sample [10, 44, 30]. In our experience recruiting minority populations (e.g., SGM, Black men who have sex with men) certain features of advertisements (e.g., rainbow colors, in-group words like “queer” or “same gender loving”) can be compelling to different groups.

Advertisements with images of couples can have a higher conversion rate (i.e., the proportion of ad viewers who respond to your ad) than those featuring a single person [45], which is consistent with our experiences even for studies not focused on couples. Sexualized images can be attention-grabbing, but their approval and acceptability depends on the platform’s culture and terms of service. Any text or audio should draw in the reader with snappy, compelling titles or headings, be written in plain language, and matched to your population and platform [20].

Be mindful of several potential issues with visual assets in advertising for HIV research. High-quality images featuring people from racial/ethnic minority and SGM communities, youth and adolescents, and people at the intersections of these communities can be difficult to obtain. Stock images can be expensive, and images that include people can appear dated quickly. As such, we have shot our own images for several studies, which itself has some costs (e.g., models, photography equipment). We have also mixed images with existing online content (e.g., GIFs, memes), self-generated graphics or videos (e.g., Canva, Adobe Spark, Vyond), or creatively repurposed content from other research studies in our ads to stretch our recruitment dollars.

Intriguing:

Recruitment materials should also make prospective participants want to learn more about your study. This is most often conveyed via text. Consistent with principles of online marketing [46], identifying intrinsic and extrinsic motivations of your target population can increase recruitment success; see Howcutt et al. [19] for a review. With communities underrepresented in HIV research, the potential to give back to one’s community and having an opportunity to share one’s voice are common intrinsic motivations [47]. Examples of extrinsic motivations include financial incentives or receipt of free sexual health information or services. What is noticeable about the ad may also be what makes it intriguing. Moreover, what is intriguing is not universal, so we advise using multiple recruitment messages highlighting different motivations.

Credible:

In an age of online scams and scandals, conveying that your research project and team are legitimate can build trust [48, 47] and may increase prospective participants’ likelihood of being receptive to your advertising [46]. Moreover, HIV research often includes populations who may have historically been mistreated in or excluded from scientific studies, making this step particularly important. Relatedly, participants may be concerned about the confidentiality and privacy implications of responding to ads that are explicitly related to HIV or communities affected by HIV [47]. If/when space permits, anticipate such concerns and address them in your recruitment and enrollment materials (e.g., study recruitment website, screening survey). Based on participant and community feedback, we have learned that ads and landing pages should appear organized and clean with high-quality visual assets, which conveys professionalism. Having recruitment websites hosted on domains such as .org or .edu, institutional logos superimposed on visual assets, or the institution named in the ad copy can lend credibility [49].

For social media advertising, consider including anonymized testimonials from former participants and endorsements from trusted sources (e.g., influences, popular opinion leaders). In our studies with limited to no face-to-face interactions with research staff (e.g., online focus group studies), some of our participants have indicated a desire to know more about the study team and the motivations for conducting the study [49]. By introducing themselves, the team can establish credibility, build trust, and increase motivation to engage with the study.

Engaging:

Noticeable, intriguing, and credible recruitment content is useless unless you are able to translate that interest into action (i.e., a conversion). For instance, use action words that explicitly state what a prospective participant needs to do in the text of your advertising materials (e.g., click or sign up here, call or text this number). This “ask” should align with what the target population is most comfortable doing, lest it be a deterrent to participation. In our studies, most adolescents and young adults strongly prefer texting about the study over speaking to someone on the phone or e-mailing; other populations may prefer other methods of contact.

Optimize recruitment-to-enrollment workflow.

Recruitment is not over once a potential participant clicks on an ad – it also includes each step until the participant begins the study, at which point the team’s focus can move from recruitment to study retention. This includes any interactions that occur between potential participants and research staff during the recruitment process (e.g., e-mails, text messages, voice calls). A fine-grained understanding of this recruitment-to-enrollment workflow, and where drop-off occurs in this pipeline, can help you understand where prospective participants are encountering potential barriers to enrollment.

Although researchers should anticipate these barriers before the study launches, barriers may not be evident until people have moved through the recruitment-to-enrollment process. For instance, are people viewing the ad and visiting your online screener but making it only partway through the screener or consent form? If so, identifying what page or what item appears to be a barrier and making changes to this process as needed can improve recruitment and enrollment. Figure 2 illustrates a recruitment and enrollment workflow diagram for one of our studies – an online multimedia HIV prevention program for sexual minority adolescent boys [50]. Figure 2a shows the workflow we planned at the beginning of the study, which included a mix of automated and staff-initiated tasks to ensure comprehension of the research study as well as to deter fake participants. However, after noticing how slowly it was taking prospective participants to enroll, we examined the proportion of prospective participants who made it through each task and identified bottlenecks where were losing more individuals than expected. Streamlining these tasks (Figure 2b) led to a noticeable increase in retention of prospective participants through the enrollment process as well as decreased burden on study staff.

Figure 2.

Recruitment-to-enrollment workflow for an online multimedia HIV prevention program for sexual minority adolescent boys as initially designed and implemented (2a) and then adapted/optimized over time (2b) based on identified drop-offs, quantitatively measured, in the process. Changes to the workflow led to increased expediency and numbers of individuals moving through the cascade.

To the latter point, optimizing workflow can also involve delineating the research team’s processes and roles during recruitment [15]. For example, who is responsible for developing, posting, and managing advertisements? For contacting participants interested in or eligible for the study? For monitoring prospective participants’ movement through each step of the recruitment process and verifying they are unique/non-fraudulent individuals? How often and at what point are the staff or faculty in project leadership roles notified of ad performance, spending, and enrollment numbers? Outlining these roles may be particularly useful in larger teams with multiple concurrent studies.

Finally, fraudulent entries and “mischievous responders” are a concern in online research [51, 52] and can threaten the validity of research findings [53, 54]. Although online recruitment should offer a streamlined user experience that does not pose undue barriers to genuine participants’ enrollment, the enrollment process should not be so easy as to facilitate bots and so-called “fraudsters’” entrance into the study. Ballard et al. [52] and Teitcher et al. [51] provide recommendations for detecting, preventing, responding to, and identifying fraudsters during internet-based recruitment, many of which we have also employed (Table 2).

Table 2.

Strategies to deter and/or detect fraudsters in internet-based recruitment.

Level	Example strategies
Study protocol design	• Asking participants to only answer once • Telling participants they will only receive incentives once • Decreasing/changing incentive structures to be less lucrative for fraudsters (e.g., lottery) • Adding multiple steps/breaks before enrollment/payment • Asking for personal data (e.g., name, phone number, address) • Contacting potential participants using video conference software
Survey design	• Using CAPTCHA or similar human authentication software • Preventing indexing in search engines • Disabling “back” buttons • Adding attention check and consistency check questions
Respondent computer information	• Collecting and blocking duplicate/ineligible IP addresses • Enabling and blocking duplicate internet cookies • Tracking referral sites (i.e., where the participant came from)
Participant non-survey data	• Checking personal data against external sources (e.g., whitepages, social media) • Identifying inconsistent/improbable paradata (e.g., time stamps) • Identifying suspicious/duplicate personal data (e.g., name, phone number) and/or computer data (e.g., IP address, geolocation)

Adapted from Teitcher et al. (2015) and Ballard et al. (2019)

Implement campaign.

The next step is to implement your recruitment campaign. If using paid ads, consider how much you are able and willing to spend, and for what period of time. For platforms that allow self-service advertising (e.g., Facebook/Instagram, Twitter, Grindr), we typically launch a 4–7-day pilot trial of our ads and closely monitor their performance while the ads are “learning” their audience. For studies with recruitment quotas for certain demographic or behavioral groups, some of which can take more time and money to recruit due to the specificity of the target group, we have approached campaigns in different ways. One approach is to launch an ad campaign targeted at a more general audience, then see how many of that group we are able to recruit with those general ads before launching ad campaigns targeted toward that specific group. Another is to launch parallel ad campaigns or recruitment strategies which target different groups (i.e., market segmentation), which we often favor when on an accelerated timeline. For instance, in one HIV prevention trial for young sexual minority men in 22 counties across the United States [55], we launched a separate ad campaign for each county at the beginning of the trial. Depending on the platform, demographic-, location-, and interest-based keyword targeting may be offered, which has been described in more detail elsewhere [56, 20]. On certain platforms, using features such as hashtags and user mentions can also broaden a recruitment message’s reach by exposing individuals following a particular user or topic/hashtag to the ad [20]. Nevertheless, in studies of various sizes and budgets that use self-service advertising, we typically launch advertisements with a modest daily or lifetime budget (e.g., $20/day on Facebook/Instagram, $350 lifetime on Grindr), then experiment with whether spending more or less substantially changed our enrollment numbers. These approaches may not be possible when using a third-party advertising service or internet panel.

Track and respond.

This final step involves monitoring and tracking recruitment campaigns, then iteratively adjusting ad content, spending, and recruitment sources as needed. We recommend monitoring recruitment performance multiple times per week or even daily, keeping in mind that potential participants may not engage with an advertisement the first time they see it, but that over time, repeated exposure to a particular ad or ad campaign may reduce its relevance to your audience.

For platforms with self-service advertising, metrics such as clicks, cost per click, and reach if available can guide decisions about adjusting your advertising. Arigo et al. [20] provide a glossary of these common terms in social media advertising, and Jones et al. [30] share a detailed case study on using these metrics to evaluate a Facebook advertising strategy for an HIV prevention trial with young women of color. Other methods include using an item in the eligibility screener to assess where participants learned of the study or using a screener URL unique to each recruitment source, either of which can provide valuable information on how each platform performs. Similarly, integrating pixels – code embedded into a website that tracks key user actions – into a study landing page to monitor where each visitor came from can also be useful. However, this approach has not been well-described in the literature, and potential ethical issues must be investigated. Nevertheless, using these methods, as well as setting up a Google alert with your screener URLs, can allow you to monitor whether and how your advertisements and screener are being shared online. Although close monitoring of your recruitment strategies and their return on investment, in terms of number of participants screened and enrolled, can take time, this process can help you understand which strategies are most effective and for which populations.

During a study, it is common for enrollment from a particular campaign to decelerate. This may necessitate refreshing ad content, reducing ad spend, and/or discontinuing ads on a particular platform and shifting your strategy to different platforms. Over time, once-productive platforms can yield fewer screened and eligible participants; in our experience, this has primarily related to dwindling popularity of the platform among our target population or changes in advertising policies or targeting on that platform.

Finally, when advertising for HIV research on platforms that allow comments on or reactions to your recruitment materials, be prepared for questions and negative or even abusive comments, and have a plan for whether and how your team will address them. It is good practice for staff who oversee recruitment to also monitor comments and reactions to advertisements. Comments may be a good sign that you are reaching your population or an indication that you are not reaching a tailored enough audience. Our study team members often respond directly to comments that reflect a misunderstanding of the study, misrepresent the study, or are legitimate questions about the study. Negative and abusive comments are often hidden or deleted as soon as possible to avoid exposure to other potential participants, and abusive comments are reported to moderators or the platform. We have also tracked the comments garnered by our advertisements in efforts to monitor how audiences respond to our content over time as well as to train staff new to recruitment about what they should expect when launching online advertising campaigns and how to best respond to different types of comments.

Conclusions

HIV/AIDS and sexual health research is reliant on online recruitment as it meets populations where they already are. However, there is a relative lack of guidance on how best to approach online recruitment, which can seem daunting as the possible avenues for internet based recruitment are ever-changing. Our practical CAN-DO-IT process model aims to address this gap, complementing existing guidance on recruitment for health research more broadly [20, 57, 56, 58, 35, 19, 16, 15] and adding to the literature concrete steps that researchers can take to develop, launch, and maintain their recruitment campaigns.

A strength of CAN-DO-IT is its applicability to a variety of online media and platforms and projects that vary in scope. However, one limitation of this process model is that it does not provide specific guidance on recruitment of populations, particularly those relatively underrepresented in HIV/AIDS research (e.g., Black cisgender women, minor adolescents, couples, families). As such, we suggest using CAN-DO-IT together with existing literature on recruitment of these populations, and we and others [20] encourage investigators to publish their methods for online recruitment, their successes, and challenges. Moreover, although online recruitment can be efficient, it does not necessarily guarantee recruitment success, sample representativeness, or generalizability [20, 44, 59, 60], so researchers should take steps to mitigate bias in ways that fit their scopes.

Although not explicitly addressed by the CAN-DO-IT model, we acknowledge that online recruitment poses unique ethical and privacy considerations that may be particularly salient in HIV/AIDS research. Several key articles and resources have provided guidance on this topic. Curtis [58] described ethical challenges in online HIV research and identifies best practices for confidentiality, privacy, and informed consent for adults and minor adolescents. In this issue, Fisher et al. [61] review ethical issues during online recruitment, data maintenance, and informed consent in eHealth HIV research and offer concrete strategies to minimize informational risk in these areas. More broadly, Bender et al. [57] proposed seven principles for online recruitment focused on protecting prospective participants’ privacy, with special attention to sensitive health conditions. Gelinas et al. [35] discussed ethical issues related to identifying, contacting, and communicating with prospective and enrolled participants via social media; they provided checklists for investigators who are proposing online recruitment and for IRBs reviewing online recruitment protocols. Finally, the Connected and Open Research Ethics (CORE) platform [62] is a freely available, web-based resource that enables investigators to ask questions about and share resources (e.g., consent form language, protocols) that may be relevant to online recruitment in health research.

In conclusion, online recruitment has arguably accelerated the pace of HIV/AIDS research in the past decade and will likely remain a key method of identifying prospective participants for the foreseeable future. CAN-DO-IT is a model that can demystify this process in an increasingly complex technological landscape. We encourage researchers to share how they addressed the CAN-DO-IT steps in their online recruitment to allow for local experiences to expand into generalizable knowledge for the field.