Recruitment Strategies of Indian Men Who Have Sex with Men in the State of Maharashtra Into an Online Survey

Abstract

In this manuscript, we report lessons learned from our efforts to recruit Indian men and transgender women/hijras who have sex with men into an online cross sectional study. Between September 2013 and May 2014, we implemented a seven-phased recruitment strategy that included the use of online and offline strategies to enroll a total of 449 participants into an online survey about recent sexual behavior and various psychosocial measures. The phases were implemented sequentially and cost-per-eligible participant was calculated. Using social media and collaborating with organizers of community events proved the most effective strategies for recruiting Indian MSM into online research.

Introduction

The widespread adoption of mobile phones and the increasing affordability of smartphones have allowed researchers to collect data from large samples of persons normally difficult to reach using offline recruitment methods. In the west, increased Internet access by men who have sex with men (MSM) has been associated with increased use of online sex-seeking websites and mobile applications. ISHKonnect was an online study that collected sexual health data from an online sample of Indian MSM and hijras/transwomen (MSM-H). One aim of the study was to determine how to successfully use online methods to recruit MSM-H. In this manuscript, we share lessons learned from efforts to recruit MSM-H into an online cross section study.

Internet penetration in India is currently estimated to be at 19% (Internet Live Stats, 2015) and the use of smart phones at 10% (International Data Corporation (IDC), 2014). Not every individual has similar access or uses the Internet or smart phones with similar frequency. Thus, before embarking on online recruitment in India, it is critical to determine if members of the target population can be recruited through online methods.

Online recruitment efforts are limited by cost and potential bias. Their convenience samples draw from a population that can range from broad to a defined, limited subset of members in a particular online community, thus potentially excluding those who lack Internet access or an account on the website being used (Sinclair et al., 2012). In India in particular, as Internet access is limited, online samples may tend to be more affluent than offline samples (Well et al., 2011). Furthermore, online recruitment may suffer from low response rates (Im et al., 2006; Jenkins, 2012; Miller et al., 2010; Tuten, 2010; Vehovar and Manfreda, 2008) and ineligible participants misrepresenting themselves to gain access to a study (Jenkins, 2012), requiring researchers to employ strong cross-validation and de-depuplication protocols.

In response, researchers have recommended several techniques when recruiting online. In tailoring messages specific to the format and the audience, researchers may more readily establish rapport with the participants and gain their trust (Hershberger et al., 2011; Riggle et al., 2005; Temple and Brown, 2011). A combination of recruitment efforts may also be used (Temple and Brown, 2011), including combining with offline methods (Cook et al., 2009; Gordon et al., 2006; Hershberger et al., 2011; McClure et al., 2006; Parsons, Vial, Starks, & Golub, 2013), recruiting via message boards or blogs (Riggle et al., 2005), or targeting website advertisements to sites frequented by the intended participant population (Graham et al., Bull, Vallejos, Levine, & Ortiz, 2008; Buller et al., 2012; 2006). Participants may also be solicited using closed email networks of willing contacts (Temple and Brown, 2011) or by offering incentives to key informants who may in turn help drive online sampling (Evans et al., 2011). Additionally, protocols have been developed to identify and remove participants who misrepresent themselves (Bull et al., 2008; Grey et al., in press; Konstan, Rosser, Ross, Stanton, & Edwards, 2005; Pequegnat et al., 2007; Wilkerson, Shenk, Grey, Rosser, & Noor, 2013)

In this report we detail the lessons learned from our efforts to recruit Indian MSM-H into an online survey about sexual behavior. When recruiting participants, we were cognizant of the aforementioned concerns and recommendations. Because MSM-H are a relatively hard-to-reach population, the number of MSM-H living in Maharashtra remains unknown, and Internet access and the use of smartphones is more common among MSM-H in the middle and upper-income brackets, the strategies we describe resulted in a convenience sample. Below, we describe the cost and success of the strategies we employed in recruiting participants. By sharing our experience, we hope to further Internet-based research in India and Southeast Asia.

Method

Study Design

Between September 2013 and May 2014, we recruited 449 MSM-H through banner advertisements on gay websites, social media advertisements and posts, and the distribution of print materials at outreach events hosted by LGBT and HIV services organizations in the State of Maharashtra, India. The banner advertisements, social media advertisements and posts, and print materials directed participants to the study website where they could complete the eligibility screen. Eligible participants self-identified as MSM or hijra/transgender woman living in the State of Maharashtra, being at least 18 years of age, having regular Internet access, and having at least one male sex partner in the previous 90 days. Once eligibility was determined, participants were provided with an overview of the study. After completing an online consent procedure (Rosser et al., 2009), the survey took approximately 30 minutes to complete. Participants could complete the eligibility screener, consent process, and survey in Hindi, Marathi, or English. Participants were compensated ₹300 (approximately $7 US) for completion of the survey, which included questions about online and offline sexual behavior and demographic characteristics. The institutional review boards of the authors’ institutions approved study procedures.

Recruitment

The recruitment of Indian MSM-H into this study occurred in seven phases. During all phases, we maintained a social media presence on Facebook^©, https://www.facebook.com/ISHKonnect. However, we did not rely heavily on Facebook^© for recruitment until phase four. During phase one and two, we contracted with an online sex-seeking website that is popular in Mumbai to run banner advertisements on their Internet and mobile website; advertisements were in Hindi and English (Figure 1). During phase one, the company provided free advertisements (impressions), and during phase two, we purchased advertisements. We asked the company to run our advertisements during evening hours, because we thought this would be when the majority of potential participants would frequent the targeted websites and a review of the websites by members of the research team counted more online profiles in the evening hours. When a person clicked on our advertisement, they were directed to the eligibility screener, consent, and if eligible, the survey. In phase two, we were charged for each click on our advertisement regardless of whether the person who clicked on it chose to provide us with their e-mail address. The decision to stop recruiting on the sex-seeking website was determined by a decrease in the number of persons clicking on the advertisements, suggesting we had identified the majority of persons motived to click on our advertisements.

Figure 1.

Examples of advertisements used to recruit participants into the study.

Phase three involved passive recruitment. On 11 December 2013, the Supreme Court of India reinstated Section 377 of the Indian Penal Code, which criminalized consensual anal sex. In anticipation of this ruling, active recruitment was paused for a period prior to and after this ruling so that the LGBT and HIV service organizations could focus community education, organization, and advocacy efforts on forming a community response to the reinstatement of the law. During this time, the ISHKonnect Facebook^© page was used by the organizations to communicate with the community. This resulted in a number of “likes” to our page. It was necessary to have a large number of persons like our page before non-paid advertisements would be effective because the success of non-paid advertising is dependent on people seeing posts in their newsfeed and, hopefully, sharing the post with others.

Phases four capitalized on the “likes” we received on Facebook^©. We continued using our Facebook^© page as a way to post news stories of interest to the LGBT community; we mixed the news stories with posts about the study with a link to the eligibility screener. We also began creating and paying for paid Facebook^© advertisements. The decision to stop paying for Facebook© advertisements was determined by a decrease in the number of persons clicking on the advertisements, suggesting we had identified the majority of persons who used Facebook^© and were motived to click on our advertisements.

Phase five occurred concurrently during part of phase four. While we were recruiting via Facebook©, the Mumbai LGBT pride festival occurred. During the festival, outreach workers affiliated with the organizations handed out palm cards that included study information and the URL for the eligibility screener.

Phases six and seven continued using Facebook^© posts but not the paid advertisements. Because about five months had passed since the end of phase two, phase six involved purchasing the same advertisements on the sex-seeking website used in phase two. Due to budgetary constraints, we were only able to purchase 1,000,000 impressions, so when all impressions had occurred, phase six ended. During phase seven, we collaborated with the organizers of the KASHISH Mumbai International Queer Film Festival, http://mumbaiqueerfest.com/. Through social media, organizers of the film festival advertised that persons who completed the online survey were offered a free pass to the festival.

We conducted our cross-validation and de-duplication protocol, adapted from previous studies (Konstan et al., 2005; Grey et al., 2015). Our protocol for verifying study samples involved comparisons of data within and between participants (Konstan et al., 2005; Bowen et al., 2008; Bauermeister et al., 2012). At analysis, we validated eligibility by comparing responses to the eligibility-screening questionnaire with responses to the survey for age, US residence, and location of IP address. We also searched for duplicates among the IP addresses, e-mail addresses, and key demographic characteristics of persons who completed the survey. We removed ineligible or duplicate observations from the final dataset.

Cost analysis

During each phase, the costs associated with the recruitment strategies employed were documented. Also documented were the number of participants who clicked on the survey link, completed consent, initiated the survey, completed the survey, and were deem eligible after our cross-validation and de-duplication protocol. To determine the cost per eligible participant, we divided the number of completed and eligible participants into the total recruitment costs for the phase. Costs were calculated in Indian rupees, US dollars, and EU euros.

Results

A total of 449 MSM and transgender women/hijras completed the survey; 96% identified as cisgender males (not transgender). The mean age of participants was 29 (SD= 8.27), with a majority living inside the Mumbai/Thane region (74.7%). Most participants had completed college (87%) and were employed (67.5%). Over three-quarters of participants self-identified as being gay or bisexual (64% & 25%, respectively), but only 19% reported being open about their same-sex attraction (out) to most or everyone in their life. Few participants self-reported living with HIV (2.3%); nearly a quarter (22.1%) reported not know their HIV status.

The cost and number of eligible participants recruited in each phase are summarized in Table 1. Costs are presented in Indian rupees, US dollars, and European euros. Also presented in the table are number of advertisement impressions purchased from the sex-seeking website. In the table is a summation of number of times someone clicked on an advertisement posted on the sex-seeking website (which automatically linked them to the survey) or Facebook^©, the number of times someone clicked on the survey link, and the number of persons who clicked on the link that completed consent and initiated and completed the survey. Because we followed protocols to remove individuals from the study who might misrepresent themselves (Bull et al., 2008; Grey et al., in press; Konstan et al., 2005; Pequegnat et al., 2007; Wilkerson et al., 2013), included in Table 1 is the number of persons from each phase who completed the study and were deemed eligible. Of the paid recruitment methods, phase seven, the offering of a free pass to the film festival, resulted in the lowest recruitment cost-per-participant. However, this recruitment method also resulted in the highest proportion of participants deemed ineligible and removed from the final dataset.

Table 1.

Results of the deduplication protocol and advertisement cost-per-participant for the ISHKonnect study, Sept. 2013 – May 2014 (N=449)

	Website banner advertisements		Website banner advertisements		Passive recruitment		Palm cards at Pride social media posts		Social media ads & posts and email blasts		Social media posts & website banner advertisements		Social media posts and film festival entrance waiver		Row Total
	9 Sep. – 27 Oct.		28 Oct. – 20 Nov.		21 Nov. – 31 Jan.		1 Feb. – 28. Feb.		1 Mar. – 1 Apr.		2 Apr. – 1 May		2 May – 30 May
Cost in Indian rupees	₹		₹	84,842.40	₹		– ₹		– ₹	186,475.60	₹	84,842.40	₹	18,300.00	₹	374,460.40
Cost in US dollarsa	$	–	$	1,380.00	$	–	$	–	$	3,033.11	$	1,380.00	$	297.66	$	6,090.77
Cost in EU eurosa	€	–	€	1,000.00	€	–	€	–	€	1,880.53	€	1,000.00	€	–	€	3,880.53
Card distributiona		–		–		–		500		–		–		–		500
Planet Romeo advertisement impressions		26,000.00		1,000,000.00		–		–				1,000,000.00		–		2,026,000
Facebook advertisement clicks		–		–		–		–		17,599		–		–		17,599
Survey link clicks		609		90		342		797		2,562		874		775		6,049
Completed consent (pi3)		145		14		34		99		132		159		162		745
Initiated survey (a1_1)		100		14		29		89		111		129		145		617
Completed survey (j200_j200)		93		8		23		65		79		90		119		477
Completed and eligble		88		8		22		62		75		87		107		449
Percentage of participants dropped as a result of the deduplication protocolb		5%		0%		4%		5%		5%		3%		10%		6%
Cost per eligble participant in Indian rupees	₹	–	₹	10,605.30	₹	–	₹	–	₹	2,486.34	₹	975.20	₹	171.03	₹	14,237.87
Cost per eligble participant in US dollars	$	–	$	172.50	$	–	$	–	$	40.44	$	15.86	$	2.78	$	231.59
Cost per eligble participant in EU euros	€	–	€	125.00	€	–	€	–	€	25.07	€	11.49	€	–	€	161.57

^aUS dollar to EU euro based on $1.38 = €1.00. US dollar to Indian rupee based on $1.00 = ₹ 61.48

^bThe percentage of participants dropped as a result of deduplication protocol = 1 − (number of observations that passed the eligibility screener/number of observations that completed but were dropped during verification).

Discussion

We relied on a variety of recruitment strategies. Purchasing advertisements on the sex-seeking website can be an expensive strategy because the study is charged for the number of impressions, not the number of persons who click on the advertisement. However, this strategy has the advantage of likely recruiting sexually active MSM. Relying on passive recruitment or the distribution of palm cards did not prove an effective method. Palm cards create a barrier to participation because they require potential participants to keep track of the card and have it accessible the next time they have computer access, enter the study URL, and complete the survey.

Social media proved an effective strategy to recruit and stay connected to current and potential participants. By creating a study-specific Facebook^© page, an outreach work could, in addition to advertisements, post LGBT content that attracts traffic to the study page and generates interested in the survey. Like the advertisements on the sex-seeking website, the Facebook© strategy targeted people who accessed the Internet through their personal computers and smart phones, the target population for the ISHKonnect study.

Another effective strategy involved collaborating with the film festival organizers to offer a free pass to persons who complete the survey, suggesting that collaborating with organizers of popular community events is a good recruitment strategy. This strategy was cost effective and resulted in the most number of completed survey from eligible respondents. Because the strategy also had the highest proportion of ineligible respondents—perhaps because it is marketed to all members of the LGBT and ally community—it highlights the need to have a protocol in place to identify and remove ineligible respondents.

A limitation of the way we documented the number of participants recruited into this study is that there were no wash-out periods between recruitment methods. It is possible that someone who heard about the study during phase 3 might have saved the URL and clicked on the survey link during phase 6, for example. In addition, our calculation of cost-per-participant included only the direct cost of recruiting a participant. Other fixed (e.g., office space and utilities) or variable (e.g., staffing) costs were not included in the calculation because these costs will vary by the location and experience of the research team. For our study, we had two staff persons who assisted with recruitment as part of their job duties. One staff member with expertise in graphic design and marketing on social media created our advertisements and generated social media posts. The other staff member assisted with administrative details, including establishing advertising contracts with companies (when necessary) and processing payments to the companies. Based on our experience, we recommend budgeting for the equivalent of one full-time employee throughout recruitment to assist with material development, posting to social media, and assisting with administrative details. When developing a recruitment budget, researchers should use the data in Table 1 and add to it the fixed and variable costs for their institution.

Because internet access is likely limited to middle- and upper-class persons, our sample is not generalizable to all MSM and hijras. We were hoping to recruit more hijras into the study. However, few hijras clicked on the URL and met eligibility requirements. The lack of hijra participation could be due to the images used to advertise the survey, which included a male torso. The lack of participation could also be due to socioeconomic differences between MSM and hijras. Until internet access becomes more available to lower-income Indians, which mostly includes hijras, online recruitment methods might be better suited to middle- and upper-income MSM samples.

We were surprised by the small number of participants living with HIV, based on self-report. The low prevalence could be due to participants not knowing their HIV status (22.1% of our sample) or because participants were uncomfortable reporting this information online. This was one of the first online surveys that asked Indian MSM about their sexual risk behavior and HIV status, so it is possible that some participants were unsure of the degree to which their health information would be secured if reported in an online survey. Future recruitment research with Indian MSM and hijras should explore the extent to which this is a concern and if so, how to educate potential study participants about efforts to increase data security.

While the generalizability of our findings to other study designs or states in India is unknown, our experience can inform the work of future researchers wishing to collect data online from Indian men who have sex with men and hijras. Online researchers are encouraged to add to the empirical literature, so that in time best practices for online recruitment in India can be identified. Our experience suggests that while researchers should rely on a variety of recruitment strategies, using social media and collaborating with organizers of community events are the most effective strategies for recruiting college-educated Indian MSM who live in the urban areas of the State of Maharashtra into online research.