What Makes a Respondent-Driven Sampling “Seed” Productive? Example of Finding At-Risk Massachusetts Men Who Have Sex with Men

Abstract

Several recent studies have sought to elaborate upon the applicability and validity of respondent-driven sampling (RDS) to find hard-to-reach samples in general and men who have sex with men (MSM) in particular. Few published studies have elucidated the characteristics associated with initial RDS participants (“seeds”) who successfully recruited others into a study. A total of 74 original seeds were analyzed from four Massachusetts studies conducted between 2006 and 2008 that used RDS to reach high-risk MSM. Seeds were considered “generative” if they recruited two or more subsequent participants and “non-generative” if they recruited zero or one participant. Overall, 34% of seeds were generative. In separate multivariable logistic regression models controlling for age, race, health insurance, HIV status, and the study for which the seed was enrolled, unprotected anal sex in the past 12 months [adjusted odds ratio (AOR) = 6.68; 95% confidence interval (95% CI) = 1.27–35.12; p = 0.03], cocaine use during sex at least monthly during the past 12 months (AOR = 8.81; 95% CI = 1.68–46.27; p = 0.01), and meeting sex partners at social gatherings (AOR = 7.42; 95% CI = 1.58–34.76; p = 0.01) and public cruising areas (AOR = 4.92; 95% CI = 1.27–19.01; p = 0.02) were each significantly associated with increased odds of being a generative seed. These findings have methodological and practical implications for the recruitment of MSM via RDS. Finding ways to identify RDS seeds that are consistently generative may facilitate collecting a sample that is closer to reflecting the MSM who live in all of the communities in a given location or study sample.

Introduction

Several recent studies have sought to describe the applicability and validity of respondent-driven sampling (RDS), a method of chain-referral sampling that utilizes initial study participants known as “seeds” to recruit their social and/or sexual network peers.^1–3 Through the use of mathematical modeling and under certain strong assumptions to compensate for potential recruitment bias, RDS has been regarded as an innovative and useful means for accessing a potentially representative sample of difficult-to-reach populations at risk for or living with HIV, including injection drug users, sex workers, and men who have sex with men (MSM), both domestically and internationally.^4–15 Prior studies utilizing RDS to sample MSM have compared RDS to snowball, targeted, and time-location sampling.^9,16 In these comparisons, RDS performed similarly on ability to recruit, demonstrating the potential to successfully reach highly marginalized populations.^8,10,11

While RDS has been shown to be efficient in recruiting participants (i.e., in both effectiveness and speed at recruiting),^9,14,16 its promise of yielding representative samples has not always been supported or achieved in research practice.^12,14,17,18 Several difficulties have been identified in implementing RDS, including the possibility that reliance on incentives could motivate individuals to participate in a study even if they do not meet eligibility criteria. For example, individuals could present themselves as men who have sex with men, when in fact they may not actually have ever had sex with another man.¹⁹ In the context of HIV prevention research, while there remains the methodological imperative to develop a means of objectively scrutinizing or verifying social networks (e.g., asking how a subsequent recruit knows a seed), a key challenge is that objectively verifying male-to-male sexual behaviors and/or identity is not feasible.¹⁹ A deeper understanding of the practical issues involved in implementing RDS as a recruitment strategy is needed. In particular, little is known about what makes a seed “generative” (i.e., recruiting two or more subsequent participants into a study).

Several studies using RDS have found peer recruiters often did not recruit anyone, and among those who did recruit, only a minority successfully recruited the maximum number of enrollees allowed by each protocol.^17,20,21 To be effective in sampling a “hidden”, marginalized, or stigmatized population, it has been suggested that RDS seeds must be connected to sufficiently dense peer networks with network members who are able to recruit others.^2,22 Understanding the characteristics associated with generative seeds has methodological and practical implications which may inform the development of criteria used for screening participants who will successfully recruit others into a study.

The goal of this analysis was to examine the demographic, behavioral, and psychosocial factors associated with original RDS seeds that were “generative” compared with those who were not. We hypothesized that factors suggesting larger social network size would be associated with being a generative original RDS seed. Understanding characteristics of generative seeds has important implications for recruiting and enrolling participants using RDS, as well as for public health interventions seeking to recruit marginalized populations.

Methods

Design and Setting

A total of 74 original seeds (i.e., participants initially selected to recruit others) were analyzed from four studies of MSM in Massachusetts conducted between 2006 and 2008 that used respondent-driven sampling. The four studies were focused on: (1) MSM experiences with partner notification services (ten seeds, total N = 189);²³ (2) MSM perceptions of HIV and sexually transmitted infection (STI) risk (10 seeds, total N = 50);²⁴ (3) MSM attitudes, knowledge, and experience with pre-exposure antiretroviral prophylaxis (33 seeds, total N = 227);²⁵ and (4) social and sexual networks, knowledge of HIV and STIs, and perceptions of risk among Black MSM (21 seeds, N = 197).²⁶ Across all studies, participants completed an interviewer-administered quantitative survey. Study activities took place at Fenway Health, a freestanding health care and research facility specializing in HIV/AIDS care and serving the needs of the lesbian, gay, bisexual, and transgender community in the greater Boston area.²⁷ The Fenway Health Institutional Review Board approved all studies.

Sample

Eligibility Criteria

Only the original seeds (i.e., participants initially selected to recruit others) from the four described studies were included in this secondary analysis (N = 74). Across studies, participants were screened by trained study staff on the telephone to determine eligibility. Seeds were eligible if they: (1) self-identified as a man who had sex with men; (2) self-reported oral or anal sex with a man in the 12 months prior to study enrollment; (3) were age 18 years or older; (4) lived in Massachusetts; and (5) were willing and able to recruit up to three to five similar peers (depending on the study for which they were enrolled) from their social or sexual network. At screening, study staff assessed individuals’ ability to recruit their peers by querying men on the total number of study-eligible men they had in their personal social and/or sexual networks. Similarly done across all four studies, potential seeds were evaluated by study staff prior to enrollment for their commitment to the goals of the study and non-randomly selected as seeds based on their motivation to recruit others into the study.

Recruitment

Respondent-driven sampling (RDS)^1,2 was used to recruit diverse samples of Massachusetts MSM. In each study, RDS was modified to terminate recruitment when the a priori desired sample size had been met. Detailed recruitment procedures have been described elsewhere.^5,24–26

Data Collection and Measures

Following an informed consent process with a trained study staff, participants completed a study visit, varying from 1 to 2 hours depending on the study. Across the four studies, participants were remunerated between 25 and 50 dollars for completing the survey and between 10 and 20 dollars for each participant they referred who successfully completed the survey. No statistically significant differences with respect to key demographics (such as race/ethnicity, income, education) were found between seeds based on compensation levels or study type; thus seeds were aggregated for purpose of analysis.

Quantitative Survey

Demographics, Sexual Risk Behavior, and Drug Use Questions Questions examining demographics (including age, race/ethnicity, health insurance), sexual risk behavior, and drug use during sex (at least monthly use of poppers, ecstasy, cocaine, marijuana, or crystal methamphetamine) were adapted from the Centers for Disease Control and Prevention’s National HIV Behavioral Surveillance Survey, MSM cycle.^28,29 Sexual risk behaviors were assessed, such as frequency of unprotected anal sex (insertive and receptive) in the prior 12 months; partner type (i.e., male and female partners); and venues where participants met sexual partners, such as social gatherings, the Internet, and public cruising areas. The survey also asked specifically about self-reported HIV status and lifetime history of one or more of the following sexually transmitted infections (STIs): syphilis, gonorrhea, or Chlamydia.

Psychosocial Risk Factors Depressive symptoms were assessed using the Center for Epidemiologic Studies Depression Scale (CES-D),^30,31 a well-studied screener of clinically significant distress as a marker for possible clinical depression (Cronbach’s alpha = 0.84). The 20 items were scored on a four-point Likert scale from zero to three. A score of 16 or greater was indicative of depressive symptoms. Problematic alcohol use was assessed using the CAGE questionnaire, a clinical screening instrument for possible alcohol dependence (Cronbach’s alpha = 0.69).^32–34 A score of three or more indicated a likely problem with alcohol, representing an 88% likelihood of having an alcohol abuse or dependence disorder.³⁵

Social Network Size Participants were asked the total number of study-eligible men they had in their personal social and/or sexual network. Network size was utilized as a continuous variable. It was also dichotomized in two ways to further examine differences by social network size: (1) greater than or equal to 15 vs. 14 or less; and (2) greater than or equal to 25 vs. 24 or less.

Data Analysis

Seeds were operationalized as “generative” if they recruited two or more subsequent participants; seeds were categorized as “non-generative” if they recruited zero or one subsequent participant. RDS study design suggests that seeds be individuals who are socially connected and highly motivated to recruit others. While we are aware of the recruitment dynamics that can provide many recruits emanating from one long chain, we determined that seeds that recruited two or more individuals would more conservatively represent those seed qualities that have been described as most desirable: socially connected and motivated to recruit others. We sought to determine how these characteristics—as represented by a higher recruitment quota—might contribute to the overall sample, both in subsequent numbers recruited and their characteristics. This categorization of “generative” was also chosen using evidence from prior RDS studies which suggest that only about two thirds of recruitment quotas are generally met.^12,18

Descriptive analyses were conducted using SAS® statistical software.³⁶ To compare generative and non-generative seeds, chi-square global tests of independence were used to examine associations between variables. Where cell sizes were small (i.e., expected cell counts less than five), Fisher’s exact tests were used to estimate proportional differences. Due to non-symmetric distributions of continuous variables (i.e., data on number of sexual partners and social network size), median and interquartile range (IQR) were reported to capture the general tendency of the data. The Wilcoxon rank-sum test, a non-parametric equivalent to the two-sample t test, was used to test the null hypothesis that the two distributions between generative and non-generative seeds were identical against the alternative hypothesis that the two distributions differ with respect to the median. Assumptions satisfied to utilize this test were that within each sample the observations were independent and identically distributed (i.e., that the shapes and spreads of the distributions were the same), and that the two samples were independent of each other. For all statistical tests, statistical significance was determined at the p < 0.05 alpha level.

Bivariate logistic regression procedures were used to examine associations between our outcome of interest and each independent variable. A separate multivariable logistic regression model was constructed for each predictor variable that was statistically significant (p < 0.05) in the bivariate models and adjusted for age, race/ethnicity, health insurance, and HIV status. We also created an indicator of “study type” to account for the study in which the seed was enrolled and adjusted for this in all final models.

Results

Description of Study Seeds

Demographic Characteristics Half of the seeds were white, 43% were African American/Black, 8% Hispanic/Latino, 8% other race/ethnicity, and 1% American Indian/Alaskan Native. The majority had a high school education/GED or less (62%) and were publicly insured (51%). Three fourths (76%) self-identified as gay; the remainder identified as bisexual (22%) and heterosexual (3%).

HIV and STI Status Nearly one in five seeds (19%) self-reported being HIV-infected and half (51%) reported a lifetime history of one or more STI diagnoses.

Sexual Risk Behavior in the Past 12 Months Overall, 78% of seeds reported unprotected anal sex in the past 12 months (68% unprotected insertive anal sex, 59% unprotected receptive anal sex) with a median of 10 (IQR = 21) male sex partners, of which a median of 5 (IQR = 25) were anonymous, and 3 (IQR = 20) were of unknown HIV status. Seeds reported using the following substances during sex in the past 12 months on a monthly basis or more frequently: alcohol (“sex while drunk”; 49%), marijuana (39%), poppers (38%), cocaine (16%), crystal methamphetamine (16%), and ecstasy (15%).

Venues Where Met Male Sexual Partners in the Past 12 Months In the past 12 months, seeds most commonly met male sexual partners at the following venues: Internet (54%), social gathering (53%), bar/club (47%), public cruising area (42%), through friends (41%), private sex party (15%), bathhouse (14%), sex club (8%), and other (18%).

Psychosocial Risk Factors Nearly one third of seeds screened positive for clinically significant depressive symptoms (CES-D 16+; 31%) and one fifth for probable alcohol dependence (CAGE 3+; 20%).

Social Network Size Seeds reported knowing a median of 19 (IQR = 15) study-eligible men they considered part of their social and/or sexual network at study enrollment. More than half (52%) of seeds knew 15 or more men who fit study eligibility criteria, and 20% knew 25 or more men who did.

Comparing Generative versus Non-Generative RDS Seeds

Overall, 34% of seeds were generative and recruited two or more subsequent participants. Demographic, behavioral, and psychosocial characteristics of generative (N = 25) versus non-generative (N = 49) RDS seeds are presented in Table 1.

Table 1.

Characteristics of generative (N = 25) versus non-generative (N = 49) seeds

	Generative seeds		Non-generative seeds
	N = 25		N = 49
Median (IQR) agea	44 (7)		37 (19)
Median (IQR) Number of sexual partners in the past 12 months
Male sex partners	9 (15)		10 (22)
HIV status unknown male sex partners	5 (13)		4 (9)
Anonymous male sex partners	3 (18)		5 (25)
Median (IQR) social network size
Number of MSM participants known who fit study eligibility criteria	18 (18)		20 (15)
	N	%	N	%
Race/ethnicity
White	9	36	28	57
Black	13	52	19	39
Hispanic	1	4	5	10
American Indian/Alaskan Native	1	4	0	0
Other race/ethnicity	2	8	4	8
Education
High school or less	18	72	28	57
Some college or higher	7	28	21	43
Health Insurance
No health insurance	2	8	4	8
Private health insurancea	5	20	25	51
Public health insurancea (Medicaid, Medicare, Veterans Administration)	18	72	20	41
Sexual Identity
Heterosexual	1	4	1	2
Gay	16	64	40	82
Bisexual	8	32	8	16
HIV and STI history
HIV-infected	8	32	6	12
STI diagnosis ever	10	40	28	57
Psychosocial
Alcohol dependence (CAGE 3+)	7	28	8	16
Depressive symptoms (CES-D 16+)	11	44	12	24
Sexual risk behavior in past 12 months
URA	17	68	27	55
UIAa	21	84	29	59
10 or more male sex partners	6	24	20	41
Substance use during sex at least monthly in past 12 months
Alcohol (“while drunk”)	15	60	21	43
Poppers	10	40	18	37
Ecstasy	3	12	8	16
Cocainea	9	36	3	6
Marijuana	13	52	16	33
Crystal methamphetamine	5	20	7	14
Any drug	20	80	31	63
Venues where met sexual partners in past 12 months
Friends	12	48	18	37
Social gatheringa	17	68	22	45
Bar/club	13	52	22	45
Interneta	8	32	32	65
Bathhouse	2	8	8	16
Sex club	2	8	4	8
Private sex party	4	16	7	14
Public cruising areaa	15	60	16	33
Other	4	16	9	18
Social network size
Participant knows 15 or more MSM who fit study eligibility criteria	13	52	26	53
Participant knows 25 or more MSM who fit study eligibility criteria	5	20	13	26

^aComparisons were made between generative and non-generative seeds using Chi-square global tests of independence and Fisher's exact tests where cell sizes were small. Median group differences were calculated using Wilcoxon rank-sum test statistics. Significance was determined at the p < 0.05 level.

Demographic Characteristics Although a higher percentage of generative seeds (compared with non-generative seeds) were African American/Black (52% vs. 39%), had a high school education/GED or less (72% vs. 57%), and self-identified as bisexual (32% vs. 16%), these differences were not statistically significant. Older age (OR = 1.05; 95% CI = 1.01–1.10; p = 0.03) and being publicly insured (compared with not) (72% vs. 41%; OR = 2.85; 95% CI = 1.03–7.89; p = 0.04) were associated with significantly increased odds of being a generative seed.

HIV and STI Status Compared with individuals who reported being HIV-negative, HIV-infected seeds were more likely to be generative (32% vs. 12%; OR = 3.37; 95% CI = 1.02–11.18; p = 0.05). Although a lower proportion of generative seeds reported having a history of one or more STIs compared to non-generative seeds (20% vs. 57%), this was not statistically significant.

Sexual Risk Behavior in the Past 12 Months Unprotected anal sex (84% vs. 59%; OR = 3.62; 95% CI = 1.08–12.16; p = 0.04) and use of cocaine during sex at least monthly in the past 12 months (36% vs. 6%; OR = 11.00; 95% CI = 2.54–49.31; p = 0.002) were associated with an increased likelihood of being a generative seed.

Median Number of Sex Partners in the Past 12 Months No significant differences were observed in the median number of male sex partners, HIV status unknown sex partners, or anonymous male sex partners reported between generative and non-generative seeds.

Venues Where Met Male Sexual Partners in the Past 12 Months Having met sex partners at a social gathering (48% vs. 37%; OR = 2.98; 95% CI = 1.05–8.46; p = 0.04) and public cruising area (60% vs. 33%; OR = 3.09; 95% CI = 1.14–8.40; p = 0.03) were associated with significantly increased odds of being a generative seed. Having met partners on the Internet was associated with significantly decreased odds of being a generative seed (32% vs. 65%; OR = 0.25; 95% CI = 0.09–0.70; p = 0.008).

Psychosocial Risk Factors Although an elevated proportion of generative seeds screened positive for clinically significant depressive symptoms (CES-D 16+; 44% vs. 24%) and probable alcohol dependence (CAGE 3+; 28% vs. 15%), differences between generative and non-generative seeds were not statistically significant.

Social Network Size No statistically significant differences were observed in median social network sizes of generative and non-generative seeds.

Multivariable Logistic Regression Models Examining Characteristics of Generative Seeds

In separate multivariable models adjusting for age, race/ethnicity, health insurance status, HIV status, and the study for which the seed was enrolled, unprotected anal sex in the past 12 months (AOR = 6.68; 95% CI = 1.27–35.12; p = 0.03), cocaine use during sex at least monthly in the past 12 months (AOR = 8.81; 95% CI = 1.68–46.27; p = 0.01), and meeting sexual partners at social gatherings (AOR = 7.42; 95% CI = 1.58–34.76; p = 0.01) and public cruising areas (AOR = 4.92; 95% CI = 1.27–19.01; p = 0.02) were each significantly associated with increased odds of being a generative seed (see Table 2).

Table 2.

Bivariate and adjusted models examining predictors of generative RDS seeds (N = 74)

	Bivariate odds ratio (95% CI)	P value	Adjusted Odds Ratioa (95% CI)	P value
Older age
Yes	1.05 (1.01–1.10)	0.029	–	–
No	1.00	–
Public Health Insurance
Yes	2.85 (1.03–7.89)	0.044	–	–
No	1.00	–
HIV-infected
Yes	3.37 (1.02–11.18)	0.047	–	–
No	1.00	–
Unprotected anal sex in past 12 months
Yes	3.62 (1.08–12.16)	0.037	6.68 (1.27–35.12)	0.025
No	1.00	–
Cocaine use during sex at least monthly in past 12 months
Yes	11.00 (2.54–49.31)	0.002	8.81 (1.68–46.27)	0.010
No	1.00	–
Met sexual partners at public cruising area in past 12 months
Yes	3.09 (1.14–8.40)	0.027	4.92 (1.27–19.01)	0.021
No	1.00	–
Met sexual partners at social gathering in past 12 months
Yes	2.98 (1.05–8.46)	0.040	7.42 (1.58–34.76)	0.011
No	1.00	–
Met sexual partners using Internet in past 12 months
Yes	0.25 (0.09–0.70)	0.008	0.35 (0.10–1.26)	0.108
No	1.00	–

^aAge, race, health insurance, HIV status, and study enrolled were adjusted for in all multivariable models

Discussion

These findings suggest that risky sexual behavior, stimulant drug use during sex, and meeting male partners in social and/or sexual venues, as opposed to the Internet, characterized generative RDS seeds in this sample. Contrary to what we expected, and what is practically done in conducting studies, self-reported larger social network size was not significantly associated with being a generative seed. Findings suggest that network size alone may be insufficient to identify productive seeds. Other factors such as the density of social networks, the strength of social network ties, or the frequency of a study-relevant behavior may be more important to consider.^17,37 Finding ways to identify RDS seeds that are consistently generative may facilitate collecting a sample that is closer to reflecting the MSM who live in all of the communities in a given location or study sample.

These results have methodological implications insofar as they may inform criteria used for screening and for enrolling original seed participants who will successfully recruit others into a study. For example, the finding that Internet users were no more or less likely to be generative versus non-generative seeds suggests that while the Internet may be useful for HIV prevention and education activities^38–41 as well as for public health functions such as partner notification,^42,43 it may not be the best subset of MSM to focus on when initiating RDS procedures. Prior to implementation of a study utilizing RDS, the social and/or sexual network characteristics of the target population should be carefully considered in designing eligibility criteria and screening tools, as well as in the purposeful selection of initial participants to serve as seeds.

A critique could be levied that enrolling a few unproductive seeds to locate a productive one is not especially cost prohibitive, representing only a small fraction of the cost of typical RDS studies. However, the 49 non-germinating seeds across four studies included in this analysis represent a sizable proportion of total seeds enrolled (66%), which may impose considerable burden from the perspective of funding staff and investigator time. Additional research is warranted to examine the diverse characteristics associated with RDS seed generativity with an eye toward the practical implications of such findings for recruitment methodologies, including cost-effectiveness.

Findings suggest that individuals of lower socioeconomic position (SEP) and/or those who regularly use drugs may be more likely to successfully recruit others into a study, perhaps because they are more motivated by the level of monetary incentives offered.¹⁹ Recently, the ethics of RDS as a recruitment methodology has been debated in the HIV prevention and research literature.^{4,19,20,44,45} In particular, it has been suggested that RDS might actually generate an informal market of research participation that enforces pre-existing economic and social inequalities that might put participants at risk for being harmed.⁴⁴ Concerns regarding coercive levels of monetary incentive among seeds of lower SEP,²⁰ as well as the challenge to recruit seeds of higher SEP who may be insufficiently motivated by the incentive level, suggest that different types of incentives may be needed, depending on the population being enrolled.

While original seeds were selected using purposeful strategies, participants across the four studies included in this analysis appear fairly representative of other published MSM research that use RDS.^9,14,46–48 Prior research suggests that study participation may be a type of perceived group membership and may potentially influence operative social norms.⁴⁹ It is possible that older, poorer, and HIV-infected communities may participate in research and represent a group unto themselves, with fewer social and sexual ties to younger, more affluent, or HIV-uninfected MSM. In fact, a characteristic of generative seeds may be past and/or regular participation in research projects, which was not assessed in the current studies. Including a metric of prior successful study participation may be helpful in guiding seed selection, assessing prior involvement in research studies, and predicting future successful seed germination.

Furthermore, seed characteristics may affect RDS estimates themselves—for example, the asymptotic behavior of RDS estimates hinge on the validity of the assumption that participants recruit a single individual chosen uniformly at random from their network of contacts.¹⁸ However, this assumption may be violated in populations with community structure (i.e., for example, among individuals who regularly enroll in RDS studies) and may lead to recruitment of people in the same social subgroup resulting in an effective reduction in sample size and larger variance estimates.¹⁸ Given the relatively modest sample size in the current study, additional research involving larger samples of RDS seeds is warranted to further investigate the role of social network configuration and study participation as a group characteristic and social norm. Generalizability of current findings are further constrained because all four studies were conducted by one agency and there may have been commonalities in site, staffing, target population, and recruitment homophily.

Results should be taken as a starting point for further prospective investigations. In particular, additional studies are warranted with larger samples of MSM and other marginalized populations to further elucidate those characteristics associated with RDS seeds who successfully recruit subsequent participants, to aid in a deeper understanding of the methodological and practical issues involved in RDS implementation. Incorporating qualitative exit interviews with original seeds may provide future insight as to why some seeds are generative and others are not, including whether any systematic bias is evident in recruiting or not recruiting. For example, fewer than 10% of individuals who received coupons from an original seed were found to be ineligible for study participation due to not meeting study criteria in these four studies, suggesting minimal systematic bias. Further assessing predictors of failing to distribute coupons or redeem coupons that were distributed may represent an area for future research efforts. To aid in the recruitment of MSM via this methodology, from these data it appears that risky sexual behavior, stimulant drug use during sex, and having met sexual partners at particular venues characterize generative RDS seeds in these samples of MSM.