Implementation and Analysis of Respondent Driven Sampling: Lessons Learned from the Field

Those who engage in illegal or stigmatized behaviors, which put them at risk of HIV infection, are largely concentrated in urban centers. Owing to their illegal and/or stigmatized behaviors, they are difficult to reach with public health surveillance and prevention programs.¹ These populations include illicit drug users, sex workers and men who have sex with men. Development and implementation of adequate prevention services targeting hidden populations requires data on risk behaviors and disease prevalence from non-biased samples. In the last two decades, a number of sampling methods have been used to collect risk behavior and disease prevalence data from highly at-risk populations and to direct survey participants to prevention services. These include venue-based time–space sampling, targeted sampling, and snowball sampling.

Time–space (also called time–location or venue–day–time) and targeted sampling provide coverage limited to population members who are readily accessible; those who are missed may differ from those who are “captured”.² Targeted sampling fares well when compared to other forms of convenience sampling in recruiting large and varied groups of drug users (DUs).^3–6 This type of sampling may allow access to a large number of non-institutionalized DUs; however, the probability of an individual’s selection is unknown and the process is non-random, thus limiting the generalizability of the survey results. Snowball sampling can achieve broad coverage of a population because respondents, including those who do not attend public venues, are reached through their social networks. However, as the respondents are not randomly selected, must be typically found by survey staff, and are dependent on the subjective choices of the first respondents, snowball samples are biased and do not provide the basis for valid generalizations to the populations from which the sample was drawn.⁷

A recent development in sampling methodology, respondent-driven sampling (RDS), was designed to overcome some of these limitations by providing breadth of coverage with statistical validity.⁸ RDS combines a modified form of chain-referral, or snowball, sampling, with a mathematical system for weighting the sample to compensate for its not having been drawn as a simple random sample. RDS is based on the premise that peers are better able than outreach workers and researchers to locate and recruit other members of a hidden population.^8,9 RDS provides methods for sample selection and evaluation of the reliability of the data obtained. As such, it allows for inferences about the characteristics of the population from which the sample is drawn.¹⁰ Similar to other chain-referral sampling methods, RDS starts with a small number of peers (called ‘seeds’) and expands through successive “waves” of peer recruitment. Seeds recruit first-wave respondents, first-wave respondents recruit second-wave respondents, and this process continues until the desired sample size is reached. Respondents recruit those with whom they have a preexisting relationship. In aggregate, respondents have been found to recruit as though they are sampling randomly from their personal social networks.¹¹ Unlike traditional chain-referral methods, procedures in RDS incorporate direct recruitment of peers by their peers, recruitment quotas, and a dual system of incentives. Respondents are usually remunerated for completing the study and also for successfully recruiting other respondents from within their networks. ¹² With its reliance on social networks, RDS has the potential, like other chain referral methods to reach individuals who do not go to public venues.

Unlike other chain referral methods, RDS also allows for the assessment of relative inclusion probabilities for members of the population based on a mathematical model of the recruitment process. This model is derived from a synthesis and extension of Markov chain theory¹³ and biased network theory¹⁴ and provides the basis for calculating both unbiased estimators and standard errors or confidence intervals.⁹ These calculations are based on information collected from respondents regarding their relationship with both their recruiters and recruits and the size of their own social networks.

The statistical theory upon which RDS is based suggests that if peer recruitment proceeds through a sufficiently large number of waves, the composition of the sample will stabilize, becoming independent of the seeds from which recruitment began, and thereby overcoming any bias the nonrandom choice of seeds may have introduced. This stable sample composition is termed the “equilibrium.” The statistical theories on which RDS is based also suggests that well-connected respondents within the target population affect the sampling process. What matters is their personal network size as defined by the number of friends and acquaintances they have who fall within the target population. Groups with larger than average network sizes are over-sampled, because more recruitment paths lead to them. Consequently, population estimates derived from RDS are weighted to compensate for this over-sampling through a process termed post-stratification.

RDS relies on social networks for successful construction of recruitment chains. The greater the density of the social networks, the better RDS performs. Network density is defined as the ratio of the number of ties in a network to the maximum number possible.¹⁵ Urban areas have traditionally been the centers of HIV transmission because of the same density of networks. Large numbers of people, increasing the likelihood of chance encounters with infected partners, be they sexual or needle sharing, increase risk of all group members. The structures of these largely urban networks make them well suited for RDS. The reports collected in this issue are based on studies conducted in urban environments. This does not mean that RDS cannot work in rural areas. This method has been successfully used among the rural mountain villages of Central Asia to provide sentinel surveillance of injection drug users (IDU) (Sharopov, personal communication).

In 2002, the Centers for Disease Control and Prevention (CDC) organized a consultative meeting to discuss these various sampling methods and also to decide what would be the most appropriate sampling method for conducting behavioral surveillance in the United States. A subsequent meeting organized by the U.S. government’s President’s emergency plan for AIDS Relief and hosted by the CDC focused on the question of conducting behavioral surveillance in developing countries. These meetings led to the development and implementation of a number of studies in the U.S. and in other countries to examine the effectiveness of RDS as a sampling strategy for behavioral surveillance among most-at risk populations. This special issue of the Journal of Urban Health is the result of these various studies conducted around the world. These studies used RDS to recruit populations at risk for HIV and other blood-borne infections. The issue highlights a small number of these studies. There are ten studies conducted in a number of countries around the world, and they provide both theoretical and practical validation of RDS in recruiting hidden populations including IDUs, commercial sex workers and MSM as well as some of the problems encountered.

The first two papers describe the use of RDS to gather samples of female sex workers. Simic et al.¹⁶ examine the heterogeneity of barriers to implementing RDS to recruit sex workers in Eastern Europe while highlighting the need for formative work prior to implementation. They report on the use of RDS in three separate surveys conducted in the Russian Federation, Serbia and Montenegro. The authors conclude that the combination of the hidden nature of sex work and weak social networks undermined the feasibility of RDS and potentially limited recruitment in Montenegro and, to a lesser degree, in Serbia. The paper by Johnston et al.¹⁷ conducted with female sex workers in two Vietnamese cities, found RDS to be very successful in recruiting diverse populations of sex workers in these two cities however, though only marginally different from a simulated time–space sample within the survey sample.

In another comparative study, Robinson et al.¹⁸ report on differences between implementing RDS and targeted sampling in three U.S. cities to recruit injecting drug users (IDUs). While both methods performed satisfactorily, targeted sampling required more staff time per-recruited respondent and had a lower proportion of screened respondents who were eligible than RDS; however RDS respondents were offered higher incentives for participation making the cost slightly greater.

Platt et al.¹⁹ compared two chain-referral sampling methods for recruiting IDUs across nine cities in Russia and Estonia. They compared RDS to chain referrals that used indigenous field workers to find referred group members. Compared to the indigenous field worker method, RDS did not recruit more marginalized IDUs and also had a higher cost due to incentives. RDS, however, had practical advantages in terms of greater recruitment efficiency and safety of field workers.

McKnight et al.²⁰ provide lessons learned from a study conducted to examine the effectiveness of RDS in recruiting drug users in New York City. The paper highlights some of the lessons learned, e.g., the number of seeds needed to initiate recruitment, managing storefront operations, begin with an appointment-only system, and meet with local business owners to inform them about the project.

The next three papers present the results and lessons learned from surveys of men who have sex with men, IDU and female sex workers in three different parts of the world. Yeka et al.²¹ report on the lessons learned from implementing RDS to recruit men who have sex with men and female sex workers in two urban centers in Papua New Guinea. The authors found RDS to be effective in terms of successful recruitment within a short period of time with lower costs and greater safety for staff and respondents.

Stormer et al.²² report on a similar survey conducted among IDUs in Russia and Albania. They found RDS to be effective in reaching IDUs in both countries. The authors found that having multiple field sites in different parts of the city facilitated recruitment. In St. Petersburg, establishing field sites near the illegal drug market helped with recruitment. Obtaining letters of support from city police officials was also helpful and it ensured that IDUs would not be harassed at these study sites. In Albania, involvement of non-government organizations’ staff in recruiting initial seeds and as members of the interviewing teams helped the participants feel comfortable about the interviewing process.

Frost et al.²³ describe the implementation of RDS for IDU surveys in two U.S.-Mexican border cities and point to the sensitivity of RDS population estimates to estimates of network size using methods modified from those previously reported in the literature. RDS was found to be an effective method to rapidly recruit IDUs in these cities.

Salganik²⁴ describes the bootstrap method for constructing confidence intervals around respondent-driven sampling estimates and analyzes their efficiency. Efficiency is measured by design effect, defined as the ratio of the variances of the RDS estimate and that of a simple random sample. He shows that RDS estimates can be more efficient than simple random samples, as reflected in design effects less than one; but in general they are less efficient, as reflected in design effects greater than one. He also analyzed the determinants of efficiency, showing that it increases as bridging ties between groups become more abundant, and hence recruitment chains have less difficulty crossing group boundaries. He concluded by offering recommendations regarding the appropriate sample size when using respondent-driven sampling, suggesting a sample size twice as large as would be needed under simple random sampling.

Tiffany²⁵ integrates RDS with participatory research. Her paper highlights how RDS can be used to design participatory research studies that include means to assess project outcomes quantitatively and how the utility of RDS can thereby be extended beyond surveillance into intervention research.

In combination, these articles document the progress and expansion of RDS-related research, to better understanding and refinement of implementation procedures; specification of the types of populations for which RDS is, or is not, an efficient and effective sampling procedure; progressive refinement of estimation procedures; and integration of the RDS sampling method into routine surveillance and intervention research.