Beyond Surveillance: A Role for Respondent-driven Sampling in Implementation Science

Abstract

We are now in the fourth decade of the human immunodeficiency virus (HIV) pandemic. Several novel prevention tools have been identified, and prevalence and incidence have declined in many settings. A remaining challenge is the delivery of preventive interventions to hard-to-reach populations, including men who have sex with men and injection drug users. Leaders in the field of HIV have called for a new focus on implementation science, which requires a shift in thinking from individual randomized controlled trials to cluster-randomized trials. Multiple challenges need to be addressed in the conduct of cluster-randomized trials, including: 1) generalizability of the study population to the target population, 2) potential contamination through overlap/exchange of members of control and intervention clusters, and 3) evaluation of effectiveness at multiple levels of influence. To address these key challenges, we propose a novel application of respondent-driven sampling—a chain-referral strategy commonly used for surveillance—in the recruitment of participants for the evaluation of a cluster-randomized trial of a community intervention. We illustrate this application with an empirical example of a cluster-randomized trial that is currently under way to assess the effectiveness of men's wellness centers in improving utilization of HIV counseling and testing among men who have sex with men in India.

Respondent-driven sampling (RDS) is a chain-referral strategy for recruiting hard-to-reach populations that has increasingly been used to assess the burden of disease, including human immunodeficiency virus (HIV) infection. RDS has been widely used in observational studies to reach populations for which there is no natural sampling frame, such as men who have sex with men (MSM), injection drug users, and female sex workers (1–3). We propose a novel application of RDS in the recruitment of participants to evaluate the effectiveness of HIV prevention interventions in the context of cluster-randomized trials (CRTs). To illustrate the practical applicability of our concept, we have included an example of an ongoing CRT.

EFFECTIVENESS STUDIES IN HIV RESEARCH

Rationale

Since the first cases of acquired immunodeficiency syndrome (AIDS) were reported in 1981 (4), tremendous advances have been made in HIV prevention and treatment (5). The efficacy of prevention interventions, such as HIV counseling and testing (6), condom promotion, syringe exchange programs, opiate substitution programs (7), and medical male circumcision (8, 9), has been demonstrated in randomized controlled trials (RCTs). Most recently, the results of the HIV Prevention Trials Network 052 study reinforced the beneficial effects of early initiation of antiretroviral therapy on both clinical outcomes (10) and secondary transmission (11).

However, the population-level impact of these interventions, especially in marginalized populations such as injection drug users and MSM in low- and middle-income countries, appears to be minimal (12), resulting in continued transmission in these groups. Indeed, according to the most recent Joint United Nations Programme on HIV/AIDS report, one of the few regions where HIV prevalence has increased (>25%) since 2000 is Eastern Europe—an epidemic driven by drug use (12). There is an urgent need to develop and evaluate effective mechanisms to deliver efficacious interventions in the “real world” to populations that need them the most. However, implementation research requires a shift from traditional epidemiologic methods used to evaluate efficacy at the individual level to methods that evaluate effectiveness at the population level.

Efficacy versus effectiveness

Efficacy of interventions is measured through RCTs—the “gold standard” for determining causality, wherein participants are individually randomized to an intervention or control condition. Traditional RCTs pose challenges with regard to extrapolation of efficacy in a trial setting to effectiveness at the population level. First, traditional RCTs do not provide estimates of intervention uptake in the real world. An efficacious intervention with minimal population uptake might have as much impact on an epidemic as an intervention with modest efficacy but high utilization. Second, participants in RCTs are often subject to stringent eligibility criteria before inclusion. Injection drug users and other mobile populations are commonly excluded from these studies to ensure optimal retention, which is essential for internal validity. However, exclusion of such populations limits external validity of the findings. Third, traditional RCTs require rigorous follow-up of participants under highly controlled conditions; such rates of retention seldom occur outside of trial settings, especially in the developing world. For example, at the Y.R. Gaitonde Centre for AIDS Research and Education in Chennai, India, a site of the HIV Prevention Trials Network 052 study, the trial retention rate was greater than 95% (13). However, in an analysis of patients in care at this same site, approximately 25% of patients dropped out of care within 1 year, with an overall dropout rate of 38.1 per 100 person-years (14).

Effectiveness studies, particularly when they are evaluating a novel service delivery mechanism, generally require a CRT wherein communities or clusters, rather than individuals, are randomized to an intervention or control condition. Two methods can be used for evaluation in CRTs (15): 1) recruitment of a cohort in intervention and control communities and comparison of outcomes between these groups after implementation of the intervention, or 2) serial cross-sectional samples before and after implementation of the intervention.

Challenges to implementation of CRTs

There are multiple challenges to implementation of a CRT. First, because a key goal of a CRT is to assess effectiveness of an intervention within a community, it is crucial that the study population be representative of the target population. This can be particularly challenging in MSM, injection drug users, and female sex workers, for whom no natural or constructed sampling frame exists that can be used to derive a random representative sample of the target population. In many lower- and middle-income countries, these behaviors are illegal and punishable by law, which drives these populations underground and further complicates sampling (16). Although enrollment and follow-up of cohorts at each site might be preferred to serial cross-sectional surveys, cohorts tend to be more expensive because they require significant resources to follow people over time. Furthermore, even with intensive resources dedicated to tracking participants, a primary challenge to fielding a cohort study remains differential losses to follow-up, which can make the population less representative of the target population over time. Among those retained, participants might modify their behavior purely by virtue of being in a cohort (17, 18). With the serial cross-sectional sample option, the primary concern is potential key differences between the baseline and evaluation samples/participants that could alter estimates of effectiveness in unknown ways.

Second, an unbiased estimate of intervention effect in a CRT requires that there be no overlap of members across intervention and control clusters; contamination can compromise the ability to detect a difference (19–22). In prior HIV prevention trials, contamination has been cited as a potential explanation for lack of efficacy. For example, in a large multisite HIV prevention CRT (23), exchange of participants in the Chennai site between intervention and control clusters (despite geographically discrete communities) was believed to have significantly contributed to the lack of difference in primary outcome (Aylur K. Srikrishnan, Y.R. Gaitonde Centre for AIDS Research and Education, Chennai, India, personal communication, 2012). Although contamination is not unique to CRTs, the statistical power lost because of contamination in a CRT is orders of magnitude higher than in individual-level RCTs given the relatively small sample sizes of CRTs. For example, it is uncommon to see CRTs with more than 50 clusters, whereas individual-level RCTs typically have 250–1,000 participants per study arm.

Third, although effectiveness studies offer an ideal setting to evaluate multilevel interventions, it is challenging to evaluate the impact of the multiple facets (e.g., individual, network, community). The primary goal of a CRT is to evaluate community-level effectiveness—that is, whether the prevalence or incidence of an outcome differs in communities that did or did not receive the intervention. However, in an intervention that targets multiple levels, understanding which component had the most impact is crucial to future implementation (24, 25).

We propose a novel approach to conducting CRTs that uses RDS to address the 3 challenges detailed above. We describe the design and the role of RDS below and illustrate its application with an empirical example of a CRT that is currently under way.

Respondent-driven sampling

RDS, a chain-referral strategy of recruitment (1, 2), has gained popularity in recent years and is commonly used to measure disease burden in hard-to-reach populations. RDS is similar to snowball sampling (26), except that in RDS, the number of referrals per participant is controlled by the investigators (approximately 2–5), and peer network information (relationship between recruiter and recruit) is documented. RDS is based on the theory of “6 degrees of separation,” and it is believed that if the approach is implemented appropriately, the study sample should be representative of the underlying target population. In addition to providing a representative sample, RDS provides network data, which can be used to examine linkages between samples from geographically distinct clusters, particularly in populations that are highly mobile.

Utility of RDS in CRTs

Our proposed design involves 4 steps (Figure 1): 1) baseline assessment and identification of discrete study sites through RDS-derived samples, 2) randomization, 3) implementation of the intervention in communities randomized to the intervention, and 4) evaluation assessment (another sample recruited by RDS). We posit that RDS can assist with the 3 previously described key challenges for a CRT.

Figure 1.

Proposed study design. CRT, cluster-randomized trial; RDS, respondent-driven sampling.

Of these 3 challenges, selection of a representative sample is the most established feature of RDS. Furthermore, RDS has been shown to be effective in recruiting hard-to-reach injection drug users (27, 28), MSM (29), and female sex workers (3) in multiple settings; these high-risk groups have been identified as populations in which the HIV epidemic has not yet stabilized or declined in lower- and middle-income countries (12). Although traditional chain-referral methods might also reach these populations, RDS uses systematically collected data about the relationships between recruiters and recruits to adjust for recruitment bias in the analysis. Furthermore, to produce estimates that are generalizable to the target population, the personal network size of each respondent is collected to allow weighted analysis through “poststratification” to compensate for potential oversampling of respondents with larger networks (1, 2, 26). RDS is initiated by selecting 4–6 “seeds,” or persons believed to be the most connected among the target population. Seed participants are given referral coupons (2–5) to hand out at random to members of their (sexual or drug-using) network, thereby initiating the chain-referral process. As recruitment continues from wave to wave, equilibrium eventually will be attained, such that the final sample will be independent of the characteristics of the subjects enrolled when recruitment began. In other words, regardless of who the seed subjects are, the final sample accrued will be similar.

What is not yet established is the reliability of RDS samples over time. A key assumption of RDS is that the target population in a particular city or region belongs to 1 large network. This assumption, though crucial in assessing the reliability of RDS samples, is a difficult assumption to test in practice. If this assumption holds within a city, then regardless of who the seeds are (in terms of geography and other characteristics), samples should be comparable as long as RDS process measures hold (e.g., equilibrium (when sample composition becomes independent of seeds) and network homophily (measure of subgroup recruitment homogeneity between different networks)). However, if the target population within a city or region does not belong to the same underlying network but rather to more than 1 nonoverlapping network, different seeds could lead to markedly different samples. Indeed, 3 studies (30–32) compared serial RDS samples over time, and in all, characteristics of participants were different at the 2 time points. Explanations for the differences have included true changes over time and changing geography. Because the hypothesis of 1 underlying network cannot truly be tested without a large sample, it is important that populations recruited at different time points target the same underlying population. This can be done by incorporating ethnography to understand in depth the network structure of the target population at each site. As an additional measure, seed participants at the evaluation assessment (step 4) should be selected to be as similar as possible to the seeds at baseline, especially in relation to region. With these additional measures, RDS samples at 2 time points should be representative of the underlying population at those time points, facilitating comparisons.

Although the theory of RDS applies to peer-driven interventions (33–35), it has not been used extensively in the context of community-level interventions to date. Thus, the role of RDS in evaluation of contamination and interventions at multiple levels (community, network, and individual) has not been extensively explored. Nevertheless, an understanding of the relationships of individuals within networks and the degrees of these connections has a clear role in the conduct of CRTs. Contamination is a major concern in CRTs because it can dilute the difference in the outcome between intervention and control arms (15). Although randomization at the cluster (vs. individual) level is thought to be less subject to contamination, contamination can still occur when extensive contact occurs between individuals across control and intervention clusters. When combined with questions about mobility in the baseline survey, RDS can provide information on potential for contamination (overlap between clusters). Although investigators might have a priori hypotheses about such linkages, RDS allows identification of network linkages in real time that might not have been expected. Knowledge of these linkages could allow randomization of truly discrete clusters, thus minimizing the possibility of contamination.

Finally, RDS provides a means of evaluating the intervention across multiple levels. It has been increasingly recognized that interventions that target multiple levels of influence achieve greater impact than those that target just 1 level (24). Inherent in most community-level interventions are 2 levels: the community and the individual. A third component that drives impact, but often goes unmeasured, is the network/peer level. Furthermore, although many interventions target multiple levels, it is often challenging to tease apart which level has the greatest impact on the outcome. Population-averaged estimates give an overall picture of community-level effectiveness, and comparisons among individuals in intervention clusters who were or were not exposed to the intervention can provide evidence of individual-level impact. These 2 levels can be evaluated in most CRTs, provided the information is captured in surveys. RDS-accrued samples provide information on a third level: the network. Evaluation of effectiveness at the network level can inform the role that peers play in the adoption of practices (precautionary and risk) among network members. For example, peer leaders with large networks could have more impact on intervention uptake than persons with small networks (33–35). The availability of network-level information in RDS samples will allow for comparisons of intervention uptake across networks of varying size and across recruitment chains and key factors within networks that drive intervention utilization.

CRT to test the effectiveness of men's wellness centers in improving HIV testing outcomes

To illustrate the utility of RDS in the context of CRTs and practical challenges that might be faced, we describe an ongoing study designed to test the following hypothesis: Establishment of gay-friendly men's wellness centers will improve adoption of routine HIV counseling and testing among MSM in India. The hypothesis will be tested through a CRT that will be evaluated through serial cross-sectional samples recruited via RDS at 12 candidate sites in India across 6 states: Andhra Pradesh, Tamil Nadu, New Delhi, Uttar Pradesh, Madhya Pradesh, and Karnataka. Our goal is to select 10 discrete study sites (5 pairs) for the CRT. The primary outcome of interest is improved utilization of HIV counseling and testing in the previous 12 months. Below, we describe the design of this trial in the context of the 3 previously described challenges.

Representativeness

MSM in India remain a hidden population with no existing sampling frame from which a random sample can be selected. Estimates of the number of MSM in India vary. The most recent estimate from the National AIDS Control Organization suggested approximately 2.35 million high-risk MSM living in India (36). By contrast, estimates of same-sex behavior among men in India range from 7.6 million to 45.8 million (37, 38). This discrepancy highlights that although male-to-male sexual contact is not necessarily unacceptable in India, the open practice of a homosexual lifestyle is uncommon (39). The primary reasons for this are: 1) the recently overturned Section 377 of the Indian Penal Code, which criminalized anal sex and has driven many MSM to live secret lives (16) (the repeal of Section 377 is still under debate in the Indian Supreme Court), and 2) the prevailing norm of opposite-sex marriage. Taken together, these barriers result in many MSM marrying to satisfy social pressures or to prove their masculinity to themselves. These legal and societal norms place MSM at risk for stigma, discrimination, and violence and pose a major barrier to recruiting MSM into research studies (40, 41).

For the proposed CRT, our interest is in reaching all MSM, regardless of how they self-identify—gay, heterosexual, or bisexual. There have been few epidemiologic assessments of HIV burden among MSM in India (42, 43). In response, we used RDS to recruit a sample of 721 MSM in South India over 2 months (44). That study demonstrated the feasibility of RDS in sampling MSM in this region. Consequently, for the present CRT, we propose the use of RDS to obtain representative samples of MSM from multiple cities across India. We will begin with 2–6 seeds that represent the diversity of MSM behavior at each of the sites and offer 2–5 referral coupons to each of these seeds and each participant subsequently recruited. We propose a sample size of 1,000 participants at each of the 12 candidate sites to ensure that the RDS will run at least 6 waves deep, at which point convergence of the sample on the basis of demographic and risk behavior has been shown to occur (1, 2). This is also sufficient to allow for an RDS design effect of 2 (i.e., RDS sample has to be 2 times as large as a simple random sample) (45).

Contamination

A key requirement for this trial is the identification of truly distinct (nonoverlapping) MSM clusters for randomization. Common convention would be to select samples of MSM from geographically distinct sites. However, we posit that geography alone does not ensure discrete samples, particularly for mobile populations. For example, it is conceivable that MSM in one city could travel to another in search of partners, services, or employment. We propose to capture information on mobility in the ethnographic phase and to use the network information that is collected in RDS to further evaluate contamination and ensure that clusters are truly discrete.

We propose to conduct baseline assessments in 12 candidate sites. The reason for the oversampling is to be able to discard or combine sites when contamination is identified. Ideally, RDS would be conducted concurrently at all sites; however, because of logistical challenges, we will run 3 consecutive 4-month RDS recruitment periods, each at 4 concurrently open study sites. We divided our 12 candidate sites into 3 groups according to potential for contamination. On the basis of formative work, we deemed that sites in the same state would have the highest probability of contamination because of common language. Therefore, we will run sites within each state concurrently, rather than grouping according to actual geographic distance.

This strategy will allow for multiple assessments of contamination. First, duplication can be monitored through a continually updated central database, which will hold all recruitment data for all sites. We can also monitor in real time whether participants visit a study site other than the site to which they were referred. To identify duplicates, we will use software that scans an individual's fingerprint and transforms it into a 128-bit alphanumeric code that is unique to each fingerprint. The actual image of the fingerprint will not be stored. The central database with this information will be updated in real time and will be available across all sites, allowing identification of duplicates through a simple fingerprint scan.

Once all sites have accrued samples, data can be examined for contamination (overlapping enrollment). If contamination is observed, we have 2 options: 1) The 2 or 3 sites where overlap was observed can be combined into 1 site, or 2) only 1 of the 2 overlapping sites is used and the other is dropped. Our oversampling of sites at baseline allows for 2 instances of contamination to occur without affecting the overall objective of 10 discrete sites. Using RDS in this capacity to select our clusters for randomization ensures that the definition of “distinct” communities is driven by the network information rather than by geographic boundaries alone. In other words, clusters are defined by communities rather than simply by geography.

Evaluation of intervention effectiveness at multiple levels

The final step is evaluation of the intervention, which in this context will be done through serial cross-sectional samples accrued by using RDS at baseline (before intervention implementation) and 2 years after implementation. The baseline RDS will provide important information on key confounders. For example, of particular importance is the baseline prevalence of HIV counseling and testing in each cluster, which will be used to pair-match clusters or conduct stratified randomization.

After the intervention (in this case, “gay-friendly men's wellness centers”) has been implemented in intervention clusters and allowed to run for an appropriate length of time (in this case, 24 months), we will conduct a second (evaluation) RDS in the intervention and control clusters. The purpose of this second RDS is to evaluate the impact of these health centers on HIV counseling and testing. We propose to use procedures similar to those used for the baseline RDS. It is important that, to obtain a sample comparable to the baseline RDS, we use seeds as similar as possible to the seeds used to recruit the baseline sample, for the reasons described earlier. We will at a minimum ensure that the characteristics of the seeds are similar to those of the baseline RDS seeds in terms of HIV status, marital status, sexual orientation, and community.

Community-level outcomes can be compared as the change in outcome of interest from baseline to evaluation among intervention versus control clusters (Table 1). Because samples at both points in time have been accrued via RDS with similar seeds, they should be representative of the underlying target population, facilitating comparison of the outcome of interest across the cross-sectional samples. Multivariate models can be used to adjust for differences across samples.

Table 1.

Evaluation of Intervention Effectiveness Across Multiple Levels of Influence

Intervention Component	Target	Outcome	Measurement of Outcome
Community
• Establish gay-friendly men's wellness centers	Availability of gay-friendly health services Provider attitudes toward MSM Stigma/discrimination	Increased frequency of HIV testing Lower levels of substance use and depressive symptoms Decreased percentage reporting unprotected anal intercourse Lower HIV/STI prevalence	Comparison of change in outcome prevalence in intervention vs. control clusters
Network
• Peer/network-based referrals to men's wellness centers	Peer service utilization HIV testing status of peers Risk behavior of peers	Percentage using men's wellness centers Percentage using specific services within men's wellness centers	Correlation of within-network utilization of centers/services vs. between-network utilization Network membership as a predictor of service utilization
Individual
• Tailor services in men's wellness centers to the individual needs of clients	HIV status STIs Substance use Depressive symptoms Perceived need for services	Increased frequency of HIV testing Lower levels of substance use and depressive symptoms Decreased percentage reporting unprotected anal intercourse Lower HIV/STI prevalence	Comparison of outcome prevalence in individuals who did and did not use men's wellness centers within intervention clusters

Abbreviations: HIV, human immunodeficiency virus; MSM, men who have sex with men; STI, sexually transmitted infection.

Beyond assessment of community-level effectiveness, RDS will allow an assessment of individual- and network-level influences. Estimation of individual-level impact likely can be achieved with other designs (e.g., cohorts or random cross-sectional samples) as well and will be assessed by comparing outcomes among individuals within intervention clusters who did and did not use the intervention by the addition of a question in the evaluation RDS that queries intervention utilization. However, an added advantage of RDS is that we can evaluate the impact of networks (33–35). Because we will have data on linkages among individuals, we can evaluate whether actual utilization within the clinics tracked across baseline network linkages as well as whether self-reported utilization in the evaluation RDS tracked across network linkages in the evaluation RDS.

Challenges

Important limitations must be considered. First, estimates from RDS tend to be less precise than those from simple random samples (46, 47). The increased variance due to RDS (design effect) is a complex function of the network structure in a population. Although common convention is that a design effect of 2 is appropriate, it has been suggested that in some cases, design effects of 5–10 are more appropriate (46). Even with oversampling, costs of serial RDS samples are likely to be lower than those associated with retaining a cohort. However, some outcomes, such as death, cannot be measured with the RDS design but could be measured if a cohort were followed.

Second, although estimates derived from RDS are expected to approximate population prevalence, a sample derived through RDS might not achieve equilibrium or might produce prevalence estimates that are different from those of the population at large (1, 2). In this case, there are methods for weighting prevalence estimates (48, 49); however, in the case of CRTs, this need for weighting adds additional complexity to the analysis, which could already be cumbersome.

Third, to date, few studies have characterized the consistency of RDS samples over time (30–32). Thus, differences observed over time in serial RDS samples might be due to intervention effects but also might reflect inherent variability that arises because of RDS methodology. It is also possible that RDS could reach only a subset of the target population (10, 15). This further reinforces the importance of choosing a well-defined geographic region from which to sample, selecting seeds that share characteristics in the baseline and evaluation RDS samples, and monitoring environmental changes.

Fourth, RDS alone might not ensure discrete samples in the context of highly mobile populations. Although RDS will assist in identifying contamination, the absence of linkages between clusters does not ensure discrete clusters. Additional steps can be taken to ensure selection of discrete study sites: 1) Conduct preliminary ethnography to collect information on mobility among members of the community, and use these data in the selection of potential sites for the CRT. 2) Ensure that the minimum distance between the clusters is such that they are not easily connected by public transport and that persons would require sufficient resources and time (e.g., ≥6 hours) to travel between clusters. 3) Select sites from different states within a country (where language can be different and distance significant) or from different countries. 4) Oversample the number of sites required for the trial to allow randomization even if 1 or more sites need to be dropped.

CONCLUSIONS

Dr. Anthony Fauci, Director of the National Institute of Allergy and Infectious Diseases of the US National Institutes of Health, referred to the lack of implementation studies in HIV research as a “responsibility gap” (50). We propose a novel design for evaluating a prevention intervention among MSM in India that will facilitate implementation. This design could be applicable to the evaluation of other HIV prevention and treatment interventions across a wide range of populations and even could be extendable to CRTs not focused exclusively on HIV.