Site selection in community-based clinical trials for substance use disorders: Strategies for effective site selection

Abstract

Background

The importance of conducting substance use disorder treatment research in real-world settings is now well recognized. While this approach to clinical trials research offers a variety of benefits, challenges also arise. Selecting high quality sites to participate is critical to recruitment, retention, and overall trial performance when conducting multi-site, community-based clinical trials of treatments for substance use disorders.

Objectives

Over the past 10 years, the NIDA-sponsored National Drug Abuse Treatment Clinical Trials Network (CTN) has strived to conduct high-quality, well-managed clinical trials. This includes developing methods for site selection to be used by investigators conducting CTN trials.

Results

Issues relevant to site selection include the clinical trial design, availability of appropriate clinical population, and organizational attributes of potential clinical research sites. Site selection strategies include reviewing regional epidemiologic data, collecting standard site selection surveys, evaluating clinic data on existing patient populations, and site selection interviews and visits.

Conclusions

This paper describes considerations for selecting research sites and identifies specific strategies to employ when selecting community-based sites for participation in clinical trials.

In 1998, the Institute of Medicine (IOM) published Bridging the Gap between Practice and Research: Forging Partnerships with Community-Based Drug and Alcohol Treatment.¹ This seminal publication set the stage for the National Institute on Drug Abuse (NIDA) in 1999 to establish the National Drug Abuse Treatment Clinical Trials Network (CTN).² As a partnership between academic research centers and community drug abuse treatment programs (CTPs), the CTN conducts community-based, multi-site clinical trials to evaluate the effectiveness of empirically-supported treatments in community treatment programs located throughout the United States.

Conducting high quality clinical trials requires rigor in study design and implementation in order to best ensure accurate conclusions are drawn from the study results. Across therapeutic areas, clinical trial investigators struggle with suboptimal trial performance.³ As described in the IOM Bridging the Gap report, community-based clinical trials present additional challenges and opportunities for investigators and clinical practitioners working at facilities whose primary intent is to deliver treatment services, rather than participate in research.^1,4 Briefly, trials conducted in community settings provide access to innovative treatment approaches not otherwise available to these patient populations.⁵ However, this access comes with the expectation that treatment conform to the standardized procedures expected in clinical trials. These procedures may not always be intuitive for clinicians.¹

While various factors associated with effective site selection have been reviewed in other clinical areas (e.g., cardiovascular disease⁶) and contexts (e.g., international trials⁷), overall there is scant literature on site selection. Further, while the importance of site factors have been identified,⁸ we are unaware of published reports that focus specifically on site selection for clinical trials of treatments for substance use disorders. To assist others conducting similar community-based research activities, we describe the current CTN model of site selection for community-based clinical trials based on lessons learned conducting 24 clinical trials during the past 10 years.

In the CTN, site selection has become an increasingly critical component of timely and successful completion of the trials as we have found it to have an influence on recruitment and retention of trial participants as well as effective, high quality study implementation. Examples from two CTN clinical trials, the Prescription Opioid Addiction Treatment Study (POATS),⁹ a combined behavioral therapy-pharmacotherapy-pharmacological trial, and the Stimulant Abuser Groups to Engage in 12-Step study (STAGE-12),¹⁰ a behavioral intervention trial to facilitate attendance at community-based 12-step meetings and engagement in self-help activities, illustrate various components of the CTN site selection strategies. We selected these two trials as we have direct experience with the site selection procedures for these trials, both trials were early examples of the current CTN site selection approach, and because, while site selection across CTN trials varies according to the study and the research team, the site selection procedures described herein are prototypical. We begin by describing site and protocol factors that are important considerations in site selection. This is followed by a discussion of specific strategies to employ when evaluating sites as candidates for site selection.

Considerations for Successful Site Selection

When identifying community-based treatment programs for multi-site clinical trials, numerous factors should be considered during site selection. These include: clinical trial design, community treatment program (CTP) organizational attributes, and clinical population characteristics.

Clinical Trial Design

Each clinical trial design has unique characteristics; a site well suited for a particular clinical trial may not be as good a fit for another. Prior to site selection, it is useful to identify design-specific factors that may influence may impact site performance and be important to consider during site selection. For example, many CTPs may be appropriate sites for a relatively straightforward clinical trial, e.g., a behavioral trial with few exclusion criteria that permits enrollment of all treatment-seeking patients. In contrast, a more complex trial design (e.g., an adaptive trial design), pharmacotherapy trial (need for a physician on staff), or special populations (e.g., adolescents with opioid use disorders) may not be suitable for participation from as many CTPs. These clinical trials may require or benefit from sites with specific research capacities, specialized staff, and/or a high volume of patients with specific clinical characteristics. Generally speaking, the more complex and specific the trial, the greater the challenges with site selection because the number of candidate sites from which to choose is often fewer than desired.

Treatment Program Organizational Attributes

We have found that a program’s organizational attributes, identified as critical for technology transfer of evidence-based practices,¹¹ are similarly important considerations when engaging community-based treatment programs in clinical research. Organizational factors that may be important in the overall success of a clinical trial include: leadership and staff attributes (leadership management style, prior research experience, (including the existence of experienced research staff), staffing levels and turn-over, staff morale), institutional support for research and evidence-based practice, organization climate (readiness to engage in new technologies), and tangible resources (financial, staff, and physical).

Staff factors may also influence the likelihood of the success in community-based research. Staff members in research-naïve programs may display wide heterogeneity in both knowledge and attitudes about clinical research.¹² For example, Forman et al.¹³ found that substantial minorities of clinicians in community treatment programs believe that random assignment is unfair, and most do not feel that a patient has a better chance at recovery by participating in a research study. Thus, educating staff members about the nature of research and examining the culture of a given treatment program as well as staff and institutional receptivity to research is critical for ensuring trial success. We have found that leadership support and staff enthusiasm for participating in a clinical trial may be just as important as other factors in successful study implementation. For example, staff support for a trial is much more likely to ensure that recruitment flyers are posted throughout the clinic and mentioned to patients at each clinic appointment than reminders from an investigator on a weekly conference call. It is also important to remember that, turnover in addiction treatment programs is quite high;¹⁴ ongoing staff education is important to ensure that new staff are oriented to research, and that any potential concerns are addressed.

It is incumbent on the investigator to engage potential sites (administrators and staff) in dialogue regarding their potential as a successful site. This bi-directional communication sets the tone for the research collaboration as the study moves forward. We have learned this is essential. An off-site investigator, no matter how experienced, cannot understand a program’s capacities better than its own clinicians and administrators. However, staff from potential research sites may not have the full complement of background knowledge and experiences to critically evaluate the viability of their organization, staff, and available patient population for consideration as a research site. Similarly, academic-based investigators lacking community research experience will may have unrealistic expectations and not fully appreciate the factors to consider when conducting clinical trials in community settings.¹ As described below, we encourage the use of site selection surveys as a starting point to the site evaluation process.

Availability of clinical population

The importance of methodical evaluation of the availability of the study population in a given treatment program cannot be overstated. As mentioned above, CTP administrators and staff may not have the prior experience or have access to necessary patient-level information to critically evaluate their respective treatment program’s capacity to enroll the desired number of research participants within the time allotted for enrollment. Successful recruitment may require identifying recruitment sources in the community that are outside of the CTP’s direct access, thus requiring significant initiative and creativity on the part of the CTP in both locating and accessing these potential research participants. Potential sites should be made aware of this and consider the need and their willingness to engage in this type of outreach.

In many instances, community-based clinical trials recruit patients from the existing patient population of the treatment programs in lieu of, or in addition to, recruiting patients through media advertising efforts that are typical in clinical trials. If funds are not available for a media campaign to support recruitment, the availability of the potential participants in the existing patient population is even more important. While treatment programs provide services to many patients, for a variety of reasons, relatively few will be eligible or have the desire to enter a trial. For example, consider a hypothetical study to examine a new pharmacotherapy for opioid dependence that requires participating sites to recruit 4 participants per month over a 12-month period. A methadone maintenance program with a patient census of 300 may seem an excellent candidate site. However, the study eligibility criteria require that patients must have a negative urine drug screen for benzodiazepine or cocaine at randomization and no psychiatric comorbidity. In addition, participants must be willing to be switched from methadone to an investigational drug, attend weekly counseling sessions, and complete weekly research visits for six months. In this case, the clinical trial design places specific demands on participating sites. Even a relatively large methadone clinic may not be suitable unless their patient population includes a sufficient number of patients who can meet the trial eligibility criteria.

Site-Selection Strategies

The 24 CTN clinical trials completed thus far have recruited over 11,400 participants at 190 sites (120 different CTPs.)¹⁴ The approach to site selection in the CTN evolved during the past 10 years. Initially, it was decentralized and varied across protocols and investigative teams. Some were more subjective, and based on mutual scientific interests between the lead investigator and CTPs. Not surprisingly this approach had mixed success. Some trials met recruitment targets on time, and performance was acceptable; however, a number of trials struggled to recruit and retain participants. Similarly, recruitment and retention rates at sites participating in the same clinical trial varied despite extensive effort by the lead investigator in selecting sites. For example, in one clinical trial, recruitment rates ranged from 3.0 to 8.3 randomizations per site per week across its 12 sites.¹⁴ Two sites recruited at a rate of 3.0 and 4.6 randomizations per week, and the remaining 10 sites enrolled at a rate of around 6, 7 and 8 randomizations per week. Reasons for this variability included weather, location, and scheduling challenges (e.g., lack of availability during clinic hours because of work constraints). This variability occurred despite all sites having expressed confidence in meeting a standard recruitment target.

As the CTN has matured, we have engaged in self-study across a variety of areas in an effort to improve performance (as evidenced by this special issue) including site selection procedures. It became increasingly clear that developing a more nuanced approach to site selection was part of a package of methods that would help optimize trial performance. Over the past four years, CTN investigators revisited the approach to site selection to move beyond an informal process (i.e., interested sites sign up for participation) to a more objective, standardized, systematic approach. Below, we review common, cross-protocol strategies adopted by CTN investigators as they were operationalized in two trials that first utilized the latter approach.

Utilize National Epidemiologic and Other Existing Data

In the POATS protocol, the clinical population to be studied was individuals with opioid dependence resulting primarily from using prescription opioids. When the trial was designed, the prevalence of individuals seeking substance abuse treatment primarily because of prescription opioids was unclear. As a result, other sources of data, including epidemiologic and criminal justice reports, were examined prior to initiating site selection. We reviewed national data from the Department of Justice, the Drug Enforcement Agency, NIDA, and the Substance Abuse and Mental Health Services Administration. Based on this information, geographic trends in prescription opiate abuse in the United States were identified and areas of the United States where the highest rates of illicit use, diversion, and treatment entry were identified. Based on this, we developed a ‘wish list’ of potential site locations (i.e., locations in areas demonstrating high incidence and prevalence of opioid analgesic abuse), reviewed the locations of existing community treatment programs in the CTN, and reached out to those programs in those areas to gauge interest in the study. This approach led us to select sites that might not otherwise have been considered. For example, a newly-established, rural treatment program with no prior research experience was selected for trial participation in part because of epidemiologic data indicating the program was located in an area with significant prescription opioid abuse. Indeed, this particular site was among the best performing sites in terms of recruitment and retention. A similar approach was used in the CTN STAGE-12 protocol in which wanted to recruit sites that treated primarily either cocaine or methamphetamine clients. The project investigators determined that there was a broad distribution of potential cocaine sites across the country while methamphetamine dominant sites were found primarily in the West.

Employ Site Selection Surveys to Standardize Review Process

Systematic collection of information about potential sites has become a standard component of CTN site selection. Table 1 provides specific examples of the items included in the site selection surveys for STAGE-12 and POATS. The actual surveys and supporting materials for the two studies are available by request to the first author. While each site selection survey is customized to the needs of a given trial, collecting information on site and staff characteristics is a common component in all CTN site selection surveys. Useful information to obtain includes: staffing levels, in general, as well as specific to the potential protocol needs, type of program (e.g., residential, intensive outpatient, outpatient, opiate substitution), program orientation (e.g., 12-step, cognitive-behavioral, open to pharmacotherapy), and availability of confidential office space for research staff as well as for external quality assurance monitors. It is also useful to consider prior research experience, particularly clinical trial experience, is useful when selecting sites. This may be even more useful when information on past performance (recruitment and retention rates) is available. While this may be more feasible in some circumstances, for example within research networks, it behooves all investigators to inquire not just about participation but performance on prior studies.

Table 1.

Selection survey domains and key questions from CTN STAGE-12 and POATS surveys

	STAGE-12	POATS
Instructions	Programs were asked to provide this information for a 6-month period by admissions per month.	Programs were asked to provide site information, administrative data, and collect patient information on key eligibility criteria prospective for 30 days.
Site Program/Policy	Are there any upcoming changes to policy or resource changes that could negatively impact the stability of the center or participation in the STAGE-12 protocol?	Are there any upcoming changes to policy or resource changes that could negatively impact the stability of the program or participation in the POATS?
	Walk through the intake process from 1st contact to beginning treatment	Does this information reflect a single clinic or multiple clinics? If multiple, how many sites?
	Describe your current treatment schedule for someone who needs outpatient services in the range of 5–15 hours per week.	Does your CTP work with ERs and/or primary care clinics? (Please include both on-and off-site facilities.)
	Describe how your program incorporates 12-step programs into therapy?	If yes, how many referrals do you receive from the following per month? Pain clinics? Surgery clinics? Emergency rooms? Primary care clinics? For each, how many of these referrals are chronic pain patients?
	What are your community resources for 12-step meetings? Do you have AA, NA, CA, and CMA meetings? Are there many of them/enough?
Study Participant Population	Please provide the total number of patients and the number of stimulant abusers admitted into outpatient treatment, providing 5–15 hours per week, who are not being provided housing as part of the treatment program.	How many opiate dependent patients did your facility admit during the prospective time period? How did you collect this information?
	What is the number of patients currently enrolled in your program receiving 5–15 hours per week?	How many of those patients also meet ALL of the following criteria? NO lifetime history of heroin injection NO lifetime history of heroin dependence In the last 30 days, four (4) days or less of heroin use
	What will you do if you are unable to meet the enrollment requirements of the study through your usual patient flow?	How many of these patients, who did not meet the three criteria described above, how many did not because of: Lifetime history of heroin injection Lifetime history of heroin dependence: In the last 30 days, five (5) or MORE days of heroin use In the last 30 days, four (4) days or less of heroin use
	In past studies, have you had to advertise for participants? What methods did you use?
Facility	Will you be conducting any other clinical trials during the course of this study? If yes, will this study compete for participants?	Are there ongoing or planned trials that would compete with this study for research participants? If yes, please specify.
	Is the proposed budget (attached) enough to run the study at your site?
	Do you have a place to allow the patient to do the assessments with a separate computer and Internet hookup?
	Do you have room for staff from the DSC, CCC and Nodes visits, possibly lasting more than one day?
	What kind of space do you have for study procedures (i.e. RA/SC office space and treatment rooms)?
Staff	Research Staff: Who will be the Site PI for the study? What experience does the PI have in clinical trials research? Will you have the resources to hire and employ research staff this summer? Does the site have an experienced Research Assistant or Study Coordinator available to help with the study?	Does your facility have physicians with buprenorphine experience available to treat study patients with buprenorphine over a 2-year period? If yes, how many?
	Therapeutic staff How many therapists do you have in your facility? What kind of burden could participating in this trial have on your therapist staffing? Do you have an idea on the interest level of your staff to participate in this trial?	Provide details for up to 3 physicians available for the trial, including for each: usage type (detoxification, maintenance, or both), months buprenorphine experience, prior drug trial experience
	Node Staff What services will your RRTC provide to your site during the course of the study?	Does your facility have staff to provide weekly individual counseling? If yes, how many counselors are available?
Institutional Review Board (IRB)	How long has it taken, on average, to get a protocol through your IRB for approval?	Information not requested
	How many IRBs do you need to go through (i.e. Node, CTP, etc)?

Review Clinic Administrative Data

Extracting relevant data from existing clinic administrative data that reports patient sociodemographic and clinical characteristics is another valuable resource for evaluating potential sites capacity to recruit. Table 1 uses examples from the POATS and STAGE-12 site selection surveys to illustrate the types of information that might be obtained from clinic administrative data including the number of patients presenting during a given period of time that appear to meet some or all of the study eligibility criteria (e.g., opioid dependence). One limitation of this approach is that information collected and easily available for extraction does not typically match a given trial’s eligibility criteria. Similarly the accuracy of this information is variable, and questions may not be asked in exactly the manner desired by the investigators. Nevertheless, taking advantage of such administrative data provides useful information for assessing candidate sites.

Consider Collecting Prospective Data for Unique or Challenging Study Populations

In some situations a study population may be novel or sufficiently unique such that administrative data may not adequately capture their presence within the general clinic population. Such was the case with POATS as we wished to identify patients dependent on opioid analgesics with very low rates of heroin use. Recognizing that this level of detail was not available in clinic administrative data, POATS investigators developed a protocol-specific survey that required sites to collect patient-level characteristics prospectively. Table 1 provides examples of the types of prospective data that POATS investigators requested from potential sites. In POATS, we selected two critical pieces of information to assess: the prevalence of opioid dependent patients using opioid analgesics and the extent of heroin use among these patients. As community treatment programs did not collect this information routinely, we requested that this information be gathered prospectively for a period of up to 30 days.

The prospective examination of the CTP patient population was helpful for both the investigators and the prospective sites. Through this process, some sites that had anticipated a large pool of potential participants identified fewer eligible patients than anticipated. As a result, several prospective sites with strong research capacities withdrew from consideration because they recognized recruitment would be challenging. Other sites emerged as excellent candidate sites because their patients evidenced low heroin use and high opioid analgesic use. Not surprisingly, these sites were located in high prevalence regions of the country that we identified by examining epidemiologic trends.

This approach should be used judiciously. As this type of activity is not likely to be funded, there should be limits on the extent of information collected. It is important to avoid overburdening staff with unrealistic expectation, and to ensure that the data collection procedure is feasible for busy clinic staff. For example, it may be useful to avoid prescribing a data collection method. In POATS, sites collected prospective information using various methods; we only asked that they describe their method for obtaining the required information. The intent is to engage the CTP staff in an exercise that will help the investigators and the site staff evaluates participant availability using specific criteria.

Consider Blinding Review of Potential Sites

As the CTN is a network of researchers and clinicians, there are often pre-existing relationships. We would expect similar relationships to exist in certain therapeutic areas and research networks like community-based participatory research or practice-based research networks. Such pre-existing relationships may contribute to unintended favoritism or bias that might result in otherwise excellent candidate sites being overlooked. For such research collaborations, blinded site selection surveys can protect against these potential problems and ensure that this data-driven approach to site-selection is employed in the most fair and useful way. This technique was used in STAGE-12. Potential sites returned the site selection surveys to an independent, central Clinical Coordinating Center (CCC). The use of the neutral CCC as the recipient of the site surveys represented an attempt to minimize bias in the subsequent site ratings and selection, since members of the STAGE-12 Executive Committee (EC) included CTP representatives and researchers from Nodes with CTPs that might potentially be applying for participation. Initially, potential site identifying information was removed before site selection survey information was distributed to members of the protocol team’s Executive Committee. EC members rated each of the blinded potential site independently on a number of a priori determined criteria and returned their ratings to the CCC, which collated the results and provided summary scores and a rank ordering of sites. The use of a priori criteria (e.g., minimum client flow, intensive outpatient program of minimum number hours per week and overall duration, minimum number of group and individual sessions) was useful in that it provided objective standards against which to evaluate applicants and minimized the likelihood of subjective biases entering into the decision process. Discussions by the protocol EC about which sites to include in a smaller pool of those having the highest ratings overall and across dimensions remained blind until the pool was determined, and it became necessary to “unblind” the Committee in order for it to identify those sites to invite to participate in the next phase of the site selection process, namely telephone or in-person site visits.

Conduct Site Selection Telephone Calls and Visits

Finally, all of the above, while helpful, are alone insufficient to evaluate candidate sites. It is also important to meet with the CTP site investigators and the research team. This may be accomplished through telephone interviews and/or face-to-face visits with the final prospective sites. These interviews are best done after the lead investigators have received and examined all preliminary site information and are preparing to make final decisions on site-selection. The purpose of these site selection calls is two-fold. First, it is an opportunity for the investigators to obtain further information regarding the suitability of the site. More importantly, it offers potential sites to obtain more information about the study, ask questions, and better determine whether there is continued interest in participating.

While face-to-face visits afford the opportunity to see the site, meet the staff, and visualize trial implementation in that setting (i.e., “boots on the ground”). A face-to-face visit can be invaluable in certain situations. For example, an original goal of the CTN was to increase the infrastructure and capacity of CTPs with less prior research experience but that are felt by the to be organizationally sound enough to successfully implement the rigorous demands of CTN protocols. However, when working with such research-naïve sites, high-risk trials, or protocols with complex inclusion/exclusion criteria, a more thorough assessment of CTP capabilities is necessary and may be best evaluated through face-to-face visits. Telephone interviews can be a more efficient and economical option, particularly when the CTP and investigative team already have familiarity with each other. Both telephone interviews and face-to-face visits can offer insight on logistical and cultural factors that may not have come up in a structured prospective site selection survey. This can be crucial for mutually determining whether a site could realistically meet the requirements of a specific protocol and provide more information regarding information obtained from responses to a site selection survey. For example, if the site is participating in multiple studies, could other studies have the potential to compete with the prospective new study for time, staff, and participants? Does the staff have a positive, professional work environment and commitment to the research?

Based on the outcome of their multi-stage selection process, the STAGE-12 Executive Committee selected three research-experienced CTPs as the first sites to implement the protocol; this was followed subsequently by a second wave of seven additional CTPs that included three research-naïve sites. The inclusion of such research-naïve sites not only addressed the original CTN goal but also increased the generalizability of findings to “real life” community programs rather than only to large, research-intensive sites.

Conclusions

In this report, we highlight some considerations in site selection for community-based clinical trials and the associated strategies to improve site selection procedures that we have found to be helpful in the CTN. A final challenge in site selection is aggregating the diverse information collected in order to make final site selection decisions. In collaboration, investigators and clinical practitioners continue to refine our methods for evaluating the relative importance of the myriad pieces of information obtained through our site selection process. As described for STAGE-12 above, we have attempted to develop objective standards but this continues to be challenging. For example, to what extent, if any, does prior research experience outweigh the availability of potential participants? Unfortunately, we do not have conclusive definitive guidelines. While we have implemented these strategies across studies, systematic evaluation of their impact on trial performance has not been investigated. It is unclear whether our recent trials would have been just as successful without employing these strategies.

Nevertheless, our approach has provided an orderly, equitable, and rational approach to site selection based on a comprehensive evaluation of site and protocol factors that appear most relevant for ensuring optimal site performance. The design and implementation of 24 clinical trials at 120 unique sites within the CTN affords the opportunity to describe cross-protocol site selection strategies employed in the CTN. The method we have adopted continues to be refined and adapted. Our experiences suggest that adopting a systematic, data-driven approach to site selection is a useful component of implementing high quality community-based clinical trials. As the CTN matures and gains further experience with site selection, we hope to better refine our site selection procedures.