Identifying and Responding to Trial Implementation Challenges during Multisite Clinical Trials

Abstract

INTRODUCTION:

The National Drug Abuse Treatment Clinical Trials Network (CTN) was initiated by the National Institute on Drug Abuse (NIDA) in 2000 with the aim of improving substance use treatment and reducing the time between the discovery of effective treatments and their implementation into clinical practice. While initial trials were conducted almost exclusively in specialty addiction treatment settings, the CTN began evolving strategically in 2010 to conduct research in general medical settings, including healthcare systems, primary care settings, emergency departments, and pharmacies, in order to broaden impact. The advantages of a research network like the CTN is not only the collective content expertise that investigators contribute to the network, but also the collective experience gained by conducting studies in the network and then applying those lessons learned to future studies.

OBJECTIVE:

To summarize trial implementation challenges encountered, and the process by which solutions were identified and implemented, within one of the last early-phase CTN Stage II behavioral intervention studies conducted in a specialty addiction treatment setting.

METHOD and RESULTS:

We describe the implementation of the CTN-0037 STimulant Reduction Intervention using Dosed Exercise (STRIDE) trial. Issues encountered during study implementation are categorized into four major areas, described in terms useful to future study teams: 1) study team infrastructure challenges, 2) participant- and site-level challenges, 3) intervention-related challenges, and 4) longitudinal study design challenges. Potential consequences of identified problems and the solutions developed to manage these problems are discussed within the context of these four areas. We propose how these implementation lessons may be extended and applied in other healthcare settings relevant to the expanding CTN.

CONCLUSIONS:

Effective study management must allow for flexible, collaborative solutions in response to expected and unexpected obstacles to study success. Implementation strategies derived from the first 15 to 20 years of CTN studies are a result of working with providers and participants, and the ongoing collaboration among CTN investigators and network staff. Timely identification and response to problems during study implementation is critical to the success of a trial, regardless of its design. We believe a collaborative approach to identifying and responding to study implementation challenges will increase the likelihood of successful adoption of relevant, efficacious interventions. As the CTN continues to expand, it is essential that the wealth of successful trial implementation strategies developed during the first 20 years of the CTN are applied and adapted to studies in broader network settings, and considered in conjunction with more formalized implementation science processes that are currently available.

1.0. Introduction

The goals of the National Institute on Drug Abuse (NIDA) National Drug Abuse Treatment Clinical Trials Network (CTN) are improved treatment for individuals with substance use and related disorders, and expedited dissemination of evidence-based treatment into clinical practice. The formative years of the CTN (2000–2010) were characterized predominantly by multi-site, Stage II clinical trials research conducted within specialty addiction treatment settings, typically consisting of behavioral, medical, or combined behavioral/medical intervention(s) evaluated in the context of a randomized controlled trial. Multisite trials are advantageous in that they have greater generalizability and statistical power and generally can be completed faster than single-site trials (Fuller et al., 1994). However, they are also more complex (Fuller et al., 1994), and even with comparatively reduced implementation time, many are still conducted over an extended timeframe, increasing the likelihood of encountering changes in critical areas such as the characteristics of the population being studied. Procedures established prior to trial start-up are intended to maximize the success of trial implementation; however, challenges arising throughout implementation may threaten recruitment, retention, adherence to the treatment(s)/intervention(s) being studied, and/or data collection and can negatively impact fulfillment of study aims. Therefore, the ability to identify and address such challenges is critical, and necessitates coordinated collaboration between local (site) and national (investigative) teams. Furthermore, information regarding both facilitators as well as barriers encountered with respect to logistical feasibility and/or clinical utility of the implementation of an intervention, if found to be efficacious, is critical to its future adoption.

The Center for the Clinical Trials Network sets clearly defined expectations for trial recruitment and retention in order to monitor CTN trial progress and increase the likelihood of achieving study milestones and goals. Using experiences from implementation of the STimulant Reduction Intervention using Dosed Exercise (STRIDE; CTN-0037) trial (Trivedi et al., 2011), this paper describes the problems encountered and the solutions developed to address those problems. STRIDE was a hybrid efficacy-effectiveness trial with restrictive eligibility criteria to prevent enrollment of individuals for whom exercise would be significantly difficult and/or contraindicated, which was conducted within residential treatment programs (i.e., a Stage II trial conducted in real-world settings). Although not specifically designed a priori as an effectiveness-implementation hybrid trial, STRIDE investigators did observe and gather information related to trial implementation to enhance recruitment, retention, and achievement of study aims, consistent with the approaches of such hybrid designs (Curran et al., 2012).

The advantages of a research network like the CTN is not only the collective content expertise investigators contribute to the network, but also the collective experience gained as a result of implementing studies in the network and then continuing to apply those lessons learned in future studies. The strategic evolution and expansion of the CTN in 2010 into larger medical settings (e.g., healthcare systems, primary care settings, emergency departments, and pharmacies) aims to achieve a broader positive impact on the treatment of substance use disorders. The ability to implement rigorous trials within these real-world settings is critical to the success of the expanded CTN. Implementation science aims to employ metrics and structured implementation frameworks to increase the relevance and uptake of scientific research into clinical practice (Bauer et al., 2015; Meyers et al., 2012: Rapport et al., 2018). The early-phase CTN trials were being implemented parallel to, and separate from, the now burgeoning field of implementation science. However, many techniques are similar. As the CTN continues to mature alongside the implementation science field, the evolving CTN network can directly benefit not only from advances in implementation science that have been incorporated into more recent CTN trials (e.g., CTN-0074 PRimary Care Opioid Use Disorders Treatment PROUD) but also strategies from early-phase, Stage II CTN trials. While practical in nature, the approach used to identify and mitigate threats to successful completion of the STRIDE trial are applicable to other clinical trials, and can be used in conjunction with more structured approaches to guide other types of trials implemented within the context of the expanded goals of the CTN. The information we describe also can inform future trials and clinical programs that aim to incorporate exercise as an intervention, and associated payer and policy implications (e.g., provider reimbursement).

2.0. Methods

STRIDE was a multisite randomized clinical trial conducted by the CTN at nine residential substance abuse treatment programs (RTPs) across the United States. STRIDE compared exercise versus health education as augmentation to substance use disorder treatment as usual (TAU) in individuals with stimulant use disorders. Site selection was completed in 2009–2010, 302 participants were recruited from July 2010 to June 2012, and the last participant completed study procedures in March 2013 (Warden et al., 2012; Trivedi et al., 2017). Recruitment rates remained at or above target for the majority of the trial but slowed somewhat in the final months of recruitment. Treatment exposure was 69% and was a major focus throughout the trial.

Randomization occurred soon after admission to maximize the days of study participation during treatment and increase the likelihood of intervention adherence. Study visits (i.e., research and intervention sessions) occurred three times a week for three months (Acute Phase), followed by six months of weekly visits (Continuation Phase). Exercise was prescribed at a dose that approximates 50-minute sessions on 3 days per week. Health education consisted of online, video, and written educational materials. The primary outcome was percent days of stimulant abstinence based on self-report and urine drug screens during Weeks 4–12, with Week 4 expected to be the first week after residential treatment discharge. Complete design and rationale are described elsewhere (Trivedi et al., 2011; Greer et al., 2012; Stoutenberg et al., 2012), as are the primary results of the trial (Trivedi et al., 2017).

A multi-directional and iterative process (see Figure 1) was used to solicit and identify problems during implementation, to determine potential consequences of problems, and to devise and implement solutions. While not corresponding exactly with a particular implementation science framework, the process used by the STRIDE team does have similarities with core concepts of a general implementation framework (Moulin et al., 2015). However, it is important to note that the primary aim of the STRIDE trial was to evaluate the efficacy of the exercise intervention, and therefore the intent of our approach was to execute the trial with the rigor and standards required to answer the research question; this can be differentiated from an implementation science framework, which is more targeted toward intervention implementation and integration into mainstream of care. The problems identified in STRIDE (see Table 1) were categorized into and discussed within each of the following four major areas: study team infrastructure challenges, participant- and site-level challenges, intervention-related challenges, and longitudinal study design challenges.

Figure 1. Process for identification of issues affecting clinical trials, the assessment of their effect on the trial, and development of solutions to address them.

For all issues, the team had to identify the problem, assess the consequences of the problem, develop a solution, and implement the solution across sites. The consequences of initiating procedural changes had to be assessed. This process can be broadly applied to many multisite clinical trials.

Table 1.

Specific problems, study consequences, and recommended solutions by four challenge categories

Problem	Study Consequence(s)	Recommended Solution(s)
Study Team Infrastructure Challenges
Inadequate communication and coordination of study procedures between research and clinical staff	Inefficiency in the implementation of study procedures Decreased rapport with and support of clinical staff	Operate as one team on a day-to-day basis Integrate the protocol into the daily clinical activities Identify a site staff member who has a leadership role in the agency and can be a “champion” for the research team Request immersion of both research and clinical staff, at least the champion, in the conduct of the study Continually monitor the effectiveness and timing of communications and procedures between research and clinical staff
Inadequate communication with outside contracted organizations conducting study procedures	Poor understanding of responsibilities at contracted organization Errors in implementation of contracted procedures Inefficiency that affects collection of study data	Establish a relationship between an accountable research team member and the organization’s representative Initiate early and frequent communication and problem solving
Multiple communications needed across study team	Staff burden if communications are ineffective Inefficiency and inconsistency in the implementation of study procedures if communications are inadequate	Provide redundant, overlapping communications via multiple venues to disseminate and discuss procedural changes to reinforce consistent adoption Reduce frequency of teleconferences when feasible
Participant- and Site-Level Challenges
Unstable housing / Plans to move from the area	Poor retention Inability to participate in intervention	Implement a detailed discussion prior to randomization of plans for housing and factors that may increase the likelihood of moving Consult participant and clinic staff frequently about housing arrangements Collect comprehensive locator information and reassess weekly Develop assessments that can be conducted by phone or off-site Add social media as communication method with participants who provide permission
Employment changes and childcare difficulties	Poor retention Study fatigue	Collect comprehensive historical data on work history and future plans for work prior to randomization Collect probable work schedule and develop contingencies Hire staff with the understanding that flexibility (e.g., weekend study visits) are an expectation of employment Explore childcare responsibilities and contingencies (e.g., family members or friends who could help; day care availability and costs)
Mental health issues	Burden to the participant if mental health issues are not addressed Poor retention	Collect comprehensive treatment and medication history prior to randomization Provide mental health referrals
Transportation difficulties	Poor retention Study fatigue	Evaluate transportation means and commute times (in the context of total visit time and overall participation) prior to randomization Evaluate opportunities for public transportation and assist with navigating complex routes Establish procedures to provide transportation assistance (e.g., bus passes, taxi vouchers, gas compensation) consistent with local standards Explore alternative transportation assistane from friends, relatives, transitional housing staff Study staff (not participants) explicitly instruct taxi drivers about dropoff locations
Court-mandated treatment and incarceration	Poor retention Inability to participate in intervention Decreased rapport with incarcerated participants given the limited contact	Obtain regulatory and facility approvals to conduct research with prisoners and to conduct study visits in prisons/jails Understand confidentiality limits within prisons/jails Determine compensation rules for research participation while incarcerated Search publicly available websites for incarcerated participants Increase efforts to maintain rapport with incarcerated participants
Relapse	Poor retention Adverse emotional response by participants	Conduct early and ongoing non-judgmental conversations about relapse potential, underscoring confidentiality protections and the separation of the study staff from RTP clinical staff Explore participant concerns about parole violations due to substance use Underscore importance of having accurate and complete data to draw conclusions about study effectiveness Praise attendance at study visits and normalize emotional experience of relapse Minimize or avoid interactions with RTP staff if important to participant (e.g., provide alternate entrances for study visits)
Treatment plan changes	Poor retention Inability to participate in intervention	Create provisions to collect data from participants if admitted to treatment at off-site locations Develop systematic method of tracking time in supervised settings (e.g., inpatient treatment, incarceration)
Study fatigue	Poor retention Adverse emotional response by participants	Incorporate tangible participation and adherence incentives using principles of contingency management Tailor study compensation to participant preferences if using gift cards Recognize study milestones and achievements (e.g., verbal praise, paper certificates, letters from Principal Investigator)
Intervention Related Challenges
Personal history negatively impacting intervention	Burden to the participant Poor retention	Evaluate current and prior biopsychosocial factors to gauge motivation and barriers relevant to interventions (e.g., school history for health education; history of exercise for exercise intervention)
Relapse	Safety issues Poor retention	Adhere to well-defined safety thresholds for interventions Allow for staff judgment to reschedule intervention, if needed
Injury exacerbation	Poor retention Inability to participate in intervention Burden to the participant if additional care is needed to address injury	Obtain thorough history of issues related to completing the intervention (e.g., injuries in the case of STRIDE)
Logistical issues of visits	Poor retention	Ensure resources are available to allow for completing intervention (e.g., access to shoes and shower facilities for exercising participants) Discuss realistic time needed for visits (e.g., shower before going to work after exercising) Increase guidance and training for completing home-based component of intervention (e.g., finding a safe place to exercise)
Safety problems not equally applicable to both interventions	Imbalanced treatment exposure across interventions	Ensure contraindications to one intervention are applied to the other intervention (e.g., health education participants could theoretically participate in their intervention if pregnant, but since exercise participants could not, all pregnant participants were withdrawn from their respective intervention)
Inadequate staff expertise on clinical characteristics of study population	Poor interventionist-participant rapport Poor retention	Provide training so interventionists have accurate expectations and skills to interact positively and effectively with participants (e.g., interventionists had training and expertise in interventions, but may not understand needs of substance users) Discussions between research staff and site clinical staff increased interventionists’ understanding of drug abuse
Longitudinal Study Design Challenges
Changes in treatment-as-usual due to funding	Reduced ability of site to recruit target population in specified time frame due to smaller candidate pool Fewer overlapping days in RTP and study for participants, thereby decreasing familiarity with intervention prior to discharge	Determine if RTP can strategically lessen consequences of funding changes (e.g., extend lengths-of-stay to correspond to protocol requirements if clinically indicated; reassign program beds to assist study in meeting goals if will not negatively impact RTP or patients)
Changes in drug use patterns	Reduced ability of site to recruit target population in specified time frame due to smaller candidate pool	Monitor and modify strategies for identifying and referring study candidates Increase treatment referrals to site (e.g., existing referral sources, advertisements, interviews with local media) to indirectly increase candidate pool
Staff turnover	Inability to conduct study visit or intervention Increased stress on remaining staff	Cross-train existing staff early to minimize disruptions Streamline training for new staff Implement formal and informal staff appreciation nationally and locally to boost morale (e.g., LT set objective monthly performance goals and recognized sites who met goals)

Note. Problems, consequences, and solutions from STRIDE are described broadly to illustrate their applicability to other clinical trials. LT = Lead Team. RTP = residential treatment program.

3.0. Results

3.1. Study Team Infrastructure Challenges

The STRIDE team was comprised of the Lead Investigator and national management team (Lead Team [LT]), and included CTN representatives, the NIDA-contracted Clinical Coordinating Center and the Data and Statistics Center (DSC). Six university-based academic nodes provided scientific, operational, and regulatory support to the nine partnering community residential treatment programs (RTPs) and affiliated outpatient programs that participated in STRIDE. RTP site study staff for STRIDE included a site Principal Investigator (PI) and other management personnel, two Research Assistants and two Interventionists - an Exercise Facilitator and a Health Education Facilitator. Information about study operations and progress came from multiple sources, such as data and monitoring reports, study logs, automated reports and communications from RTP and node staff. This comprehensive structure provided the LT timely identification of site-specific and study-wide issues and allowed consideration of the potential effects on implementation, protocol integrity, and burden to participants and staff.

3.1.1. Coordination between research and clinical site staff

It was expected that close collaboration between the research staff and the RTP clinical staff not directly involved with study activities would be needed. However, it became essential for the independent staffs to operate as one team on a day-to-day basis and that the protocol be fully integrated into the treatment site’s daily activities. These collaborative relationships and integration worked best when a research staff member (e.g., site PI) (Robbins et al., 2010) with a leadership role in the agency was available to “champion” the study. At some sites, clinicians were concerned that research procedures might interfere with participants’ treatment. The champion helped research and clinical staff identify, discuss, and resolve concerns so the study was more acceptable to site clinicians.

A positive, collaborative working relationship between the RTPs’ clinical and research staff in which clinical staff were equally invested in the successful implementation of the study was essential to identifying and resolving emerging problems during implementation. For example, site meetings were used by RTP clinical and research staff to proactively identify prospective participants and to determine the ideal time to approach them in order to fit within the site’s workflow and clinical needs while also meeting study timelines. Consistent communication of study candidates’ expected RTP discharge date and post-discharge plans were important due to study enrollment criteria. As the study progressed, however, some procedures became discontinued inadvertently (e.g., clinical staff could forget to complete a study candidate referral form when rushed) or became ineffective due to changes in the site’s operations. Thus, the LT found it important to involve research and key clinical staff in more frequent monitoring of these intersecting research and clinical procedures.

Research staff had to flexibly and creatively implement solutions without disrupting the study’s or site’s clinical goals. For example, some study teams became concerned that candidates inadvertently were not being referred, so they developed customized monitoring tools to track treatment admissions. Some participants left treatment against clinical advice or discharged early without warning. It was important to develop new site-specific procedures to quickly inform research staff of these discharges. Without full integration of the study in the daily life of the site and day-to-day collaboration between research and clinical staff, solutions to the above issues would have likely not been identified and implemented.

3.1.2. Communication with contracted organizations conducting study procedures

In order to complete exercise stress tests to evaluate participants’ medical safety prior to randomization, most sites contracted with an outside medical organization (e.g., cardiologist), which presented unique challenges. Timely completion of required research training for contracted medical staff was challenging. Additionally, research documentation requirements were often greater than clinical practice requirements, creating concerns for medical staff burden and demanding research staff scrutinize forms. Medical personnel unavailability (e.g., due to illness, vacation) also was problematic when it delayed tests and subsequent randomizations. Issues were best addressed through an established relationship between a specific research team member and designated representative from the contracted medical organization, but responsibility for identification of issues ultimately fell to the site research team.

3.2. Participant- and Site-Level Challenges

The importance of retention and the need to conduct trials more efficiently were noted challenges for clinical trials of substance use disorders as reflected in the first 10 years of the CTN (Carroll et al., 2011), but are applicable to trials of any type in any clinical population. Significant heterogeneity is observed with respect to reasons why participants become less adherent during study participation or terminate study involvement (Northrup et al., 2017). The interplay between less stable characteristics of the participant population (e.g., housing availability) and study sites (e.g., funding authorizations for treatment) is a pervasive retention issue. Ongoing evaluation and reassessment of retention barriers and responsive efforts to bolster retention can positively affect critical markers of study success.

3.2.1. Scheduling

Several proactive strategies to minimize participation barriers in STRIDE were developed at study start-up, such as scheduling flexibility (i.e., weekends, early or late hours), a behavioral adherence program to encourage intervention attendance (Stoutenberg et al., 2012) and a pre-randomization discussion of participation barriers (Northrup et al., 2017). However, even with proactive strategies in place, site and node staff tracked changes in participants’ life circumstances that created motivational and logistical barriers to attendance, many of which affected long-term retention. For example, randomization soon after admission maximized the days of study participation during the RTP stay, yet many participants did not have post-discharge housing plans prior to randomization. Close consultation with the participant and clinical staff assisting with housing was critical. Similarly, most randomized participants entered treatment unemployed (68.5%) or underemployed, making it difficult to consider the impact of future employment on study participation. Staff obtained historical details to help gauge and plan for emerging needs throughout the study.

3.2.2. Transportation

In addition, each study site faced unique challenges regarding participants’ commutes and a range of transportation options had to be explored. Non-study transportation assistance (e.g., from transitional housing staff, relatives, friends, public transportation) was fully explored pre- and post-randomization. Participants in rural or large metropolitan areas often had longer commutes and some who relied on public transportation had complex routes with multiple transfers and long wait times. Participants with vehicles were burdened by rising and consistently high gas prices during the study; in response, gas compensation was provided by the study, and was needed for almost all participants at one urban site. Taxis were utilized at sites with poor public transportation, which led to new challenges from participants attempting to use them for non-study transport. Effective verification processes were needed to ensure transportation compensation was used appropriately.

3.2.3. Relapse

Relapse is a major barrier to adherence in clinical trials for drug abuse treatment (Herbeck et al., 2005), and was also a major barrier for STRIDE. Consistent messages to participants prior to randomization and throughout the trial underscored: a) the potential for relapse; b) nonjudgmental attitudes of study staff; and c) confidential data collection. Research staff praised attendance to reinforce participation regardless of abstinence status and reiterated that continued study participation was important even if relapse occurred. Study staff also assisted participants with alternative entrances for study visits to reduce the likelihood of interacting with treatment staff when this was important to participants. Frequent reminders and increased assistance helped retain some participants who relapsed after RTP discharge.

3.2.4. Readmission to residential treatment

Some participants were readmitted to their original RTP during the trial, which provided an opportunity to either easily continue or resume attending study visits. Admission to other facilities not affiliated with the study often posed challenges. Many facilities had leave restrictions and study teams in some cases successfully arranged for participants to resume study visits. Changes were often interrelated (e.g., a change in employment could create childcare and transportation issues), making problem-solving more complex. In response, study teams developed additional strategies to improve visit adherence. Of note, solutions at a given site can be highly dependent on local factors (additional details in Table 1).

3.2.5. Addressing Multiple Challenges with the “Fireside Chat”

Although regular pre-randomization discussions of anticipated barriers were planned at study start, higher than expected early dropouts at some sites indicated the need for more formalized discussions of possible barriers with potential participants. The “Fireside Chat” was a strategy developed at a local site, and then adopted study-wide to engage multiple local site personnel in actively identifying and problem-solving potential participant-specific barriers to successful study participation. The content, methods, and example questions used during the “Fireside Chat” are thoroughly described in Northrup et al. (2017). The “Chat” occurred prior to randomization, and consisted of study personnel engaging with potential participants in a nonjudgmental way to identify potential barriers and possible solutions to address those barriers (e.g., if child care plans were unknown or unreliable, asking a family member for help with child care could be a potential solution; for an individual who may have unknown or unreliable means of transportation, site personnel could help determine a bus route to the RTP). The “Chat” allowed the study team to both reinforce the participant’s understanding of study responsibilities reviewed during the informed consent process, as well as determine what barriers may arise for that participant during the study and may therefore threaten retention.

Although we were unable to quantify the study-wide impact of the “Chat”, data from the site where the “Chat” was developed evaluated acute phase intervention and follow-up attendance in 29 participants (Northrup et al., 2017) before (N=15) and after (N=14) implementation of the “Chat”. Individuals who participated in the “Chat” had significantly greater attendance in acute phase intervention visits, follow-up phase intervention visits, and research visits, compared to individuals who participated prior to “Chat” implementation (Northrup et al., 2017). Anecdotal evidence across multiple study sites indicated a positive impact of the “Chat”.

Despite the proactive strategies to address as many issues as possible prior to randomization through open, honest dialogue between study staff and potential participants, unexpected circumstances arose after randomization (e.g., incarceration, relocation) that impeded participants from attending study visits. After consideration, telephone and off-site research visits were allowed if confidentiality could be protected and IRB and LT approvals were obtained. Although interventions continued only on-site, data collection flexibility retained some participants who may otherwise have been completely lost to follow-up, with some participants eventually returning to the site for later visits.

3.3. Intervention Related Challenges

Issues unique to the exercise and health education interventions created specific barriers to intervention participation. While some are specific to exercise, many issues are relevant to any behavioral intervention and are described here and in Table 1. Both interventions required concentration and distress tolerance. For example, health education was designed to be interesting and interactive, but it involved learning about potentially distressing topics (e.g., HIV). Gradually increasing exercise demands occasionally created physical and mental stressors that Facilitators actively worked on with participants. Both interventions were also impacted by participants’ histories. Prior performance in school impacted attention and motivation for health education. Prior exercise experience and injuries impacted motivation for exercise. As study staff became increasingly aware of factors influencing adherence, they paid closer attention to each participant’s history during the “Fireside Chat.”

3.3.1. Relapse

Relapse was discussed previously as a barrier to study visits, but it also posed unique challenges for the interventions. Substance use immediately prior to a study visit reduced a participant’s ability to engage. Regardless of intervention assignment, facilitators were asked to reschedule visits if a participant seemed unable to fully participate. Exercise visits in which a participant’s heart rate or blood pressure exceeded study-defined safety thresholds were rescheduled, regardless of recent substance use status, due to safety precautions specifically for exercise.

3.3.2. Issues Unique to Physical Activity Interventions

In addition to potential injuries and physical discomfort, logistical considerations unique to exercise also had to be addressed. It was anticipated that some participants would not have appropriate shoes and clothing; therefore, each site ensured these were available. Not having facilities to shower and change clothes was a common barrier, and sites secured these when feasible or factored in additional time to go home to shower prior to work when planning visits. A more detailed discussion of these issues was then incorporated into the “Fireside Chat.” During the continuation phase, exercise participants completed two weekly off-site exercise sessions on their own. Exercise Facilitators worked with participants to determine feasible locations for these sessions. Safety and avoidance of relapse triggers when exercising off-site were routinely assessed. Additionally, weather changes necessitated new approaches, for example, walking in the mall instead of outdoors.

3.3.3. Avoiding Imbalance in Exposure to the Interventions

Some circumstances, such as pregnancy, use of excluded medications, or injuries, contraindicated exercise but not health education. Similarly, health education visits could have been completed off-site but exercise on treadmills could only be done on-site. Site research staff often inquired about completing health education sessions off-site. A guiding principle was to ensure interventions were treated equally so treatment exposure was not differentially impacted. Thus, any safety or logistical issues potentially resulting in discontinuation for one intervention resulted in discontinuation in both interventions. Staff were continually reminded and encouraged to reschedule visits for both interventions if acute effects of substance use was an issue; however, this required less judgment for exercise due to objective safety measurements. While attempts were made to avoid imbalance, it was hard to avoid and may need to be considered when evaluating outcomes.

3.3.4. Staffing Considerations

Intervention facilitators were required to have prior experience consistent with their role. Few had experience in research or with substance using populations. Additional training was needed to understand addiction so facilitators could better assist participants, particularly with respect to barriers to completing intervention sessions. For example, Exercise Facilitators were adept at addressing exercise-related barriers (e.g., injury) as compared to barriers related to substance use (e.g., not communicating during a relapse episode). These situations likely impacted retention as well as staff satisfaction and burnout. Several nodes implemented training on pertinent characteristics of drug use such as comorbid conditions and cycles of abstinence and relapse.

3.4. Longitudinal Study Design Challenges

Even in trials completed quickly, substantial time may pass between finalizing the protocol and completing the study. During this time, expected changes such as staff turnover occur and are routinely considered during study planning. However, unexpected changes in both study sites and the population may occur. Examples of such changes experienced during STRIDE are presented below and are summarized in Table 1.

3.4.1. Changes in Treatment-as-Usual

Sites were selected that provided residential treatment for an average of at least 14 days, ideally with discharge to outpatient treatment at approximately 21 to 30 days, allowing participants to complete 2–3 weeks of the study while in RTP. During implementation, sites reported that insurers began to authorize fewer residential treatment days, sometimes seven or fewer at a time, with additional interim reauthorizations needed. This required study teams to vigilantly minimize time from admission to randomization to maximize participants’ time in the study while in RTP. In addition, some sites reported fewer treatment authorizations for stimulant users in general, significantly reducing the potential participant pool. Overcoming these challenges required close collaboration between research and clinical site staffs and study championing.

3.4.2. Changes in Drug Use Patterns

An additional challenge was the change in prevalence of specific drugs used among patients presenting for treatment. Sites were carefully selected for adequate stimulant abusing or dependent clients with no opiate dependence (a study exclusion). However, many sites had substantial increases in opioid-dependent stimulant users, consistent with a national shift in drug use during the study (SAMHSA, 2012), and coinciding with the start of the third wave of the opioid epidemic (CDC, 2020). Recruitment efforts became more intensive and included continuous monitoring and modification of strategies to ensure no candidates were overlooked, and increasing the client pool by utilizing existing and new referral sources.

4.0. Discussion

Effective study management must allow for flexible, collaborative solutions in response to both expected and unexpected obstacles to study success. Trial implementation strategies derived from the first 15 to 20 years of CTN studies are a result of working with providers and participants, and the ongoing collaboration among CTN investigators and network staff. While many of these early CTN trials predate more recent formalized implementation science approaches to the conduct of pragmatic clinical trials, such as mixed methods designs (Palinkas et al., 2011), the information gleaned from their implementation is informative to the next phases of CTN research. This paper described various trial implementation problems encountered during a complex multisite trial that was conducted in real-world community treatment settings to evaluate two behavioral interventions. The process by which emerging problems’ potential consequences to the study were assessed, potential solutions were developed, and the way in which solutions were implemented (Figure 1) may be a useful paradigm for others implementing and managing clinical trials.

Several important points applicable to the conduct of multisite clinical trials can be derived from how the STRIDE team addressed problems that threatened recruitment and retention. First, consistent, collaborative communication between national study leadership, local site leadership, and site staff allowed all personnel to contribute to problem-solving. While perhaps not surprising, the take-home message is that when procedures are problematic or implementation targets are not met, soliciting information from all parties involved is essential, including taking non-research clinicians’ practical needs and research familiarity into account (Tai et al., 2011). This collaborative approach is also important for team building and enhances the investment and motivation of the entire team. Furthermore, significant and ongoing effort is needed to keep open lines of communication and active problem-solving among all staff. Identification of a champion for the study at the clinical site is also critical to this process. Future studies would ideally employ a process evaluation to determine the ideal “champion” at the clinical site (Elwy et al., 2019), and those individuals could participate in the facilitation of formal querying of site personnel regarding potential barriers to the use of exercise as a treatment strategy, as opposed to the more informal consultation and discussion with the clinical site staff that occurred in STRIDE.

Second, implementing novel solutions for specific problems participants are unable to solve is a significant retention aid. This observation is supported by other studies empirically examining factors associated with retention that have found, for example, that sites with the highest retention of female participants offer childcare (Pinto et al., 2011). Creative solutions could broadly impact multiple problems and effectively address site-level, participant-level, and intervention-related challenges. The “Fireside Chat” used in CTN-0037 (Northrup et al., 2017) is an excellent example of how a local study team problem-solved local barriers to retention, developed a strategy to prospectively mitigate retention problems, and worked with the national leadership team to formalize and adopt the process across all study sites.

Finally, careful monitoring of participant and site characteristics must occur in longitudinal studies that include populations and study sites that are acutely susceptible to change over time. For STRIDE, the notable changes were in national drug use trends and local substance use disorder treatment financial coverage; these changes are likely to occur within all foreseeable studies of substance use disorder populations. An analysis of 24 previously completed CTN studies did not identify any quantifiable trial design characteristics significantly associated with recruitment and retention and noted qualitative factors such as study site clinical operations, study staff morale, and unanticipated challenges in studies taking place over several years may better explain variability in trial implementation (Wakim et al., 2011). The types of problems encountered in STRIDE are consistent with this hypothesis. As such, the problems and solutions implemented during STRIDE that are most likely to be encountered in other behavioral intervention research were described so that others may benefit during planning and implementation of future trials.

This paper reports challenges and problem-solving strategies specific to an RCT of behavioral interventions conducted within addiction specialty care settings (i.e., residential treatment). The principles of how problems were identified and subsequently addressed can be broadly applied to a variety of trials conducted in a variety of settings, and can support, inform, and augment formalized implementation science approaches. Bringing evidence-based interventions to the community is a challenge across all disciplines in medicine. Several infrastructures have been supported over time to bridge this gap and to facilitate adoption and implementation of evidence-based interventions into care. The mission of the CTN has been supported specifically to accomplish these goals. Other notable infrastructures, such as the NIH Collaboratory are tasked with, through implementation research, bringing solutions to the health care delivery and health care system. Given the NIH Collaboratory’s mission of identifying and collating best-practices in the conduct of large-scale pragmatic clinical trials, there is certainly much overlap between aspects of the NIH Collaboratory and the lessons we learned in the conduct of the STRIDE trial. These include the need for a planning phase, and the fact that novel challenges will arise despite good planning, and must be addressed as they occur within implementation; the need for ongoing engagement with clinicians when data are collected in the context of clinical care; and the potential for unbalanced and unanticipated challenges within the comparator groups under study (Weinfurt et al., 2017). Other lessons learned from initial NIH Collaboratory trials, such as leveraging existing electronic data, including electronic health records (Weinfurt et al., 2017), were not encountered due to the design and aims of STRIDE, but certainly this issue is of great relevance to ongoing and future CTN research.

The expansion of the CTN introduces additional stakeholders into study implementation teams, as well as recipients/beneficiaries of CTN research (e.g., primary care physicians, healthcare networks, payers). Furthermore, CTN trials that aim to establish screening for substance use disorders must evaluate threats to successful screening processes rather than treatment exposure; trials that are conducted in settings beyond addiction specialty treatment have different potential threats to workflow integration and local communication strategies. The lead investigative team must identify, facilitate and clearly articulate the following: 1) who are the key staff members, teams, and stakeholders involved directly and indirectly in study implementation?; 2) are there clearly defined leads/champions for each team?; 3) is there a mechanism by which local issues and threats to study implementation can be clearly communicated to the lead investigative team?; 4) is there a structured, collaborative process in place to facilitate strategic problem-solving and adoption of solutions across local sites/networks/teams? Consistent, collaborative, and clearly communicated procedures to identify threats to successful trial implementation are needed regardless of the specific study team membership, settings, or aims. Furthermore, developed solutions must maintain the integrity of the protocol and adherence to study timelines. We believe a collaborative approach to identifying challenges to study implementation and their associated solutions will increase the likelihood of successful trial completion and complete data collection. As the CTN continues to expand, it is essential that the wealth of successful trial implementation strategies developed during the first 20 years of the CTN are applied and adapted to studies in general medical settings.

Highlights.

• Effective trial management requires input from all study team members

• Collaborative, creative, and ongoing problem-solving leads to novel solutions

• CTN trials have expanded from specialty settings to larger general medical settings

• Historical implementation lessons from earlier CTN trials can meaningfully inform future trials

• We describe CTN0037’s infrastructure, participant- and site-level, intervention, and longitudinal design challenges

• Historical information coupled with structured monitoring can increase trial impact