Mechanisms to Expedite Pediatric Clinical Trial Site Activation: The DOSE Trial Experience

Abstract

Background:

Timely trial start-up is a key determinant of trial success; however, delays during start-up are common and costly. Moreover, data on start-up metrics in pediatric clinical trials are sparse. To expedite trial start-up, the Trial Innovation Network piloted three novel mechanisms in the trial titled Dexmedetomidine Opioid Sparing Effect in Mechanically Ventilated Children (DOSE), a multi-site, randomized, double-blind, placebo-controlled trial in the pediatric intensive care setting.

Methods:

The three novel start-up mechanisms included: 1) competitive activation; 2) use of trial start-up experts, called site navigators; and 3) supplemental funds earned for achieving pre-determined milestones. After sites were activated, they received a web-based survey to report perceptions of the DOSE start-up process. In addition to perceptions, metrics analyzed included milestones met, time to start-up, and subsequent enrollment of subjects.

Results:

Twenty sites were selected for participation, with 19 sites being fully activated. Across activated sites, the median (quartile 1, quartile 3) time from receipt of regulatory documents to site activation was 82 days (68, 113). Sites reported that of the three novel mechanisms, the most motivating factor for expeditious activation was additional funding available for achieving start-up milestones, followed by site navigator assistance and then competitive site activation.

Conclusion:

Study start-up is a critical time for the success of clinical trials, and innovative methods to minimize delays during start-up are needed. Milestone-based funds and site navigators were preferred mechanisms by sites participating in the DOSE study and may have contributed to the expeditious start-up timeline achieved.

1. Introduction

Clinical trials are instrumental and necessary in improving public health. Despite their integral importance, successful completion of randomized trials is hampered by several known challenges [1]. One key determinant of a trial’s success is timely start-up. Delays during trial start-up are common; up to 86% of trials experience delays in the start-up process [2]. These delays lead to increased costs in trial execution, postponement in the provision of novel therapeutics or interventions to the bedside, and increased risk of trial failure [2, 3].

Key factors associated with start-up delay often relate to regulatory processes, but contracts and budgeting are commonly identified pain points, in addition to delays from site identification and selection, site activation, and inefficient processes [4, 5]. Studies assessing start-up times among academic medical institutions have shown that study start-up can take an average of 4 to 6 months, and in some cases more than a year [6, 7]. Moreover, available literature on trial activation timelines is primarily from trials in adults; data regarding start-up metrics from pediatric clinical trials are sparse. Additionally, many known challenges in the execution of pediatric randomized clinical trials exist; therefore, understanding and mitigating the factors that delay or place successful trial completion at risk can directly benefit investigators, prescribing clinicians, and children in need of these products.

Here, we describe the implementation of three innovative trial mechanisms designed to optimize trial start-up, and the subsequent start-up times in comparison to previously conducted pediatric trials. We describe the perceptions surrounding the mechanisms employed and report the subsequent success of the participating sites.

2. Materials and methods

The Dexmedetomidine Opioid Sparing Effect in Mechanically Ventilated Children (DOSE) trial (NCT03938857) was a Phase 1b randomized, double-blind, placebo-controlled, dose-escalation trial of dexmedetomidine in critically ill children requiring mechanical ventilation. This trial sought to determine whether the mean daily fentanyl dose in mechanically ventilated children could be reduced by the addition of dexmedetomidine.

DOSE was designed as part of a collaborative effort among investigators of the National Center for Advancing Translational Sciences Clinical and Translational Science Awards (CTSA) Program-Trial Innovation Network (TIN) [8]. The TIN aims to both improve the execution of clinical trials and serve as a national laboratory to study, understand, and innovate the process of conducting clinical trials by leveraging a single institutional review board (IRB) system, master contracts, and evidence-based strategies for recruitment and patient engagement. Furthermore, the TIN supports the coordinating center and recruitment/retention activities for a component of the National Institutes of Health (NIH) Helping to End Addiction Long-term (HEAL) Initiative^SM [9], namely the HEAL Pain Management Effectiveness Research Network (ERN) [10]. The HEAL Pain Management ERN was established to perform trials that compare effective existing therapies that prevent or manage pain while reducing the risk of addiction. The national opioid addiction crisis motivated TIN investigators to design the DOSE trial as a proof-of-concept operational framework in preparation for upcoming HEAL Pain Management ERN trials.

The DOSE study start-up process implemented novel start-up mechanisms across participating sites. In the current study, these mechanisms were examined for their impact on study start-up metrics, and site personnel were surveyed for their perceptions about the novel mechanisms. This study received a determination of not human subject research by Duke University IRB.

2.1. Interventions employed to expedite start-up

To uphold the expeditious goal in the HEAL Pain ERN, an activation timeline of three months was agreed upon for the DOSE trial. During project development, TIN principal investigators (PIs), project leaders, and stakeholders agreed upon three mechanisms to expedite study start-up processes. The interventions were chosen based on opportunity and prior experience within the TIN. Additional network funds were available to use towards milestone payments, thus securing this mechanism; and there was some early collective experience between the TIN PIs using the other mechanisms.

First, the trial used “competitive activation,” whereby more sites were selected than would be allowed to fully activate and enroll participants. The selection process started with the CTSA Hubs and their Liaison Teams distributing a site-selection feasibility survey with instructions explaining that of the 20 sites initially selected, only the first 15 sites to complete study start-up activities would be chosen to activate and enroll in the study. The last five sites to complete start-up activities would be “back-up” sites in case enrollment was slow or a more rapidly activated site dropped out.

Second, two site navigators, already knowledgeable in start-up efficiencies, were assigned to guide 10 sites each through the start-up process. The site navigators were oriented to the customized sequence of the DOSE trial start-up workflow and funding strategies. They then engaged sites and built trusted partnerships with site teams using a 12-week, standardized workflow plan that guided the sites through the completion and submission of regulatory documents, central and local IRB submissions, execution of contracts, and study-specific trainings. Site navigators engaged with sites weekly by telephone to answer questions and assist in the completion of all necessary study activation steps. Every site was guided through the identical time-sensitive activation sequence using dissemination-ready weekly slide presentations, manuals, and automated checklists supported by progress tracking reports.

Third, additional funds were earmarked in the budget to sites that achieved a priori start-up milestones. These funds were distributed regardless of whether sites were ultimately selected to fully participate in enrollment, which created a “no lose” situation for sites to participate. Four reimbursement milestones were developed by TIN PIs, project leaders, and stakeholders during project development:

1. Regulatory documents complete – site provides all start-up regulatory documents (except IRB approval) within 25 days of receiving the regulatory packet.

2. Context complete – site provides human resources protection local context language within 30 days of receiving the clinical trial agreement (CTA).

3. Partially executed contract complete – site signs the contract and the coordinating center confirms receipt within 31 days of receiving the CTA.

4. Site activation complete – site is activated (IRB approval, site training, fully executed contract, complete regulatory documents) by 55 days of receiving the CTA (receives a full fund disbursement) or by 84 days (receives partial disbursement).

Achieving milestones 1–3 awarded equal dollar amounts, while achieving site activation afforded the highest funds. Milestones for reimbursement are displayed in Figure 1.

Figure 1. Example Timeline for Activation Milestone-driven Reimbursements.

Four potential reimbursement milestones were offered as part of DOSE. Note: Because the regulatory packets and CTAs were received on different dates, Day 0 had to differ for certain milestones, represented by the different colors of shading on the top and bottom half of the figure. Abbreviations: CTA, clinical trial agreement; DOSE, Dexmedetomidine Opioid Sparing Effect in Mechanically Ventilated Children; IRB, institutional review board.

2.2. Site start-up survey evaluation

Following site activation, the DOSE coordinating center developed and distributed a REDCap survey to site PIs, study coordinators, and regulatory coordinators. One additional email reminder was sent to complete the surveys. Survey distribution was IRB exempt. Survey questions included multiple choice, ordinal (Likert) scale, and free-response prompts concerning overall perceptions of start-up experience, motivating and de-motivating factors during the start-up period, and satisfaction with specific portions of the start-up process. Survey questions are shown in Supplemental Figure 1. Survey respondent demographics and survey responses were descriptively analyzed. Free response data were analyzed by categorizing comments as positive, negative, or neutral in sentiment; these categorizations were independently scored and subsequently agreed upon by two authors of this paper.

3. Results

3.1. DOSE trial site demographics

The DOSE trial selected 20 sites through feasibility questionnaires. Sites included academic children’s hospitals, both university and community based. All sites had a pharmacy that could prepare blinded study drug 24 hours per day and had a method for storing frozen pharmacokinetic samples. Most sites (18/20; 90%) used dexmedetomidine and fentanyl per standard of care in patients requiring ventilator support. Most sites PIs (12/20; 60%) reported having conducted > 5 pharmacokinetic studies in the past, though a quarter of site PIs had not previously performed a pharmacokinetic study at their current site. The selected sites had a median (quartile 1 [Q1], quartile 3 [Q3]) of 20 (14, 25) pediatric intensive care unit beds. Most sites (19/20; 95%) had at least one dedicated research coordinator who was available for the trial. Sixteen sites (80%) did not require IRB approval before a contract or budget was in place.

3.2. Site activation milestones metrics

All 20 sites met the benchmarks of submission of regulatory documents within 25 days and completion of local human resource protections context within 30 days. Eleven sites had partially executed contracts within 31 days from the time the regulatory package was received by the site. Contracts were fully executed by a median (Q1, Q3) of 50 days (34, 63) from receipt of the CTA. Only one site (5%) met the goal of activation within 55 days following the receipt of the CTA, but 10 (50%) met the goal of activation by 84 days from receipt of the CTA. Across all participating sites, the median (Q1, Q3) time from receipt of regulatory documents to activation was 82 days (68, 113), and the days to activation across all sites ranged from 55 to 237 days. The probability density distribution of time to 1) regulatory packet completion, 2) contract execution, and 3) site activation across all sites is shown in Figure 2.

Figure 2. Distribution of Key Start-up Activities by Site.

Probability density plot of time to start-up activities from n=19 sites. Time from regulatory packet completion (blue) was measured from the date that the regulatory packet was sent. The remainder of milestones shown were measured starting from the date that the clinical trial agreement (CTA) was sent. The dashed lines represent the median time for each milestone. Abbreviations: CTA: Clinical trial agreement.

Ultimately, the competitive activation mechanism was abandoned after the first ten sites had activated, and all 20 sites were allowed to complete activation. One site chose not to continue to activation because the dose proposed in the initial dexmedetomidine cohort was too discrepant from their usual practice, and the rapid start-up processes employed in the trial were not suitable at their site. Consequently, 19 sites continued participation.

3.3. Subsequent patient recruitment

The first patient was enrolled 63 days after the first site was activated, and the median (Q1, Q3) time from activation to first enrollment across all sites was 113 (61, 188) days. Across individual sites, the number of days from activation to first enrollment ranged from 20 to 210 days. The median (Q1, Q3) time from study initiation (regulatory documents sent to sites) to first enrollment across all sites was 194 (171, 276) days.

3.4. Site start-up survey results

The post-site activation survey was sent to 70 individuals across all activated sites. Thirty respondents (43%) completed the survey, including 11 site PIs, 13 study coordinators, 4 regulatory coordinators, and 2 who described their role as “other”, but did not specify their role within the survey. Rating the overall experience with study start-up on a 10-point scale (with 10 being the best experience possible), the average (standard deviation) score across all respondents was 7.9 (± 2.4).

Respondents were asked to rank the most effective motivating factors for rapid start-up. The additional funds to meet milestone-based start-up activities was most frequently ranked the highest motivating factor, while the competitive activation mechanism was most frequently ranked the lowest (Table 1).

Table 1: Site-reported Ranking of Motivating Factors for Efficient Start-Up.

Ranking was calculated using the mean score for each factor, on a 1–5 ordinal (Likert) scale of least to most motivating.

Start-up activity	Rank Order	Mean Rank (SD)
Additional funds for completing milestones by specific dates	1	3.93 (1.41)
Site PI’s interest in the particular scientific study/question	2	3.80 (1.33)
The weekly calls with site navigator	3	3.53 (1.31)
The one-on-one attention provided by site navigator start-up specialist	4	3.50 (1.28)
The competition to complete activation within a limited number of weeks	5	3.33 (1.42)

Abbreviations: PI, principal investigator

When asked about satisfaction of different study start-up components, sites were most satisfied with weekly site navigator calls and least satisfied with competitive activation (Table 2). Forty-three percent of respondents stated they would partake in another trial using the competitive activation mechanism (13/30), while 43% were unsure (13/30) and 13% would not (4/30).

Table 2: Site-reported Ranking of Satisfaction with Various Study Start-up Factors.

Ranking was calculated using the mean score for each factor on a 0–10 (Likert) scale of least to most satisfied.

Start-up activity	Rank Order	Mean Rank (SD)
The weekly calls with your Site Navigator to keep your site on schedule	1	9.03 (2.46)
The additional funds paid for completing the start-up milestones	2	8.83 (2.93)
Single IRB process	3	8.20 (2.75)
The contracting process	4	7.97 (2.37)
The compressed timeline of completing activation within a few weeks	5	7.80 (2.69)
The competitive activation process	6	6.43 (3.32)

Free-text responses were requested as part of the survey querying the three novel start-up mechanisms. Of the 12 responses about the milestone-based funding allotments, one was negative, one was neutral and 10 (83%) were positive. Several of the positive comments focused on the milestone-based funding allotment as an incentive for efficient activation; for example, one respondent commented, “despite this being a rough first attempt at the accelerated study startup process, I do believe there is a ton of potential with this system. The results are undeniable, this is the fastest a study has ever been started in our office, and there was absolutely a strong sense of determination to get the study started quickly. The process needs a lot of work and improvement, but it should absolutely be iterated on and used to help research get off the ground quickly and effectively.”

Site navigators, overall, were perceived as helpful for a more expeditious start-up process; of 12 free responses, nine (75%) were positive. Site navigators were often viewed as a helpful resource: “consistently responsive in email/phone communication, and always appropriately referred questions to others if unsure of the answers. Having weekly calls was a great way to keep on track and set goals for the week…” Others, however, described a need for agendas during the weekly calls and ensuring that the site navigators were organized and not creating additional inefficiencies in the process. Although useful, several survey comments about site navigators noted repetitive requests for paperwork. One respondent commented that “there were many times where we would have to repeat information that [was] already collected… it was frustrating.”

Competitive activation was overall negatively perceived and not considered a motivator for efficient start-up. While a few responses had positive sentiments (2/14, 14%), most were negative (10/14, 71%). For example, one respondent reported that the competitive activation portion “certainly did increase motivation to speed the process along locally. However, it created an unreasonable amount of stress on the site. Many factors of the activation process … are beyond the site’s control and created the fear that we might not be able to participate because of those factors…”

4. Discussion

Study start-up can be critical to the success of a trial, but delays in start-up are frequent. Delays remain notable even where established research networks provide scientific, technical, and administrative infrastructure to conduct trials safely, effectively, efficiently, and ethically. Given this critical need, we implemented three mechanisms in DOSE to enhance the efficiency of study start-up. Adding competitive activation, site navigators, and milestone-based funding to the start-up process led to a median start-up time of 82 days. This is roughly one-third the start-up time observed in other pediatric clinical trials (unpublished data provided by Best Pharmaceuticals for Children Act Data Coordinating Center for the Pediatric Trials Network, June 3, 2021) [11], and shorter than the average 120 to 180 days in adult trials reported in the literature [5–7]. Moreover, of the 20 selected sites, 55% met the goal of site activation within 84 days from receiving initial regulatory documents. This demonstrates that the additional mechanisms executed in the DOSE trial may expedite study start-up processes.

Though the individual impact of each of the three mechanisms employed in DOSE is difficult to determine, the survey results indicated that milestone-based funding allotments and site navigators were perceived as very useful. Although useful, several survey comments about site navigators noted repetitive requests for paperwork. A possible way to address the issue of repetitive paperwork requests is using clinical document exchange portals [12, 13]. Centralizing and streamlining document exchange through document exchange portals has been shown to reduce start-up time and save costs, [12] and may be a worthwhile intervention when expeditious start-up is needed.

Although not a formal intervention, an additional highly ranked motivating factor was site PI interest. The feasibility survey distributed by the CTSA Hubs and their Liaison Teams allowed for sites to self-select and opt-in, however, it was obvious through personal communications with the sites, that the study question was pertinent to the site investigators, many hoping to generate data to guide clinical decisions at the bedside. We think this factor could be an untapped resource in clinical trials as it is the site level study team that drives the ultimately imperative site-specific activities such as screening and enrollment. Although best practices for site engagement have not been extensively studied, examples of novel and effective site engagement practices that differ across study phases have been reported in a longitudinal multi-site clinical trial [14]. This identified motivating factor also prompted our creation and implementation of a Professional Practice and Quality Improvement (Part 4) of Maintenance of Certification project [15], in response to the decline of screening and enrollment rates during COVID. This quality improvement project provided credit applied towards American Board of Pediatrics certification and was targeted to engage sites to improve site screening and recruitment.

Conversely, competitive activation was ranked poorly in terms of motivation and satisfaction. Though not preferred by sites, it is difficult to determine how much it contributed to sites completing milestones quickly. Ultimately, the competitive activation mechanism was abandoned shortly after the second activation deadline date passed (day +84). Ten sites had already activated by that time point, and initial recruitment was slower than anticipated. Study leadership recognized that this study would have challenges with enrollment, as it targeted critically ill children; additionally, since a substantial investment had already been made in the sites that were proceeding with start-up, the decision was made to abandon the competitive activation mechanism and allow all sites to activate and enroll. As a generally less favored mechanism by sites from free text survey feedback, we would recommend consideration of other potential motivating factors for expeditious start-up.

Furthermore, although study start-up is critical, ensuring subsequent enrollment is just as important for successful trial completion. Despite rapid start-up, the time to the first DOSE patient enrolled was unexpectedly long and less than half of sites met recruitment goals. Slow enrollment was addressed through changes to the study design, specifically changes targeting inclusion and exclusion criteria, adding the creation of an electronic consent platform, and implementation of a Professional Practice and Quality Improvement (Part 4) of Maintenance of Certification project as noted above [15].

There are limitations to the data presented herein. First, the survey response rate was modest, and there may be response bias in those that opted to respond. Additionally, multiple interventions were implemented simultaneously in an attempt to facilitate start-up efficiency; therefore, which individual intervention (milestone-based reimbursement, competitive activation, or site navigators) had the greatest impact on start-up cannot be determined. Nevertheless, site-specific opinions and preferences for these interventions clearly reveal a preference for milestone-based reimbursement and site navigator assistance. It should also be acknowledged that there are other potential interventions that could be considered to expedite study start up, including the use of consortium master agreements. Contracts were executed 1:1 for each site for the DOSE trial; using a master agreement might have afforded additional time savings. Second, the attitudes towards the various start-up mechanisms are subject to response bias, but whether responses differed by role at the site could not be determined. Finally, the current DOSE experience was compared to historical PTN trial data because readily available published literature on durations of pediatric clinical trial start-up metrics are lacking; however, PTN trial start-up metrics may not be characteristic of pediatric trials outside of PTN. Despite these limitations, the study start-up lessons learned from DOSE have important implications for the planning and execution of future pediatric trials.

5. Conclusion

Innovative methods can and should be implemented and analyzed to improve efficiencies in clinical trials. The interventions used in DOSE may have effectively shortened the duration of time to activation, but each intervention had clear benefits and downfalls. Although the novel start-up mechanisms in DOSE proved successful, opportunities to improve remain. Pediatric researchers must continuously improve clinical trial operational standards to facilitate rigorous, novel, data-driven therapeutics for our pediatric patients.

Highlights:

• Three novel incentives reduced study start-up times in a multi-site clinical trial

• Funds assigned for achieving start-up milestones were favored by study sites

• Site navigators effectively guided sites through start-up

• Competitive activation was the least preferred incentive by study sites