Site engagement for multi-site clinical trials

Abstract

Multi-site clinical trials are essential within medical practice to help drive reliable and generalizable knowledge on advancing medical treatments. Although the success of multi-site trials is significantly dependent on local clinician and site research teams, best practices for engagement of site teams, or “site engagement,” has not been extensively discussed. Site engagement centers on including sites in the planning and implementation of clinical trials to promote trial enrollment, compliance, and applicability to local contexts. Using a case example from the RadComp Trial, a longitudinal, multi-site clinical trial, novel site engagement practices are provided across three major research phases. In the Planning Phase, site engagement builds partnerships and commitment by active elicitation of information on site specific processes and feedback on trial design. In the Conducting Phase, sustained engagement encourages bi-directional communication and facilitates learning networks for enhanced site performance. In the Dissemination Phase, site and community partnerships are leveraged to create locally designed dissemination plans for broader scientific reach and impact. Site engagement practices discussed in this paper can be replicated or molded for application in other multi-site clinical trials.

1. Introduction

Clinical trials play a central role in determining how medicine is practiced [1]. National attention has focused on engaging patients in the design and conduct of trials to increase accrual, retention, and applicability to patient populations [[2], [3], [4]]. However, the engagement of clinicians and research teams at trial sites has not been extensively discussed despite their primary role in ensuring clinical trial success. Engagement of clinicians and research teams, what we term “site engagement,” is especially important in large multi-site clinical trials where variability in performance of trial operations across sites can dramatically impact participant accrual and retention, data compliance, and even internal validity of research findings [5]. Multi-site clinical trials have driven the approval of new drugs and devices and important changes in medical practice [[6], [7], [8]]. Compared to single-site trials, multi-site clinical trials offer key advantages, such as larger and potentially more representative samples [[9], [10], [11]]. Yet, prominent challenges include the need to standardize across multiple sites with different personnel and clinical practices, and maintain scientific integrity and commitment [6,9,10,12]. These challenges necessitate active site engagement throughout the study lifespan.

The Patient-Centered Outcome Research Institute (PCORI), is an independent, nonprofit organization that funds research aimed at providing patients, caregivers, and clinicans with evidence-based information to aid in healthcare decision making. The PCORI Engagement Rubric can be used to guide engagement practices in clinical research [13]. The rubric focuses on engagement of "patient partners" and "stakeholder partners" and presents potential engagement activities within three major phases of research: planning phase, conducting phase, and dissemination phase. This article expands on the PCORI Engagement Rubric to present novel approaches for site engagement and lessons learned from a multi-site, pragmatic trial, across the three study phases.

The RadComp Trial is a large, multi-site trial funded by PCORI, entitled: “Pragmatic Randomized Trial of Proton vs. Photon Therapy for Patients with Non-Metastatic Breast Cancer: A Radiotherapy Comparative Effectiveness (RADCOMP) Consortium Trial.” RadComp is composed of 25 proton networks nationwide with their affiliated treatment facilities and a Coordinating Center at the University of Pennsylvania. Participants are randomized to receive either proton radiation therapy or photon radiation therapy and are followed longitudinally for cardiovascular morbidity and mortality, cancer control outcomes, and health related quality-of-life [14]. Participant enrollment will be complete by February 2022 and then participants will be followed for ten years for long-term study outcomes. The trial is currently in Year 5 and actively recruiting participants.

The RadComp Coordinating Center serves as the central contact point for all sites and manages the overall progress of the trial, including: facilitating IRB approvals, monitoring study accrual, compliance, and patient data, reporting clinical events and study deviations, and engaging study participants, site teams, and stakeholders. Local sites are responsible for enrolling participants to the trial, treatment delivery, and clinical follow-up. With RadComp's pragmatic design, sites are not confined to treat according to strict, study specific treatment guidelines and are allowed flexibility to treat patients per their institution's clinical practice, leading to diversity in site practices [15]. Site engagement efforts are managed by the Coordinating Center with the overall aim to facilitate communication and collaboration with sites to promote study accrual, retention, and compliance.

The purpose of this article is to present novel approaches to site engagement using examples from the RadComp Trial. Site engagement activities are framed within the three major study phases, as delineated in the PCORI Engagement Rubric. While this case example is specific to clinical and comparative effectiveness research these strategies can also be applied to non-clinical multi-site trials.

2. Site engagement by study phase

The lifespan of a clinical trial can be broken up into three major phases: planning, conducting, and dissemination [13]. The Planning Phase sets the foundational framework for trial conduct, including building partnerships with sites, ensuring standardization and integrity, and setting up the IRB process. The Conducting Phase, in which the trial is actively accruing participants, involves continued IRB coordination and maintaining a variety of touch points with local sites. Lastly, in the Dissemination Phase, study findings are concluded and communicated to the general public with the intention to inform medical practice. Various site engagement activities should be utilized depending on the needs and goals of the study phase (see Table 1, Table 2, Table 3).

Table 1.

Planning phase site engagement activities for multi-site clinical trials.

Activity	Study Role Target	Conventional Approach	Novel Approach	Evaluation Criteria
Institutional Review Board (IRB) Coordinationa	•Clinical Research Coordinators (CRCs) •Local IRB •Regulatory Specialists	•Sites have the option to rely or not rely on the Coordinating Center's IRB •Non-relying sites are responsible for submitting regulatory documents to their IRB	•Coordinating Center tracks IRB deadlines and sends reminders to site •Coordinating Center provides document templates for site teams to fill in with site specific details	•Time to IRB approval and renewal •Compliance with IRB policies
Site Initiation Engagement	•CRCs •Site Principal Investigators (PIs) •Research Nurses •Data Managers •Regulatory Specialists •Physics team •Supervisors/Research Managers	Coordinating Center trains site teams on study protocols, such as during site initiation visits	•Encourage two-way communication amongst sites and the Coordinating Center •Site initiation visits include presentations with an interactive question format •Open membership of technology specialist committees (e.g. physicists)	•Attendance •Time to activation •Report of problems related to site activation
Initial Site Survey	•Site PIs •Regulatory Specialists •Clinical Research Coordinators •Supervisors/Research Managers	Not required	•Set of pre-determined questions provided to each new site •Elucidate site opportunities, challenges, and communication preferences	•Response rate •Follow-up survey and/or calls to elucidate site changes

^aOngoing activity across all study phases.

Table 2.

Conducting phase site engagement activities for multi-site clinical trials.

Activity	Study Role Target	Conventional Approach	Novel Approach	Evaluation Criteria
Refresher Training	•CRCs •Site PIs •Data Managers •Regulatory Specialists •Supervisors/Research Managers	Not required	•Designed for new site personnel to address staff turnover •Gather site feedback to help tailor future trainings	•Attendance •Change in accrual or compliance rates before and after training
Site Visit	•CRCs •Site PIs •Medical Directors •Research Nurses •Data Managers •Regulatory Specialists •Supervisors/Research Managers	Not generally required	PI and/or Project Manager meet with site study personnel in-person, discuss progress, and troubleshoot challenges	•Number of site visits •Change in accrual or compliance rates before and after visit
Monthly Newsletters	•CRCs •Site PIs •Research nurses •Advocacy partners •Stakeholders •Supervisors/Research Managers	Not required	•Provide study updates •Recognize site study personnel •Disseminate new research •Feature new sites and/or site personnel •Highlight advocacy resources and events that site teams can share with patients •Give call-to-actions	•Email open rate and click-through rate •Follow-up actions on call-to-action items •Number of times newsletter features were forwarded
Monthly Group Calls	•CRCs •Site PIs •Research Nurses •Data Managers •Physics team •Regulatory Specialists •Supervisors/Research Managers	•Site members share study updates and reminders •Periodic calls to train staff on protocol modifications	•Shared learning opportunity to troubleshoot challenges and streamline workflow •Individual sites lead discussion on areas of expertise •Disseminate lessons learned back to site teams	•Attendance •Frequency of bi-directional discussion vs. one-way instruction •Number of shared lessons or resources
Individual Site Calls	•CRCs •Site PIs •Supervisors/research managers	Not required	•Hold calls with site PIs and at least one CRC from each site •Summary of findings provided back to sites through in-person meetings, monthly calls, and tip-sheets	•Attendance on calls •Number of calls held •Number of follow-up calls needed to expand on identified opportunities or challenges
Acts of Gratitude	•Site PIs •CRCs •Advocacy Partners •Stakeholders •Executive committee •Physics team •Adjudicators •Reviewers •Any other supporting staff	•Thank you letters •Presentation of formal awards •Inclusion in “acknowledgement section” of publications	•Formally addressed thank you letters •Monthly recognition emails •Newsletter feature •Social media feature	•Number of letters, cards, emails, and awards given

Table 3.

Dissemination phase site engagement activities for multi-site clinical trials.

Activity	Study Role Target	Conventional Approach	Novel Approach	Evaluation Criteria
Presentations and Publications	•Executive committee •Regulatory committee •Physics committee •Advocacy partners •Stakeholders •Site PIs •CRCs	Physician preparation of conference posters and presentations at large academic and medical conferences	Site team representatives and advocacy partners present at advocacy and local community organization conferences	•Number of paper and poster submissions •Number of acceptances •Number of reposts or public shares •Open and click-through rates on virtual campaigns
Multi-Media Material Development	•Executive committee •Regulatory committees •Advocates •Stakeholders •Site PIS •CRCs	•Publication in scholarly texts •Formal reports to funding agency	•Provide results in a brief, easy-to-read format •Use a mixture of formats •Site teams share findings on their organization's webpages, social media, or other media outlets	•Number of reposts or public shares (physical or electronic) •Open and click-through rates on virtual campaigns •Engagement on social media posts (likes, comments, shares)
Advocacy and Community Group Involvement	•Advocacy partners •Stakeholders •Site PIs •CRCs	Advocacy groups take the initiative to share latest research to patient populations	•Actively elicit input on drafted materials •Advocates advise on content and format of dissemination materials •Identify appropriate dissemination outlets	•Number of reposts or public shares (physical or electronic) •Open and click-through rates on virtual campaigns •Engagement on social media posts (likes, comments, shares)

2.1. Planning phase

Building strong partnerships with trial sites starts by engaging them early on during the initial Planning Phase of the trial. During this phase, one recommendation from the PCORI Engagement Rubric is to design the study in a way that minimizes disruptions to patient and stakeholder study participants [13]. Similarly, RadComp's novel site engagement approach during this phase includes activities that gather information from sites and streamline study processes for local site teams (see Table 1).

One of these approaches focuses on Institutional Review Board (IRB) approval, which from the onset, is a key area of early collaboration in multi-site trials between local sites and the Coordinating Center. To date, trial sites have had the option to rely or not rely on the Coordinating Center's IRB. The National Cancer Institute recently introduced the Central Institutional Review Board (CIRB) Initiative that will further streamline this process, but up to this point the standard practice is that the Coordinating Center provides necessary documents to sites to be formatted and submitted to their IRB [16]. RadComp takes an extra step to track site specific IRB deadlines and send reminders to sites when deadlines are approaching. Templates of study documents are also supplied in which sites can fill in site specific details. This engagement helps alleviate administrative burden and delays on local site teams and is carried out throughout the lifespan of the trial, as study modifications are needed (see Table 1).

In multi-site trials, all sites must participate in and abide by a set structure of trainings and procedures to ensure a level of standardization, good clinical practice, and scientific integrity. Engagement with site teams during the initiation process helps recognize site teams as valuable members of the trial consortium. RadComp leverages the site initiation process as an opportunity for active, two-way communication. For instance, during site initiation visits the study Project Manager and National Principal Investigator uses presentations that include an interactive question format to prompt site responses. These visits can be conducted in-person or as a virtual meeting. Presentation materials are provided to site teams ahead of time for preparation. Also during the site initiation process, site physicists present their physics plan to the RadComp Physics Committee and receive feedback prior to opening the trial at their site. Membership to the RadComp Physics Committee is open to all study site physicists. Encouraging discussion in this way promotes buy-in and commitment from site teams.

Another early novel engagement activity utilized by RadComp is the provision of an initial site survey to each new site. The surveys asked site investigators and clinical research coordinators for their insights related to site specific contexts and challenges (see Table 1). It contained a series of pre-determined questions to elucidate site strategies for: promoting the trial, participant engagement, potential challenges, and site preferred methods of contact from the Coordinating Center. This allows the opportunity for the Coordinating Center and individual sites to work together, at the forefront, to strategize on how to address possible challenges and optimize opportunities. For instance, sites have identified opportunities such as partnerships with local organizations to leverage participant recruitment and language translation services to reach minority populations. Preemptively eliciting feedback from sites early on is important for success given each site will have its own unique opportunities and challenges.

2.2. Conducting phase

The nature of engagement takes a shift in the Conducting Phase. The PCORI Engagement Rubric states activities for this phase include: revising study materials and protocols, study recruitment, data collection and analysis [13]. Emphasis is placed on participant recruitment protocols, ongoing IRB coordination, and monitoring data compliance. The RadComp Trial is currently in the Conducting Phase with ongoing participant enrollment and data collection. Novel approaches during this phase include activities that maintain ongoing communication, promote shared learning across sites, and recognize sites for accomplishments for sustained commitment (see Table 2).

The conventional approach in multi-site trials to maintain communication in this phase is through monthly calls to provide study updates, reminders, and next steps. Since Year One of the trial, RadComp has held bi-weekly or monthly group calls with investigators, technology specialists (e.g., physicists), and research coordinators. In a novel way though, RadComp's calls are leveraged for active discussion between sites to promote shared learning. Sites are encouraged to provide their own experiences, through the opportunity to present on the call and ask questions of one another. Eliciting site communication in this way has effectively created informal “learning networks” across sites leading to important changes in trial conduct. For instance, open discussion in Year One led to the removal of a pregnancy test as a study entry requirement, because sites noted it was redundant to standard clinical practice. Another example of the impact of learning networks is from the collaboration amongst the RadComp Physics Committee. Physicists have learned best practices from one another on how to treat patients with existing tissue expanders in the breast, rather than necessitating surgery to remove the expanders prior to radiation – a previous barrier to enrolling into the trial. Thus, RadComp study sites nationwide have expanded radiation treatment to include tissue expanders.

RadComp also holds periodic Refresher Trainings and Individual Calls during this phase to maintain ongoing communication. Refresher Trainings were implemented to mitigate communication gaps that can occur during staff turn-over. The trainings elicit administrative changes at sites, build relationships with new staff, and address questions or concerns. Individual calls have also been conducted to allow sites to share personal experiences in a more private space. RadComp conducted sixteen individual calls with physicians and fourteen calls with coordinators in Year Three and twenty-three calls with site investigators in Year Four. Findings from site principal investigator calls identified insurance barriers, patient treatment preferences, and travel to treating facilities as challenges to enrollment. To address these challenges, sites were supplied a medical necessity template with study specific language to submit to insurance providers, language for how to communicate the trial to patients with equipoise, and information on temporary lodging resources provided by a national advocacy partner. Individual coordinator calls promoted shared learning as coordinators provided on-boarding techniques for new coordinators, a patient eligibility checklist, and answers to frequently asked questions from patients. These strategies were then compiled by the Coordinating Center into a tip-sheet and disseminated back to coordinators for reference.

Aside from virtual communications, RadComp also conducts one in-person site visit to each site. While some multi-site trials conduct site visits as part of the initiation or monitoring process, it is not the norm. RadComp site visits are conducted for monitoring purposes and to check in with sites on general study progress. Members of the Coordinating Center most familiar with data compliance meet with the site principal investigator and clinical research coordinators. On-site visits are beneficial to multi-site trials to help facilitate more targeted, in-depth discussions with site teams. During these visits, sites with higher accrual have commonly reported difficulty maintaining high compliance rates due to the heavier workload. In response, the Coordinating Center worked with these sites to develop a plan that breaks up tasks and establishes a data entry timeline to boost compliance.

In sum, to date, site engagement has been utilized to collaboratively address site challenges during the Conducting Phase through active two-way communication and shared learning. The last novel site engagement element in this phase is acknowledging sites for best practices and achievements. Traditionally, trials have taken a retrospective approach by acknowledging contributors at study completion; whereas RadComp recognizes sites on an on-going basis using a variety of site engagement activities for positive reinforcement (see Table 2). Sites are acknowledged for best practices, achievements in enrollment, compliance, and patient engagement, and notable milestones (e.g. 20, 50, 100 participants enrolled). Since Year One, study newsletters have been emailed monthly to the trial consortium, including stakeholders, site physicians, clinical research coordinators, physicists, and other clinical research staff across study sites, to relay study updates and site accomplishments. Newsletters routinely experience a significant 30% open-rate and 20% click-rate on content. RadComp also sends out monthly recognition emails and delivers formal thank you letters to principal investigators and high enrolling physicians (e.g., top 20% of enrolling investigators) directly from the trial's National Principal Investigator. Lastly, achievements are also shared on the trial's social media pages.

Sustained site engagement throughout this phase has boosted study enrollment, retention, and compliance. RadComp's average monthly accrual and compliance rates have increased. Most recent available data demonstrates an average monthly accrual increase of 2.9% in Year Three and another 0.7% in Year Four. Form submission data compliance also increased: 5% at baseline (85%–90%), 23% at 1-month (66%–89%), and 35% at the 6-month time-point (56%–91%) in Year 3, and another 7% at baseline (90%–97%), 10% at 1-month (89%–99%), and 4% at the 6-month time-point (91%–95%) in Year Four. According to one site coordinator, the trial's deliberate engagement efforts and open-door policy during this phase has been unique compared to other multi-site trials.

2.3. Dissemination phase

When disseminating trial methods and results, the PCORI Engagement Rubric suggests that patient and stakeholder partners participate in the dissemination of study results using manuscripts, presentations, and other opportunities they may identify [13]. A conventional approach is to involve site investigators as co-authors of publications and poster abstracts at large academic and medical conferences and formal reports to funding agencies. Unlike single-site trials, multi-site trials stretch across larger geographic and cultural regions, which could necessitate alterations in the language and presentation of research findings to fit local populations.

The RadComp Trial will enter the Dissemination Phase once target enrollment is complete. At that time, secondary study outcomes for health related quality-of-life and cancer control can be reported, then primary outcomes of effectiveness of proton therapy vs. photon therapy with long-term follow-up. Novel approaches including partnership with active advocacy organizations and development of multi-media materials will be utilized in addition to the aforementioned conventional approaches (see Table 3).

RadComp will engage other site team representatives and study advocacy partners to present at local advocacy conferences that cater more towards patient populations and lay audiences. Research staff can present at grand round presentations at community health systems and academic medical centers to influence patient care. Local advocacy partners will also help reach a broader network of communities. The study will actively elicit advocate and stakeholder feedback on drafted dissemination content and gather advice on appropriate formats and outlets for disseminated material.

The format of disseminated content will be expanded beyond the conventional journal article and poster format to digestible, multi-media formats. RadComp will partner with advocacy organizations to provide patient populations with information in lay friendly formats, such as media briefs, pictures and graphs, and video representations that are more accessible than academic text. Longitudinally, RadComp's social media pages will also serve as an accessible, public outlet for disseminating study findings and continuing site engagement. Social media posts can be shared on local site and advocacy partner's webpages to help disseminate findings to their followers. Taken together, site engagement will be utilized to incorporate input from local communities to help craft messages appropriate for their context, in easy-to-digest formats, for a more robust dissemination plan.

3. Conclusion and recommendations

Site engagement should be a priority for multi-site clinical trials as it can enhance the performance of individual sites; and thus, the overall success and applicability of the trial. Site engagement practices should be maintained throughout the duration of the study with activities shifting depending on the study phase and research role targeted and continuously involving sites in a collaborative manner. Multi-site trials are encouraged to think beyond working with local sites by way of regulatory requirements only and experiment with innovative site engagement ideas to promote performance.

In the Planning Phase, engaging with sites early and eliciting their input will build relationships that carry throughout the remainder of the trial. Novel strategies include assisting sites with IRB coordination by supplying templates and tracking deadlines, encouraging two-way communication during site initiation visits, and provision of an initial site survey. In the Conducting Phase, site engagement should be maintained to boost enrollment and compliance. Sites should be encouraged to share their unique experiences and facilitate a learning network for sites to glean lessons from one another. Novel site engagement activities, such as refresher trainings, on-site visits, monthly group calls, and individual calls, will help maintain ongoing communication and promote shared learning across sites. Furthermore, attention should be paid to highlighting site best practices and achievements to maintain site commitment. For the Dissemination Phase, novel approaches include the active involvement of local site teams and advocacy organizations to craft messages and provide guidance on multi-media formats and outlets appropriate for their context.

A mixture of conventional and novel site engagement approaches should be implemented in multi-site trials. The RadComp Trial's approach to site engagement has culminated in collaborative, learning networks that promotes the trial's high monthly study accrual and boosted data compliance rates. The collection of site engagement practices provided here demonstrates a blend of conventional and novel approaches that can be translated to other multi-site trials to boost overall site performance. A next step would be to determine which engagement strategies foster the best balance between increasing site performance while minimizing burden on busy research teams.

Funding/support

Research reported in this work was partially funded through Patient-Centered Outcomes Research Institute (PCORI) Award PCS-1403-12804. The views, statements, and opinions presented in this work are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee.

Declaration of competing interest

Authors have no conflicts of interest to disclose.