The Complexity of Conducting a Multi-Center Clinical Trial: Taking it to the Next Level Stipulated by the Federal Agencies

Abstract

Randomized controlled trials (RCTs) are becoming increasingly difficult to organize and conduct efficiently. This, in turn, hinders the ability to derive highest level of evidence. Often investigators forget or remain unaware of essential practices that will help them fulfill their study goals. This manuscript emphasizes the common issues that a study team experiences during the planning and conducting of RCTs. We sought to share remedies to overcome these issues with the experience garnered in conducting several multicenter clinical trials and observational studies. Additionally, we list resources from sponsors like the National Institutes of Health and the United States Food and Drug Administration that study teams can apply to undertake studies effectively,

Randomized controlled trials (RCTs) provide the highest level of evidence to compare treatments or to evaluate therapeutic efficacy. Evidence, however, is only as good as the study that generated it. RCTs can take many years to plan and successfully fund, but when it comes to execution, there are still many roadblocks that a study team will encounter. RCTs can also yield biased results if they lack methodological rigor.^1–5 A systematic review of RCTs in plastic surgery concluded that despite an increase in the number of RCTs published, the methodological and reporting quality still needs familiar improvement.⁶ Therefore, it is critical for investigators to conduct studies with high quality and be prepared when pitfalls arise.

This manuscript aims to depict our experiences in conducting RCTs.^7–9 It elaborates on how we planned, executed, overcame the encountered issues. WRIST (Wrist and Radius Injury Surgical Trial) is a multicenter RCT evaluating three types of distal radius fracture (DRF) treatment. WRIST was conducted at 24 centers in the United States, Canada, and Singapore in patients age 60 years or older with a DRF. This manuscript also discusses the resources provided by the most common funding agencies. The National Institutes of Health (NIH), the largest biomedical research funding organization, strives to improve health, promote longevity, and reduce disease burden. NIH have several clinical research resources to achieve its goals.¹⁰ (Table 1) Beginning in the 1970s, the United States Food and Drug Administration (FDA) also instituted regulations for the conduct of clinical trials involving human subjects.¹¹ These regulations include measures of good clinical practice and human subject protection.

Table 1.

NIH Clinical Trial Resources

NIH Clinical Research Trials and You

• A website to help those who want to learn more about clinical trials

Children and Clinical studies

• Provides information about who can participate in trials, benefits, risks, and trial safety

Clinicaltrials.gov

• A database of privately and publicly funded clinical studies conducted around the world

NIH Office of Human subject Research

• Provides information for NIH Research staff, funded investigators, and interested volunteers

Clinical Research and Bioethics Policy

• Provides research and training on all bioethical-related issues

Issues prior to the beginning of a RCT

Site recruitment

Sites commit to a study for altruistic reasons, a chance to make an impact and to influence practice by helping patients. However, the less enticing parts of the study such as rigorous dedication to the study protocol and an intense commitment to recruit every eligible patient, in particular, the older patient in the WRIST trial, are more difficult to maintain through the study period. Sites also back out when they think funds provided are insufficient for them to conduct research in accordance with their institution requirements. Overall, successful site recruitment relies on effective communication about the expected time commitment, funds provided, and the expected role and responsibilities for the study progress. The WRIST principal investigator (PI) recruited sites based on the site investigators’ experience, research abilities, resources at their facility, and their performance on prior collaboration projects. WRIST PI was familiar with the experience and research abilities of site investigators owing to working in the same specialty and being aware of their prior work through their publications. However, he relied on the site with regards to facility resources such as adequate staff to conduct research.

Another issue to be prepared for is investigator changes. For example, an investigator may move to a different institute after the study starts. Be prepared to decide whether to continue at the existing site with a new investigator or to move the study to a different site with the familiar investigator. Either way, this will result in delays with regards to regulatory approval and onboarding new personnel.

WRIST planning and Kick-off meetings

WRIST was developed with many years of PI’s experience, vision, and collaboration with other investigators who shared the same research goal. Years of caring for DRF patients, literature reviews¹², and ra publications^13–16 translated into a unique multicenter RCT. WRIST secured a planning grant from the NIH to fund project development. Inputs from other site investigators were obtained through meetings at national conferences. A dedicated WRIST planning meeting was executed to discuss study feasibility and finalize the protocol and study assessments. Before starting enrollment, WRIST investigators, coordinators, the study statistician and epidemiologist met at a kick-off meeting. PowerPoint presentations informed the WRIST team regarding the study conduct from screening, enrolling, follow-up, study assessments, protocol, Adverse Event (AE) documentation, data entry and communication between sites and coordinating center. These meetings helped the group to get familiar with the study and understand its importance. The Agency for Healthcare Research and Quality provides a resource on how to conduct a kick-off meeting that includes meeting logistics and a sample meeting agenda.¹⁷ (See Document, Supplemental Digital Content 1, which demonstrates an example kick-off meeting agenda, INSERT HYPERLINK HERE) The kick-off meeting is also a good venue to setup study steering committees (participant eligibility, protocol, and safety advisory committees) depending on the needs of the study. Finalize the role of the committee members with regards to their time commitment and effort to review, oversee, and make decisions when required. WRIST set up a protocol review committee prior to the beginning of the study.

Pilot studies

An important part of the RCT planning process, a pilot study can be started at only one site, usually the Coordinating Center. The study team can get a first-hand glimpse of any potential barriers to enrollment or problems implementing the study protocol as written. If WRIST had a pilot study period, we could have performed trouble-shooting on our biggest challenge – enrollment. Given this extra time we could have involved additional sites, used more surgeons per site at existing sites or implemented alternative recruitment methods and materials.

Sample size estimation

Recruitment goal can be assessed by identifying the actual number of patients treated for the clinical condition at participating sites over the last few years. Obtain this information from their health system records; relying on investigators’ memories will lead to misestimation of potential participants. Additionally, allow room for patients who will be ineligible, lost to follow-up, or withdraw.^18,19 WRIST encountered an issue with slow recruitment 2 years after enrollment began. The Data Safety and Monitoring Board (DSMB) advised futility assessment by conducting a conditional power analysis based on the observed effect size at the interim. The study statistician revised the required sample size while maintaining ability to detect differences between comparative treatment groups as proposed in the study protocol and WRIST was able to achieve this goal successfully.

Regulatory approvals

With WRIST, the time to receive a final Institutional Regulatory Board (IRB) approval from our institution was much longer than typical 3–6 months. This also delayed participating sites, many of whom required coordinating center approval for their institutional IRB. For WRIST, Canadian sites obtained their Research Ethics Board approval and had Federalwide Assurance registration to participate in research that is US based.²⁰ The process of obtaining these approvals was similar to that of an IRB approval in the US. Allow ample time for obtaining regulatory approvals. Alternatively, start applying as soon as the study gets funded. The new NIH Common Rule that enables multisite studies to work under single IRB approval will be an efficient way save effort and time previously devoted to regulatory issues.²¹ This policy is applicable to domestic sites of NIH-funded studies where each site will conduct the human subjects research according to same protocol.

Site compliance

Sites are involved in study planning, protocol design, and attending investigators and kick-off meetings prior to the beginning of the study. A thorough understanding of study protocol regarding eligibility criteria, study assessments, follow-up schedule, and AE documentation is essential for success. (Table 2) In addition, maintaining effective and timely communication regarding participants’ progress and transmitting study documents is vital. The WRIST coordinating center ensured site compliance through periodic newsletters and regular emails communicating study recruitment goals, site progress, DSMB recommendations. and other troubleshooting to keep WRIST team abreast of study status. The WRIST PI also held periodic conference calls with site investigators and coordinators to troubleshoot or strategize whenever an issue arose.

Table 2.

Responsibilities of Coordinating Center and Participating Sites

Responsibility	Coordinating center	Participating sites
Manual of operations and procedures (MOOP)	Develops, maintains, and distributes to sites	Review and make suggestions to improve MOOP
Study regulatory binder	Creates and distributes to sites Maintains and updates with coordinating center information	Receive from coordinating center, maintain and update with site information
Participant recruitment	Screens and recruits participants, guides sites regarding participant eligibility issues	Screen and recruit participants
Randomization	Collaborates with study statistician to devise randomization scheme	Follow the study randomization scheme
Data flow	Develops and implements processes for data transfer and data tracking	Adhere to the data flow requirements per the study protocol
Data entry and data quality checks	Develops web-based data entry system Verifies data entered by site personnel	Follow the data entry process and enter data as soon as collected form participants
Communication	With sites through email, newsletters, conference calls and investigators’ meetings at national venues	With coordinating center through emails, conference calls and investigators meeting at national venues
Adverse events	Will receive reports from sites on adverse and serious adverse events. Reports them to data safety and monitoring board within 48 hours of receipt	Provide the adverse events and serious adverse events reports to the coordinating center and all supporting information immediately as these events are noticed
Site visits	To ensure protocol adherence and provide any training as required	Help the coordinating center in arranging for site visits when necessary
Report creating	Creates enrollment, adverse events, participant status reports by site. Will distribute them to data safety and motoring board and to sites	Provide additional information required for adverse events or serious adverse event reports that occur at sites

Issues during study conduct

High site coordinator turn-over

Because the surgeon investigators are busy providing care to their patients, the study execution depends heavily on the research coordinator. Their role includes, but is not limited to, developing a thorough knowledge of the study protocol, screening patients, recruiting participants, performing study assessments, and assisting the investigators in participant follow-up. Additionally, they communicate the study data to the coordinating center. A major problem WRIST encountered was their high turn-over, primarily because coordinators left the position for a new position, further education or for other reasons. It was often not possible to hire a replacement immediately, resulting in essential time loss to achieve study goals. Ways to mitigate coordinator loss include informing them about the time commitment and the duration of the study so they can make a decision prior to the beginning of the study, hiring experienced coordinators, assigning backup coordinator, and preparing coordinators for time flexibility and travel if transportation is a burden to participants. Study budget should include ample funds for site coordinators’ salaries to increase retention until study completion.

Patient screening and randomization

Recruiting participants who fit the strict eligibility criteria of an RCT is a crucial step in the study conduct. WRIST coordinators implemented a best practice advisory that will send email alerts to the study coordinators whenever a patient with a DRF is seen at the emergency room. This process first set up at the coordinating center was later adopted at sites by using keywords and phrases “fall on outstretched hand” and “fracture.” This practice saved time and effort for coordinators so they did not have to manually screen health system records every single day and also ensured that all potential eligible participants were screened and recruited. Randomization scheme, prepared by the study statistician prior to the start of enrolment, was stratified by study site using random block sizes of 3, 6, and 9. Sites obtained the treatment assignment for enrolled participants through a web-based system. Participants were blinded to the treatment assignment to the extent possible.

Participant retention

WRIST instituted several measures to engage participants and increase retention including scheduling follow-up appointments before participants left the clinic or providing a check-out sheet to participants that had information about the next visit window dates and which provider to see so they can make next appointment. We also arranged to see participants as they arrived to minimize their wait time, administered surveys while they were waiting to be seen, and sending appointment reminder letters for 12- and 24-month visits or after missed visits. We were always sure to thank participants for their time and to emphasize that the study’s success was entirely dependent upon patient volunteers. Sending holiday cards and birthday cards can engender the importance of participants to the study and increase retention.

Data management and quality checks

A routine practice instituted to verify the data entered by another study staff prevents data entry errors. Interim checks of exported data will ensure that there are no data outliers, prevent missing data or data discrepancies to have high quality data. In addition, confirm that units are consistent across sites. For example, grip strength can be measured in kilograms or pounds; prior decision regarding which measuring system will be used can prevent data errors. In case any data issues arise, inform the site coordinator to contact the participant immediately and try to resolve the issue immediately. WRIST also encountered issues related to procurement of x-rays from sites and access to the web-based data entry system for site coordinators during the study period.²². The WRIST team at the coordinating center was ultimately able to resolve these issues by working with information technology staff at the coordinating center. Prior to the study commencement, the coordinating center and sites should finalize the method of data transmission. WRIST used web-based data entry system for data transmission that enabled periodic checks for missing data.

DSMB

The DSMB acts as an advisory board to the study sponsor, whose members oversaw the study conduct, patient safety issues, and made recommendations to the study sponsor on the continuity of the study. The coordinating center sent monthly reports about the study status and progress to DSMB through a consulting group. WRIST also had a phone conference with DSMB members, the study PI, study statistician, and study coordinators twice every year for the duration of the study to review study materials.

It is very important to identify the role of DSMB in study initiation, participant recruitment, data collection, complication and AE documentation, and reporting of results. The study team should understand that the DSMB may find it necessary to meet more frequently than initially scheduled. This in turn may lead to increased workload for team members and the study statistician to gather and prepare data for such meetings.

Study monitoring visit

WRIST had one study monitoring visit performed by the study sponsor consulting group. This was a detailed and close examination of study documents, regulatory binder (Table 3), participant consent forms, data collection forms (surveys and assessment forms), and AE documentation. Study protocol and manual of operations were also assessed for consistency. WRIST had room for improvement in several areas. First, specifying the source from where eligibility information was obtained. For example, WRIST participants had to be 60 years or over, rather than simply noting that this is true, include on the eligibility checklist the source from where the patient age was verified from. This is especially important for eligibility criteria that is verbally relayed by the site PI or if the study PI approves the enrollment of an otherwise ineligible participant (i.e. inclusion criteria requiring enrollment within 14-days of fracture. This was later changed by the protocol review committee). A signed, written record of such information must be placed in the participants’ file. Communication from participants and/or their families and communication between sites and the coordinating center should also be documented with the date and time and retained. Finally, any reminder letters sent to participants should be sent via certified mail so that a “receipt” can be retained in the participants’ file.

Table 3.

Regulatory Binder Component Checklist

Delegation of Authority Log

Study Manual of Operations and Procedures

Institutional Regulatory Board approval

Informed Consent

Clinical trial registration

Study Protocol

Institutional Federalwide Assurance

Study personnel Curriculum Vitae

Investigators’ medical licenses

Study personnel human subjects protection training certification

Data Safety and Monitoring Board reports

Site visit/auditing log

Second, AEs should be documented meticulously, even seemingly innocuous patient complaints such as seasonal allergies. For serious AEs, hospital discharge reports should be included in the report. If multiple diagnoses are cited as the reason for hospitalization, each one should be reported on a separate form. If a participant is hospitalized at an outside hospital, every effort should be made to obtain those records. For participant deaths, a death certificate should be requested from the family (a copy, not an original, is sufficient). If there is no available information to confirm the death, including hospital discharge reports or a copy of a published obituary in the participants’ file is a best practice.

Finally, attention to details is best. For example, some WRIST participants signed the consent form on the line designated for the PI. In such cases, the participant should be asked to cross out their signature with a single line and to then sign again in the appropriate place. WRIST participants who had fractured their dominant hand would sometimes ask a spouse to sign for them. This is incorrect. The participant needs to personally sign the form; they may use the non-dominant hand, but must sign and not print. Another detail we sometimes overlooked was including study ID and visit number on every page of surveys, rather than only on the first page. These items may seem insignificant, but including them in a study will improve the quality of your study data.

General issues to consider for efficient study conduct

Good Clinical Practice (GCP) training

Investigators and staff involved in the design, conduct, oversight, or management of clinical trials must have training in GCP. It has 13 principles that ensure a trial is conducted in accordance with ethical principles, clear detailed protocols, and appropriate record keeping and that the study benefits outweigh risks,.²³ Compliance with GCP increases the credibility of a trial and assures the public that the rights, safety, and well-being of trial participants are protected. (Table 4) This training can be attained through NIH or via a class or course, academic training program, or certification from a recognized clinical research professional organization.

Table 4.

NIH Requirements During The Lifecycle of a Clinical Trial

Item	What to do	How it will help
Good Clinical Practice (GCP) training for investigators and NIH staff involved	Gain appropriate knowledge about the design, conduct, monitoring, recording, analysis and reporting of clinical trials	Provides consistent and high quality standard for a clinical trial conduct
Clinical trial protocol template use: protocol templates are developed through a collaboration between NIH and FDA	Provide consistent information to NIH, FDA, and IND	Helps avoid delays and inefficiencies; Helps investigators prepare protocols efficiently to render timely review by IRBs and be compliant with FDA and IND application guidelines
Single IRb (sIRB) review	Use NIH standardized agreements for single IRB of record in grant application and contract proposal. A single IRB is selected. This is usually an existing IRB that agrees to serve as the IRB of record for a particular study. It can be an independent or unaffiliated IRB, or a Central IRB organized to review specific projects.	Prevents duplicative IRB review in multicenter studies; prevents delays in often conflicting and multiple reviews that do not enhance human subject protection
Register with Clinicaltrials.gov as a mandate	Register no later than 21 days after enrolling first subject. Update the status on study progress at least once every 12months, and share the results no later than a year after study completion. Failing to comply with above may result in civil monetary penalties (by FDA) in addition to withholding future funding to the institution	Benefits scientific/biomedical community, informs public about clinical trials, apprise research participants, investigators interested in the specific topic. Shared results will prevent duplication of similar studies, inform about unsuccessful strategies/research methods and promote innovations in future clinical trials
Data Retention	Federal regulations require research records to be retained for at least 3 years after the completion of the research. Check for additional sponsor, institutional, or specialty specific requirements.	Ensures appropriate storage of participants’ PHI. Data can be retained for future research.

NIH: National Institutes of Health; FDA: Food and Drug Administration; IND: Investigational New Drug; IRB: Institutional Regulatory Board; PHI: Protected Health Information

Meetings and workshops

Every year NIH and FDA host conferences, meetings, and workshops focused on improving rigor, reproducibility, and transparency in clinical research. Investigators and study staff are educated about clinical trial requirements, compliance, and good clinical practices. Attending such events will help in keeping abreast of latest initiatives and provide guidance to investigators on common pitfalls. NIH website also provides video modules for basic training on these topics.²⁴ NIH institutes that are study sponsors provide templates for development of protocol, DSMB reports, AE forms, and other items on their websites for free use by study teams.²⁵ This practice helps the study team to adhere to required formats and avoid errors.

Another such initiative is Clinical Trials Transformation Initiative (CTTI), established by the FDA as a public-private multisector partnership to identify and promote practices that will enhance the quality and efficiency of clinical trials.^26,27 CTTI uses a systematic approach that includes literature reviews, surveys, qualitative interviews, focus groups, and multi-stakeholder meetings, with the goal of making recommendations that will improve the conduct of clinical trials. CTTI investigates ineffective or inefficient practices related to clinical trials and initiates projects to address them. CTTI projects led to improvements in single regulatory approval for multicenter studies, data monitoring committees, electronic healthcare data, informed consent, investigator community and qualification, and opioid use. The Grading of Recommendations Assessment, Development and Evaluation laid out the criteria for assessing certainty of evidence and strength of recommendations from systematic reviews.²⁸ This can be achieved when evidence from a well conducted study is included in the systematic review. In addition, there will be issues during study close-out such as site closing, data storage, and fulfilling if any sponsor or study specific requirements. (Table 5)

Table 5.

Issues During Study Close-out

Item	How to do it
Clinical trial site close-out visit	Occurs after all the enrolled participants complete the scheduled visits, all data has been collected, and all queries regarding missing data and AEs/SAEs have been resolved. All regulatory documentation, subject and source information, and other applicable study records are well-organized and available for future review. If site close-outs happen in a staggered fashion, update other sites on lessons learned to prevent repeating mistakes.
Data storage and disposal	Data storage after the study close out should be the same as during the study conduct to ensure no risk to subjects’ identity occurs after they complete the study. If required, paper records should be destroyed by shredding and recycling. Electronic copies should be completely erased from hardware using appropriate software under the guidance of information technology personnel.
Document tracking	Make a master list of location of each essential document (with site or with coordinating center), unused supplies, materials, equipment returns, final close-out report, and if any, sponsor-required close-out documents. Always ensure that the close-out process meets the sponsor and institutional requirements.

Conclusion

NIH and FDA provide several resources for better conduct of clinical trials as outlined in this manuscript. Investigators should make use of these resources to enhance research integrity. Despite experience in conducting RCTs, investigators are prone to encounter issues owing to the unique nature of each study. Therefore, it is beneficial for young as well as experienced investigators to expand their knowledge on how to overcome these issues and utilize the increasing resource pool from study sponsors. In addition to generating highest level of evidence with quality, a well-conducted RCT will ensure future funding to investigators in conducting rigorous clinical trials.

Supplementary Material

Supplemental Digital Content

Document, Supplemental Digital Content 1. Example Kick-off Meeting Agenda