Bolstering the complex study start-up process at NCI cancer centers using technology

Abstract

Background

The study startup process for interventional clinical trials is a complex process that involves the efforts of many different teams. Each team is responsible for their startup checklist in which they verify that the necessary tasks are done before a study can move on to the next team. This regulatory process provides quality assurance and is vital for ensuring patient safety [10]. However, without having this startup process centralized and optimized, study approval can take longer than necessary as time is lost when it passes through many different hands.

Objective

This manuscript highlights the process and the systems that were developed at The University of Kansas Comprehensive Cancer Center regarding the study startup process. To facilitate this process the regulatory management, site development, cancer center administration, and the Biostatistics & Informatics Shared Resources (BISR) teams came together to build a platform aimed at streamlining the startup process and providing a transparent view of where a study is in the startup process.

Process

Ensuring the guidelines are clearly articulated for the review criteria of each of the three review boards, i.e., Disease Working Group (DWG), Executive Resourcing Committee (ERC), and Protocol Review and Monitoring Committee (PRMC) along with a system that can track every step and its history throughout the review process.

Results

Well-defined processes and tracking methodologies have allowed the operations teams to track each study closely and ensure the 90-day and 120-day deadlines are met, this allows the operational team to dynamically prioritize their work daily. It also provides Principal investigators a transparent view of where their study stands within the study startup process and allows them to prepare for the next steps accordingly.

Conclusion/future work

The current process and technology deployment has been a significant improvement to expedite the review process and minimize study startup delays. There are still a few opportunities to fine-tune the study startup process; an example of which includes automatically informing the operational managers or the study teams to act upon deadlines regarding study review rather than the current manual communication process which involves them looking it up in the system which can add delays.

1. Introduction

Institutional Review Boards (IRBs) are an essential aspect of ethical medical research in the United States as they are responsible for the careful review of study protocols [2,10]. This review process involves many members of diverse working backgrounds to ensure a broad perspective [1]. In addition to the requirement that study review is overseen by an IRB, the National Cancer Institute requires NCI-designated cancer centers utilize a local institutional scientific review process as well [12]. Institutional Scientific Review Committees (SRCs) function to review a study's design, scientific value, catchment area fit and ethics before IRB review [3,5]. The SRCs function as an initial filter to ensure only high-merit studies are progressed in the startup process; however, the operation of SRCs comes with challenges. Barriers such as limited staff size, and logistical and operational challenges can lead to delays in scientific review [8]. These problems are further exacerbated when a research institution has multiple different SRCs reviewing their studies [4,16].

Despite the delays that scientific review can introduce to the study review process, there are factors identified by researchers that can improve the function of SRCs. These factors include broad-based communication, external motivators, senior-level support, and improved staffing [8,15]. Two aspects of the scientific review process were identified as opportunities for improvement. The first aspect was enhanced communication, which could be improved through a centralized system for communication between the different SRCs. Centralized research systems have been shown to reduce staff time and resource usage [7]. The second aspect was limited staffing, which could be addressed by automating processes within that system that were historically performed manually. It is shown that the ‘human factor’ plays a large role in IRB processing time so automation will help to reduce staff impact [9].

The University of Kansas Cancer Center utilizes a series of SRCs which each review a unique aspect of potential studies. These SRC reviews occur in succession before the studies undergo submission to the IRB. This introduces additional complexity to the process of study review by requiring efficient communication not only within each review committee but between review committees. In consideration of this, a system to streamline the study review process would additionally have to facilitate the process of review within committees and simplify the process of passing studies from one committee to the next. By having all committees utilize the same system, the process of review and transfer of studies between them is centralized. Cancer centers and other medium to large organizations that must deal with similar study startup processes could benefit from a robust process backed up by the technology described in this manuscript.

In August 2020, the University of Kansas Cancer Center implemented a system that allows review boards to seamlessly review and approve therapeutic trials, thereby ensuring appropriate studies are opened in a timely fashion. This system is referred to as Trial Review and Approval for Execution (TRAX) and was developed through a partnership between the clinical trials office and the Biostatistics and Informatics Shared Resource. The software base of the TRAX system was built in partnership with WCG Velos. We aim to evaluate how we can efficiently track studies during the study startup stage and monitor the review process. A strong process backed up with technology will allow reviewers to document availability and accessibility, improving the review's ease of use, speed, and accuracy.

1.1. Process

With the adoption of the TRAX system, every cancer-related clinical trial that is conducted by The University of Kansas Cancer Center is tracked at each stage with all the key performance indicators are captured and provides a holistic view to the cancer center leadership team. The first stage of the review process begins with the Disease Working Group (DWG) evaluating the need and the importance of the study. Once DWG approval is obtained, the study team uploads all the relevant documents into the system before submitting the study for ERC and PRMC review. These documents typically are study protocol, contractual document, consent, investigational pharmacy manual, medication information, and other supporting documents.

TRAX is a heavily modified version of WCG Velos standard eCompliance solution, which is part of their Clinical Trial Management System (CTMS). Our customization to incorporate KUCC regulatory process included checkpoints (submission criteria), custom fields, custom pages, automated letters, automated notifications (for communications), review meeting agenda, meeting minutes and dashboards to track key metrics, which all help to automate and cut down the labor involved in the startup process. This helps to improve efficiency and accuracy in the startup process.

1.2. DWG review

The DWG members review all protocol documents at their monthly meeting and decide the outcome of the study that is being proposed. Possible outcomes from the DWG monthly meeting for any study protocol are Approved, Rejected, or Tabled. Studies that are Tabled could require more justification in their proposal or the study may be being deferred due to competing ongoing studies. Before a study is reviewed by the DWG chairs, it is the investigator and research team's responsibility to submit the protocol document along with any other documents that help with the approval process. The supplement documents typically include funding information, the scientific rationale of why KU should participate, catchment area need, intra-disease and shared resources collaboration, consent documentation, and other strategic documents that illustrate the plan to recruit participants to expedite the clinical trial process.

The site development team along with the project managers attend the monthly DWG meetings to capture the outcome of the study review along with the approval date within the TRAX system. The approval from the DWG is the trigger for the study to proceed to the next stage of initial study startup, Executive Review Committee (ERC), for their review and feedback. The entire review process is illustrated under Fig. 1.

Fig. 1.

Therapeutic study review process at The University of Kansas Cancer Center.

1.3. Study scoring mechanism

The University of Kansas Cancer Center has built a scoring algorithm (aka. Study scoring mechanism) to assign studies with a score based on the following key metrics – scientific importance (first in human or phase I trial, phase II or phase III trial, drug registration trial), portfolio (fills the gap, competing study, competing study with large cohort), significance (contribution to science, R01, NIH, EDDOP, NCORP), funding (Externally, Internally, no funding), study type (IIT, consortium, Federal, Industry) and projected annual accrual.

This form is filled out at the DWG level by the project manager and PI and a numerical score between 1 and 100 is automatically generated based on their responses. Anything above 50 is considered a high priority, 30–50 is treated as a medium priority and below 30 is a low priority. The score plays a vital role in deciding which protocols must be launched versus the ones that could be tabled. Given the limited resources, especially during the COVID pandemic, the priority score provides an objective metric which the DWG can use to prioritize studies to move through the approval process. This algorithm helps ERC reviewers from the resourcing standpoint to make an informed decision as to which protocols are important from the DWG and Cancer Center's standpoint.

1.4. ERC review

Once a new protocol is approved by the DWG, the project manager submits the study to the ERC and PRMC committees by completing the ERC and PRMC submission forms which capture preliminary information that would be beneficial for the reviewers to make an informed decision based on the merit of the study.

Information under these forms captures the funding sources, the type of study that is being proposed, the sites where this study is expected to be launched, the recruitment plan and the feasibility of the sample size, the cancer center shared resources that will be utilized along with the latest protocol document that describes the science that is being studied. Every University of Kansas Cancer Center study must utilize Cancer Curated Clinical Outcomes database (C3OD) to obtain a rough estimate of the potential eligible participants [14].

Listed below is the built-in checklist that is required for every study before it can be submitted to obtain ERC and PRMC approval. Without any one element on the checklist, the project manager/study team is not able to proceed with their study submission. The check and submit logic that is required is represented under Fig. 2.

Fig. 2.

The Check and Submit logic screen.

The ERC review is conducted on a fixed schedule every Monday. With that schedule the PRMS manager ensures the studies for the upcoming Monday are submitted by the previous Thursday and assigns reviewers to each study. The TRAX system automatically generates a report that is sent to the ERC Chairs based on the assigned studies for review for that week. This allows them to see the study priority score before making their decision and discussing the outcome with the committee.

The PRMS manager runs the ERC meeting every week and documents all the studies that are assigned for the upcoming meeting. The TRAX system allows the PRMS manager to compile the agenda once the study assignment is complete. During the meeting, the PRMS manager documents the notes and the outcomes for each study and pushes the studies into the post-review outcome stage.

Under the post-review outcome stage, the PRMS manager can fill out a custom form with discussion points that were addressed during the ERC meeting. After completing the form, these custom review points are automatically transferred to the ERC outcome letter which is generated and sent to the DWG chairs, PI, and the study team. If the reviewers need more information prior to approval, the ERC letter specifies the additional information needed to address the reviewer's concerns. If the study is rejected, then the review process ends, and the outcome letter is generated and sent to the PI. Once ERC approved, the study is assigned to PRMC reviewers and meeting agenda.

1.5. PRMC review

Studies that have an ERC approval or any cancer-related study done at KUCC are submitted to the PRMC for independent scientific review. This committee is managed and conducted independently of the Clinical Trials Office and meets bi-weekly to discuss and decide, unbiasedly, on the review outcome from a scientific, statistical, and ethical standpoint.

The feedback from the other two committees, DWG and ERC, acts as input for the PRMC to make an informed decision. The PRMC additionally addresses specific questions related to science, medication, and the statistical methodology that is being proposed under the study.

The PRMC also reviews every study at least annually to ensure the study is on track from an accrual standpoint, patient safety perspective, and that the science is still timely and appropriate. PRMC has the sole authority to close or terminate a study if the study is underperforming or no longer meets the goals of KUCC. At the annual review, PRMC can also decide if the study should be put on a six-month probation giving the study team an opportunity to address the accrual concern prior to study closure.

2. Results

Tracking all the information within a single system allows our cancer center to capture and analyze metrics that our operational and regulatory teams use to assess the year-by-year trend and strategize improvements for the upcoming year. Most importantly, we can track days from DWG to ERC to PRMC to study activation. We can see how long a review committee takes to review a study upon receiving it. We can audit submissions and data entry. These metrics help identify slowdowns and allow our teams to address bottlenecks at each stage.

In case of an audit, the PRMS administrator can more easily review and gather information from the system as all the information through each review step is documented electronically. Additionally, generated reports allow the cancer center to obtain real-time metrics. One of the examples of the metrics is listed below. Table 1 depicts the cross-tabulation between the funding source and the ERC committee outcome. Similarly, Table 2 depicts the cross-tabulation between the funding source and the PRMC committee outcome.

Table 1.

ERC types of protocols reviewed under TRAX system (8/2020–8/2021).

	Reviewed	Approved	Tabled	Withdrawn	Disapproved
Institutional	41	24	14	0	3
External Peer-Reviewed	6	5	1	0	0
NCTN	62	51	2	7	2
Industry	137	63	27	22	25
Total	246	143	44	29	30

Table 2.

PRMC types of protocols reviewed under TRAX system (8/2020–8/2021).

	Reviewed	Approved	Tabled	Withdrawn	Conditionally Approved	Disapproved	Terminated
Institutional	36	19	3	0	4	0	10
External Peer-Reviewed	22	5	1	0	6	0	10
NCTN	66	48	1	0	3	5	9
Industry	90	62	4	2	16	4	2
Total	214	134	9	2	29	9	31

The system also allows the PRMC coordinator to conduct the post-review process of adding comments from the reviewers and populating other key feedback into the form which then allows the PRMC administrator to auto-generate the appropriate outcome letters. Based on the review outcome different letters can be generated. An example of ERC and PRMC approval letters is shown below under Fig. 3a, Fig. 3ba and b.

Fig. 3a.

ERC outcome letter.

Fig. 3b.

PRMC outcome letter.

Table 3 and Table 4, in comparison to Table 1, Table 2, show that we had more studies reviewed by the ERC committee post-TRAX implementation, 162 studies pre-TRAX implementation versus 246 studies post-TRAX implementation by the ERC committee. PRMC had reviewed fewer studies post-TRAX implementation, which is expected because the ERC committee filters out more studies that are not feasible, saving PRMC resources. More importantly, the study type, such as Institutional sponsored study, external peer-reviewed study, NCTN, or Industry-sponsored, can be traced using TRAX. Additionally, the outcome is well defined under TRAX. Table 1, Table 2 are more in line with NCI's recommendations.

Table 3.

ERC reviews BEFORE TRAX system (8/2019–8/2020).

	Reviewed	Approved	Withdrawn
Total	162	153	9

Table 4.

PRMC reviews BEFORE TRAX system (8/2019–8/2020).

	Reviewed	New Trials	Renewals	Amendments	Closure
Total	702	191	231	177	103

2.1. Outcome letter

.

2.2. Automated email

The system sends automated alert emails to PIs as their studies progress through the startup process. This keeps them up to date on how their studies are doing. For active studies that are under probation or ongoing, the system automatically triggers reminder emails 30 days before their due date as shown in Fig. 4b, so PIs can prepare.

Fig. 4.

PRMC reminder email.

2.3. Study start-up dashboard

Another tool available to researchers is the study startup dashboard. The study startup dashboard provides an overarching summary of where all the studies are in the startup process, as shown in Fig. 5. This provides a big picture view of efficiencies/inefficiencies in the study start up process within Clinical Trials Office. The dashboard also allows users to filter for specific disease working groups, PIs, process stages, and date ranges. Using these filters users can narrow down to see specific metrics and answer questions that are relevant to them. An example of this can be seen below.

Fig. 5.

Study start-up dashboard.

3. Discussion

Establishing a robust process for our study initiation allows our teams to collaborate, coordinate and execute the various steps that are involved with study startup. Regulation is a vital component of the study initiation process [11]. Our process described above allows our team at The University of Kansas Cancer Center to diligently review every study with the utmost care and attention to detail.

Every Principal investigator has a clear picture of where their study stands within the review process as they receive email updates and real-time information. Through the startup dashboard, CTO management has an overall picture of each stage in the review process. Collectively, our process acts to instill confidence and transparency in our review process among the researchers.

Each oncology study is different and there are varying complexity scores across every study that goes through the study initiation process. Regulatory complexity allows our teams to ensure resources are allocated appropriately to meet our goal of timely study activation. Our review process allows us to ensure we stick to the NCI Operational Efficiency Working Group goal of a 90-day turnaround on study initiation regardless of complexity (“Report of the Operational Efficiency Working Group Clinical Trials of the and Translational Research Advisory Committee”, 2010).

Apart from the study complexity, another key to the quick turnaround is communication across different teams. The dashboard and the automatic notifications that are built within the system allow the different teams to keep track of activities that must be performed within specific timeframes. One of the key examples of the automatic notification is when the annual review is due for a study, the TRAX system sends out an email to the PI, study team, and regulatory team notifying them about the annual review that is due at 45 and 30 days before the due date. The flexibility of our review platform allows us to evaluate and add other vital notifications which would improve study initiation efficiency and compliance maintenance. An example of this is sending a notification if a study has been pending for more than 90 days.

Limitations: Specific details around the sponsor negotiations and contract information gathering timeline information is not tracked within the system at this point and is instead maintained in a separate spreadsheet. There could be a few missing data elements that were not available for certain types of studies or were not provided to our study startup team by the sponsors or research at the time of study initiation[6,13].

Conflict of interest

None to Report.

Funding

This study was supported by the National Cancer Institute (NCI) Cancer Center Support Grant P30 CA168524 and used the Biostatistics and Informatics Shared Resource (BISR).

Availability of data and materials

The datasets generated and analyzed during the current study are available from the corresponding author (MB) on reasonable request.

Ethics

The IRB at the University of Kansas Medical Center has waived the need for approval for this study as this is considered as a quality improvement study.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.