The Rockefeller University Navigation Program: A Structured Multidisciplinary Protocol Development and Educational Program to Advance Translational Research

Abstract

The development of translational clinical research protocols is complex. To assist investigators, we developed a structured supportive guidance process (Navigation) to expedite protocol development to the standards of good clinical practice (GCP), focusing on research ethics and integrity. Navigation consists of experienced research coordinators leading investigators through a concerted multistep protocol development process from concept initiation to submission of the final protocol. To assess the effectiveness of Navigation, we collect data on the experience of investigators, the intensity of support required for protocol development, IRB review outcomes, and protocol start and completion dates. One hundred forty‐four protocols underwent Navigation and achieved IRB approval since the program began in 2007, including 37 led by trainee investigators, 26 led by MDs, 9 by MD/PhDs, 57 by PhDs, and 12 by investigators with other credentials (e.g., RN, MPH). In every year, more than 50% of Navigated protocols were approved by the IRB within 30 days. For trainees who had more than one protocol navigated, the intensity of Navigation support required decreased over time. Navigation can increase access to translational studies for basic scientists, facilitate GCP training for investigators, and accelerate development and approval of protocols of high ethical and scientific quality.

Introduction

Designing and implementing a human research protocol has grown increasingly challenging due to the increasing expectations about scientific and biostatistical rigor, the growth in regulatory requirements, and the increasing subtlety of bioethical review.1 As a result, the successful design of a human research protocol requires inputs from a variety of content experts in addition to the investigator. We hypothesized that having an expert and experienced leader (Navigator) coordinate and integrate those inputs in a structured program could expand opportunities for, and accelerate, translational research by: (1) lowering the barriers for basic investigators to develop human subjects protocols, (2) providing practical and comprehensive education in the principles of good clinical practice (GCP), (3) speeding IRB approval, and (4) insuring scientific quality.

Methods

Navigation overview

The eight steps of the Navigation process implemented in the Rockefeller University's Center for Clinical and Translational Science (CCTS) are outlined in Figure 1 (left), along with the expert personnel that participate in the process (Figure 1, right).

Figure 1.

Navigation of research protocols involves a concerted process consisting of a series of semistructured meetings spanning the protocol development process from concept initiation to the submission of a completed protocol. Content experts from the cores supported by the Rockefeller Center for Clinical and Translational Science (CTSS) and both Rockefeller University Hospital and Rockefeller University departments participate in protocol Navigation, providing interdisciplinary expertise for development of the protocol. Navigation is provided free of charge to all investigators.

Navigation infrastructure

The staff

The lead Navigator (D.B.) has more than 10 years of experience developing, conducting, and coordinating clinical trials, including Investigational New Drug (IND)/Investigational Device Exemption (IDE) studies and investigator‐initiated research. She leads three to five highly experienced clinical research coordinators who serve as research facilitators and participate in Navigation. As a member of the Rockefeller University Hospital and CCTS senior leadership, she collaborates closely with the leadership of the other CCTS core resources.

Electronic platform

The Rockefeller CCTS selected iRIS (iMEDRIS; Redlands, CA, USA) as its comprehensive electronic application for protocol writing, IRB review, and study/subject management. The protocol writing component of iRIS was extensively customized by the CCTS senior leadership and Rockefeller University IRB to incorporate the elements of GCP in a guided protocol writing pathway. iRIS is designed with mandatory fields that insure that all GCP‐required elements are included in the protocol and thus the process provides new principal investigators (PIs) with a comprehensive introduction to GCP during Navigation.

Navigation process

Navigator Preparation for the first meeting includes researching the topic to provide context, reviewing the investigator's clinical research experience, and providing the proposed PI access to relevant electronic systems. The Navigator tracks the needs, action items, and progress of the project throughout the process. The feasibility of conducting and completing the protocol is assessed initially and reassessed at each subsequent meeting to identify and resolve potential obstacles to protocol completion. Meetings generally last 1 hour, and conclude with assignments for the next meeting. Minutes are recorded and progress is assessed weekly.

The first meeting with the investigator is a Concept Initiation Meeting, during which the Navigator explains the services that the Facilitation staff can contribute and assesses the PI's clinical research experience. The investigator presents the research idea, which may range from a preliminary concept to a fully articulated hypothesis backed up by preclinical and/or clinical data. The Navigator performs an initial feasibility assessment based on the research experience of the PI and the other members of the research team and the available facilities, equipment, and supplies. The financial resources available to support the study are identified and evaluated relative to the projected costs. The roles of collaborators and consultants are defined. Based on the discussion, the PI and Navigator decide whom to include in subsequent meetings.

The Concept Refinement Meeting, attended by the PI, Navigator, and a Research Coordinator from the Facilitation Office, is used to create a first draft of the protocol. Details of the study are refined, including aims, objectives, eligibility criteria, and study procedures. A CCTS Recruitment Specialist usually attends this meeting to provide input on the feasibility of identifying and enrolling the requisite number of participants. Discussions at this stage focus on risks and methods to minimize risks, anticipated adverse events, the likely burdens and benefits experienced by participants, and plans for data and safety monitoring.

After the scientific goals have been established, a Biostatistical Navigation Meeting takes place between the PI and a staff biostatistician to review the study design, estimate sample size based on the desired power, and develop the data analysis plan.

Based on the results of the above meetings, the Navigator may recommend a Scientific Refinement Meeting to more fully explore scientific questions such as reframing the hypothesis, reconsidering the experimental approach, or deciding whether a Phase 0 or Phase I study is more appropriate. The Navigator and PI usually enlist the aid of several expert researchers at The Rockefeller University or from other institutions to better define the scientific aspects of the project. The IRB Chair, the CCTS Research Hospitalist, who is a physician trained in intensive care with experience in the conduct of human subjects protocols, and the head of the regulatory support core (R.K.), who is a physician with experience in research and human subject protections, usually attend. If the PI is a trainee in the CCTS Master's degree Clinical Scholars program, the trainee's mentor is also invited to this meeting.

Depending on the complexity of the study, One‐on‐One Meetings between the PI and content experts may be recommended by the Navigator. Meetings with Rockefeller leaders in research nursing, bioethics, contracting, technology transfer, research pharmacy, and/or bionutrition are arranged as required.

The Completion of the Draft Protocol Meeting is interdisciplinary, affording PIs exposure to a select group of experts from different departments (Figure 1, right). At this meeting, the full protocol draft is reviewed, with a focus on the study population, methods and procedures, and study visits. One specific goal is to map a participant's flow through the protocol so as to minimize inconvenience and encourage participant compliance with the protocol. This may include preparing written instructions, adjusting the timing of visits, and organizing study visit procedures to maximize the participant's comfort and encourage protocol compliance. Other topics addressed at these meetings include technical details of study procedures, expected adverse events (and how they may be prevented or treated), lab testing specifics, and the recruitment plan. Informed consent forms are formulated at this meeting.

If the Navigator believes the protocol still needs further refinement, the Navigator will recommend a final Interdisciplinary Navigation Meeting in which all content experts who previously participated in developing the protocol have an opportunity to provide additional input into the final protocol. Prior to the meeting, the Navigator prepares a list of issues that require review. The meeting begins with the PI summarizing the scientific question and then the group as a whole reviews the inclusion and exclusion criteria one‐by‐one, along with the methods and procedures. Each of the open issues is resolved and the study‐specific roles of the individuals who will conduct the study are detailed.

All protocols have a Finalization and Protocol Submission Meeting, during which the PI and the study coordinator make final revisions to the study and submit it for review.

At Rockefeller University, protocols are initially reviewed by the Advisory Committee on Clinical and Translational Science (ACCTS), the governing body of the CCTS, primarily for their scientific validity, statistical soundness, and data safety and monitoring plan. ACCTS members include investigators and CCTS senior staff leadership members with broad experience in clinical investigation. Once approved by the ACCTS, protocols are reviewed by the IRB, primarily for the elements of informed consent, bioethics, risk/benefit ratio, and participant safety.

Tracking and evaluation

The Navigator uses a detailed series of checklists to track the progress of the protocol and ensure that all issues are resolved before the protocol is submitted. After the protocol is submitted, the Navigator records the areas and intensity of Navigation services that were required to develop the protocol. Using the Navigator's checklist entries and other notes as a guide, each protocol is scored as to which of the 16 different categories of services listed in the Table S1 were required. We assign a score of “0” to those challenges or needs that required minimal support and a score of “1” to those that needed intensive or extensive support. A summary “Needs Score” is calculated by compiling the scores across the categories; thus, Need Scores range from 0 to 16.

The evaluation of the Navigation program is based on assessing its contribution to five specific goals.

1. Enabling inexperienced PIs to develop protocols: To assess the impact of Navigation on empowering those who are otherwise inexperienced in human subjects investigation to develop and conduct human research we tracked the number of new protocols initiated, the research orientation of the originating laboratory (basic science or clinical science), the academic degree of the lab leader (PhD, MD, or MD‐PhD), the training and career stage of the PI (e.g., trainee, junior or senior PhD, MD, MD‐PhD, or RN investigator), and whether trainee investigators were able to complete their studies before the end of their training period.

2. Educating trainees. To assess the educational value of Navigation for trainees, we analyzed whether they needed less intense Navigation support (i.e., had lower “needs scores”) with their second (or subsequent) protocol relative to their initial protocol.

3. Expediting protocol development. To assess the impact of Navigation on expediting the protocol development and approval we tracked: (1) the percentage of Navigated protocols that resulted in a completed, submitted protocol; (2) the time from the first Navigation meeting to the date the IRB accepted the submitted protocol; (3) the time from IRB submission to IRB approval; and (4) the percentage of protocols approved within 30 days of submission, indicating approval in the first round of IRB review.

4. Insuring protocol quality. To assess whether Navigation insured that protocols were of high ethical, regulatory, and scientific integrity, we tracked whether the protocols were approved by the ACCTS for scientific validity and the IRB for bioethical integrity.

5. Providing valued support for the PI and research team. We developed an online anonymous satisfaction survey so that PIs and research team members could rate their experience with the Navigation process.

Results

Number of protocols navigated and submitted for IRB review

Table 1 presents data on the growth of the Navigation program since its inception in 2006 according to the PI's background and the scientific orientation of the sponsoring laboratory. A total of 164 protocols underwent Navigation, of which 144 (90%) were ultimately submitted for review. Among the reasons why Navigated protocols were not submitted for review included: a review of the literature indicated the scientific question had already been answered; a review of the preliminary data indicated a need for additional preclinical studies; the investigator left the institution before completing the submission; and the investigator could not secure the desired collaboration.

Table 1.

Number of protocols navigated per year according to characteristics of the PI and the sponsoring research laboratory's orientation

Characteristics of investigator and research laboratory		Grant year						Total
		2008	2009	2010	2011	2012	2013
PI degree	PhD	4	8	21	10	11	13	67
	MD	5	8	16	14	10	4	57
	MD/PhD	0	3	6	7	5	2	23
	Other	0	3	4	4	4	2	17
								164
Laboratory orientation	Basic science	2	8	17	14	11	15	67
	Clinical science	6	12	21	17	15	5	76
	Other	1	2	9	4	4	1	21
								164
Previous navigation and clinical research experience	Neither clinical research nor navigation experience	9	16	22	20	16	11	94
	Clinical research experience without navigation experience	0	1	10	9	5	3	28
	Clinical research experience with navigation experience	0	5	15	6	9	7	42
								164

Enabling inexperienced PIs to develop protocols

Data on the 35 protocols Navigated in 2011–12 by laboratory orientation and PI degree are given in Figure 2 A. During this time period, 41% of the PIs held PhD degrees and 14% held MD‐PhDs degrees; 41% of the Navigated protocols were from basic science labs. The utilization of Navigation overall and the number of open CCTS protocols led by basic science labs rose steadily from 2007 to 2012 (Figure 2 B). By 2013, 22 of 53 PhD Laboratory Heads had an active human subjects protocol.

Figure 2.

(A) Selected demographics for protocols navigated in FY 2012 (N = 35). (B) Number of navigated open protocols from basic science laboratories.

Expediting protocol development

The duration of protocol development under Navigation varied across studies as measured by the time elapsed from the Concept Initiation Meeting to the date of IRB submission of the protocol. From 2008 to 2013, the median time decreased from 26 to 8 days (Figure 3).

Figure 3.

Time elapsed from the Navigation Concept Initiation Meeting to submission of the protocol to the IRB for review.

In the 3 years prior to instituting Navigation (2003–2006), the median time from IRB submission to protocol approval was (41 days; n = 29). Since the Navigation program was initiated, there has been a steady decrease in the overall median time to protocol approval, reaching 17 days in the most recent year (Figure 4). For protocols Navigated from 2009 to 2012, more than 50% received approval within 30 days of submission, that is, they were approved after one round of IRB review (Figure 5).

Figure 4.

Time elapsed from submission of Navigated protocol to final IRB approval.

Figure 5.

Navigated protocols submitted for review that received IRB approval in 30 days or fewer.

Educating trainees

The Navigation Needs Scores for the 11 trainees who underwent Navigation for more than one protocol are shown in Figure 6. Seven of the 11 had a lower Needs Score with their second protocol and this downward trend continued with additional protocols. The change from the first to the second protocol showed a downward trend (p = 0.057 by nonparametric Wilcoxon test), and when all of the data, including protocols beyond the second one, were fitted to a generalized linear model describing the Needs Scores as a function of subsequent navigations (assuming a Poisson probabilistic distribution for the Needs Scores) the model had a slope of –0.324 (p = 0.0005), supporting a downward trend in Needs Scores with subsequent Navigations.

Figure 6.

Intensity of Navigation support needed by trainee investigators (n = 11) for their first and subsequent protocols.

Insuring protocol quality

All of the protocols that underwent Navigation and were submitted for review were approved by the ACCTS and the IRB.

Providing valued support for the PI and research team

The results from 41 anonymous survey responses (out of 100 invitations) are shown in Figure 7. PIs and research team members rated Navigation as either “Very Helpful” or “Helpful” more than 70% of the time on every element.

Figure 7.

Investigators’ ranking of the value of the Navigation process in the development of different elements of the protocol. Navigation User Satisfaction Survey responses (n = 41) expressed as a percentage of those responding after removing responses indicating that the service was not needed.

Discussion

The Navigation program we developed is designed to assist investigators in developing human subjects protocols, speed the process of protocol development and approval, educate investigators in developing protocols, and insure the quality of the protocols. An ancillary goal is to encourage PhD basic investigators who had no previous experience in conducting human subjects research to develop protocols that would advance their basic and translational science goals. This outreach to basic investigators is especially important at our University since it is composed of 73 different laboratories, each of which has a head of laboratory (HOL), and 53 (73%) of the laboratories are led by a PhD (or D Phil) HOL and another 11 (15%) by an MD‐PhD HOL. The number of PhD HOLs with human subjects protocols increased steadily after Navigation was implemented and currently 22 of the 53 laboratories led by a PhD (or D Phil) HOL (40%) participate in one or more approved protocols. In addition to the Navigation program, we try to lower the barrier for basic scientists to conduct clinical research by supporting a Research Hospitalist and Research Nurse Practitioner who can help with the medical aspects of studies developed by PhD investigators, place physician‐scientist Clinical Scholar trainees in labs led by basic investigators, and support a pilot project awards program. Some of the elements of our Navigation program are similar to the “Studio” program developed by the Vanderbilt CTSA,2 but whereas that program assembles three to six faculty research experts from diverse academic disciplines to meet with an investigator, our program is broader in its multidisciplinary representation and tightly focused on the goal of producing a final protocol.

Navigation was successful in reducing the time from protocol submission to IRB approval and increasing the percentage of protocols that are approved within 30 days of their review by the ACCTS and IRB. Navigation has also been an integral educational component of our Master's degree in Clinical Scholars program and our other training programs in clinical investigation, providing practical experience in the principles of protocol development under GCP. Moreover, the multidisciplinary nature of the Navigation process provides trainees with an introduction to effective team science, one of the major goals of our program and of the CTSA program as whole.3 The decline in the need for intense Navigation support for individual trainee's subsequent protocols attests to their increased expertise in designing studies. In addition, the Navigation program also provided quality control information on individual investigators. For example, the Navigation “needs scores” for some nontrainee investigators remained high despite their undergoing Navigation on multiple protocols, indicating that they had difficulty in comprehending and/or incorporating key elements of protocol development.

The Navigation program has also supported the commitment of the CTSA leadership to generating protocols of the highest quality before submission for review. All of the protocols that completed Navigation and were submitted for review were approved by the ACCTS and IRB.

Based on qualitative judgments, Navigation also appears to have a positive impact on protocol implementation by engaging key departments, such as Research Nursing and Bionutrition in protocol development, thus helping them prepare to rapidly implement the protocol.

Navigation requires a major commitment of resources to support the multidisciplinary team that participates in the process, but we have not, as yet conducted a formal cost–benefit analysis. We recognize that Rockefeller University differs from most academic medical centers in being much smaller and having a high percentage of basic investigators. We also benefit from the willingness of our IRB chair to devote approximately 2 hours per week to the Navigation program. Thus, the structure and cost of our Navigation program may not be scalable in larger institutions. We also have a high percentage of investigator‐initiated rather than sponsored protocols. Thus, our Navigation program may need to be modified for larger institutions with fewer basic scientists and a higher percentage of sponsored protocols. It is possible, however, that a single large academic department (or a consortium of several smaller departments) in an academic medical center could adapt our Navigation system to their needs. Finally, we would emphasize the importance of the lead Navigator to the success of the program. In addition to content expertise, the leader needs to have expertise in team building, teaching, and scientific collaboration, as well as a high level of political sensitivity and institutional responsibility. In conclusion, we believe that a structured multidisciplinary Navigation program has the potential to improve, speed, and expand human subjects research, as well as encourage participation by basic investigators and provide crucial practical education for trainees.

Supporting information

Disclaimer: Supplementary materials have been peer‐reviewed but not copyedited.

Supporting Information