Protocol Development Program

Under the current climate of globalization and increasing legal and regulatory requirements, finding new ways to improve and streamline processes has become paramount to the future success of clinical research in the United States. In a recent report, the Institute of Medicine found that the regulatory review process for clinical trials conducted by the National Cancer Institute’s Clinical Trials Cooperative Group Program is lengthy and duplicative.¹ It is estimated that approximately a third of the cost of conducting clinical research is used to address protocol compliance with local, state, and federal regulations.²

This has contributed, in part, to the growing trend of conducting clinical trials outside the U.S. For example, 80% of the drugs and biologics approved by the Food and Drug Administration (FDA) in 2008 included data from foreign clinical trials, and 78% of all the subjects who took part in those trials were enrolled at foreign sites.³

This article provides an overview of the successful launch and implementation of the Protocol Development Program (PDP) now in use at the National Institute of Allergy and Infectious Diseases (NIAID), with an emphasis on the program’s approach to target adequacy of resources within the institute.

Barriers to Clinical Research

At the National Institutes of Health (NIH), the Medical Executive Committee of the Intramural Working Group was charged with the task of identifying major barriers to clinical research at the NIH. Responding to the challenge, a survey was sent in November 2007 to 546 intramural investigators. The results were summarized based on the frequency of response and ranking, and responders represented 16 institutes across the NIH. With a nearly 50% response rate, the survey revealed four main areas perceived as major barriers to clinical research—adequacy of resources, industry and technology transfers, institutional review board (IRB) and ethical approval process, and scientific review process.

NIAID’s Response

NIAID was the first institute to implement changes in response to the results generated from the NIH barriers to clinical research survey.

To target the adequacy of resources barrier, the director of the Division of Clinical Research (DCR) at NIAID appointed the managers of the Regulatory Affairs Branch to design and implement a solution. The branch managers sought feedback from the different intramural laboratory groups within the institute about specific areas lacking the resources and/or the expertise to advance clinical protocols through the regulatory and NIH-specific requirements. Incorporating this feedback into an evolving structure, the PDP was instituted in November 2009 within the Regulatory Affairs Branch in DCR to provide early interventions in protocol development by streamlining processes, managing administrative and regulatory requirements, and delivering writing support.

The PDP was designed to offer services free of charge to NIAID intramural investigators conducting domestic and international clinical research in biomedical disciplines related to infectious diseases, immunology, and allergy. The PDP is uniquely structured as a team of protocol navigators and medical writers.

Protocol navigators are central to the program, facilitating the protocol development process through a series of interactions with internal and external groups (see Figure 1). Medical writers are responsible for writing and editing clinical protocols from study concepts (hypotheses, study objectives, design), informed consent documents, amendments to protocols and informed consents, standard operating procedures, and publications. The team consists of two protocol navigators, three medical writers, and a project manager, with further growth anticipated in the future.

Figure 1.

Protocol Navigator Interface

Legend: NIH = National Institutes of Health; NIAID = National Institute of Allergy and Infectious Diseases; FDA = Food and Drug Administration; IND = investigational new drug application; OHSRP = Office of Human Subjects Research Program; DRS = Division of Radiation Safety; OBA = Office of Biotechnology Activities; OTD = Office of Technology Development; IRB = institutional review board.

Process Steps and Milestone Mapping

Metrics to evaluate the program’s capabilities and performance output were developed shortly after the program was launched. Using Six Sigma, which emphasizes the phases of the DMAIC approach (define, measure, analyze, improve, and control)⁴ for process improvement, several activities were defined, including project and team functions, past work, and stakeholder needs (see Table 1; Phase I); trends in protocol development practices at other institutions were also identified^5–10 and compared against our results. Likewise, process steps and milestones were identified and mapped as an instrument of measure for the program’s performance.

Table 1.

DMAIC Phase I and II Stages and Activities

Project	Stage	Activity
Phase I	Define	Project and team functions
		Past work and related studies
		Voice of PDP users and stakeholders needs (e.g., investigators, IRB, etc.)
	Measure	Process mapping
		Key performance indicator development
	Analyze	Evaluate process capability; identify opportunities for improvement
		Key performance indicators; baseline/trend compared to internal goals and external benchmarks
Phase II	Improve	Identify best practices to achieve benchmark performance
		Develop and implement improvements (small- and large-scale)
	Control	Systemize key performance indicator tracking and management
		Store knowledge gained on improvements
		Spread improvements and best practices

The process mapping resulted in the creation of a detailed functional flow-chart that tracks 48 steps and eight milestones involving 12 departments/ functions, delineating the maximum number of possible steps (see Figure 2 for the “lifecycle”). Some of the milestones determined through the process mapping, such as submissions to the scientific review and IRB committees, were similar to those reported by others.^5,6,9

Figure 2.

Protocol Development Process

Notes: The figure depicts an example of the process flow for an investigational new drug study; the process varies slightly for other types of studies. Legend:

^a A quality control check of the document(s) is performed by a second medical writer prior to submission to the scientific review committee or the IRB, or both.

^b This step may occur either at the scientific review submission stage or just prior to submission to the IRB. The review may occur twice for extensively revised documents.

^c If the pre-IRB review occurs at the scientific review submission stage, an e-mail correspondence is sent to the PI by the medical writer; if the pre-IRB review occurs prior to submission to the IRB, an e-mail correspondence is sent to the PI by the medical monitor. If the review occurs twice, the process will occur as detailed in the latter example above.

IND = investigational new drug application; RSC = Radiation Safety Committee; RDRC = Radioactive Drug Research Committee; IBC = Institutional Biosafety Committee; RAC = Recombinant DNA Advisory Committee; PDP = protocol development program; MW = medical writer; PN = protocol navigator; QC = quality control; FDA = Food and Drug Administration; SR = scientific review; stips = stipulations; PI = principal investigator; MM = medical monitor; OTD = Office of Technology Development; IC = informed consent; IRB = institutional review board; PFH = personal financial holdings; DSMB = data and safety monitoring board; PRIA = protocol resource impact assessment; REG = regulatory group; OPS = Office of Protocol Services.

During protocol development, the PDP team collaborates closely with the principal investigator (PI) to ensure compliance with regulatory requirements and NIH policies, project time-lines, and overall accuracy and quality of the content. The process begins with the PI drafting a study concept for the planned protocol. Once the concept is approved, an initial meeting is scheduled with the PI to identify the project’s objectives and generate a time-line. Because the PDP is a voluntary program, investigators can choose the level of support needed (i.e., participating throughout the process or merely a part of it). Then, the medical writer drafts the clinical protocol in collaboration with the PI prior to submission to an intramural scientific review committee.

Every intramural laboratory group conducting clinical research within the Division of Intramural Research (DIR) is responsible for assembling a scientific review committee that evaluates the scientific merit, originality, and relevance of the proposed study. DIR investigators conduct either basic or clinical research at the NIH, with funding received from the respective intramural laboratory groups to which they belong within NIAID.

Concurrent with the scientific review, or prior to the IRB review, the study documents are submitted for a pre-IRB review, which is an NIAID requirement for all studies conducted intramurally. The pre-IRB review consists of a team of experts in clinical trials management, regulatory, and safety oversight, which ensures compliance with regulations (e.g., FDA and Department of Health and Human Services) and guidelines (e.g., Good Clinical Practice). If regulatory concerns exist, they are communicated to the PI, who must resolve them before proceeding with the rest of the process. The pre-IRB review may occur twice if the study documents are extensively revised following the scientific review.

The PDP team remains actively engaged throughout the process, attending scientific review and IRB meetings in person to keep current with the viewpoints of approving bodies, participate in project-related discussions, and help respond to stipulations. The protocol navigator guides the PI through the regulatory and administrative requirements to facilitate the submission process, thus eliminating the need for sequential submissions and avoiding unnecessary delays. For example, based on the individual requirements of each study, the protocol and informed consent documents might be submitted for a simultaneous review to one or more of the following: the FDA, Radiation Safety Committee, Institutional Biosafety Committee, a data and safety and monitoring board, or others.

As part of the process, the PDP team also maintains an open dialog with the IRB administrative office for clarification of any potential issues that could be resolved at an earlier stage prior to submission to the IRB. Throughout, the medical writer works closely with the PI to help address any comments and stipulations received from the various review committees, while the protocol navigator ensures that all the logistical requirements and timelines have been met.

Key Performance Indicators

The Six Sigma approach and the Malcolm Baldrige National Quality Award’s “Education Criteria for Performance Excellence”¹¹ were used to identify a set of key performance indicators measuring the effectiveness and capabilities of the program; these included the percentage of PDP-supported protocols of the total number approved annually by the NIAID IRB, the volume and distribution of protocols by intramural laboratory group and study type, the number of repeat users of PDP services, the volume of IRB stipulations received per protocol and per informed consent, and the total cycle time for each project and interim measurements between milestones.

Similar to the DMAIC approach, the Baldrige criteria provide guidance on developing performance indicators with a focus on leadership, strategic planning, customer focus, information and knowledge management, workforce focus, and process management.¹¹ Collectively, these two methodologies established the structure for developing a balanced set of indicators for ongoing tracking and sustainable performance.

The PDP has facilitated the development of 30 new studies through 2011, ranging from natural history studies to Phase 0–IV domestic and international clinical trials, and it has supported nine laboratory groups within the DIR, the Collaborative Clinical Research Branch within the DCR, and the Center for Infectious Disease Imaging at the NIH (see Table 2).

Table 2.

PDP Key Performance Indicators Through 2011

Key Performance Indicators	2010 and 2011
No. of NIAID IRB-approved protocols
Protocols not supported by PDP	57
Protocols supported by PDP	19a
No. of PDP protocols by study type
Natural history	17
Phase 0	2
Phase I	4
Phase I/II	1
Phase II (there were no Phase III during 2011)	4
Phase IV	2
No. of PDP protocols by group
Clinical infectious diseases	11
Allergic diseases	2
Parasitic diseases	4
Immunogenetics	3
Immunoregulation	4
Molecular immunology	2
Infectious diseases	1
Center for infectious disease imaging	1
Collaborative clinical research branch	2
No. of repeat users of PDP services	5
Multiple-time users	2b

^aEleven additional protocols supported by the PDP are in the process of being approved by the IRB.

^bEach repeat user completed three studies.

Seventeen of the 30 studies were natural history and 13 were Phase 0 through Phase IV clinical trials. At the NIH, natural history studies are defined as clinical research studies conducted in human subjects (or on material of human origin, such as tissues, specimens, and cognitive phenomena) for which an investigator directly interacts with the subjects to determine the mechanisms of a specific human disease.

Nearly 30% (10 of 33) of the studies approved by the NIAID IRB in 2010 and 38% (nine of 24) approved in 2011 were supported by the PDP. The remaining 11 of the 30 PDP studies are at various stages of development. Each study presented inherent challenges, which the PDP team has had to address, for example, ensuring compliance with specific domestic or international regulations; facilitating technology/material transfer agreements, reliance agreements, and credentialing; and assessing the availability of resources.

Five investigators were repeat users of PDP services. Of those, two investigators were multiple-time users, having used PDP services for three different studies each. Collection of the remaining key performance indicators, such as the volume of IRB stipulations received and cycle times, is currently under way.

Investigator Satisfaction

A customer feedback tool was also developed to collect data about investigator satisfaction with PDP services. The tool is structured into three sections; the first and last sections capture information about the users and serve as a narrative part for additional comments, respectively. The second section consists of nine questions used to determine satisfaction with PDP services, and responses are rated on a five-point scale, with 5 being the highest score possible.

To date, a total of 18 respondents from 15 studies approved through 2011 have completed the customer feedback tool. Six of the nine categories captured in the tool were rated with an average score of 5.0, while the other remaining categories were rated with an average score of 4.9 (see Table 3).

Table 3.

Customer Feedback Summary

Investigator Satisfaction	Average Scorea (5=high; 1=low)
Assistance provided	5
Communication provided	5
Issue resolution	5
Availability of PDP staff	5
Improvement of IRB submission	5
Improvement of scientific review submission	4.8
Support provided prior to scientific review	4.9
Improvement of consent development	4.9
Overall satisfaction	5
Respondent Characteristics	No. of Respondentsa
Experience (years)
1 to 5	14
>10	4
Experience (no. of studies)
1	5
2 to 5	9
6 to 10	2
>10	2
Reasons for Using PDP Services	No. of Responsesb
Assistance with drafting documents	16
Hasten approval process	13
Lack of resources	10
Busy schedule	9
Unfamiliar with approval process	8
High-priority study	7

^aBased on a total of 18 respondents from 15 studies approved through 2011.

^bBased on a total number of 63 responses, with multiple responses allowed per respondent.

The majority of the respondents (78%; 14 of 18) noted they had one to five years of experience in conducting clinical research at the NIH and 22% (four of 18) had more than 10 years (Table 3). Similarly, 50% (nine of 18) of the respondents answered that they completed two to five studies as lead investigators, 28% (five of 18) completed one study, 11% (two of 18) completed six to 10 studies, and an additional 11% (two of 18) completed more than 10 studies.

Most of the respondents (78%) remarked that they were first-time users of PDP services. The chief reason for using the services was to receive assistance with drafting protocol and consent documents (25%; 16 of 63), followed by hastening the approval process (21%; 13 of 63), lack of resources (16%; 10 of 63), busy schedule (14%; nine of 63), unfamiliarity with the protocol development process (13%; eight of 63), and conducting a high-priority study (11%; seven of 63).

Predominantly positive comments were received in the narrative portion of the customer feedback tool, which were consistent with the high overall scores received for the questions rating PDP services. Moreover, 100% of the respondents noted that they would use the PDP services again and would recommend the services to other investigators.

We are currently in the process of comparing baseline key performance indicators against internal goals and external benchmarks, seeking possible opportunities for improvement (see Table 1).

Summary and Conclusions

The concept of protocol navigation has been previously reported in the literature;¹² yet, to the best of our knowledge, the NIAID PDP is uniquely structured, making use of specialized groups, such as protocol navigators and medical writers, in the context of additional support provided by clinical trials management, regulatory, and safety oversight teams. Collectively, these groups offer a multidisciplinary level of expertise, working together to address the adequacy of resources within NIAID.

Different approaches have been explored on ways to identify, measure, and target impediments curtailing the success of clinical research.^5,6,9,12,13 Our program uses process mapping, milestones, key performance indicators, and a feedback tool to measure the performance output, productivity, and quality of the services provided. By looking at parameters such as the volume of protocols supported and stipulations received during a specific time period, and the average cycle times for the different metrics used, the capabilities of the program will continue to be refined to achieve the best possible outcome.

Since the process-mapping exercise was completed, the protocol development process has been further streamlined to enhance efficiency. For instance, concurrent submissions to the scientific and pre-IRB reviews have shortened the timeline between the scientific review approval and submission to the IRB. Responding to concerns earlier in the process has proven beneficial and, in many cases, has eliminated the need for a second pre-IRB review. In contrast, studies not using the PDP could be delayed during the pre-IRB review due to unforeseen or unresolved issues.

The PDP has facilitated the development of 30 new studies through 2011, spanning a wide range of clinical phases and laboratories within NIAID. Thirty percent of the studies approved by the NIAID IRB in 2010 and 38% approved in 2011 were supported by the PDP. Additionally, the PDP has provided assistance with amendments, which consist of extensive revisions to the protocol and consent documents.

Preliminary results from the feedback tool are encouraging, suggesting that the program has had a positive effect on those using its services. The majority of the respondents were first-time users of PDP services who completed two to five studies as lead investigators, and had five years or less of experience in conducting clinical research at the NIH. The primary reasons for using PDP services were to receive assistance with drafting the protocol and consent documents, to hasten the approval process, and for support due to a lack of resources.

These results mirror the findings from the 2007 survey, which highlighted adequacy of resources and the approval process as main barriers to clinical research at the NIH. The high overall satisfaction scores and positive responses provided in the narrative portion of the feedback tool support the idea that our program fulfills an unmet need for NIAID intramural investigators.

The PDP organizational structure and scope of services can be adapted to fit the individual needs of other institutions. Seeking input from researchers throughout the process and ensuring management support are essential to the successful implementation of a program. Undoubtedly, barriers to clinical research will ensue during a program’s implementation phase; however, clinical research professionals and institutions must evolve through the process by overcoming barriers with innovation and a commitment to getting research done in the most efficient manner possible.

Biographies

Tracey J. Miller, RN, CCRP, is manager of the NIAID Protocol Development Program and has 13 years of clinical research experience. Prior to working in clinical research, she was a neonatal intensive care nurse. She contributed to the acquisition, analysis, and interpretation of the data, as well as the revision and final approval of this article. She is an employee of the Clinical Research Directorate/CMRP, SAIC-Frederick, Inc., National Laboratory for Cancer Research in Frederick, Md. She can be reached at trmiller@mail.nih.gov.

Vali S. Sevastita, MS, is a medical writer with nearly 13 years of writing and editing experience in domestic and international clinical research. She earned an Advanced Curriculum Certificate from the American Medical Writers Association and has drafted numerous peer-reviewed publications. She contributed to the acquisition and interpretation of the data, as well as the drafting, revising, and final approval of this article. She is an employee of the Clinical Research Directorate/CMRP, SAIC-Frederick, Inc., National Laboratory for Cancer Research in Frederick, Md. She can be reached at sevastiv@mail.nih.gov.

Doreen G. Chaitt, RN, MPH, has served as the oversight manager of the NIAID IRB Administrative Office since 2004. Previous to her current role, she worked at the NIH HIV Clinic and has held different positions within the Regulatory Compliance and Human Subjects Protection Branch, Division of Clinical Research, NIAID. She contributed to the program’s conception and design, the interpretation of the data, and the revision and final approval of this article. She can be reached at dchaitt@niaid.nih.gov.

Jorge A. Tavel, MD, is the medical director for Global Product Development at Genentech. In his previous position as deputy director in the Division of Clinical Research at NIAID, he provided the conceptual leadership for the Protocol Development Program, and he contributed to the revision and final approval of this article. He can be reached at tavelj@gene.com.

Jerome F. Pierson, RPh, PhD, is chief of the Regulatory Compliance and Human Subjects Protection Branch, Division of Clinical Research, NIAID. Prior to joining NIAID, he served in the United States Army in a variety of assignments. His research focus has ranged from topics in pharmacy practice and pharmaceutical economics to regulatory science and public policy. He contributed to the Protocol Development Program’s conception and design, the interpretation of the data, and the revision and final approval of this article. He can be reached at piersonjer@niaid.nih.gov.