Increasing Burden of Institutional Review in Multicenter Clinical Trials of Infertility: the Reproductive Medicine Network Experience with Pregnancy in Polycystic Ovary Syndrome (PPCOS) I and II Studies

Narrative Abstract

Many clinical investigators feel that the burden of institutional review board (IRB) requirements has been consistently increasing over recent years, though there are few objective data describing these trends. Over a period of 7 years the Reproductive Medicine Network observed a significant increase in the size and requirements of IRB submissions, and significant variability of IRB performance in reviewing multicenter trials. These additional regulatory and administrative demands represent substantial burdens to researchers and to the IRBs themselves. It is timely to consider whether these changes better protect the interests and safety of human research participants.

Introduction

Prior to the second half of the 20th century, the performance of scientific research was primarily governed by personal conscience and ethical standards. Over the past 50 years, increased attention and widespread discussion of research ethics have given rise to specifically articulated guidelines for the performance of clinical research, most notably the Declaration of Helsinki in 1964 and the Belmont Report in 1979. These and other related documents have established the defining principles that clinical research studies must undergo review by an independent committee, and that informed consent must be obtained from human subjects participating in clinical trials. In the intervening years the process has evolved as studies have become more complex and the clinical research community has become much larger and more sophisticated. Other factors including increased attention to medical-legal risk, financial oversight, protection of personal health information, and the widespread reliance on electronic record systems have all contributed to what seems to be an ever-expanding mountain of regulatory and oversight documentation. The increased burden produced by these many requirements is daunting to many researchers, perhaps most compellingly to those who participate in multicenter trials that are subject to frequently inconsistent and variable reviews by local institutional review boards (IRBs) (1, 2). In such trials, each site must negotiate not only their own institutional review process, but in addition, may be required to modify protocols or policies based on determinations of IRBs at other sites. Thus, the challenges at any one site are often multiplied by the requirements of others even though the submitted protocol is identical at all sites. Ultimately, this ungainly and inefficient process can consume significant time and resources from both the investigative teams and the IRB staffs (3), produce mistakes to the protocol and consent forms (4), and significantly increase the time required to initiate and conduct a clinical trial.

Performing clinical research on “vulnerable populations” necessitates additional review and produces even more burdens (5). In the United States, vulnerable populations include pregnant women, fetuses, neonates, prisoners, and children who require “additional safeguards” which are not specified in the federal code but left to the discretion of local IRBs (6). For example, even a simple observational trial in a pediatric population requires additional documentation which is often assessed differently by local IRBs, and may ultimately discourage some investigators from participating (7). Because infertility studies often include two vulnerable populations, i.e. pregnant women and fetuses, researchers in the field of reproductive medicine must typically satisfy these additional requirements, made all the more difficult if they are participants in a multicenter trial.

The Reproductive Medicine Network was established by the National Institute of Child Health and Human Development in the 1990's in order to design and conduct multicenter trials which would address critical questions in the field of reproductive medicine. The centers chosen to participate in the RMN have varied over its several iterations; since 2007 the RMN has been comprised of seven clinical sites and one data and coordinating center (DCC). The RMN Steering Committee, consisting of the experienced principal investigator (PI) from each site, the PI of the DCC, plus an NIH-appointed project officer and committee chair, develop and prioritize all clinical protocols. Approved protocols are the critiqued and refined by an Advisory Board as well as a Data Safety Monitoring Board, both convened by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), prior to submission to the local IRBs and ultimate implementation. The goal of this study was to obtain objective data describing the impact of evolving institutional review requirements on initiating clinical research in RMN-sponsored multicenter trials.

Materials and Methods

The RMN Steering Committee collected data from the institutional reviews of two multicenter clinical trials for the treatment of infertility in women with polycystic ovary syndrome, a series of studies now referred to as Pregnancy in Polycystic Ovary Syndrome (PPCOS) I and II. PPCOS I (NCT00068861) was initiated in 2002 and we have since reported our main outcomes (8). PPCOS II (NCT00719186) was initiated in 2009 and is now completing its final phase of enrollment. The two studies were extremely similar in design; both compared a new ovulation induction medication (metformin in PPCOS I and letrozole in PPCOS II) to the existing first-line ovulation induction drug (clomiphene citrate) in a double-blind manner. Both were powered to examine live birth as the primary outcome. There were minimal differences between the two studies as regards study design, none of which was associated with a significant difference in the risk/benefit analysis or other major informed consent issues. PPCOS II did include consenting of male partners while PPCOS I did not.

The formal RMN protocol submitted to the individual IRBs was identical, though there was significant variability of submitted documents at each site due to differences in local requirements and format. In order to perform the present study, each of the seven RMN sites retrospectively reviewed the documents submitted to its local IRB for PPCOS II, and the four (of seven) sites that also participated in PPCOS I provided the same information for that trial. The data were then analyzed in order to examine both the variation between sites as well as the changes that have occurred over time. We chose to obtain four specific sets of data which we thought would reflect the challenges each site faced in obtaining IRB approval. These included: 1) the total number of pages required for the submission to the IRB. The total number of pages excluded the formal RMN protocol but did include the locally required protocol in the necessary format, consent forms, attachments, disclosures and administrative documents, and requests for advertisement approval. Case report forms prepared by the DCC and any investigator drug brochures were excluded from this page count; 2) the total page length of the final, approved informed consent document; 3) the number of attachments required by the IRB to address specific research or administrative questions; and 4) the number of days from submission of the completed IRB package to final approval by the IRB.

Because the process of developing the protocols (peer review in the application process, consensus of RMN investigators, review and refinement by the advisory board, and approval by the data safety monitoring board) as well as the protocols themselves were so similar, we hypothesized that additional IRB burdens which evolved between submission of PPCOS I and II would be reflected by increases in the factors described above. Additionally, because four of the seven sites that participated in PPCOS II also participated in PPCOS I, we hypothesized that these established sites might have experienced less delay in obtaining IRB approval. Furthermore, we thought that reviewing the experience of this small subset of four established sites would provide a direct measure of the changes that have occurred over the 7-year period between the two studies.

Statistical analysis

We performed Student t tests of overall data to compare the experience of the seven present RMN sites in the PPCOS II study to that of the four sites which participated in PPCOS I. We then performed pair-wised Student t tests to compare the parameters of the four clinical sites that participated in both PPCOS I and II.

Results

The total length of the IRB submission and the number of attachments were both significantly greater for the PPCOS II study compared to PPCOS I (Table 1). Most notably, the length of the consent form was dramatically greater for PPCOS II. Interestingly, there was no significant difference in the time from submission to approval between the two studies though we did note that the mean time required was about 2 weeks longer (21.9%) for PPCOS II. When we compared the experience from only the four established sites we again observed the same relationships, i.e. longer submissions and consent forms, more attachments, but no statistically significant difference in time from submission to approval (mean increase in time to approval 17.8 days or 26.6%). Furthermore, we found no significant difference in these parameters when we compared the experience of the four established sites to that of the three new sites in PPCOS II. Total pages were 104.5 ± 45.3 vs. 85.7 ± 49.9, number of attachments was 13.0 ± 6.6 vs. 9.7 ± 7.0, consent form pages were 23.5 ± 4.0 vs. 25.3 ± 6.5, and submission to approval days were 84.8 ± 67.2 vs. 77.7 ± 29.3. We observed a high degree of variability for most of the parameters studies across the sites which appeared to be much greater for the PPCOS II trial as compared to PPCOS I.

Table 1. Parameters reflecting the institutional review board submissions of RMN sites in Pregnancy in Polycystic Ovary Syndrome I and II (PPCOS I and II) studies.

	PPCOS II (n=7) M ± SD (range)	PPCOS I (n=4) M ± SD (range)	P Value*	PPCOS II (n=4)** M ± SD (range)	P Value***
Submission to approval (days)	81.7 ± 50.6 (42-185)	67.0 ± 9.9 (53-75)	0.51	84.8 ± 67.2 (42-185)	0.60
IRB Submission (total pages)#	96.4± 44.2 (42-166)	40.8 ± 22.4 (18-65)	0.03	104.5 ± 45.3 (64-166)	0.03
Consent form (pages)	24.3 ± 4.5 (18-32)	11.3 ± 7.4 (6-22)	0.006	23.5 ± 4.0 (18-27)	0.03
No. of Attachments	11.6 ± 6.4 (3-21)	3.5 ± 4.4 (0-9)	0.05	13.0 ± 6.6 (5-21)	0.04

Plus-minus values are mean ± SD. Numbers in parentheses are the range

^#Excluding the formal RMN study protocol, case report forms, and any investigational drug brochures

^*T test of overall data between all sites that participated in PPCOS I and PPCOS II (irrespective of participation in the other)

^**Data regarding PPCOSII form sites that also participated in PPCOS I

^***Paired T test for comparison of data from sites that participated in both PPCOS II and PPCOS I

Discussion

We found that the administrative burden experienced by the Reproductive Medicine Network, as measured by the length and requirements of the IRB submission, has increased significantly over the past 7 years. Further, we observed that there was significant variability in the performance of individual IRBs, though none ultimately demanded significant changes in the clinical protocols. While there is a widespread sense that the administrative and regulatory burdens are increasing in infertility research, we are not aware of any previous data that demonstrate these trends in a quantitative manner. We acknowledge that our study is based on data from a small number of sites. Further, we appreciate that it is difficult to be completely certain that all sites included and excluded exactly the same material in determining their total page counts despite clear definitions and our best efforts. We have much greater confidence in the uniformity and accuracy of the other three parameters studied. Finally, we acknowledge that time from submission to approval could have been prolonged if investigators were not efficient in providing the necessary responses and clarifications to their IRB in a timely way. However, we confirmed that the turn-around times for investigators corresponding with IRBs were efficient at all sites. Delays did not appear to be unduly long; in practice such delays are part of the review and approval process and, as such, should be considered in a comprehensive analysis of the present system. We accept that these are all potential weaknesses of our study, but nevertheless feel that these findings, being among the very first hard data on this subject, can provide important observations to aid in our analysis of the present state of research review in infertility trials.

Several important observations are clear. First, the amount of information required by IRBs is increasing significantly. There can be no question that amassing all the required information and completing the increasingly numerous attachments in the necessary format takes more time and effort by investigators and their teams. Interestingly, we noted that the new RMN sites did not take any longer from the time of submission to approval than did the established sites, suggesting that more experience did not translate to more rapid or efficient completion of the review process. It is possible that new sites required more time to compile their materials before actual submission to their IRB as compared to the established sites, but we did not have a way to ascertain these data.

A somewhat hidden but critically important question is whether the IRBs themselves are reviewing the submitted material as carefully as they have in the past. If the total pages submitted and the length of the consent form have more than doubled, and the number of required attachments has tripled, we wonder how IRBs can review all this additional material in roughly the same amount of time as they have in the past. We must speculate that either IRB members are more efficient at reviewing material now than they were 7 years ago, or that some or even much of the submitted material is not being scrutinized as closely as it was previously. We hope that it is the former, but fear that it is the latter. If the review of research protocols is more superficial than it has been in the past, the entire research community must ask if this increasingly bureaucratic process is likely to be more or less effective at the most basic goal of the IRB process, that is, protecting the interests and safety of research participants.

A second important observation is the dramatically increasing length of consent forms, a problem which some RMN members have attributed to IRB “mission creep”. Our data show that our informed consent documents are more than twice as long as they were 7 years ago. While a small portion of this can be explained by the additional pages required to consent the male partners of our female participants (another important and controversial question to consider), we are struck by the increasing and often numbing detail required by IRBs in designing consent forms. Many elements of consent forms as presently required by numerous IRBs appear to be aimed more at legal indemnification than explaining the nature of the proposed research with an emphasis on the point of the research and a description of potential risks and possible benefits. Further, many have suggested that the level of detail used to explain each and every minute detail of numerous research interactions as is now required by many IRBs is not only unnecessary but may well adversely impact the ability of potential participants to fully understand the research study to which they are enrolling (9,10). We also fear that increasingly long and detailed consent forms have the potential to erode the important personal relationships between subjects and members of a research team. By making the process excessively bureaucratic, we wonder if there may be an adverse effect on the “team” feeling often shared by subjects and researchers which is so important to participant interaction and retention. In any case, it behooves the research community to thoughtfully consider whether lengthier and more detailed informed consent is better informed consent.

The first two points relate to review of any research protocol in any individual institution. Our third key observation relates specifically to the particular challenges in obtaining approval for multicenter trials. As shown in Table 1, there is a huge amount of variability between the various sites. This observation does not appear to be unique to our multicenter trials (11). Given that all sites submitted exactly the same protocol to their local IRB, we must question whether they are all consistent in their evaluations and their concerns. Based on discussions at meetings of the RMN Steering Committee, it is our sense that the major barriers to IRB approval were quite different at each site. Furthermore, despite the increased burden to the investigators and the apparent inefficiency of the process, none of the IRBs substantially altered the study protocol. We must conclude that there is significant variability in the information and format required by IRBs, that the issues which provoke greatest concern may well be site-specific, and that the multiple IRB reviews of the same protocol do not seem to have much impact on the final form of the study.

There has been increasing recognition of the unbridled growth in regulatory burden and a call for reform by investigators (12), and perhaps most importantly by the federal government (13, 14). The administrative challenges which individual investigators experience in obtaining IRB approval for a research study may be exponentially increased by those participating in multicenter trials. Therefore, as the research community addresses the issues discussed above, there should be a similar effort at reducing the regulatory burden for multicenter trials. Effective approaches could include accepting cooperative agreements between local IRBs with one IRB taking on the main review burden (2), or the creation of independent central IRBs such as have been instituted by the National Cancer Institute for cancer trials (15). While these remedies can generate their own unique set of problems (16, 17), they may well represent a significant step forward for both investigators and potential study subjects.

Thoughtful investigators have expressed concerns that IRBs have become overly expansive in interpreting regulatory requirements, have increasingly focused on inconsequential details, and have lost sight of their mission as they have become bureaucratically stilted and progressively unresponsive (18, 19). Indeed, in a recent commentary in the Journal of the American Medical Association, Christine Grady posed the question “Do IRBs Protect Human Research Participants?” (20) The research community should strive to develop strategies to answer this most critical question. Unfortunately, in our study, we do not feel that we could point to any outcome data which would allow us to address this issue. We believe that the observations described in this study demonstrate the degree to which the burden of research review has increased over a relatively short period of time, and how the experience at individual institutions can be extremely different in considering identical, well-designed multicenter trials. The RMN strongly endorses initiatives intended to streamline research review for multicenter infertility trials as well as efforts to explore whether the increasing regulatory burden experienced by many investigators actually leads to improvement in the protection and safety of participants in clinical research studies.