Navigating the Institutional Review Board Approval Process in a Multicenter Observational Critical Care Study

Abstract

Background

Factors that contribute to variation in Institutional Review Board (IRB) evaluation and approval of multicenter studies are not well understood.

Objective

To characterize variation in the IRB application process of a multicenter, observational critical care study.

Design, Setting, and Participants

Survey analysis of 36 investigators who applied for participation in the United States Critical Illness and Injury Trials Group: Critical Illness and Outcomes Study (USCIITG-CIOS), an observational study of 69 adult intensive care units.

Measurements

Analysis of investigator-specific characteristics, IRB process, application and approval dates, and level of difficulty in obtaining approval.

Main Results

Surveys were analyzed from 36 (95%) sites that applied for IRB approval. Level of review ranged from full board, expedited, to exempt. Seventy-five percent of applications were submitted by an experienced investigator while 25% were submitted by a less experienced investigator. Median time to IRB approval was 30 days (IQR 14, 54) and ranged from 5 days to 5.5 months. Time to approval was 29 days (IQR 17, 48) for applications submitted by an experienced investigator compared with 97 days (IQR 25, 159) for those submitted by a less experienced investigators (p=0.08). Subjective level of difficulty was significantly higher for less experienced investigators 4/10 (IQR 2,8) versus 2/10, respectively (IQR 1,3) (p=0.04). Four sites cited IRB concern regarding waiver of consent as a major barrier to approval and were required to perform revisions or participate in Board meetings regarding this concern.

Conclusions

In a multicenter, observational critical care study, significant variation was observed between sites in all aspects of the IRB evaluation and approval process. The level of difficulty was significantly higher for less experienced investigators with a trend toward longer time to IRB approval. Variation in IRB interpretation of waiver of informed consent regulations was cited as a major barrier to approval.

Introduction

Wide variation has been reported in the pathways of Institutional Review Board (IRB) review for multicenter studies [1-5]. Under Title 45 of the Code of Federal Regulations, IRB approval is required at each participating site, including review of a research protocol and determining need for informed consent [6]. Study approval is subject to variation in the interpretation of federal regulations at each site, in addition to many other potential factors including investigator-related characteristics, institutional standards, IRB structure and function, clinical expertise of the reviewing committee, and individual ethical and methodological standards [7-10]. While giving each IRB the discretion to identify local concerns related to a standard protocol, multisite IRB approval may increase inefficiency, inconsistency, and lead to delay [7, 9-13].

Little is known about which factors contribute to variation in the IRB approval process. Multicenter, observational trials are well suited for study of IRB variation because they are generally considered minimal risk and are designed to identify important trends in epidemiology, risk factors and outcomes [14]. Together, these characteristics suggest that efficient and consistent IRB review is a reasonable expectation for this type of study design. Importantly, a potential consequence of variation and barriers to conducting this type of work is delay of findings that could significantly affect patient outcomes [10].

Numerous researchers have reported variation in the approval process across sites of multicenter studies, even when they are observational, quality improvement-oriented and minimal risk [1-3, 5, 15-17]. However, only one of these types of studies reported IRB variation in a critical care setting but was neither observational nor minimal risk [17]. To better understand the factors that affect the process and timeline of approval of a multicenter observational study of critically ill patients, we characterized variation in a large multicenter, observational critical care study and attempted to identify specific factors associated with variation in the IRB approval process. We utilized the United States Critical Illness and Injury Trials Group: Critical Illness Outcomes Study (USCIITG-CIOS) to test our hypothesis that wide variation in the evaluation process would exist between sites, and that IRB concern surrounding waiver of informed consent regulations would pose a significant barrier to IRB approval.

Materials and Methods

The United States Critical Illness and Injury Trials Group: Critical Illness Outcomes Study was a prospective, observational study designed to examine the organizational and structural factors present in 37 sites including 69 adult intensive care units in the United States, and to determine whether these factors were associated with patient related outcomes [18]. A standard submission template was available to all sites as an aid for IRB application, and all patient data was devoid of personal identifiers. Patient-level data collection fell into the following major domains: demographics, diagnoses, severity of illness, organ failure, use of mechanical ventilation, medications, treatment factors, infections, and clinical outcomes. Study investigators applied for waiver of informed consent following standard criteria for such a waiver[19] .

The USCIITG-CIOS investigators who applied for IRB approval at each site were electronically surveyed about their IRB experience using Survey Monkey, an online software program. The survey was developed in an iterative, consensus process by a group of USCIITG-CIOS primary investigators to explore the IRB approval process at each site. One site that applied for IRB approval was excluded from analysis because the research coordinator left the institution after the IRB application was submitted. We have presented our primary results including the remaining 36 sites. As a sensitivity analysis, we also analyzed the data with the additional site included. Survey question categories included information about the submitting investigator, IRB submission process, application timeline, level of difficulty in obtaining IRB approval, and specific barriers to approval. Investigators who did not initially respond to the electronic survey were contacted via email and telephone. We defined experienced investigators as people who had previously submitted 5 or more IRB applications, and less experienced investigators as people who had previously submitted less than 5 IRB applications.

Continuous variables were reported as median values with interquartile ranges and categorical variables as proportions. Data was analyzed using Student’s t test for continuous data that was normally distributed, the Kruskal-Wallis test for variables that did not appear normally distributed, and by chi-squared tests for categorical data.

All analyses were performed using Stata 11.0 software (Stata Corporation, College Station, TX). A 2-sided p-value <0.05 was used to determine statistical significance. Approval for this study was obtained from the University of Arizona Institutional Review Board.

Results

We surveyed all 39 sites that applied for IRB approval of the study. Two sites did not respond, and one site that did not pursue IRB approval due to inadequate staffing was excluded prior to analysis. Sensitivity analysis including this site revealed no difference in the results. Survey data was analyzed from the remaining 36 sites, including 1 site that did not obtain IRB approval. The survey response rate was 95% (36/38). Of these, 78% (28/36) of surveys were complete. Sixty-nine ICUs at 37 sites were represented within the study, including 25 medical, 24 surgical, and 20 mixed ICUs. Seventy-one percent of sites used the submission template provided by the USCIITG-CIOS work group to submit their IRB application.

Submitting Investigator

Sixty-one percent (22/36) of IRB applications were submitted by the primary investigator, while 19% (7/36) were submitted by a research nurse. Nineteen percent (7/36) of applications were submitted by someone else who was not otherwise specified. Seventy-five percent (27/36) of IRB applications were submitted by an experienced investigator, while 25% (9/36) of applications were submitted by an inexperienced investigator, defined as someone who had previously submitted fewer than 5 IRB applications (see Figure 1).

Figure 1.

Participation, level of review and experience of surveyed sites.

IRB Committee Submission Process

Sixty-seven percent (24/36) of sites underwent expedited review, 11% (4/36) underwent full board review and 22% (8/36) were deemed exempt by the IRB (see Figure 1). Sixty-four percent (23/36) of sites submitted IRB applications electronically, and 66% (23/35) of applications were reviewed by an IRB analyst prior to submission. Of sites that used an IRB analyst, 65% (15/23) reported that analyst review was helpful, 30% (8/23) reported that analyst review was not helpful and 4% (1/34) did not report whether review was helpful. The meeting frequency of IRB committees varied: 27% (10/37) met weekly, 30% (11/37) met every other week, and 35% (13/37) met monthly. Nineteen percent (7/37) of sites had their applications reviewed by a focused IRB committee. Of these, five focused on biomedical research, one focused on cardiovascular, pulmonary, and hematologic research, and one focused on low-risk protocols.

Of approved applications, fifty-nine percent (20/34) were approved on the first submission, 26% (9/34) on the first resubmission, 15% (5/34) on the second resubmission. Five percent (2/37) were denied.

Application Timeline

Complete timeline data were unavailable for 6 out of the 36 survey responses. In those responses with timeline data available, the median time from original IRB submission to first formal reply was 26 days (n=31; IQR 6,45). The median time from submission to final approval was 30 days (n=30; IQR 14,54) with an absolute range from 5 to 169 days. It took less experienced investigators a median of 97 days (IQR 25,159) to obtain IRB approval as compared to experienced investigators who obtained approval after a median of 29 days (IQR 17,48) (p=0.08). There was no association between time to application approval and the type of submission, IRB committee meeting frequency, review by committees with or without a focus, or review by an IRB analyst prior to submission.

We asked survey responders to rate the subjective level of difficulty in obtaining IRB approval for this study measured on a Likert scale of 1-10, with 1 being very easy and 10 being very difficult. The median level of difficulty for experienced investigators was 2 (IQR 1,3), versus 4 (IQR 2,8) for less experienced investigators (n=26 vs 9, respectively) (p=0.04).

Barriers to Approval

Four sites noted that their IRB committee expressed concerns regarding waiver of informed consent and specifically whether the study was only practicable without consent. Other barriers included additional, in-person meetings with the IRB and specialty committees as well as finding the time and funding support to complete the application. Selected narrative comments are as follows: “…it took an extraordinary effort and a face to face meeting with [the IRB] for them to agree with doing the study under waiver of informed consent…”; “this research involved no more than minimal risk and the criteria for waiver of consent was met. The rights and welfare of the research subject were not determined to be adversely affected and the research could not practicably be conducted without the waiver of consent”; and “I would urge CIOS to consider seeking IRB approval for centralized access to health systems’ EHRs via data exchanges so that we can study larger ICU populations, in particular those such as those outside academic health centers”.

Sites Not Approved

Three percent of sites (1/36) did not obtain approval for the study. Partial survey data was obtained from this site along with a subjective account of the process. The request for waiver of informed consent was cited as the reason for IRB denial. The primary investigator at this site met with the IRB to make a case that the study met Title 45 CFR 46.116(d) guidelines for waiver of informed consent. The basis for appeal primarily included the minimal risk nature of the study and that the study could not be practicably carried out in a prospective nature if informed consent was required from patients who were frequently intubated and sedated, with variable presence of surrogates.

The site that was denied had submitted an electronic IRB application and applied for expedited approval. The application was submitted by an experienced investigator and was reviewed by an IRB committee that met every other week.

Discussion

This survey analysis of 36 sites that applied for participation in a prospective, observational critical care study suggests that significant variation in the IRB approval process occurs in response to the same protocol delivered to different Institutional Review Boards, despite the minimal risk nature of this study. The type of review, speed of review, and requirement for review by an IRB analyst varied between sites. Less experienced investigators found it significantly more difficult to obtain IRB approval than experienced investigators, and there was a trend toward longer time to approval for those with less experience. These data suggest there is a significant learning curve involved in the skill of submitting IRB applications, and that interventions such as training tools for less experienced investigators and streamlining processes may improve efficiency for all investigators, experienced in IRB processes or not.

Variation has also been reported in the IRB approval process of a number of other multicenter research projects, including minimal risk and quality improvement studies [1-3, 5, 7, 16, 17]. Similar to our results, Thompson and colleagues found variation in the type and speed of IRB review among five sites participating in a cardiovascular surgical process study. Others who perform minimal risk, multicenter research have reported variation in the primary care, emergency medicine, and surgical settings [2, 5, 7]. Although barriers to IRB approval may be similar between these groups, it is not clear that these factors are generalizable to the critical care setting.

To our knowledge, this is the first report of IRB variation and barriers in a large multicenter, observational critical care study. IRB variation in a multicenter critical care trial has previously been reported [17]. However, that study was not minimal risk and involved informed consent. The report largely focused on changes made by different IRBs with regard to the informed consent document. In contrast, USCIITG-CIOS was a minimal risk, prospective, ecologic study that only involved de-identified data collection from study participants.

Indeed, factors other than those captured by our survey analysis may also play a role in the variation observed in our analysis. Such factors may include a poorly written application, use of confusing language, and lack of supporting documentation which may lead to confusion on the part of the IRB. In fact, one might surmise that the trend toward a longer time to application approval for less experienced investigators may be due, in part, to the quality of the IRB application. However, at both sites where the IRB application was denied, the submitting investigators had previously submitted 5 or more prior applications, and the Board at one of these sites specifically denied the application because they determined it did not meet waiver of informed consent criteria.

In our study, the high survey response rate is an important feature that provides a valid sample of critically ill patients in both academic and community settings, as well as in medical and surgical intensive care units. In addition to characterizing the types of variation present, individual factors were evaluated to determine whether they were significantly associated with this variation.

Weaknesses of our study include the potential for recall bias introduced by the survey design as well a retrospective analysis of the data. In addition, only a small proportion of community hospitals were included in the parent study, possibly limiting the generalizability to this type of research environment. However, many clinical trials are done in academic institutions which suggest our results may be similar to that of other clinical studies. Finally, qualitative data about IRB revision requests were not surveyed..

It is important that we emphasize the importance of the work done by IRB’s; our study focus in no way intends to minimize the essential work of ensuring patient safety by these groups. However, we wish to highlight the variable response to a similar protocol at different sites. By design, our study does not allow determination of whether investigator or IRB characteristics were responsible for variable IRB response. Our observational study only allows us to identify and describe the association of the variable IRB response, rather than impute causation.

In an effort to improve efficiency and productivity of clinical research, the NIH Roadmap suggests increased establishment of and access to clinical research networks (CRN) [20]. A large scale centralized IRB process, currently modeled by the National Cancer Institutes, Canadian Centralized IRB, and the American Academy of Family Practitioners, represents one method to streamline the IRB application process for multicenter trials [21, 22]. While a centralized IRB oversees the general approval process in this model, the local IRB retains the right to review how a protocol will perform in a particular environment and modify accordingly. In addition, a recently started collaboration between some IRBs, called IRBShare, has provided another framework for standardizing and expediting the approval process for multicenter studies. IRB Share allows sites to rely on the initial review of another site to get a study started, yet retain overall IRB oversight at the local level for amendments, adverse events, protocol violations, and continuing reviews. Inherent in the IRB Share process is the sharing of documents, including redacted committee meeting minutes and IRB risk to benefit and vulnerable population determinations.

The potential benefits of such a process include increased efficiency, consistency, collaboration among IRBs, and access to committees whose members have clinical areas of expertise [21, 23]. At a time where up to 18 revisions to a multicenter protocol result in 12 percent of sites withdrawing their applications [24], significant reform will be necessary to accomplish the goals set forth by the NIH Roadmap.

There are many potential ways to improve the IRB application and approval process such as the creation of a centralized IRB that would be responsible for review and approval of multicenter trial protocols. The centralized IRB would then work in concert with local IRBs that would focus on logistical aspects of protocol implementation as well as ensuring that adequate resources are available to safely and ethically carry out the protocol. We propose that an optimal path to IRB approval of low-risk, multicenter studies is composed of several components: 1) the preservation of the health and welfare of human subjects 2) that additional human and non-human resources such as online, educational modules are available for less experienced IRB investigators throughout the IRB application and approval process, and 3) that efforts are made to increase consistency of interpretation and application of federal regulations, including potential centralization or sharing of IRB duties and determinations.

Conclusion

We have shown that significant variation exists in the IRB approval process of a multicenter, observational critical care study, including the IRB submission process, application timeline, as well as IRB committee concern regarding waiver of informed consent. The use of training tools for less experienced investigators and the potential use of a centralized IRB or increased IRB collaboration are two potential methods to both facilitate a more expeditious IRB approval process and broaden the base of potential sites for inclusion into multicenter, observational critical care trials.