Transitioning to the NIH Single IRB Model: Piloting the Use of the SMART IRB

Abstract

Background/Aims

Obtaining ethical approval from multiple institutional review boards (IRBs) is a longstanding challenge to multi-site clinical trials and often leads to significant delays in study activation and enrollment. As of January 25, 2018, the National Institutes of Health (NIH) began requiring use of a single IRB for U.S. multi-site trials. To learn more and further inform the research and regulatory communities around aspects of transitioning to single IRB review, this study evaluated the efficiency, resource use, and user perceptions of a nascent IRB reliance model (SMART IRB).

Methods

This research was embedded within INVESTED—a multi-site trial of two influenza vaccine formulations. In the first year of the trial, a sample of sites agreed to use the developing SMART IRB model and participated in its evaluation. In keeping with a least burdensome approach, short surveys were developed and obtained from each reporting entity (relying sites, non-relying site, lead site, and reviewing IRB). Data regarding time to IRB approval and site activation, costs, and user perceptions of reliant review were self-reported and collected via the survey form.

Quantitative and qualitative analyses were performed, with costs analyzed as actual vs. estimated due to lack of established baseline cost data.

Results

Thirteen sites ceded review and received IRB approval. Mean time to approval was substantially faster in sites that ceded review using the SMART IRB model vs. the one site that did not cede review (81 vs. 121 days). The mean time to approval was also faster than published averages for academic medical centers (81 vs. 103 days). Time to first enrollment was faster for ceding sites vs. the non-ceding site, and also faster compared to published averages (126 vs. 149 and 169 days, respectively). Costs were higher than estimates for local IRB review and approval. Nearly half (47%) of stakeholders reported being very satisfied or satisfied with the reliance experience, although many noted the challenge related to institutional culture change.

Conclusions

Implementation of a single IRB represents a shift in practice and culture for many institutions. Evaluation of the reliance arrangements for this study highlights both the potential of, and challenges for, institutions as they transition to single IRB review. Although efficiencies were observed for study start-up, we anticipate a learning curve as institutions and research teams implement necessary process and resource changes to adapt to single IRB oversight. Findings may inform research teams but are, however, limited by the relatively small number of sites and lack of a control group.

Introduction

For studies that fall under the purview of federal regulations, institutional review board (IRB) review and approval is required for non-exempt research involving human participants. Review of human subjects research has become more challenging with the evolving research landscape, particularly with the increasing number of multi-site studies.¹ In recent years, there has been growing concern that IRB reviews of multi-site research have become duplicative, inconsistent, and often a barrier to timely study initiation. It has also been noted that multiple IRB reviews frequently do not result in substantive changes with regards to human research protections, but rather focus on administrative or organizational requirements.²

In an attempt to address these concerns and foster more collaborative and efficient research, the National Institutes of Health announced a new policy for the use of a single IRB. This policy took effect on January 25, 2018, with the expectation that “all sites participating in multi-site studies involving non-exempt human subjects research funded by the National Institutes of Health (NIH) will use a single Institutional Review Board (sIRB) to conduct the ethical review required by the Department of Health and Human Services regulations for the Protection of Human Subjects at 45 CFR Part 46.”^3,4 The NIH further clarified that if a multi-site study involves both domestic and foreign sites, the domestic (U.S.) sites will be expected to use a single IRB with the foreign sites allowed to use their own IRBs or Ethics Boards.³

In advance of this policy, the NIH’s National Center for Advancing Translational Sciences (NCATS) funded initiatives to develop tools and workflows to help institutions and research teams transition to sIRB review. The first NCATS initiative, launched in 2014, concentrated on developing consensus regarding a master IRB authorization agreement (IAA) that could be adopted nationally to promote reliance arrangements and eliminate time spent in negotiating IAAs. This initial effort, called IRBrely, was transformed in 2016 into the Streamlined, Multisite, Accelerated Resources for Trials IRB Reliance (SMART IRB) Platform to assist the research community in implementing the sIRB policy. The core of SMART IRB is a master common reciprocal IAA that allows participating institutions to easily enter into reliance arrangements with other SMART IRB-participating institutions without needing to negotiate IAAs on a study-by-study basis.⁵

Working in collaboration with the SMART IRB team, the National Patient-Centered Clinical Research Network (see PCORnet.org) coordinating center evaluated SMART IRB as implemented in the context of the Influenza Vaccine to Effectively Stop Cardio Thoracic Events and Decompensated Heart Failure (INVESTED) trial—a large NIH-funded trial comparing two doses of flu vaccine in high-risk cardiovascular disease patients. The SMART IRB model, which was still under development at the time, was piloted during the first year of INVESTED.

Methods

Procedure

Fifteen PCORnet sites participated in the first year of the INVESTED trial. After consultation with the study principal investigator, the University of Wisconsin – Madison (UW) site agreed to serve as the reviewing IRB. The UW Health Sciences IRB director worked with the INVESTED principal investigator to identify key resources that would be required to support reliance arrangements for this study, including involvement of the UW Office of Clinical Trials to provide regulatory support, an IRB facilitator, and a lead study team.

Once this key infrastructure was decided upon and put into place, implementation of single (SMART) IRB began. First, the UW Office of Clinical Trials was responsible for preparing and submitting IRB applications for all relying site study teams to the reviewing IRB. As required by SMART IRB standard operating procedures, the principal investigator identified a lead study team to coordinate communication between the reviewing IRB and relying institution study teams. Second, the IRB facilitator was a member of the UW Health Sciences IRB office and provided key support for communication between the reviewing IRB, lead study team, and relying institution points of contact (POCs). Finally, the lead study team identified POCs for each of the relying site study teams. As illustrated in Figure 1, the lead study team played a key role in facilitating communication between the reviewing IRB and relying site study teams and relying site POCs, all of whom were personnel from IRB offices in this case.

Using a draft version of the SMART IRB master agreement, the IRB facilitator for the reviewing IRB reached out to relying institution POCs to determine interest in a reliance arrangement for the INVESTED trial. If the reliance arrangement was acceptable, the lead study team helped collaborated with the POCs from relying site study teams, working with their institutions to provide the required information that would allow them to finalize a determination regarding a reliance arrangement. Given the novelty of the situation, the principal investigator and UW Health Sciences IRB director conducted two webinars to educate the potential relying institution POCs and site study teams about the study and proposed reliance arrangement. If the sites agreed to cede review, the following occurred:

• Because the SMART IRB master agreement had not been finalized, a study-specific “joinder” to the draft agreement was sent to the designated relying institution POC at each ceding site for them to work with the appropriate organizational official to sign. (Time for this step would have been eliminated if the SMART IRB master agreement had been launched and all institutions participating in INVESTED had already joined.)

• The ceding site was asked to provide local context information through an online survey, created by the UW Health Sciences IRB in collaboration with the lead study team, about local considerations (e.g., relevant state laws or institution policies that could affect IRB review, whether the institution had “unchecked the box” on its Federalwide Assurance [FWA], and institution-specific required informed consent language).

• The UW Health Sciences IRB provided a letter documenting that it would serve as the reviewing IRB on behalf of the relying institution.

Data collection and analysis

The efficiency of the single IRB model using the draft SMART IRB reliance agreement was evaluated by calculating achievement of, and time required to achieve, select study milestones at the 15 sites. The milestones included time to sign a pilot-specific joinder to the draft SMART IRB master agreement, time to obtain IRB approval for each relying institution, time to site activation, and time to enroll the first patient.

Activities related to the single IRB process were captured and reported by a specific POC at relying sites, the non-relying site, the Reviewing IRB and the lead study team. Relying institution POCs kept logs of the time (hours) spent deciding to cede, time to review local contextual surveys, and time to submit materials to the reviewing IRB. The one non-relying site reported hours to prepare, submit and receive approval from the local IRB. The reviewing IRB reported staff times (hours) required to review local contextual survey data and to review changes in the research (such as addition of site via amendment and personnel changes). The lead study team reported time (hours) required to create the informed consent template, time spent finalizing the study protocol, and time spent pre-reviewing ceded site information prior to submission to the reviewing IRB.

The POC for each reporting group (above) was asked to report cost (hourly wage) for the time (hours) associated with the activities mentioned. Costs were then estimated by multiplying this time by site-reported average POC hourly wage. POCs were also asked to estimate costs of what they expected were the research not reviewed under single IRB.

The cost is reflective of the person(s) performing the tasks mentioned above. This was usually the POC. In instances of more than one person performing these tasks, the salary information was averaged to produce one “hourly wage.”

User perceptions of the SMART IRB process were collected via a mix of self-administered and interviewer-facilitated surveys with key POCs, including those from the reviewing IRB, reviewing Office of Clinical Trials, a sample of the relying IRBs, lead study team, and a sample of relying site study teams. First, POCs were given a survey and asked to rate (for example, their overall satisfaction with the process) on a scale of 1–5, where 1 indicates “least satisfied” and 5 “most satisfied.” Respondents were asked for their role in the study, their overall satisfaction with the single (SMART) IRB model, overall satisfaction with the Joinder Agreement process, and overall satisfaction with the division of responsibilities using this model. Additionally, they were asked if the single IRB model affected their workload, and if so, in what ways. Respondents were also asked for feedback regarding challenges, perceived barriers, and recommendations for the use of the SMART IRB platform. Phone interviews were conducted with those who were unable to complete and or return the survey and with those who submitted surveys that were unclear or required additional information. The POC-provided survey and interviewer notes served as the underlying data.

Quantitative data were descriptively analyzed; for cost data, both means and medians are reported to capture variation in responses and outliers. Qualitative data were reviewed for repeating factors and summarized accordingly. Data were collected between August 2016 and April 2017 (the first year of the trial).

Given the lack of a control group in this project, published comparison data were used to benchmark the time to IRB approval and time to first enrollment metrics. Briefly, the comparison data used in the results section are median days reported by 19 organizations for 5,396 studies.¹

Results

Efficiency.

With a small number of participating sites, the project was not designed to be a controlled sampling of relying vs. non-relying sites. Fourteen out of 15 participating PCORnet sites ceded review during INVESTED’s first year. Thirteen of those received IRB approval during the first enrolling year and their data are included. For those that received approval, it took an average of 50 days from the time the site received the joinder to the draft SMART IRB master agreement to the time it was approved by the reviewing IRB. One site chose not to cede, so their review was performed by their local institutional IRB. Responses were received from the reviewing IRB (n=1), lead site (n=3), office of clinical trials (n=1), ceding sites (n=13), ceding IRBs (n=8), and non-ceding site (n=1).

From the date the site received the study package, the average time to IRB approval was 81 days for ceding sites, compared with 121 days for the non-ceding site (and 103 as reported as a published comparator for academic sites).¹ Similarly, ceding sites achieved first enrollment faster than either the non-ceding site or published averages¹ (126 for ceding site, 149 non-ceding site, and 169 published comparator for academic sites) (Table 1).

Table 1.

Time (in days) to approval and enrollment of first INVESTED study participants

	Ceded sites	Non-ceded site
	Mean (SD)	Mean (SD)
From study team’s receipt of protocol to IRB approval	81(31)	121(-)
From IRB submission to IRB approval	22 (11)	27 (-)
From receipt of protocol to site activation	116 (23)	142 (-)
From receipt of protocol to enrollment of first participant	126 (25)	149 (-)
From IRB approval to enrollment of first participant	41(16)	28 (-)

IRB, institutional review board; SD, standard deviation.

Initial Ceding Site Cost.

Mean cost incurred per ceding site for initial review was $2,357; of this cost, an average of $623 was spent making a determination to cede review, while an average of $1,734 was spent completing the actual work of providing local context requirements to the IRB (Table 2). Thus, for the 13 sites that received IRB approval, a total of $32,998 was spent (using imputed figures) at the site level. The average estimated (hypothetical) cost if the site had not ceded review was lower, at $861 per site ($11,193 extrapolated to 13 sites) The site that chose to maintain local IRB review instead of ceding review incurred a cost of $345 in processing its local IRB review.

Table 2.

Time and cost for ceding and lead sites, and reviewing IRB

Ceding site (n=11/14)*	Site hours	Site cost	Total cost
	Mean (median) SD		Actual Imputed*
Determination to cede	8 (6.0) 7.8	$623 ($270) $779	$6,850 $8,722
Providing local context	20 (10) 28	$1,734 ($540) $2,975	$19,079 $24,276
Estimated time and cost of preparation for local IRB review, if choosing not to cede	14 (15) 10	$861 ($675) $856	$7,751 $12,054
Lead site (n=1)	Hours/site	Cost/site	Total cost
	Mean (median) SD		Actual Imputed*
Prepare consent form	3 (2) 2	$100 ($64) $72	$1,296 $1,400
Finalize protocol	9 (6) 11	$298 ($192) $352	$3872 $4,172
Reviewing IRB (n=1)	Hours	Cost/hour	Total cost
			Actual
Educating and preparing the lead study team and relying site IRBs, and advising IRB review staff	48	$54	$2,600
Initial review of the study	34	$44	$1,500
Initial review of the study, including primary reviewer preparation and committee discussion	3	$767	$2,300
Changes of protocol to add sites	7.5	$43	$325
Changes of protocol for other ceded site changes	5	$43	$217

^*11 of 14 ceding sites provided cost data. Cost data are available from lead sites for 13 of 14 ceding sites. Where indicated, total costs are imputed for missing sites based on mean cost.

IRB, institutional review board; SD, standard deviation.

Lead Site Cost.

The total cost incurred by the lead site and its Office of Clinical Trials for preparing the initial submission was $5,168, which included time to both prepare a template consent form and finalize and approve the protocol (Table 2). When following the SMART IRB standard operating procedures, the lead site is also responsible for submitting study-wide and site-specific amendments to the reviewing IRB; for this study, there were 10 amendments (which included the addition of enrolling sites and site-specific personnel changes), and the total cost was $672. Of note, these figures are for the first year of review. An additional 9 amendments were since submitted for review, including several that affected consent form local content information.

Initial Reviewing IRB Cost.

The total reported cost for the reviewing IRB in obtaining IRB approval was $6,400 (Table 2). Of this cost, about $2,600 was spent in educational activities aimed at helping the lead study team, UW Office of Clinical Trials, and relying institutions understand the SMART IRB process and requirements. Without this educational requirement, the initial cost was $3,800. For the 10 amendments that were issued, a total of $542 was spent in the first year.

Due to the timeline of the study’s first enrolling year, continuing review data are not represented here. Additionally, there were no reportable unanticipated problems during this time. The amendments represented were for the purposes of adding study personnel or minor/moderate revisions to study documents. The non-ceded site had no amendment activity to report.

User perceptions.

At the end of the study period, 17 respondents representing the relying sites, the non-relying site, the lead study team and the reviewing IRB, participated in self-administered surveys and interviews. Of these respondents, 47% reported being satisfied or very satisfied with the SMART IRB experience; 24% were dissatisfied or very dissatisfied and 29% were neutral. The same proportions also reported being satisfied or very satisfied with the new division of responsibilities as they pertain to the single IRB process. The majority (71%) reported an increased workload as a result of ceding review. Of the sites that responded the work was unchanged, one respondent reported that, “It’s not more or less work, but different work.”

Sites were asked to describe benefits, challenges and local barriers, and suggestions for the future. Themes surrounding benefits of SMART IRB methodology include enhanced collaboration and communication, standardization of processes, consistency of information collection, and decreased site implementation time overall. In terms of the challenges reported, respondents cited institutional cultural barriers, as study teams and IRBs were accustomed to local IRB review and reluctant to spend time learning a new system. In at least one reported instance, the local IRB conducted its own full IRB review in addition to the review by the sIRB. Other challenges involved a need for educating study teams regarding the sIRB process, extra time involved in learning a new process, and understanding the effect of a sIRB on local context requirements (Table 3). Common reported suggestions for the future included the need for greater education around sIRB review, increased need for standardization across sIRB processes, and harmonization across IRBs to reduce review burden.

Table 3.

Challenges and local barriers

Relying Sites (study team and IRB) n= 14	Reviewing IRB n=1	Lead study team n=1
Challenges
Additions to or changes in personnel took longer than anticipated (5)	-	-
Burdensome method of obtaining local context for reviewing IRB (2)	-	Increased time for the collection of recruitment information (2)
Need education of local study teams with regard to single IRB process (2)	Communicating reviewing IRB requirements to local sites	-
Creation of user accounts to access local eIRB resulted in added start-up time in some instances	-	Creation of net IDs to access local eIRB resulted in added start-up time in some instances (2)
Even though sites ceded review, local IRB required submission and review, which increased workload at ceded site (2)	-	-
Negotiation of HIPAA language within ICF	-	-
Currently do not have best practices and internal process improvements in place at local institution with regard to single IRB	-	-
Local barriers
Negotiation of HIPAA language within ICF	Negotiation of HIPAA language within ICF	-
Communicating reviewing IRB requirements to local sites; need for education of all local participating entities on single IRB processes	-	-
Clear definition of “engaged in research” and whom should be listed as key personnel on study documents; burdensome approach to obtaining local institutional context; local IRB issues with vetting and reporting of COI (2)	-	Customization of an unapproved consent form, which led to greater start-up time for some sites; sites wanted to modify more parts of the consent than were allowed with the template they were given
-	Increased time for the collection of recruitment information	-
-	Even though sites ceded review, local IRB required submission and review, which increased workload at ceded site	Some local IRBs required review of study-specific documents in addition to review by reviewing IRB

Numbers in parentheses reflect number of times reported; similar themes reported on same row.

COI, conflict of interest; HIPAA, Health Insurance Portability and Accountability Act of 1996; ICF, informed consent form; IRB, institutional review board.

Discussion

This study of early implementation of reliance arrangements used a draft of the SMART master agreement and standard operating procedures and suggests that the model yields promise for streamlining IRB review in multi-site trials. Time to IRB approval and first enrollment was faster for SMART IRB sites than that reported in the literature or for the one site that did not use SMART IRB. User perceptions of SMART IRB were mixed, but users provided actionable feedback that may further improve the process. As expected, it cost more to implement SMART IRB methodology than to not implement it, yet the cost is expected to decrease as all parties become more familiar with the process and as efficiencies are leveraged.

Two of the challenges in implementing the sIRB approach, as outlined in The New England Journal of Medicine,⁶ came to bear in this study. First, local IRBs needed to adapt their communication and information systems as well as protocols to align with the sIRB model, which required additional staff time and resources. Second, efficiencies in study approval gained by the streamlined approach were mitigated because some local IRBs required duplicative review.⁶

Overall, inefficiencies in this study’s use of a SMART IRB were likely related to the need for culture change among institutional IRBs and study teams. With new processes, roles, and responsibilities, it will take time to adapt and review institutional and other policies to ensure alignment. In many cases, local and institutional policies are based on administrative—and not legal or safety—requirements, and thus are subject to ongoing review and refinement. Additional educational resources and support will be needed as more studies start to adopt SMART IRB. SMART IRB and other sIRB models must be rigorously evaluated and open to adaptation, and the respective responsibilities of each stakeholder should also be carefully considered.^7,8 Indeed, since the time of this evaluation, many additional resources have been developed for SMART IRB and its many stakeholders, including educational webinars, online resources such as templates and instructions, and budgeting information, and committees to continue to harmonize the process (see www.smartirb.org).

This evaluation was subject to a number of factors that should be considered. As stated previously, the evaluation was not designed as a controlled sample of relying vs. non-relying sites. With a comparison of only one non-relying site, it is difficult to gauge site-specific factors that may have increased or decreased efficiencies. It is also important to recognize that the participating institutions were exclusively academic; thus, findings may not reflect the experience of working with community-based sites. Further, as an influenza vaccine study, the INVESTED study is unique in its short window for seasonal enrollment and related timelines. It remains unclear whether the reported timelines would be observed in trials that do not have the same pressures of time and, therefore, motivation. For the evaluation, the cost data may not reflect all associated costs of implementation, as respondents reported difficulty in tracking time specific to SMART IRB vs. other activities. The lead study team, for instance, provided significant support to sites in completing the local context questionnaire; however, this time was not clearly differentiated from general site start-up support, and so not fully reflected. Additionally, there are noted methodological challenges with collecting hypothetical cost data. Since this was a new process, baseline data were limited; hence the study team approach was to estimate what costs were incremental above the local IRB review process.

Now that the NIH policy for a sIRB is in effect, researchers, institutions, policy-makers, and funders must continuously revisit existing processes and regulations to ensure that research participants are protected, resources are utilized effectively, and important research questions are answered. The number of pragmatic trials and multisite trials continue to increase, further compounding the need to simplify IRB and other processes and standards that affect timelines for initiation of research studies. Networks including PCORnet and the Trial Innovation Network (TIN),⁹ an initiative of the Clinical and Translational Science Awards (CTSA) program, have identified SMART IRB as one mechanism for conducting trials in a more pragmatic and sustainable manner. Yet, as regulatory experts have noted, this may not be a “panacea”.¹⁰ Improvement can and must be continuous, informed by results, and targeted to ensure safe and impactful research.