Reducing the Single IRB Burden: Streamlining Electronic IRB Systems

Abstract

Electronic institutional review board systems (eIRBs) have become an integral component in ensuring compliance with Human Research Protection Program (HRPP) and IRB requirements. Despite this, few of these systems are configured to administer the single IRB (sIRB) process mandated by the National Institutes of Health (NIH) for multisite research. We interviewed 103 sIRB administrators, chairs, members, and staff members about their experiences with sIRB multisite research review. We observed three main obstacles to adapting existing eIRB systems to accommodate the sIRB process: (1) Existing systems are not designed for sIRBs and are not configured to administer sIRB responsibilities, (2) they are not interoperable, and (3) resources to improve existing systems are lacking. Our findings suggest that IRBs that act as an sIRB will need major changes to their electronic systems in order to accommodate sIRB processes. These difficulties threaten both the ability of IRBs to focus on ethical rather than bureaucratic problems and the efficiency of multisite trials.

Introduction

The National Institutes of Health (NIH) Policy on the Use of a Single Institutional Review Board for Multi-Site Research (NIH sIRB Policy), effective on January 25, 2018, mandates single IRB¹ (sIRB) review of domestic NIH-funded multisite studies (National Institutes of Health 2016). There have also been other efforts to encourage the use of sIRBs (21st Century Cures Act 2016). These modifications have changed the scope and workflow of institutional review boards (IRBs). Previously, local IRBs (LIRBs) conducted their own reviews of the same multisite protocol, even when another IRB had already reviewed and approved the protocol. In the sIRB model, a single IRB serves as the reviewing IRB for all sites involved in a research study. The NIH sIRB Policy was intended to streamline the IRB system, increase efficiencies, reduce workload and duplication, and thereby accelerate approval of studies with multiple research locations (National Institutes of Health 2015); whether it will achieve these goals already has been hotly debated (Ervin et al. 2016; Moon 2017; Taylor and Ervin 2017).

One component of simplifying and making the overall IRB process more efficient has been the introduction of electronic IRB systems² (eIRBs) (He et al. 2014). These systems have refined and, in some ways, significantly modified IRB workflows, thereby probably making IRBs more efficient. He et al. (2015) highlighted the benefits of eIRB systems in enhancing efficiency and streamlining the research review workflow through automation and electronic distribution of materials to IRB members and staff, and tracking protocol-related events and activities (He et al. 2015). Responsibilities that are commonly tracked and incorporated into an eIRB system include, but are not limited to:

• Protocol development, protocol submission, tracking of existing protocols, and new and continuing reviews (Turisco et al. 2005; He et al. 2014; He et al. 2015; Association for the Accreditation of Human Research Protection Programs 2017).

• Conflict of interest disclosure.

• Investigator education and training (Turisco et al. 2005; He et al. 2014; He et al. 2015; Association for the Accreditation of Human Research Protection Programs 2017).

• Regulatory compliance, monitoring, and reporting (e.g., for adverse events) (Turisco et al. 2005; He et al. 2014; He et al. 2015; Association for the Accreditation of Human Research Protection Programs 2017).

Because of the NIH mandate to use sIRBs, local IRBs have had to incorporate new and additional tasks, including:

• Developing, executing, and monitoring reliance agreements (with each study site).

• Maintaining institutional profiles and confirming that local requirements are being met.

• Tracking site-specific consent forms.

These items are not easily incorporated into an eIRB system. More broadly, it may become difficult to capture adequately all responsibilities necessitated by the sIRB model across multiple sites if they are not part of a previously established and interconnected eIRB system. If the sIRB cannot reliably get the information it needs from individual study sites in an efficient way, it may not be meeting its responsibilities as a reviewing IRB and lapses in participant safety may occur.

Most academic institutions engaged in research utilize eIRB systems in some capacity, as do many commercial IRBs³, (Association for the Accreditation of Human Research Protection Programs 2017). According to the Association for the Accreditation of Human Research Protection Programs (AAHRPP), 91% of all AAHRRP-accredited IRBs use an eIRB system to distribute materials, 74% utilize an online IRB application, and 74% use an online system to facilitate protocol review (Association for the Accreditation of Human Research Protection Programs 2017). Despite the widespread use of eIRB systems, little is known about how these systems are integrated or linked between different institutions—a critical step needed for the efficient functioning of sIRB processes. In this article we report a study exploring the potential of, and barriers to, implementation of eIRBs for sIRBs.

Methods

Data collection

As part of a larger study of sIRBs, we interviewed 103 participants, including sIRB staff, chairs, and members from 20 active IRBs that participate in or intend to participate in sIRB reviews of multisite studies. Participants’ levels of IRB experience ranged from less than 1 year to more than 20 years of experience (see Table 1). We enrolled three types of sIRBs: commercial, academic, and government-based. Recruitment and interviews were conducted from March 2015 until August 2017.

Table 1.

Demographic background of interviewees (N = 103).

Role	n	%
Chair	20	19.4
Director	27	26.2
Member	30	29.1
Staff	26	25.2
IRB experienced,a years
<1	3	2.9
1–4	22	21.4
5–9	23	22.3
10–19	29	28.1
20+	16	15.5

^aData missing on 10 participants.

Our recruitment processes included sending at least two e-mails, followed by a phone call, inquiring about sIRBs’ willingness to participate in our research. IRBs were identified using listings of relevant IRBS from (1) the Association for the Accreditation of Human Research Protection Programs (Association for the Accreditation of Human Research Protection Programs 2018), (2) the Office for Human Research Protections’ registration portal (Office for Human Research Protections 2018), (3) the Public Responsibility in Medicine & Research’s website (Public Responsibility in Medicine & Research 2018), and (4) a list of commercial IRBs maintained by circare.org (Citizens for Responsible Care and Research 2020). In total, we contacted 49 sIRBs by e-mail: 30 commercial, 11 academic, and 8 government-based. Ultimately, we enrolled seven commercial sites, nine academic sites, and four government-based sites. The site response rate for participation was 40.8% (see Table 2 for response rates by type of IRB).

Table 2.

Participation rates by type of single IRB.

					Noncommercial
	Total		Commercial		Academic		Government		Total
	n	%	n	%	n	%	n	%	n	%
Totals for participation	20	40.8	7	23.3	9	81.8	4	50.0	13	68.4
Totals for nonparticipation	29	59.2	23	76.7	2	18.2	4	50	6	31.6
Total	49	100%	30	100%	11	100%	8	100%	19	100%

Procedures

Interested participants were asked to participate in a 1-hour, semistructured, audiorecorded interview by phone or in person. The interviews examined participants’ experiences with conducting sIRB reviews, including benefits and limitations of the sIRB process, establishment of sIRB procedures, interactions with local IRBs, resources required for operations, and their experiences with electronic submission systems. Further information on study methods is detailed in Lidz et al. (2018).

The study received approval from the University of Massachusetts Medical School IRB and the New York State Psychiatric Institute IRB.

Data analysis/coding

All interviews were transcribed and any identifying information (e.g., site, location, names) was redacted. A codebook was developed by the investigators and three coders. It was based in part on the codebook from a previous NIH-funded study of local IRBs (Candilis et al. 2012; Lidz et al 2012). Three coders, all master’s-level senior staff, initially coded the interviews separately. Interrater reliability using Cohen’s kappa for individual interviews ranged from .71 to .85. Disagreements were resolved by review with the principal investigator and a co-investigator. Codes analyzed for this paper include “resources required for operations,” “discussions of electronic submission systems and other electronic IRB systems,” and “protocol’s process” (e.g., how a protocol is distributed and dealt with). Demographic information is provided in Table 1.

Findings

Interview participants reported that one subset of issues they experienced was obstacles in implementing sIRBs, in part due to problems with electronic IRB systems. Specifically, they indicated that eIRB systems were not designed for sIRBs and are ill-equipped to manage responsibilities and procedures that a sIRB must deal with on a day-to-day basis (e.g., maintaining structured protocol records and communications across multiple study locations, capturing local context and knowledge) (He et al. 2014; National Institutes of Health 2017). Another difficulty is achieving interoperability, or, more specifically, how to integrate eIRB systems to work with one another. Interoperability would allow for computer-to-computer communication between sites regardless of the platform or software version that is used. Finally, reengineering and implementing these systems is costly, is time-consuming, and requires special expertise.

1. Current eIRB systems are not designed for sIRBs. Acting as an sIRB is a new role for many IRBs. Participants noted that most eIRB systems have been developed to suit institution-specific needs to review local research:

   • We were trying to be nice and let everyone keep their own systems and I think that’s part of the trouble [with] IRB authorization agreements. We’ve all developed local [electronic] systems based on our own needs and our own populations and our own research portfolios and our own funding. (Director 1, Academic IRB 11)
     Participants noted that if systems are not adapted for the sIRB role, efficiency may be reduced, as certain requirements may need to be tracked using other, less ideal methods:
   • It may seem a little trivial but one [of] our big hurdles is adapting our electronic administration system to administer a central [single] IRB … it’s a pretty much state-of-the-art electronic system, but it wasn’t designed to administer [a] central IRB. And we spent a lot of time trying to make it work to do that, doing workarounds: maintaining Excel spreadsheets outside of the database to keep track of things. (Chair 1, Academic IRB 4)
     Even when eIRBs are designed to manage sIRBs, they may have problems with different models of sIRBs. Some sIRBs may expect a lot of input from the relying local research sites regarding the protocol review, while others approach reviews in a top-down manner. Some sIRBs are commercial and emphasize efficiency, while those in government and academic settings must deal with unique internal organizational rules. Agreements made between the reviewing and relying IRBs regarding the parties’ roles and responsibilities throughout the sIRB process may also vary on a study-by-study basis (Lidz et al 2018). Since little is known about the best way to implement an sIRB (Greene and Geiger 2006; National Institutes of Health 2017), sIRBs may change their model or processes when they find that another method works better. This creates additional problems for eIRB systems:
   • I think the biggest challenge is you don’t know what you don’t know until it’s happening. We stopped using paper and started using a new electronic system … and then we started doing a different process (the sIRB model) than what was in mind when that system was designed. (Director 1, Academic IRB 4)

2. Interoperability of systems.

   Interoperability refers to the extent to which a system or systems can share and use information produced by another system. Simply put, to say something is interoperable means that two different computer systems can communicate with each other and utilize the data that one or the other produces. There are two components to this challenge:

   1. How to permit relying investigators and staff to have access to the reviewing IRB’s system.
   2. How to link disparate systems used by different institutions.
      Participants reported that permitting relying institutions to have access to the reviewing institution’s system, and vice versa, is a challenge. Most academic institutions have stringent data security requirements and, as a result, are unlikely to give outside personnel access to their systems. Even if the relying institutions’ personnel can get into the other system, they may be unfamiliar with its operation and may have problems using it.
   • No one’s using the same electronic systems and we still have some major issues with that, because it’s all behind a firewall … and right now we don’t have an ability for outside investigators to do this [get into the system]. This all has to be centralized. (Director 1, Academic IRB 2)
   • The Pis [principal investigators] don’t have access to our system. They can’t get in there directly, such is the way it was set up. (Staff 1, Academic IRB 4)
     Another related problem is the difficulty of achieving interoperability between eIRB systems so that protocol-related information does not have to be entered separately into both the reviewing and relying institutions’ systems. Some sites at which interviews were conducted were able to work around this issue by using procedures to incorporate outside users into their systems. Participants noted that commonly a contact person or liaison would be designated for both the reviewing and the relying institutions. This individual is typically responsible for transmitting information from the relying site and then inputting it into the reviewing IRB’s electronic system:
   • We ask them [relying institution] to file a user access form, which is what we call our permissioning for our electronic system. They designate a liaison from their institution to have access to all the studies that are in our system. So they can go in at any time and view records and run reports as well. (Director 1, Independent IRB 4)
     This process, however, is labor-intensive and time-consuming, and may result in avoidable errors that occur in transition from the relying to the reviewing site. Even when eIRB systems are theoretically interoperable, difficulties can arise when one institution is using a different version of the platform or has made modifications to the platform to adapt it to its specific needs.

3. Limited resources.

Some IRBs that sought to update their systems to accommodate sIRB processes reported lack of resources as a challenge. This problem was twofold: institutions either (a) lack financial resources for system upgrades and/or (b) do not have the technological expertise and/or training to develop and implement a more robust, interoperable eIRB system.

Financial resources

Many institutions have already invested time and money in eIRB systems for their own IRBs:

• About two-thirds of the academic centers now [2015] have electronic submission systems if you look at the 2014 AAHRPP reports. Those systems are not set up to have external people log in and do stuff. So it means that there’s a huge cost to reengineer those systems to have those IRBs function as a central [single] IRB and not everybody is going to want to do that. (Member 1, Academic IRB 12)

Those institutions that have not yet transitioned from a paper system may not be well suited for the sIRB role. Participants noted that financial resources can be a barrier:

• So I would love to improve some things that only capital would be able to improve: our website and our electronic submission [system] which is right now a joke. Those all require money and we don’t have it. (Director 1, Independent IRB 1)

• The main PI (principal investigator) has to have the infrastructure to facilitate all of this. They have to collect all of the information and submit it. So institutions are going to have to have resources to start getting these outside people in. (Chair 1, Academic IRB 12)

• If this goes through [the NIH sIRB requirement for multisite studies] we’re going to have to invest some money in the electronic system … somehow those things [sIRB-related activities] have to come in through our system. (Director 1, Academic IRB 2)

Another issue is adequate staffing, as the sIRB will need to manage and track a wide array of protocol- and site-specific documents that go beyond what is required of a traditional local IRB:

• The staffing shortage and not having a fully electronic system, I think a lot of our other issues would be solved by those two issues being solved. (Director 1, Government IRB 13)

• I think it’s, it’s underappreciated by the larger institutional aspect of just how much brain power and resources and energy goes into these reviews, and I think, I’m constantly surprised but the lack of resources that our office is given [to process these reviews]. (Member 1, Academic IRB 1)

• I think that the amount of resources and time that it takes to put these arrangements together, and also the resources to provide reviews for each study as a central IRB is tremendous and doesn’t seem to be something that’s in the NIH’s budget. And then, when you are relying on another site, there is so much of this behind the scenes work that maybe isn’t captured in real IRB work stuff [like] going to a committee [meeting]. The amount of time that it’s taking for IRB staff, general council’s offices, other institutional officials in reviewing documents and arrangements. That is not something that has been captured in any of the data that’s been presented … I think that’s the biggest thing: the time, the energy, and the cost obviously. (Director 1, Academic IRB 9)

Technological expertise/training

Technological expertise is required to redesign or install necessary systems and to maintain and adapt those systems for use with different IRBs.

• If you’ve got an IRB that doesn’t have exposure to working with other institutions or central IRBs, it’s asking a lot for them to develop systems that are going to support those efficiently for their institution. (Director 1, Academic IRB 12)

It also takes time to learn how to use an electronic system. eIRB systems are no exception. Participants noted that an appropriate amount of time must be allocated to training and practice so that IRB personnel are able to use these systems effectively and efficiently:

• When we switched to the electronic system that we’ve had in place for a few years now, it was so different than what we were doing and we had to spend a few of those [meetings] just to help and navigate them [users of the system] through and give them tips. (Staff 1, Academic IRB 4)

  sIRBs are more complex and have many more users, so additional time and resources for training may be needed.

Discussion

We have described the reports of IRB members and staff concerning three main challenges to utilizing eIRB systems for sIRB reviews of multisite studies. Our data suggest that it may be difficult to achieve the goals of the NIH sIRB Policy to foster a more efficient and effective IRB process without well-designed, interoperable eIRB systems. As already discussed, current eIRB systems are not designed for sIRBs but rather for traditional IRB roles and managing local responsibilities. Because of this, eIRB systems have been customized to do a broad variety of tasks based on institution-specific needs. Due to such variability, it will be challenging to ensure that a redeveloped system is engineered appropriately to suit the sIRB’s role. eIRBs’ systems must likely incorporate a more robust series of data types to accommodate variations in institutional structures and to expand the systems’ capacity to handle such data types on a multisite scale. There are many types of relationships between sIRBs and the institutions relying on them for IRB review, and eIRBs are not yet configured to manage the different types of information required. However, there are certain documents that every eIRB system should accommodate, including:

• Standard operating procedures for central/single IRB functions.

• Reliance agreements with each study site.

• Institutional profile and procedures from study sites—this is important for local context considerations (e.g., standard of care may vary from one hospital system to the next; hospitals with religious affiliations may have specific requirements or restrictions; etc.).

• Master protocol and consent form—this must be approved by the reviewing IRB and distributed to each relying institution. The distribution of this document should be documented within the system.

• Site-specific consent forms.

• Confirmation of whether or not local ancillary committee requirements, or other requirements outside the scope of a sIRB, are met by each relying institution (e.g., biosafety, investigator training/credentials, etc.).

• Progress reports/continuing review information from every relying institution.

• Reports of unanticipated problems, adverse events, violations, and other forms of noncompliance from each institution, which must be reported as they occur.

• Reviewing the sIRB’s responses to progress reports, noncompliance, unanticipated problems/adverse events, and violations from relying institutions.

Moreover, given that systems are customized for specific institutions, an interoperable solution for all parties in the sIRB process will be difficult to realize. One potential means of achieving interoperability would be to have uniformity in standards and data types. Initiatives such as the Clinical Data Interchange Standards Consortia (CDISC) have great potential, but have yet to be fully implemented nationally. The CDISC Protocol Representation Model (PRM) simplifies interoperability using standardized, validated protocol fields (The CDISC Protocol Representation Group (PRG) 2009). If these standards become widely established in eIRB system architecture, they may facilitate interoperability across disparate systems (CDISC 2018). Implementing such a system will reduce the time and costs associated with developing, tracking, and maintaining multisite protocols (Hood et al. 2005; Vadakin and Hinkson 2012; Bian et al. 2014; He et al. 2015).

Data-sharing platforms may prove to be a viable yet temporary solution to interoperability concerns by providing a shared locus where both reviewing and relying IRBs can upload and download data. Although they do little to reduce duplication in sIRB work, these data exchange portals do aid in compiling and tracking necessary information for the reviewing IRB, the relying IRB, and the study team. One such platform, the IRB Reliance Exchange Portal (IREx),⁴ is designed to support the reliance process and sharing of information throughout the life of a study (IRB Reliance Exchange 2018). Once a reviewing IRB is chosen, IREx can track reliance agreements among participating sites, capture local context and local issues, and facilitate communication and notification of required actions across sites, in addition to serving as a central repository to manage protocol approval and supplemental documentation (University of Vanderbilt 2018).

In spite of the many advantages of IREx and other data sharing platforms, they are not a long-term solution. The portals do not link disparate systems, but instead facilitate exchange of information that can then be inputted into in-house eIRB systems to permit tracking of protocols from initial approval to study closure. It would be ideal for data exports from these IRB data exchange platforms to be imported directly into existing eIRB systems, as, for example, data from the Research Electronic Data Capture (REDCap) system can be imported easily into a number of statistical analysis packages. This may become feasible with more widespread adoption of CDISC recommendations for data standards. However, these data-sharing platforms are built for specific reliance networks. Those institutions that are not within a specific reliance system (e.g., SMART IRB) may not have a readily available mechanism for sharing information and, as a result, study information may be e-mailed back and forth rather than inputted into a secured shared system. This method may have both implications for staff time and security.

Finally, we described lack of financial and time resources as a major roadblock to integration of systems and efficient review processes. Our participants reported inadequate funding for eIRB system development, and some stated that their institutions lack funding for even basic eIRB systems. Others have invested in systems that are not built to manage a sIRB and may be hesitant to update their systems to ones better suited to sIRBs. Modifying/upgrading these systems is costly and time-consuming. Transitioning to a commercially available system may be even more onerous. Since some systems have been used by institutions for many years, the efforts required to switch now are even greater, as they would potentially require complete overhauls of existing methods for managing research reviews.

Integration of a system designed for sIRBs will require stakeholders to dedicate time, personnel, and resources (with attendant costs), as well as their regulatory and technical expertise. NIH has agreed that some sIRB activities can be included in grant budgets as direct costs if the costs are unique to sIRBs. Since eIRB systems will need to be adapted for the sIRB process, some costs associated with advancing these systems could be interpreted as direct costs (National Institutes of Health 2016). However, since appropriate budget justification is needed, not all institutions may be eligible. In any case, many costs will not be covered. If an institution lacks the means and financial resources to function as a reviewing sIRB, it has the option to outsource review to an external sIRB when required.

Even with the continued interest in informatics, data sharing, and interoperability in the clinical research domain, research on informatics in the IRB world is lacking (He et al. 2014). Despite the widespread use and availability of eIRB systems, the three challenges highlighted in our data identify a need for a consensus on data standards to aid the integration and interoperability of such systems. There is a major need to better define models for integration of sIRB processes into eIRB systems. Estimates of costs for development and implementation of a well-functioning interoperable eIRB system for use with sIRBS are also needed. If unresolved, these issues threaten to undermine the goals of the transition to sIRBs, which are largely based on increased efficiencies in the review process.