Changes in the institutional review board submission process for multicenter research over six years

Abstract

While collaborative research across sites is essential to increase the statistical power and generalizability of research findings, the need to undergo multiple IRB reviews is a challenge. The purposes of this paper are to describe changes in the IRB submission process in two national multisite studies before and after the implementation of the Health Information Portability and Accountability Act (HIPAA) Privacy rule (2002 and 2008), and to discuss implications for policy and practice related to human subjects research. In the second study there was a shorter mean approval time and reduced variability in the decision about the level of review, the mean number of pages per application doubled, and an increased proportion of IRBs required conflict of interest and data use agreements. Possible approaches to further enhance the efficiency and streamlining of the research review process are suggested.

Oversight of human subjects research is an ethical as well as a legal imperative. Such oversight is generally delegated to local institutional review boards (IRBs), whose mandate is to assure that regulatory and ethical standards are upheld in research protocols.^1,2 Although federal regulations regarding research are mandated only for protocols receiving federal funding, these same standards are generally applied to all research, regardless of whether a project receives external funding or is unfunded. Several recent reviews have reported wide variation and substantial challenges in how IRBs handle multisite studies.^3-5 To encourage standardization of human subject protections in research, IRBs or independent ethics committees must be registered through the Office of Human Research Protections (OHRP). There are currently thousands of such registered boards. In California alone, for example, there are >2,000 registered IRBs and in New York there are >1,400.⁶

In addition to OHRP oversight, the Health Information Portability and Accountability Act (HIPAA) Privacy Rule⁷ enacted in 2003, has resulted in major changes in how personal health information is handled. The IRB is charged with the responsibility of monitoring adherence to HIPAA regulations as part of the approval process for research protocols. In order to better understand the current status of the review of human subjects research and the impact of HIPAA on the IRB process, we examined our experiences in two national studies. The purposes of this paper are to describe changes in the IRB submission process in two national multisite studies before and after the implementation of the current HIPAA regulations, and to discuss implications for policy and practice related to human subjects research.

BACKGROUND

The Office of Human Research Protection (OHRP), http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.htm#46.110, is the federal lead office overseeing the system that protects the rights, welfare, and well-being of subjects involved in research and helps ensures that such research is carried out in accordance with the regulations described in the Federal Registry. In addition to maintaining the IRB registry, OHRP provides interpretation and guidance, educational programs and materials on human subjects research, and maintains regulatory oversight.

As defined by OHRP, the main purpose of the IRB is to protect the rights and welfare of human subjects who take part in research. The IRB reviews research in accordance with current OHRP and Food and Drug Administration (FDA) regulations (http://www.circare.org/info5.htm), and with Good Clinical Practice Guidelines (http://www.fda.gov/ScienceResearch/SpecialTopics/RunningClinicalTrials/default.htm). All research involving human subjects which is conducted within an institution is reviewed by the local institutional IRB or an external IRB with which the institution has an agreement.

The first two questions the IRB faces in reviewing a new protocol is whether the activity involves research, and second, whether it involves human subjects. Research is defined by the regulations as “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge” (45 CFR 46.102(d)). Human subjects are defined by the regulations as “living individual(s) about whom an investigator (whether professional or student) conducting research obtains (1) data through intervention or interaction with the individual, or (2) identifiable private information” (45 CFR 46.102(d)). It is at this second juncture that the new HIPAA regulations intersected with review of research done by the IRBs.

There are three levels of IRB review defined by federal regulations: exempt, expedited and full review. Research is considered ‘exempt’ if it meets one of five specific categories such as research involving the use of educational activities or research using existing data which is publicly available or collected in such a manner so that subjects cannot be identified. There are at least nine categories of research which are considered to be minimal risk. These are classified for expedited review and do not require discussion at a meeting of the full IRB committee (http://www.hhs.gov/ohrp/humansubjects/guidance/expedited98.htm). When any research falls outside the exempt or expeditable categories or is considered to be greater than minimal risk, full review necessitates a meeting of the entire IRB committee (http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.htm#46.110).

The HIPAA regulations (http://www.hhs.gov/ocr/privacy/index.html) did not change the interaction between investigator and subject, nor did they change the way in which the IRB reviews potential studies, but they did change the definition and allowable disclosure of protected health information. There are three parts to the HIPAA guidelines: (1) the definitions of what constitutes protected health information, (2) the Privacy Rule which provides federal protection for health information and (3) the Security Rule which specifies the administrative, physical, and technical safeguards which health providers, health plans and health care clearinghouses (known as covered entities) must use to assure the confidentiality, integrity, and availability of electronic protected health information. Initial HIPAA regulations were passed in 1996. In 2002, the Department of Health and Human Services published the final rule which articulated the federal definitions and guidelines for the Privacy Rule (http://www.hhs.gov/ocr/privacy/hipaa/administrative/privacyrule/privrulepd.pdf), and compliance with the Security Rule was required as of April 2005 in large health plans and April 2006 in small plans (http://www.hhs.gov/ocr/privacy/hipaa/administrative/index.html).

METHODS

In this article, we compare our IRB experiences in Study 1 (“Outcomes of ICU Working Conditions,”⁸ Agency for Healthcare Research and Quality (AHRQ), R01HS 13311401), conducted in 2002, and Study 2 (“Prevention of Nosocomial Infections and Cost Effectiveness Analysis,” National Institutes of Health (NIH), R01NR010107), conducted in 2008. In both studies, similar study protocols were submitted to multiple IRBs across the nation and hospital recruitment was similar in terms of the nature of the health information accessed. Briefly, both multisite observational studies were designed to understand various factors related to elderly patients risk for healthcare associated infections. In both studies, we obtained a data use agreement (as required) with the Centers for Medicare and Medicaid Services and requested that hospitals provide health information on elderly Medicare patients (i.e., Medicare number, age and gender).

All hospitals in both studies were participants of the Centers for Disease Control and Prevention's (CDC's) National Healthcare Safety Network (NHSN) (formerly the National Nosocomial Infections Surveillance System). See Stone et al for detailed recruitment methods.^9,10 Importantly, both studies had similar human subjects concerns and involved the collection of retrospective administrative data. A waiver of informed consent was requested for both studies. In Study 2, which occurred post HIPAA, a waiver of HIPAA authorization was obtained under the requirements set out by the HIPAA Privacy Rule. Initially, before recruiting hospitals, the study protocols were approved by IRBs of each primary investigator (Columbia University Medical Center, the CDC and RAND Corporation).

The hospital eligibility criteria for both studies were identical. In order to participate, a hospital must have conducted device-associated surveillance of healthcare associated infections in an adult medical, medical/surgical, or surgical intensive care unit (ICU) according to CDC protocols, the ICU must have had a minimum of 500 device days, and IRB approval had to be obtained at each participating hospital. In both studies, there was a designated site investigator for each hospital who participated in informational conference calls to ensure consistency in data collection, assisted with the IRB process, and facilitated the data collection process at their institution. The project coordinator worked closely with the site coordinator at each site to prepare and submit the IRB applications.

While obtaining these multiple IRB approvals the following data were collected from the participating sites: availability of application forms online, number of pages per application and number of copies required, time from submission of protocol to approval, requirements related to human subjects research training including evidence of human subjects research training, conflict of interest statements, curriculum vitae for principal investigator, IRB fees, requirements for data use or business associate agreements, and level of IRB review. Data on hospital characteristics such as geographic region and hospital bed size were also obtained since these characteristics may potentially affect the approval process. To gauge the time requirements related to the IRB process in Study 2, we also tracked the time spent by the project coordinator to complete and review the IRB applications for each of the participating sites, the need for site-specific HIPAA and human subjects training, and revisions requested by the IRBs.

Data Analysis

First, to ensure the comparability of the hospitals in terms of region and bed size we examined descriptive statistics and computed Chi-square and t-tests analyzing the hospital characteristics from both time periods. In order to assess changes in the IRB process, we compared characteristics related to the IRB submissions from Study 1 and 2 using Chi-squared statistics, Fisher's Exact Tests and Student's T-tests. To examine how hospital characteristics were associated with the IRB processes, analysis of variances (ANOVAs) were conducted for the 2002 and 2008 studies separately. In the ANOVAs we compared the level of review by review time and hospital bed size.

RESULTS

Sixty-eight hospitals were recruited to participate in Study 1 and 50 hospitals were recruited to participate in Study 2. Table 1 shows the hospital characteristics in terms of region and bed size. While the two samples were not necessarily the same hospitals, there were no statistically significant differences in these demographic characteristics. Overall, the preparation of all of the IRB protocols took an estimated 44 hours of work for the project coordinator in Study 2.

Table 1.

Comparisons of Hospital Characteristics in the Two Studies

Characteristic	2002 Study 1 Pre HIPAA (n = 68)	2008 Study 2 Post HIPAA (n = 50)
Region
Northeast	20 (29.4)	20 (40.0)
South	16 (23.5)	12 (24.0)
Midwest	16 (23.5)	12 (24.0)
West	16 (23.5)	6 (12.0)
	Mean (SD)	Mean (SD)
Hospital Bed Size	465 (367)	401 (199)

Note: Chi square and t-test conducted. No statistically significant differences in hospital characteristics

The IRB submission characteristics from Studies 1 and 2 are presented in Table 2. The mean number of pages per application in Study 2 was significantly greater than the mean number of pages per application in the Study 1 (11.1 and 5.3 respectively, p < 0.001). In Study 2, the mean time from IRB submission to approval was shorter, 35.2 days (range 0 to 193) as compared with 45 days (range: 1-303) in Study 1 (p = 0.045). The availability of forms online did not change over time (44.1% vs. 44.0%, respectively), but in Study 2 significantly more IRB submissions were performed electronically via email or online (40.0% vs. 29.4%, p = 0.048). The proportion of hospitals requiring evidence of human subjects research training doubled in Study 2 (p = 0.0053). An increasing proportion of IRBs required a fee in Study 2 compared to Study 1 (18.0% versus 1.5%, p = 0.001); however, in all cases the fee was waived after submission of a letter stating that this was a federally funded study and that IRB fees were not budgeted. Over one- third of the IRBs in Study 2 required a conflict of interest or financial statement as compared with only 10% observed in the previous study (p = 0.0009). In Study 2, seven (14%) of the IRBs requested that a data use or business associate agreement be drafted before the hospital was permitted to provide data for the study, which never occurred in Study 1. Additionally, 12% of the IRBs required that the principal investigator receive HIPAA training specific to their institution, which was a new requirement not relevant to Study 1 since it was pre HIPAA. There were no statistically significant differences between the two studies in the proportion of IRBs requiring the curriculum vitae of the principal investigator or requesting revisions to the initial IRB submission.

Table 2.

Comparisons of Characteristics of IRB Submissions in the Two Studies

Characteristic	2002 Study 1 Pre HIPAA (n = 68)†	2008 Study 2 Post HIPAA (n = 50)
	Mean (range)	Mean (range)
Number of pages of application	5.3 (0-31)	11.1 (3-45)*
Time from submission to approval	45 (1-303)	35.2 (0 to 193)*
	n (%)	n (%)
Forms available on web site	30 (44.1%)	22 (44.0)
Number of printed copies required: 0 (electronic submission)	20 (29.4)	20 (40.0)*
1	35 (51.5)	21 (42.0)
2-3	9 (13.2)	2 (4.0)
4-8	3 (4.4)	2 (4.0)
9-20	0	2 (4.0)
Missing data	1 (1.5)	3 (6.0)
Required evidence of human subjects research training for research team members	18 (26.5)	29 (58.0)*
Required revisions	10 (14.7)	8 (16.0)
Required conflict of interest statement	7 (10.3)	17 (34.0)*
Required investigator curriculum vitae	5 (7.4)	9 (18.0)
IRB Fee	1 (1.5)	9 (18.0)*
HIPAA/human subjects training specific to institution	0	6 (12)*
Required data use or business associate agreement	0	7 (14)*

^†Results previously published.⁸

^*Significantly different between the two studies at the p <0.05 level.

Table 3 presents the results of the analyses examining variations in review time and bed size stratified by level of review. In Study 1, expedited review required more time from IRB submission to approval than either full or exempt review, and larger institutions were more likely to give expedited reviews. These trends were not observed in Study 2 in which there were no significant differences in mean review time or bed size by level of review. However, in the analysis that was stratified by review level, the mean review time for expedited review was significantly lower in Study 2 as compared to Study 1 (p < 0.034).

Table 3.

Variations in Mean Review Time and Mean Bed Size by Level of IRB Review

	2002 Study 1†			2008 Study 2
Variable	n (%)	Mean Review time (days)	Mean Bed Size	n (%)	Mean Review time (days)	Mean Bed Size
Level of Review
Exempt	14 (20.6)	10.8	320.9	1 (2.0)	33.0	310.0
Expedited	42 (61.8)	54.8*	458.4	42 (84.0)	31.1*	420.3
Full	12 (17.6)	47.1	340.4	3 (6.0)	49.3	324.7
Ethics/Research
Review**	NA	NA	NA	4 (8.0)	21.8	301.3

^†Results previously published⁸

^*Significantly different between the two studies at the p <0.05 level

^**Some hospitals did not have a formal IRB, but rather an informal ‘ethics/research’ review process in which the level of review was not specified

DISCUSSION

Changes in IRB processes over time

To our knowledge, this is the first analysis of the IRB processes in multisite studies before and after the implementation of the current HIPAA rulings. Evident in Study 2 was an increase in training requirements and statements regarding conflict of interest of researchers; several IRBs required a written data use or business agreement between the hospital and the investigators’ home institution in addition to IRB approval. This seems appropriate and advantageous, but also resulted in doubling of the mean number of pages per application. There were a number of other improvements over time. First, the total time to approval and the average time required for review when the protocol was deemed expeditable were significantly shorter in 2008 when compared to 2002. Significantly more IRBs in the later study deemed the protocol as expeditable (84% and 61.8%, respectively).

Others have reported considerable variation in IRB reviews of multisite studies. In one report from the United Kingdom, the overall level of agreement regarding 18 protocols among three review committees was only slightly better than chance (kappa=0.29).¹¹ In an observational health services research study conducted by Green et al., which met criteria for expedited review, 72% (31/43) of IRBs required full board review, 28% (12/43) requested changes that increased patient risk, and one IRB rejected the protocol. Median time to approval was 286 days and 15% of the IRBs required 3-6 revisions, most of which were editorial rather than substantive.¹² Other investigators attempting to initiate multisite protocols have also reported prohibitive time delays as well as inconsistent (and sometimes contradictory) expectations and requirements,^4,5,13,14 which adversely affect the quality and rigor of the science.⁵ Compared with these reports, our experience was considerably less variable.

Sugarman and colleagues in 2005, reported results of a survey of 63 U.S. medical schools and calculated that the average cost for full or expedited reviews was slightly more than $1000/protocol;¹⁵ but, a letter to the editor in response to this paper suggested that these costs were seriously under-estimated.¹⁶ Speckman et al estimated the annual costs of IRB activities to total between approximately $500,000 to almost $2 million/institution.¹⁷ In a 43-site observational study conducted over 19-months within Veterans Affairs primary care clinics, an estimated 4,680 hours of staff time were required to deal with the IRB process.⁵ Costs have been reported to vary widely across institutions, indicating that some processes may be inefficient or of poor quality.¹⁸

While we estimated the time the project coordinator spent on the IRB processes in Study 2, this did not include the time spent by the principal investigator to review submissions, the time needed to complete all of the required institution-specific trainings in human subjects research or the time spent by the site coordinator and IRB personnel at each participating hospital. From an institution's perspective, the costs of managing the human subjects review process include the hiring of staff knowledgeable about the regulatory aspects of research and the time and expense required by IRB members to conduct reviews and participate in regular meetings. Additionally, like the researchers, the IRB members are required to complete human subjects research training, which is generally provided on-line or in didactic courses. The development and oversight of this training, which may require passing several examinations, is an additional cost to each institution. Columbia University, for example, has a professional fulltime staff of about 28 individuals and five IRBs, each of which includes 11-12 members. They manage more than 2,500 protocols each year at a considerable cost to the University.¹⁹ Given the resources required to manage the IRB process, it is not surprising that an increasing number of IRBs are charging a fee to defray some of the costs of managing and maintaining their activities. Since the review process requires considerable expenditure of time and financial resources, a process as efficient as possible is important.

The regulatory process for research is sometimes perceived by researchers as being a hindrance or a hurdle to overcome. Despite the possible need for streamlining and harmonizing, however, IRBs are essential not only for governance purposes but they also have an important social function and can facilitate discourse regarding important issues such as risk, trust, and confidence.²⁰ While some have argued that IRBs are inappropriately paternalistic, they also play a role in preventing exploitation of vulnerable individuals and assuring that there is no coercion to participate.^21,22 Furthermore, in addition to the importance of IRBs in protecting patient safety and choice, most biomedical journals require evidence of IRB review and approval before considering a manuscript for publication, giving investigators further motivation to obtain appropriate IRB approvals. Based on the challenges and barriers faced by investigators, particularly those attempting to conduct multisite studies, it is clear that the process can be improved, and many have called for reform.^4,13,23

Considerations for practice and policy

Two issues in particular warrant consideration. First, there is a longstanding tradition that review at the local level is essential because local conditions must be considered when assessing the ethics of human research.²⁴ Others have argued that ‘local’ issues regarding research ethics can be appropriately and adequately handled by centralized IRBs and, that there are, in fact, ‘no such things as local issues in research ethics’.²⁵ The results of this and other studies demonstrate that local reviews do reflect differences in local culture with variations in opinion, values or and preferences by reviewers (e.g. editorial changes in consent forms). While it seems appropriate for local IRBs to consider issues such as cultural appropriateness of language and social mores regarding information sharing, decisions such as the level of review required or regulatory compliance should be standardized and consistent across IRBs. Such decisions should have little variation in the review process for multisite studies, but rather require knowledgeable, trained board members and reviewers who are well versed in the regulations.

In addition to whether reviews for multisite studies should be centralized or local, the second issue is the extent to which the current standards and regulatory oversight are effective or whether additional structure is needed. Numerous efforts to harmonize the research review process and improve the reliability and efficiency of reviewers have been made and/or are in process, including those initiated by the FDA and OHRP in the U.S. and the Central Office for Research Ethics Committee in the United Kingdom.²⁶ Many of these guidelines and codes are summarized by the Citizens for Responsible Care and Research (CIRCARE).²⁷ Several organizations exist to actively promote education, policy, and certification to improve and standardize the conduct and review of human subjects research. Such organizations include Public Responsibility in Medicine & Research (PRIM&R, http://www.primr.org/) in the U.S. and the International Council for Science (ICSU, http://www.icsu.org/index.php), which has members from almost 150 national and international scientific bodies. Huang and Hadian have suggested that it is the clarification, uniformity and interpretation of current standards that is needed, not necessarily more regulatory oversight.²⁸ To help investigators manage the increasing complexity of research governance, there is a burgeoning of guidebooks and instructions about how to manage the IRB process.^29-32

CONCLUSIONS AND RECOMMENDATIONS

While collaborative research across sites is essential to increase the statistical power and generalizability of research findings, the need to undergo multiple IRB reviews is a challenge. It is a time consuming and sometimes tedious process that occurs just at the moment when researchers are most eager to begin the actual work of the study. It should be possible to design a system that protects human subjects while also minimizing the redundancy in the current system. Based on our findings and the results of others, the following approaches are suggested to facilitate an efficient IRB review process:

• Clarify the specific purposes of local review of multisite studies—e.g. to assure cultural appropriateness and consistency with local social mores and expectations.

• Consider central review mechanisms for multisite studies to assure regulatory compliance.

• Assure that IRB members are trained and well versed in the regulatory requirements as well as the ethical principles of research.