EFFECTS OF LOCAL-IRB REVIEW ON PARTICIPATION IN NATIONAL PRACTICE-BASED RESEARCH NETWORK (PBRN) STUDIES

Abstract

Objective

To describe the process and outcomes of local-IRB review for two Pediatric Research in Office Settings (PROS) studies.

Design

PROS conducted two national studies concerning sensitive topics: 1) Child Abuse Recognition Experience Study (CARES) – observational study of physician decision-making; and 2) Safety Check (SC) – violence prevention intervention trial. IRB approval was secured by investigators’ sites, AAP, and practices with local IRBs. Practices were queried about IRB rules at PROS enrollment and study recruitment.

Setting

PROS practices in 29 states.

Participants

88 PROS practices (75 IRBs).

Main Exposure

Local-IRB presence.

Outcome Measures

Local-IRB presence, level of PROS assistance, IRB process, study participation, data collection completion, and minority enrollment.

Results

Practices requiring additional local-IRB approval agreed to participate less than those that did not (CARES: 33% vs. 52%; SC: 41% vs. 56%). Of the 88 practices requiring local-IRB approval, 55 received approval, with nearly 50% needing active PROS help, many requiring consent changes (e.g., contact name additions, local-IRB approval stamps), and 87% beginning data collection. Median days to obtain approval were 81 (CARES) and 109 (SC). Practices requiring local-IRB approval were less likely to complete data collection but more likely to enroll minority patients.

Conclusions

Local-IRB review was associated with lower participation rates, substantial effort navigating the process (with approval universally granted without substantive changes), and data collection delays. When considering future reforms, the national human subject protections system should consider the potential redundancy and effect on generalizability, particularly regarding enrollment of poor urban children related to local-IRB review.

BACKGROUND

Over the past two decades, primary care practice-based research networks (PBRNs) have arisen to conduct multi-site studies addressing community-based practice and aimed at improving the effectiveness of primary care.¹ Concurrently, problems in the protection of human subjects at a few large research universities have resulted in increased scrutiny of research by local Institutional Review Boards (IRBs), which are charged with assuring that studies are ethically conducted, and meet the federal requirements governing the conduct of investigations involving human subjects (the Common Rule).² Although the regulations themselves are national, the interpretation and implementation of these rules are left to local discretion. As a result, PBRN research typically requires that multiple IRB applications be submitted for local review, even when the studies are of minimal risk and have received prior approval by the investigators’ IRBs.

Pediatric Research in Office Settings (PROS), the American Academy of Pediatrics (AAP) national practice-based research network, is a PBRN that includes over 1,800 practitioners in more than 700 practices located in all 50 states, Canada, Puerto Rico, and District of Columbia. Subject enrollment and data collection by multiple practitioners in geographically dispersed practices presents challenging IRB logistics for all PROS studies. Local IRBs can vary considerably on important issues, even when assessing identical clinical trial protocols.³ IRB review processes at different sites may be divergent enough to prevent national clinical research projects from obtaining approval in a timely and efficient manner,⁴ which can, in turn, compromise the size and diversity of the study's sample as well as introduce geographic bias among participating practices and patients. Variations in IRB requirements have also been shown to impact response rates (e.g., length of time between IRB submission and approval) and generalizability of health services research studies.^5,6

Little has been done to research the problem or determine solutions. The National Institutes of Health (NIH), the Office of Human Research Protections (OHRP) and other key groups convened workshops on alternative models of IRB review in 2005 and 2006 ^7-8 and the NIH Roadmap has cited the need for PBRNs to facilitate large-scale studies in a more timely fashion.⁹ PBRNs have attempted collaboration to improve the IRB process (e.g., AHRQ hosts a yearly three-day conference that routinely includes sessions dedicated to IRB issues). One PBRN, the American Academy of Family Physicians National Research Network (AAFP NRN), has begun discussions of an alternative model of central IRB review.¹⁰

Little empiric research has systematically explored the process and outcomes of review by local IRBs. This study reports the process and outcomes of local-IRB review for two large federally-funded PROS studies, both of which had first received IRB approval from at least two IRBs -- the PI's institution(s) and the AAP's IRB. Analysis was directed at determining: (1) local IRB prevalence among participating practices; (2) level of PROS staff assistance required to apply to local IRBs; (3) local-IRB process (e.g., submission/approval time, consent changes); and (4) the impact on study participation, data collection completion, and minority patient enrollment by practices covered by local IRBs.

METHODS

Two PROS studies were included in our assessment. One study was observational and the second included a counseling intervention and distribution of materials (firearm locks, kitchen timers). PROS completed the Child Abuse Recognition Experience Study (CARES) - an observational study of physician decision-making,¹¹ and Safety Check - a violence prevention intervention trial.¹² CARES study subjects were the providers themselves, and the study required consent only by practitioners enrolled in the study. Safety Check included both patient and provider outcomes and required consent from families and participating practitioners. Both of these studies were considered minimal risk. The AAP IRB served as the responsible IRB for unaffiliated practices.

Description of CARES

The aim of CARES was to describe factors influencing the diagnosis and reporting of child physical abuse. Participants provided data concerning themselves, their practice environments, and their management of 40 consecutive injured children using pocket-sized encounter cards. No patient data were identifiable either on the encounter card or in a database. A subsample of practitioner participants provided more detailed information through surveys, telephone interviews and chart audits.

Practitioners gave written consent prior to beginning data collection. IRB approval, including waiver of parental consent, was given by the AAP and multiple investigator institutions (Tufts New England Medical Center (Boston, MA), Children's Memorial Hospital and University of Illinois at Chicago (Chicago, IL)).

Description of Safety Check

Safety Check tested the effectiveness of an office-based violence prevention intervention on patient behavior over time using a cluster randomized-controlled trial design. PROS practices that agreed to participate were randomly assigned and initiated patient recruitment of 30 consecutive patients for either an office-based violence prevention intervention or reading promotion control arm (educational handout on literacy promotion provided). Consent was obtained from practitioners after they had been randomly assigned to a group.

Eligible parents were asked to consent to join the study in the office at the time of check in. Parents of children ≥6 years old who agreed to participate and signed the consent were asked to tell their children about the study and encouraged to get their children's ideas about the survey questions. Assent from age-appropriate children was obtained at this time. Institutional Review Board approval for the study was obtained from the AAP and the principal investigator's institution (Wake Forest University School of Medicine (Winston-Salem, NC)).

Procedures for Obtaining Local-IRB Approval

Prior to study participation, practices were asked to complete and return an IRB Assurance Form that indicated whether they required local-IRB review. Unaffiliated practices were covered by the AAP's IRB, which also required that participating practitioners complete human subjects training and sign unaffiliated investigator agreements. Practices owned by, or affiliated with, entities with their own IRBs (e.g., universities and hospitals) were required to obtain local approval before enrolling in a study. The local-IRB approval process was facilitated by the PROS staff, who developed a comprehensive “IRB Packet” containing all necessary study information based on the AAP's IRB application in electronic form for use in preparing the local application, which was mailed to the practices requiring local-IRB approval. Within a week of the mailing, PROS staff contacted these practices to determine if someone on-site was able to complete the application or if they wanted assistance from PROS staff. At the request of local practitioners, PROS staff facilitated and coordinated the application process for the practice and communicated directly with either the IRB administrator or another contact person until final approval was secured at each local site.

PROS staff tracked practice recruitment data, IRB specifics (e.g., correspondences, level of aid, changes to protocol and/or consent), data collection (e.g., completion status, number enrolled), and practice and patient demographic data (e.g., practice setting, patient race/ethnicity).

Measures

IRB Review/Approval Time

Time to local-IRB approval was calculated as the number of days from PROS sending the comprehensive “IRB Packet” to practices until the date of IRB approval, as indicated on the IRB letter.

Level of PROS Assistance

PROS staff offered different levels of assistance in completing the local-IRB process, ranging from sending the “IRB Packet” to actively completing the application and communicating directly with the IRB administrator or another contact person (estimated at 15 hours/application).

Informed Consent Modifications

The type and number of changes required for the informed consent by a local IRB were identified by comparing the originally approved consent by the AAP's IRB to the locally approved consent.

Study Participation

Practice/practitioner participation was measured by calculating the number of patients enrolled to determine data collection completion. Beginning data collection included enrolling at least one patient into the respective study. Minority patients were defined as Hispanic/Latino, African American, American Indian or Alaska Native, Native Hawaiian or other Pacific Islander, or Asian.

Statistical Analysis

Three research assistants and two project managers reviewed the consents for changes and tracked participation status, local-IRB need, level of assistance, and data collection status in an extensive database. Frequencies and other descriptive statistics were calculated for number of practices requiring local-IRB approval, assistance in completing IRB process, changes to consent, and IRB approval time. Chi-square tests were conducted to determine the significance of observed differences in study participation, data collection completion, minority patient enrollment, and practice setting between participating practices that obtained local-IRB approval versus those that were covered by the AAP's IRB.

RESULTS

IRB Review/Approval Time

Table 1 provides detail for each study regarding participation rates according to whether local-IRB approval was necessary. Eighty-eight of the 838 (includes duplicates) practices that were approached regarding participation in CARES and/or Safety Check required additional local-IRB approval. A slightly higher percentage of Safety Check practices agreed to participate compared to CARES practices (41% vs 33%).

Table 1.

Study Participation According to Local-IRB Approval Requirement

		Local-IRB Approval Required
Participated in Study		YES	NO
YES	CARES	37 (33%)	146 (52%)
	Safety Check	51 (41%)	180 (56%)
NO	CARES	75 (67%)	134 (48%)
	Safety Check	73 (59%)	142 (44%)

The 88 participating practices requiring local-IRB approval were located in 29 states and associated with 75 different IRBs. The majority of the participating practices were located in urban areas (68%) and connected to medical schools or hospitals/clinics (48%). Of these 88 practices, 33 dropped out of the study (CARES=12; Safety Check=21) and 55 (CARES=25; Safety Check=30) received local approval. Chi-square analyses indicate that for both studies the requirement of additional local-IRB approval made a difference in practice study participation (CARES: χ²=11.73, p=.001; Safety Check: χ²=7.82, p<.01).

Local-IRB approval took on average 81 days for CARES and 109 days for Safety Check (see Table 2), with the shortest approval time being 13 days for CARES and 25 days for Safety Check. Longer approval time was related to the number of requested changes and follow-up questions by the local IRB. Examples of other factors related to an extended approval time were practice changes (e.g., staff turnover), holidays, timing of IRB meetings, and delay in producing/submitting IRB application.

Table 2.

IRB Process for 88 Practices Agreeing to Participate and Requiring Local Approval

	# recd local approval	# needed active application help	# reqd changes	Median days to submit & obtain approval (intraquartile range 25%, 75%)
CARES (N=37)	25 (68%)	9	8	81 (48, 284)
SC (N=51)	30 (59%)	17	24	109 (65, 182)

Level of PROS Assistance

The Safety Check practices were more likely to require active application help (57% vs 36%) (see Table 2), possibly reflecting the higher complexity of an intervention (vs. observational) study design.

Informed Consent Modifications

Table 2 also indicates that Safety Check practices also required more changes to the informed consents (N=24 vs. 8), which included mostly additions of contact names and approval stamps. Appendix A includes examples of requested changes according to study. None of the IRBs requested revisions that questioned either the studys’ scientific merit or whether their protocols were ethical.

Study Participation

Of the 55 practices receiving local-IRB approval, 87% began data collection (CARES=22; Safety Check=26). Overall, 64% of the CARES practices and 60% of the Safety Check practices completed data collection. Table 3 indicates significant differences in completing data collection were only found for CARES between practices requiring local-IRB approval and their counterparts (CARES: χ²=26.72, p<.001; Safety Check: χ²=.56, p=.45).

Table 3.

Data Collection Completion According to Local-IRB Approval Requirement

		Local-IRB Approval Required
Completed Data Collection		YES	NO
YES	CARES	17 (46%)	125 (86%)
	Safety Check	18 (35%)	74 (41%)
NO	CARES	20 (54%)	21 (14%)
	Safety Check	33 (65%)	106 (59%)

Analyses also indicate differences in patient populations and practice setting (e.g., urban, suburban, rural) depending on need for local-IRB approval. CARES sites requiring local-IRB approval enrolled more minorities as part of their patient sample compared to sites covered by the AAP's IRB (39% vs 23%; p<.0001). Minority patients were also more frequently enrolled in Safety Check practices that required local-IRB approval (51% vs 36%; p<.0001). In addition, practices requiring local-IRB approval were more likely to be located in an urban setting than those covered by the AAP's IRB (CARES: 76% vs 28%; p<.0001; Safety Check: 69% vs 34%; p<.0001).

DISCUSSION

This study suggests that the need for local-IRB approval appears to be an impediment to participation in PBRN-based research, may discourage the inclusion of minority and urban patients, and seems to result in little if any significant change in the research protocols. Similar effects were seen in these two studies, although they differed greatly in design - one an observational study of provider behavior and the other a randomized controlled intervention trial that assessed both provider and patient outcomes. The latter study required a longer time for IRB approval and a larger dedication of study resources in the form of staff time for active assistance. In no case did any IRB determine that either study posed more than minimal risk to subjects.

Many poor and minority patients receive primary care at university or hospital affiliated sites. These are the sites that typically require local IRB review, even if other IRBs have previously reviewed and endorsed the protocol. Moreover, practices requiring additional local IRB review were more likely to terminate participation in the research study. These findings raise concern for limited participation from diverse patient populations. For the two PROS studies reviewed here, significant additional resources in both time and money were needed to achieve an adequate balance of urban sites and minority patients in order to reduce these disparities. In many studies, additional funding is not available for such efforts, allowing a significant portion of American children to be under-represented in clinical research. PROS studies, like other PBRN-conducted multisite studies, offer specific benefits such as enhanced external validity by increasing the generalizability of research results and rapid patient recruitment. However, as previously suggested by Gold et al, delays resulting from an unnecessary multisite ethics review system can influence the timeliness of this research, which ultimately affects the subsequent health care policy and decision making process.¹³

To achieve the goal of facilitating large-scale PBRN studies effectively and efficiently, and to ensure that research results have high external validity, IRB processes need to be streamlined, particularly in the case of minimal risk studies. The local IRBs that approved practices to participate in CARES and Safety Check undoubtedly spent time and effort in reviewing these applications, resulting in financial costs. Although local control of the human subjects process must be maintained, several steps could be taken to improve the process. These include: (1) consideration by local IRBs to the existence of multiple prior IRB approvals with the text of those reviews made available to local IRBs; (2) standing arrangements between local and central IRBs ceding detailed scrutiny of proposals to the central body, with final review by the local IRB; and (3) creation of local IRB expertise in PBRN studies to facilitate more rapid local review. Federal initiatives should be undertaken to identify innovative protocol review policies and processes for multisite network trials that reduce redundancy of efforts. The NIH Roadmap has identified this issue as well. In fact, the Clinical and Translational Science Awards (CTSAs) sites are working together to identify alternative IRB models that would speed multi-CTSA institutional site implementation of clinical research protocols.¹⁴

LIMITATIONS

This report covers two recent national PBRN studies involving child health. Our participating practices and study sample may not be representative as the practices were chosen from a domain of volunteer practices in a PBRN. However, research has shown that patients seen in these practices do not differ significantly from national samples of patients.¹⁵ Studies involving other conditions and other populations may not encounter the same experience. The additional effort and cost of local IRB reviews were not quantified in this study beyond the tracking of time taken to review the protocol. In future studies, quantitating the burden of these efforts relative to changes made in the protocol is warranted.

CONCLUSIONS

When considering future reforms, the national human subject protections system should consider the potential unproductive redundancy and effect on generalizability, especially regarding enrollment of poor urban children related to local-IRB review, and particularly in PBRN studies that are deemed minimal risk. As per the NIH Roadmap, this is the time to create efficiency in the research process to develop and apply scientific results to improve health in a timely manner. The time for creating a streamlined IRB process is now.

Abbreviations

PROS

Pediatric Research in Office Settings

PBRN

practice-based research network

CARES

Child Abuse Recognition Experience Study

SC

Safety Check

AAP

American Academy of Pediatrics

PI

principal investigator

OHRP

Office of Human Research Protections

AAFP

American Academy of Family Physicians National Research Network

NIH

National Institutes of Health

AHRQ

Agency for Healthcare Research and Quality

Appendix

Appendix A.

Examples of Requested Consent Changes

Study	Requested Change
CARES and Safety Check	Name of local practitioner listed as PI with contact information and co-investigators
CARES and Safety Check	Addition of local-IRB approval and/or expiration date stamp
CARES and Safety Check	Name of practitioner or local IRB office person listed under Participation-Termination section as additional contact to answer questions about rights as research subject or protection of human subjects
Safety Check	Inclusion of Patient Bill of Rights text
Safety Check	Addition of question regarding whether parent is participating in any other research studies
Safety Check	Inclusion of text related to specific institution not being responsible for research and medical care by other institutions or personnel participating in the study (research related injury)
Safety Check	Addition of subject's and/or parent's and/or physician's initials on each page
Safety Check	Addition of sections related to alternative treatments, significant new findings, compensation, number of participants, right of investigator to withdraw parent from study, risks to being female, funders of study
Safety Check	Inclusion of distribution of tangible tools (gun locks, timers)
Safety Check	Inclusion of assent of minor section requiring signature
Safety Check	Text under Confidentiality section bolded
Safety Check	Inclusion of signature of person conducting consent discussion and/or witness in addition to signature of investigator
Safety Check	Text moved to different section within the consent
Safety Check	Description of patient incentives (e.g., book) included under Costs and Compensation
Safety Check	Description of securing and destroying data collection surveys at PROS
Safety Check	Formatted on institution's letterhead