Variations among Institutional Review Board Reviews in a Multisite Health Services Research Study

Abstract

Objective

To document the Institutional Review Board (IRB) review process and to explore the impact of different patient notification procedures.

Data Sources/Study Setting

Review of IRB application and correspondence records prospectively collected during a multisite study of health care quality involving telephone interviews of 3,000 participants across 15 primary care sites.

Study Design

Records were reviewed to ascertain: (1) the type of IRB review conducted, (2) the number of days from submission to approval of the IRB application, (3) whether the IRB required patient notification and/or consent prior to the release of names, and (4) patient participation rates.

Data Collection/Extraction Methods

The study coordinating center prepared a common study protocol for IRB submission and assisted sites with submission. The application, correspondence with the IRB, consent script, and patient letters were collected, reviewed, coded, and analyzed.

Principal Findings

IRBs at the 15 sites and survey center varied in the type of IRB required and the number of days from submission to approval (range of 5–172 days). Four sites required patient notification in advance of the study; 2–11 percent of patients refused in opt-out sites and 37 percent in the single opt-in site. Participation among contacted patients did not appear to be related to patient notification procedures.

Conclusions

Variations in IRB requirements can affect response rates and sample generalizability.

Obtaining Institutional Review Board (IRB) approval for health services research (HSR) is increasingly challenging. The Health Information Portability and Accountability Act of 1996 (HIPAA 1996) called for the development of Standards for Privacy of Individually Identifiable Health Information (Standards for Privacy of Individually Identifiable Health Information, Final Rule 2002). These regulations impose substantial limits on researchers' access to medical record data and penalties for institutions that do not take steps to protect the privacy of patient data. These concerns are particularly heightened given recent scrutiny of human subjects research (Steinbrook 2002). Restrictions on access to data because of privacy concerns are of special importance in HSR because of the need for access to population-based samples of patients (Institute of Medicine 2000). HSR often uses existing databases such as insurance claims and medical records or patient interviews about health care experiences, health risks, and other factors (Brook, McGlynn, and Cleary 1996). In most health services studies, risks are likely to be associated with subjects' discomfort with disclosing personal information or the potential loss of privacy. The potential benefits are to improve care quality through enhanced vigilance, new standards or practices, or education. Still, the uncertainty about the possible harmful impact of disclosure of health information in HSR can create great concern (Burman et al. 2001; Friedrich 2001).

Previous research has documented widespread variation in how IRBs handle multisite studies involving clinical research (Goldman and Katz 1982; Silverman, Chandros, and Sugarman 2001; Stair et al. 2001). Less is known about the variations in how IRBs handle HSR studies (Lynn, Johnson, and Levine 1994; Friedrich 2001). A handful of previous studies reported increased protocol changes and more frequent requirements for advance authorization from potential participants before patient contact. These changes led to delays in data collection and publication (McCarthy et al. 1999; Bennett, Sipler, Parada et al. 2001; Nelson et al. 2002). Research to date fails to capture in a prospective fashion the variability in the process and content of IRB deliberations.

A federally funded study evaluating clinical care in the National Centers of Excellence (CoEs) in Women's Health provided an opportunity to examine variations in IRB review of HSR with prospective tracking of the IRB review process. This report (1) documents the IRB review process, including the type of review, whether revisions were requested, and the length of time from initial submission to approval, (2) describes IRB decisions regarding patient notification and recruitment, and (3) explores the impact of different recruitment procedures on patient participation in research.

Methods

Description of the Larger Study

The Federal Office on Women's Health within the Department of Health and Human Services (HHS) funded the parent study, whose purpose was to evaluate patient perceptions of care in 15 primary care sites designated as CoEs in Women's Health (Anderson et al. 2002). The survey collected data on demographics, satisfaction with care, receipt of preventive services, and health status. An independent university-based survey research center conducted the telephone surveys in English and Spanish.

To obtain the target of 200 completed surveys, each site was asked to draw a simple random sample of 400 patient names from an administrative database. Women aged 18 and older, who had made at least one primary care visit within the previous year were eligible for the survey. Interviewers read an informed consent script, indicated that they were calling on behalf of the site, and obtained oral consent. No identifiable patient data were either recorded in the survey database or retained in an administrative or analytic database.

Procedures for Obtaining IRB Approval

The coordinating center developed a common study protocol and cover letter requesting an expedited review with a waiver of the requirement to document written informed consent based on HHS regulations regarding the conduct of research with human subjects (Protection of Human Subjects 1991). These documents were distributed to each site's principal investigator (PI) for use in preparing the local IRB submission. The coordinating center staff tracked IRB queries and assisted site PIs in preparing responses. Upon receipt of local IRB approval, the site PI was asked to provide a copy of the original IRB submission packet, all correspondence with the IRB, and any consent forms or patient letters. Changes in procedures requested by the local IRBs were implemented for the individual site only.

Measures

IRB Review

Using the IRB applications and correspondence, we identified the type of IRB review as exempt, expedited, or full-board review, and we noted whether any changes from the common protocol were made by the site PI or requested by the IRB. Time to approval was calculated as the number of days from the official IRB submission to the date of the IRB approval letter.

Release of Patient Names for Study Recruitment

Recruitment procedures were grouped as (1) opt-in: patients had to indicate in writing and in advance their willingness to be contacted about potential research studies; (2) opt-out: patients were given an opportunity by telephone or mail to remove their name from the list of potential subjects; (3) notification: patients were notified in advance of the research without an opt-out opportunity; and (4) no notification: the patients received no prior notification of the research.

Informed Consent

Changes required for the informed consent script were identified by comparing the IRB approved script to the common protocol script.

Response Rates

Response rates were determined at two different stages of recruitment. For sites using an opt-in or opt-out process, the contact refusal rate was calculated as the number of persons who declined research contact out of the total number of patients approached. The cooperation rate, calculated for all sites, was the number of completed interviews out of the total number of completed interviews, partial interviews, and refusals among eligible respondents.

Analysis

Two authors (K. D. and S. S.) reviewed the IRB application materials and correspondence independently. Where materials were unclear, site PIs were contacted for further clarification. We report the average and range for the time to approval and cooperation rates.

Results

Type of Review

Nine of the 15 CoE sites obtained IRB approval through an expedited process as suggested in the coordinating center's protocol. The independent survey research center was granted an exemption from review by its IRB. At the remaining six clinical sites, the protocols received a full IRB review. One site required full-board review for all initial applications. Among the three sites that initially requested expedited approval of their protocol but were required to undergo full-board review, two sites had concerns about release of patient names and another about the content of the survey questions. The IRB at the fifth site discussed the protocol in a full-board review and determined that the protocol was exempt because the project was an evaluation of clinical care. In the sixth site, the IRB initially approved an exemption for the study, but as described below, the IRB subsequently required protocol resubmission and full-board review.

Changes Required and Time to Approval

Both exempt protocols and five of the other expedited protocols received IRB approval without revisions (Table 1). The remaining nine sites asked for changes or clarifications. The length of time between IRB submission and approval ranged from 5 to 172 days. In Site #8 the protocol received expedited approval with no changes; however, the approval took more than 4 months (132 days), due apparently to a backlog of protocols.

Table 1.

Summary of Review Process: Type of Review and Rationale, Revisions Requested, and Time to Approval

Site ID	Type of Review	Rationale for Type of Review (if Specified by PI or IRB)	Time to Approval (Days)
1	Expedited		51
2	Expedited		39
3	Expedited		57
4	Expedited		20
5	Expedited	Initially submitted as exempt then IRB required resubmission as expedited because patients would be contacted.	23
6	Expedited		8
7	Expedited		72
8	Expedited		132
9	Expedited		5
10	Full	New projects are not eligible for expedited review.	32
11	Full	Initially submitted as expedited, then full-board review requested because of concern that questions regarding counseling on illegal drug use or physical abuse could place subject at risk.	72
12	Full	Initially submitted as expedited, then IRB requested full-board review because of patient contact.	105
13	Full	Initially submitted as expedited, then IRB requested full-board review.	172
14	Exempt then full	Initially approved under exempt review procedures.	Initial: 15
		Resubmission with full-board review required after patient complaints.	Revised: 22
15	Exempt	Reviewed by full-board, which recommended submission as exempt protocol because it was an evaluation of the services offered at the clinic.	25
16	Exempt survey center	45 CFR 46.101 (b)2	10

IRB=Institutional Review Board; PI=principal investigator.

Release of Patient Names for Study Recruitment

Eight clinical sites and the survey center approved the release of patient names to the survey center without patients' receiving prior notification of the research (Table 2). Two IRBs had questions that were addressed by the PI or by adding language to the consent script. Three sites voluntarily sent a letter to potential participants informing them of the research. The remaining four sites objected to the release of patient names to the research group without the patients' knowledge. Three sites required an “opt-out” process. Contact refusal rates ranged from 1 percent to 5 percent at sites that offered a telephone “opt-out” option and 11 percent at a site where patients could call or return a card in a postage-paid envelope to opt-out. In the single site that required patients to “opt-in” for possible future research projects by completing a form at the point of care, 37 percent of patients did not complete an “opt-in” card.

Table 2.

Release of Patient Names for Research: Contact and Cooperation Rates

Site ID	Patients Contacted in Advance?	Contact Refusal Rate (%)	Cooperation Rate (%)
1 Expedited	No notification	—	78.6
2 Expedited	Notification	—	80.3
3 Expedited	No notification	—	69.4
4 Expedited	Opt-out letter	10.6	80.7
5 Expedited	Notification	—	89.2
6 Expedited	No notification	—	83.5
7 Expedited	No notification	—	82.2
8 Expedited	No notification	—	75.8
9 Expedited	Notification		83.3
10 Full	No notification	—	78.2
11 Full	No notification	—	79.1
12 Full	Opt-in	36.6	87.1
13 Full	Opt-out letter	1.3	81.8
14 Full	Opt-out letter	5.3	85.0
15 Exempt	No notification	—	90.4
16 Exempt Survey Center	No notification	—	—

Contact Refusal Rate: Number of patients who did not want to be contacted about research, out of total number approached.

Cooperation Rate: Number of completed interviews, out of total number of completed interviews, partial interviews, and refusals.

The issue of patient privacy led to the suspension of IRB approval at one site. After consulting with local IRB staff, the PI submitted and received approval for the study as an exempt protocol. After two patients complained about being called by the survey research center, the IRB suspended approval, and data collection stopped. The IRB stated that the protocol could not be considered exempt because patient names would be released to a separate research group. The PI submitted a new protocol that included an “opt-out” process. A new sample was drawn to ensure that patients were not contacted again, and data collected under the exempt protocol were destroyed.

Cooperation rates—the proportion of surveys completed out of eligible respondents contacted by the survey center—overlapped by the type of recruitment procedure. The range of cooperation rates was similar in “opt-out” or “opt-in” sites (80.7–87.1 percent); where informational letters were sent (80.3–89.2 percent) and where patients were not notified in advance of the study, rates varied from 69.4 to 90.4 percent.

Informed Consent and Other Changes

All IRBs approved a procedure to obtain oral informed consent, as proposed. IRBs at five of the 15 clinical sites requested changes in the informed consent script, and a sixth raised questions that were addressed in an informational letter. Changes to the script addressed: the location of interviews, project description and purpose, source of patient names, survey content, data uses, payment(s), rights and protections of research participants, PI contact information, and participation assent. The original script took approximately 90 seconds to administer; the longest version of the script took 150 seconds.

Discussion

Based on the CFR that sets standards for IRB review of study protocols, the research team argued that this study could be considered to pose minimal risks because (1) the only risks were related to invasion of privacy and breach of confidentiality, (2) information obtained from each of the participating CoEs was limited to patients' names and contact information, and (3) the information collected during the survey process did not include sensitive information (Protection of Human Subjects 1991). There was no linkage to medical record data, and all identifiers were removed from the case record after completion of the survey. Thus, there was no possibility that subjects could be placed at increased risk of social or legal harm from the accidental disclosure of information.

Most of the IRBs agreed with the research team. The cautious handling of this study in some locations may reflect different local climates regarding use of patient information. Out of more than 3,000 patients contacted to participate in this survey, the research team learned of only a few cases, at two sites, where patients called their clinical center to inquire about the survey. Neither of these sites had notified their patients about the survey in advance. In one site, clinic staff did not confirm the legitimacy of the survey and the patient complained to the IRB. The IRB determined that the initial exempt review was not concordant with their standard procedures for prior notification. At the second site, the patients' concerns were addressed by clinic staff, and the research protocol was not changed.

Variations in the Content of the IRB Review

This study illustrates how recruitment procedures designed to protect the privacy of patient health information may also limit the generalizability of research, and therefore limits both the burdens and potential benefits from that research. A third of the clinical sites required opt-in or opt-out procedures, leading to moderate-to-large loss of potential patients (11–37 percent) in two sites and minimal loss in two others. Other investigators have found that opt-out protocols significantly reduce patient participation in research (Verboncouer et al. 2000). Given existing literature showing lower research participation among minority and low-income persons, patient notification procedures may set up additional hurdles to participation for these groups (Giuliano et al. 2000; Kressin, Meterko, and Wilson 2000; Shavers, Lynch, and Burmeister 2000; Corbie-Smith, Thomas, and St George 2002).

When faced with conflicting interests, researchers and IRBs have considered the “risk-to-benefit” ratio in which the participant's right to privacy or right not to be harmed was balanced against the societal gains to be realized from research (Skolnick 1998). Given the expanding array and capabilities of information technologies, privacy concerns are warranted. These risks must be balanced against the societal benefits of HSR, including improved service delivery, improved health outcomes, and enhanced efficiency of the medical care system. Historically, IRBs and investigators have been able to find ways to protect privacy, while permitting researchers access to personal information. However, in today's environment, IRBs may also consider institutional economic and legal liability concerns in the risk-to-benefit equation (Finkelstein 2003).

Still, the price of greater attention to privacy and longer consent scripts may be a lessening of the IRB's ability to ensure that the benefits and burdens of research are equitably distributed. If results are limited by the lack of generalizability to the full population, especially low-income and minority groups, then health policy decisions may be made with inadequate and unrepresentative data. Furthermore, although advance contact was possible in our study, it may not be feasible in many forms of HSR.

Strengths and Weaknesses of the Study

The strengths of this study include the prospective assessment of IRB response to a particular research protocol and the availability of detailed records of correspondence between the IRB and investigator at all study sites. However, because study investigators were free to change the standard protocol prior to submission, the availability of different information may have influenced the IRB decisions. In addition, results may be influenced by differences in IRB structure, staffing, skill or interest of investigators, the health of participants, regional ethnic variation, and operation across study sites. Differences in participation rates across study sites may be influenced by unassessed patient factors, because we have no data on the comparability of samples drawn from the different settings.

Implications and Recommendations

Our data give evidence of the variability in IRB approval processes, in particular relating to privacy concerns. AcademyHealth and others have warned that HIPAA privacy regulations could hinder HSR, pointing out that some health plans and providers were not releasing any data to researchers out of fear of being sued (HSR Funding Update 2001). While this research was undertaken before the implementation of the HIPAA regulations, it illustrates issues likely to be occurring in the post-HIPAA research environment. In addition, efforts are needed to understand the impact of variability in IRB reviews on the conduct of research. While differences in IRB reviews are expected and encouraged by federal research regulations, the variability is an enormous challenge to the internal and external validity of HSR. Variations in patient recruitment procedures may yield study groups that differ from target populations in unknown ways that could threaten the validity of the findings. Efforts to minimize the burden of research on poor and minority populations may actually result in their inability to benefit from the findings. This is a particular issue in HSR, where studies of access, quality, and outcomes consistently show racial/ethnic and socioeconomic disparities (Jha et al. 2001; Bach et al. 2002).

Completion of the IRB process is time-consuming and expensive. It would be helpful to have more detailed information to track how IRBs function, both to save investigators and IRB staff the unnecessary work involved in multiple protocol revisions and reviews, but also to make the process more transparent and accountable to the research community and the public at large.

Ultimately, the issues about patient privacy and willingness to participate in research reflect the need to educate consumers about the value of HSR. A recent report found that legislators generally ignored HSR because they could not understand it (Sorian and Baugh 2002). Concerns about privacy loss and abuse of research may be more palpable to consumers than the benefits of studies on quality or access to care. Better efforts are needed to help consumers as well as legislators and policymakers to understand the methods used in HSR and how HSR data can help formulate policy decisions and consumer choices.