Principal Investigator Views of the IRB System

Abstract

We undertook a qualitative e-mail survey of federally-funded principal investigators of their views of the US human subjects protection system, intended to identify the range of investigator attitudes. This was an exploratory study with a 14% response rate. Twenty-eight principal investigators responded; their comments were analyzed to show underlying themes, which are here presented along with supporting quotations.

There was consensus that it is important to protect human subjects from research abuse, but disagreement over how well the IRB system is functioning. Some researchers felt that the system is effective and serves its purpose well. Of those who support the system, some endorse its methods, purpose, and daily functioning, as they experience it, without reservation. Others, while expressing some frustration, feel that the purpose is important and their local IRB does its best to make a difficult system work well.

Those investigators who were more harshly critical commented on multiple flaws in the system, including (1) consent forms that are inappropriate and incomprehensible, (2) an emphasis on minutiae, and (3) concern with protecting the institution more than research subjects. Respondents told us that the IRB system is a particular burden for research in neurology, emergency medical conditions, repositories, and social sciences in general; a more comprehensive study might identify other problematic areas. Significant concern was expressed about the cost, inefficiency, and irrationality of IRB review. The IRB system works well for some researchers, but our results indicate that other investigators feel the costs outweigh the benefits.

INTRODUCTION

The human subjects protection system is ever-enlarging yet its quality and efficiency are cast in doubt by experts, ¹ review bodies, ² and field- or discipline-specific committees. ³ In a complex system with multiple stakeholders, one important source of insight is with those who are regulated—the principal investigators. The views of investigators are therefore important in understanding the system as it functions today.

Previous work suggests that the opinions of investigators toward the IRB system are mixed. An informal survey of members of several social science organizations conducted in 2000 drew mixed results. Some of the researchers who were polled reported excellent rapport with their local IRBs; others were frustrated by a system that delayed and obstructed their research. ³ Borris and Moss studied the views of a group consisting primarily of federally-funded principal investigators; they identified significant problems with the system. ⁴ Investigators whose research requires data collection in more than one location have been especially vocal about the shortcomings of the IRB system. ^5-7

We undertook a study of researchers' attitudes toward the IRB system, using a confidential e-mail questionnaire format to encourage candid responses. We limited our sample to investigators with current NIH funding in human subjects research. We recognized that this unfunded work could not attempt to paint a comprehensive picture of researchers' attitudes and experiences; our more modest goal was exploratory, to gather the range of attitudes from a small sample of people with in-depth knowledge of the system.

METHODS

Surveys were sent via e-mail in two phases in 2003, using NIH-funded principal investigators listed in the CRISP database whose project abstracts indicated that they used human subjects. The pilot phase used a single e-mail solicitation only, including a six item questionnaire. This was sent to 34 investigators; four had invalid e-mail addresses, and two responses were received, for a response rate of 7%. The second phase survey was sent to 193 investigators who were NIH-funded to do research in human subjects in 2003. This phase included an e-mail alerting potential subjects to the study, an e-mail containing the survey, and two follow-ups for those who did not respond to the initial solicitation. Thirteen of our subjects did not have valid e-mail addresses; of the remaining 180, valid responses were received from 26. We combined respondents from the first and second phases, giving us a final sample size of 28, which is adequate for the purposes of qualitative analysis. Because this was an exploratory study, we asked open-ended questions and analyzed the responses qualitatively. The project was approved by the Baylor College of Medicine Institutional Review Board.

The final survey included these items:

1. What has been your experience with the human subjects protection system in general? What do you like? What would you change?

2. What has been your experience with the informed consent process with potential subjects? Is there anything you would change?

3. Do you feel your IRB does its job well? If so, what helps you the most? If not, how could it improve?

4. Do you feel that your IRB usually understands your protocols adequately?

5. Have you ever served on an IRB?

6. What other thoughts do you have on informed consent and the human subjects protection system?

RESULTS

I. Positive comments

A. Toward the system

Some respondents emphasized their conviction that the human subjects protection system is an essential safeguard in clinical research. Respondents identified several significant benefits of the overall system, including its function as a safeguard against overoptimistic investigators (subject 6), protecting subjects against both nonphysical and physical harms (subject 6), the promotion of “social justice” (subject 6), and the provision of online ethics training by the NIH (subject 22).

I think that the informed consent process is an essential element of both research and treatment. Risks occur in many situations that are not commonly perceived as such and individuals deserve protection. (subject 6)

Another respondent noted that self-interest is also a motivation:

My experience has been good to excellent overall. The time I have to spend on consent forms and IRB approval is bothersome at times but I gladly do it knowing that this system protects every researcher. I don't want my research shut down because some other researcher was sloppy and a subject was harmed. The IRB system is the only protection I have against that. (subject 15)

B. Toward individual IRBs

Individual IRBs were lauded for taking their work seriously (subjects 6, 13, 18, 20, and 23), for being sensible (subject 24), and for having a service orientation, including frequent meetings (subject 8), prompt responses (subject 6), bringing in outside expertise when appropriate (subject 6), and providing individual help and feedback (subjects 8, 11, 14, 15, and 16). Some respondents praised their IRB for excellence in helping researchers within a set of constraints that are challenging:

[Our] IRB has been good to work with, and I don't have the same horror stories that I've heard from others. They labor under the same antedeluvian mindset that has been imposed on everyone, but have accommodated to it in a way that is helpful to researchers in general. Most important, our IRB has avoided the pitfall of rewriting protocols (subject 8).

II. Negative comments

A. The system as a whole

Not all of our subjects felt that the human subjects protection system works adequately.

Overall, I find the people working in IRB offices to be trying very hard to keep a sane approach in what is rapidly deteriorating into an very insane process. The constraints placed upon investigators and indeed research subjects by OHPR and so-called ethicists approaches the absurd. I am both a PI and have participated in clinical trials [as a subject]. Getting protocols approved gets worse each year because you have to document more and more and more USELESS stuff. For most of my work I receive coded samples devoid of patient identifiers, yet I have to fill in all sorts of crap and REPEAT OVER AND OVER AND OVER AGAIN that I couldn't track down these subjects if i tried. Examples of this absurdity include: mandating IRB approval for a collaborator to look at the viral sequences from patients samples that were devoid of all identifiers. He may be good but I doubt he could look at someone and figure out who they were by the sequence of their virus. (subject 27)

B. Consent forms

Informed consent, by which our subjects mean the consent forms that are the focus of much IRB effort, was treated with widespread derision, with 15 out of 28 of our subjects providing criticisms such as “unlikely to be read,” or “incomprehensible.” (subjects 1, 4, 8, 10, 11, 12, 14, 15, 16, 17, 24, 25, 26, 27, 28) Excessive length and detail were commonly cited, resulting in reduced understanding (subject 11) and potentially reduced enrollment. (subjects 14, 15, 16, 17) Researchers were troubled by requirements to mention the risk of death even in minimal risk procedures (subjects 4, 17, 25) and felt that inappropriate language, such as a paragraph on the risk of death in a minimal risk procedure, was inflammatory and unduly frightening. (subjects 4, 17, 25)

Most informed consents include FAR TOO MUCH INFORMATION and the average person doesn't understand what is in them. They don't inform, at best they bore and confuse. Informed consents have become "legal" documents to protect the institutions from OHPR and have little to do with informing subjects let alone protecting them. (subject 27)

C. IRBs

IRBs were often portrayed as slow or cumbersome. (subjects 1, 4, 7, 12, 14, 16, 28) Caution was reflected in the refusal of a Department of Health IRB that refused to approve smoking-related research because it was “too politically hot.” IRBs were seen by some as not competent in the area under review (subject 4), not understanding the protocols (subject 4) or the analytic methods (subject 27) submitted.

Some IRBs were seen as applying federal regulations and their own local custom in arbitrary and even irrational ways. One IRB required that the informed consent form be written at the sixth grade reading level even when the subjects are college students. (subject 19) Another respondent commented,

I believe that most subjects skim through the incredibly long informed consents, believing that most of it is simply bureaucracy. They often are most puzzled by statements such as denial that an institution will pay for medical costs if they are injured in a study, when their participation simply includes filling out questionnaires. (subject 10)

Two respondents spontaneously mentioned fear of IRBs.

There is…quite some show of 'muscle' and if a member has a score to settle with an investigator, woe to the whole study. I have seen when investigators are frustrated but they dare not speak out lest the committees sit on their studies for ever (with a what-can-you-do attitude). You should see they way the letters to IRBs are full of 'pleading' words, and this is characteristic of both the local and the US IRBs I have had the misfortune of working with in other studies! (subject 28)

A final comment: All of our emails may be monitored, and although I have nothing to say that I would not share with any official of [my institution], and email is convenient, I wonder if this is the wisest way to conduct this survey? (subject 10)

D. Protection . . . for whom

Seven subjects, one quarter of our sample, commented that IRBs do not seem to have protecting subjects as their highest priority. (subjects 8, 9, 10, 12, 14, 27, 28)

Many IRB members are out more to protect themselves (from possible future lawsuits) than the hullabaloo about protecting study subjects. Many make decisions that harm clients in the long run without caring at all, as long as THEY feel safe. (subject 28)

Informed consent has clearly transmogrified from protection of subjects to protection of the institution. We are required to use language that is incomprehensible to most research subjects (I do street based work in STD/HIV and many of our clients are not fluent readers, so the problem is greater for me and my colleagues). This absurdity has recently been compounded by HIPAA, and the extra pages that clients must now wade through. In short, the informed consent process has become a travesty. (subject 8)

E. Specific areas of research

The most impassioned comments were made by investigators who believe that entire lines of research are made difficult or impossible by the human subjects protection system. (subjects 1, 9). Respondents identified four specific lines of research that are so burdened.

1. Neurology

A researcher who works in Neurology pointed to the difficulties posed by the human subjects protection system and raised a corollary concern, that the system discourages young researchers from entering careers in human research:

If parents of children dying of some of the diseases we encounter in neurology understood how much more cumbersome and expensive research is as the result of this system, the system would be in big trouble. Even worse if they realized how these problems conspire to dampen enthusiasm of young people who might otherwise have gone into clinical/translational research. (subject 2)

2. Social science research

Three respondents commented that the system (both the regulations and their application by individual IRBs) seems to them to work badly for social science research. (subjects 10, 11, 26) One respondent, who had served as chair of an IRB, commented:

What I would like to see changed is the federal law/policy that was developed specifically for biomedical research. It needs to be adapted for social science research. . . . For instance, it makes no sense to mention alternative procedures when conducting a lab experiment or field survey. … Some common sense could be applied to develop an appropriate parallel set of guidelines to fit the needs of social science. (subject 11)

3. Urgent or emergent medical conditions

Obtaining informed consent according to usual standards was a particular problem in research in emergency medical conditions (subjects 14, 16):

I am involved in acute stroke clinical trials, and informed consent with a confused patient and family in the ER under a very tight time limit is very difficult. (subject 14)

4. Repository research

A repository or data bank is a collection of medical information, tissue or blood samples, or other information from many individuals.

The constraints on repository material are a major set back in science and I hope all those at OHPR and the ethicists die of diseases that we could have made significant progress on if we had these valuable research. USE OF THESE SAMPLES IN NO CONCEIVABLE way harms the study subjects. It is a bureaucratic legalistic impediment to research that only HURTS people and protects no one. (subject 27)

F. A system under stress

Some of our subjects granted that it is essential to protect human subjects yet questioned the viability of the current system. One respondent, a former chair of an IRB, commented:

We've reached a point . . . where the logic of increasing improbability holds sway. If something could happen, it might happen, or perhaps it will happen, and we must defend against it. The work of the IRB has become the intellectual (jesuitical? talmudic?) exercise of trying to see what could go wrong and taking steps to prevent it, whatever the probability and whatever the cost. This is the same logic that drives airport security. To stave off an event with nanoprobability but enormous potential impact, we have to do everything to everybody, and such an imperative has an inescapable logic. You begin at Point A, and have logical justification for each step along the way to Z, but the situation you end up in, at Z, is ludicrous. . . . The system is clearly in a downward spiral, as the . . . addition of HIPAA requirements demonstrates. This involution takes us deeper into a bureaucratic morass, from which there is no visible exit strategy at the moment. I think all of us will just have to wait until it becomes apparent that useful research on human subjects cannot be performed, and there is a sea change in the way we approach the issue of protecting people from us. (subject 8)

DISCUSSION

All of our subjects explicitly or implicitly recognized the importance of treating human subjects appropriately. All of our subjects have worked with the human subjects protection system, and forty percent, including two respondents whose comments were entirely negative, were present or former members of IRBs.

Where investigators perceive problems, it is important to distinguish flawed implementation of regulations by the local IRB from problems inherent in the IRB system or in the governing regulations themselves. Some of our respondents complained, perhaps appropriately; others were happy to point out ways in which their IRB met its ethical obligations in a timely, courteous, and efficient manner.

This was an exploratory study with a low response rate, which imposes significant limits on the conclusions we can draw. These data do not support quantitative inferences. Our results show clearly that researchers range from very satisfied to extremely dissatisfied with the IRB system, but we cannot tell how these attitudes are distributed across the many thousands of investigators doing human subject research today.

Sometimes there is tension between protecting human subjects and permitting research to go forward. Our system correctly prohibits research that sacrifices the interests of subjects even if society might benefit. ⁸ Sometimes, however, the risks (to the individual) and benefits (for society) are not easy to compare; this is most apparent in our subjects' discussions of research in neurology, using repository material, in the social sciences, and in emergency and neurological medical conditions. We did not intend a comprehensive study of the impact of the IRB system on different areas of research, and did not inquire as to the field of research of our respondents, learning this information only when it was volunteered. We suspect that a larger survey would have identified more areas in which the IRB system causes severe difficulties, as well as other research methodologies that are particularly burdened (the problems with IRB review of multicenter research are already well documented).

The current exception for research in emergencies works, at least for subject 14, well enough to make research in acute stroke possible, but still imposes a substantial burden. In some circumstances, time spent obtaining informed consent of any kind (for research or for clinical treatment) directly reduces the time available for prompt and effective intervention.

Social science is a particularly hard case. IRBs work hard to protect the subjects of social science research. It is, however, clear that some of the federal regulations pertaining to research were not designed to apply to social science research, and that some IRBs do not understand this type of investigation.

The most important protection against abuse of human subjects is the high ethical standard of individual researchers. Some of our respondents understand that protecting human subjects is more than using an approved consent form. One respondent commented that his experience with the human subjects protection system has been “. . . positive but cumbersome. I like going through the process of being forced to think concretely about all possible ethical dilemmas associated with a project in advance.” (subject 4) This expresses nicely the attitude of a conscientious investigator for whom the protection of human subjects is an ideal. Many of our subjects feel that the bureaucratic process now in place does little to bring that ideal to reality.