Regulating Research with Human Subjects—Is the System Broken?

Abstract

There has been a great deal of criticism of the system to regulate research and protect the interests of research participants. Structural problems in the system result from the nature of current research and the regulatory process that was created over thirty years ago. Procedural problems exist because local IRBs show wide variation in practices, resources, quality and experience. Assessment of performance is perhaps the most important problem facing the system since there are no standard measures of outcome or performance for the system as a whole or to assess local IRBs.

Is the system broken? No, but it is straining under the weight of a changed research environment and inadequate resources. It has the capacity to respond to the concerns of both the research community and the public by the thoughtful application of present regulations and the creation of performance assessment strategies.

Introduction

Over the past several years, criticism of the system to regulate research and protect the interests of human subjects has reached crescendo pitch, arising from such disparate sectors as the federal government, sponsors, investigators, academic institutions, research subjects and their families, media, and the general public. This appears to be related to reports of unexpected deaths of normal volunteers in clinical investigations (1,2,3), alleged conflicts of interest that affected research participants (4,5), federal stopping of all clinical research at major Universities because of noncompliance with regulations to protect human subjects (3,6), a court decision that questioned the authority of parents to consent for children to be enrolled in studies that include any level of risk without “therapeutic” benefit (7), reports of healthy children exposed to “risky” medications solely for research purposes (8), and allegations that pharmaceutical companies suppress negative data including adverse outcomes during clinical trials (9).

These concerns have led many to argue that the system is broken and in need of radical restructuring and stronger laws and enforcement in order to protect the interests of human subjects (10, 11). This article will examine the background of the current federal regulatory system that governs research involving human subjects, the concerns of the various stakeholders about the ability of the regulatory system to facilitate clinical research while protecting human participants, and address some proposed solutions.

Background

Clinical research has always been an important aspect of modern medicine. In the late nineteenth and early twentieth centuries, well-intentioned physicians often “experimented” with new approaches to illness and disease and reported their experiences at research meetings and in the medical literature. Sometimes, the subjects of these innovative approaches were made aware of the experimental nature of the treatment and were asked for informed consent. More often, the patients trusted their physicians, understood the uncertainty of the outcome of any treatment for serious disease, and accepted the approach provided without question (12).

The advent of World War II brought two very important events in the history of medical research. First, illness, disease, and trauma among soldiers overseas created an urgent need for medical research to examine the most effective approaches to saving lives and maintaining a healthy fighting force. This resulted in substantial federal funding for research and greater organization of the research enterprise both in the military and in academia. Second, numerous experiments were conducted by Nazi doctors on prisoners in concentrations camps during the war. This research had dubious, if any, scientific basis and caused tremendous suffering, severe disabilities, and numerous deaths. These atrocities resulted in twenty-three physicians and Nazi officials who were involved in experiments being prosecuted for their crimes at the Nuremburg trials. In addition to finding fifteen of the defendants guilty, the American judges issued the Nuremberg Code, a list of ten governing principles to guide medical experimentation with human subjects (13). The code mandated sound scientific methods be used in any research involving humans, that the risks of the research be proportionate to the benefits accrued to the subjects, and that the voluntary consent of the subjects be required.

Post World War II, the Nuremberg Code was generally ignored in the United States, while federal funding for clinical and laboratory research increased dramatically. In an attempt to standardize clinical research practices throughout the world, in the early 1960s, the World Medical Association set about creating a universal set of professional guidelines to aid investigators in conducting ethically sound research. These guidelines, called the Declaration of Helsinki, were published in 1964 (14). The Declaration permitted both “therapeutic” research, performed on patients and intended to benefit them in the medical context, and “non-therapeutic” research, performed on healthy subjects and intended to gain generalizable knowledge.

In the United States, after it was discovered that the use of the experimental drug thalidomide by pregnant women caused severe birth defects, the Food and Drug Administration (FDA) mandated in 1962 that experimental drugs be tested in standardized trials using formal consent procedures. In 1963, the Director of the National Institutes of Health, the federal agency that had become the major funder of medical research in the U.S., created a committee to examine research sponsored by the Institutes to study unethical practices and recommend guidelines for the future.

The work of this committee along with several reports in the 1960s describing unethical practices in human subjects research, resulted in 1966 in a memorandum from the US Surgeon General, William H. Stewart, in which he stated that no research grants will be funded by the Public Health Service involving human beings unless the grantee has indicated that the grantee institution has provided prior review by a committee of “institutional associates” to assure an independent determination of the rights and welfare of the individuals involved, the appropriateness of the methods used to secure informed consent, and the risks and potential benefits of the investigation (15).

The next decade saw increased public concern about the protection of human subjects of research with the publication of several journal and newspaper articles that concluded that there were unethical procedures in human subjects research and the informed consent process (16, 17, 18, 19). This highly charged atmosphere resulted, in 1974, in the creation of the National Commission for the Protection of Human Subjects in Biomedical and Behavioral Research through the National Research Act, passed by Congress as Public Health Law 93-348. The Commission published a series of important reports from 1974–1978 that are the basis for the present federal regulatory structure of research involving humans. In its report on IRBs (20) the Commission advocated for local institutional review and accountability to assure the protection of human subjects.

The federal regulations that were promulgated in the late 1970s and approved in the early 1980s reflected the nature of the research enterprise of that era and were consistent with the recommendations of the Commission (21). The vast majority of research involving human subjects funded by the federal government at that time was individual investigator initiated clinical trials conducted at a single academic institution. Thus, the regulations required that each institution create a set of guiding principles and policies to protect the rights and welfare of human subjects of research at that locale. These regulations, with minor changes, remain in effect today.

The goals of the federal regulatory structure, created in the aftermath of unethical practices and public scandals, are to protect the subjects of research, and to assure accountability for ethical oversight on the part of institutions in which investigators work. Regulations were never meant to facilitate research or to enhance the acquisition of knowledge. Rather, regulations were imposed in order to control inappropriate practices in research and to reassure the public that if federal dollars were being spent to benefit health, those altruistic and sometimes vulnerable individuals who would volunteer to take part in these studies were not being placed at undue risk.

Problems with the System

There is a paucity of data documenting the breadth and depth of clinical research in the United States. There are few data on the numbers and types of research studies being conducted, how many subjects are being enrolled, how many serious and unexpected adverse events occur, and how many participants die from research-related causes (22). There is also no systematic data that quantify the problems with the current system. The Institute of Medicine in a recent report (23), echoing the sentiments of the National Bioethics Advisory Committee and the DHHS Inspector General's Office, noted that the absence of systematic data on the human subjects protection system in the U.S. is a serious problem that precludes any potential to identify and solve problems concerning protection of human subjects.

A recent publication of a group of academic and industry scholars in research ethics sponsored by the Doris Duke Charitable Foundation concluded that the problems with the current system of research oversight can be classified into three broad areas: structural, procedural, and assessment of performance (24).

Structural problems derive from the current way research is conducted and the present federal regulations. Federal regulations do not apply to all research involving human subjects in the U.S. The regulations apply only to research funded by seventeen government departments that have adopted the “Common Rule,” and to research seeking FDA approval of drugs, biologics and devices. Although institutions that seek certification of their research protection infrastructure from the DHHS Office for Human Research Protection (OHRP) must promise to review all human subjects research consistent with the federal regulations regardless of funding source, there are many independent entities that do not seek federal funding or certification and are not bound by federal regulations.

A new model for performing clinical research has evolved in the United States that no longer utilizes academic medical centers as primary sites for this work. The pharmaceutical industry has turned to commercially organized networks of community-based physicians called contract-research organizations (CROs). Over the last decade, the number of physicians in private practice office settings who engage in clinical research has more than tripled (25). Although all of this research is regulated by federal authority and IRBs constituted by the pharmaceutical company or the CRO have reviewed and monitor the studies, the inherent conflict of interest in the review process raises concerns and requires scrutiny. These conflicts of interest are not solely of concern outside of academic institutions. Even within universities and academic medical centers that are clearly subject to federal regulations, there is an inherent conflict of interest in research review in as much as the institution and its professional staff are benefiting from the funding of the research and at the same time are responsible for assuring compliance with existing regulations. Moreover, in recent years the academic medical center and its faculty have engaged in financial arrangements with industry that raise many concerns about financial conflicts of interest (26).

Additional structural problems flow from the local nature of the institutional review process. Regulations require local institutional to be responsible for oversight of human subjects research (27). The virtues of local review and responsibility are clear. Local committees are thought to be familiar with the actual conditions surrounding the conduct of the research. They reflect the customs and moral wisdom of the region and are supposed to know the capacity of the investigators and the institution to fulfill the obligations of the study and be culturally sensitive to participants from the various local communities. A local review infrastructure also adds a “culture” of education and accountability for the ethical conduct of clinical trials. However, with all these potential benefits of local review, research trials conducted in multiple sites require repetitive reviews that are redundant and time consuming but have never been shown to be beneficial to subjects.

Procedural problems in the process of review also result from the local nature of the human subjects protection system. IRBs show wide variation in practices, resources, quality, and experience. Local IRBs have been criticized for significant inefficiency, lack of needed expertise, poor quality of reviews, insufficient knowledge of the regulations, providing too little education in research ethics to investigators and board members, insufficient ongoing review and attention to adverse events, and a general lack of resources (22). In addition, multi-center trials have created new challenges for the human subjects protection system and have resulted in serious criticism of the system in general as failing to adapt (28). Local IRBs have contributed to inordinate delays in initiating trials, exclusion of some sites from participation in a trial, substantial duplication of effort and extraordinary time commitments by core personnel and trial sponsors (29).

Local IRBs have been criticized for excessive focus on the informed consent form, rather than the process, and for neglecting other equally important aspects of ethical review such as conflict of interest and confidentiality. Local review can result in wide variation in the content of the informed consent documents in multi-center trials and may even result in differing eligibility criteria between sites. In addition, once a research project is initiated, IRBs tend to do little to monitor the actual performance of the research. IRBs also have little ability to evaluate adverse event reports and in multi-center trials, often lack full appreciation of the extent of the problem.

Assessment of performance is perhaps the most important problem facing the system for human subjects protection since there are no standard measures of outcome or performance for the system as a whole or to assess IRB performance or quality.

Proposed Solutions

There are numerous institutions and organizations representing the federal government, academia, and the private sector that are concerned about the quality and functioning of the system for human subjects protection. Table 1 lists a number of these groups. There have been many proposed solutions to reform or enhance the present system, including legislative proposals that require all research involving human subjects to conform to federal regulations, broad plans for educating IRB members and staff, and accreditation programs for IRBs. In addition there have been recommendations to centralize multi-site reviews and to enhance efficiency and quality of local IRBs.

TABLE 1.

Organizations and Institutions Concerned with Human Subjects Protection

Federal: Congress Office of Inspector General, DHHS National Bioethics Advisory Commission Office for Human Research Protections, DHHS Office for Research Integrity, NIH Office for Good Clinical Practice, FDA National Human Research Protections Advisory Committee, DHHS Responsible Conduct of Research Education Consortium, DHHS and FDA Human Subjects Research Subcommittee, White House Secretary's Advisory Committee on Human Research Protections

Private: Public Responsibility in Medicine and Research Applied Research Ethics National Association Duke Foundation Consortium to Examine Clinical Research Ethics (CECRE) Association of American Medical Colleges Institute of Medicine—National Academy of Sciences National Committee for Quality Assurance Association for the Accreditation of Human Research Protection Programs

In order for the government to be able to regulate and monitor all research initiated within the U.S. that involves human subjects, regardless of funding source or investigator affiliation, there needs to be new legislation. No government agency presently has authority to provide mandatory oversight of all human subjects research. However, the vast majority of human subjects research is now under the regulatory authority of the government. The creation of a new office responsible for all human subjects research could create and maintain a standardized system for collecting data on the entire clinical research enterprise including every research protocol involving humans, the number of subjects enrolled, adverse events, etc. However, OHRP could go a long way toward accomplishing that goal if it began the process with a comprehensive data collection system for all IRBs presently registered with the government.

A permanent federal advisory committee to OHRP or to a new office with broader responsibilities could address major ethical concerns that arise from time to time and could develop performance measures to assess each aspect of the human subject protection system. There is a federal advisory committee to OHRP, the Secretary's Advisory Committee on Human Research Protections that has been working on many important initiatives since its inception in 2003. This group shows great promise as a forum for the development of recommendations and guidance for the research community. The tenure of many predecessor ethics advisory committees has been very short and this group would benefit from the knowledge that it will have a more sustained existence.

It is important to note that the most frustrating procedural problems faced by investigators in relating to local IRBs are not a result of the regulatory framework for protection of human subjects. Rather, these problems are caused by inadequate resources provided by institutions to local IRBs, IRB staff members who are unfamiliar with regulations, IRBs that are comprised of individual members who are not knowledgeable about the federal regulations and do not have needed expertise to evaluate protocols, and increasingly paranoid administrators who fear repercussions from federal reviewers for minor infringements of the regulations.

Even now, OHRP can provide needed leadership and guidance to assist IRBs and institutions to provide more efficient and higher quality review and monitoring of human subjects research. Guidance documents can clarify many of the terms and definitions within the regulations, encourage local IRBs to cede authority and responsibility for review of multi-site protocols to a “central” or “lead” IRB through the use of “cooperative research agreements,” and assist IRBs to interact with Data and Safety Monitoring Boards.

National organizations can help by setting guidelines for the resources needed by IRBs dependent on volume and complexity of protocols. Voluntary programs for credentialing IRB staff and educating IRB members can be helpful; information management systems that streamline the interactions between IRBs and investigators have become available and should be utilized widely. Federal and industry sponsors can recognize the important role IRBs play by creating a direct cost line item as part of each grant budget for protocol review and human subjects protection, while academic institutions supplement these dollars to enable research offices to provide efficient and high quality services.

Institutional human subjects protection programs have three inter-related goals that do not need to be in conflict. First, the human subjects protection program must protect the rights and interests of those altruistic humans who participate in clinical research by being conscientious and knowledgeable in the review process and by developing innovative methods to monitor the research after IRB review. These institutional programs must also ameliorate all financial conflicts of interest in clinical research so that both the reality and perception of conflict is eliminated.

Second, the local program should facilitate research by creating an atmosphere that recognizes the clinical investigator as the “customer” of the program. This customer is not “always right” but deserves respect and an efficient process that assists in performing the many federal regulatory requirements and local procedures and does not obstruct ethically sound clinical research. Third, the human subjects protection program has the responsibility to assure that the institution is in compliance with all federal and regional regulations, but this effort should not be so all encompassing as to result in compromising the other two goals.

Some critics think the research protection system is broken. They argue that it is inadequate to protect the interests of the participants of research and can only be fixed by radical overhaul. Is the system broken? No, it is straining under the weight of a changed research environment and inadequate resources to perform its job, but it has the capacity to respond to the concerns of both the research community and the public.

Critics argue the need for radical restructuring, and stronger enforcement as the only way to protect the interests of research subjects. New legislation and additional regulation will not solve all the problems in the human subjects protection system; it will only create a new set of challenges and more bureaucracy. Infusion of resources and energy can provide needed rejuvenation without total overhaul. Each of the stakeholders, government regulators, sponsors, investigators, IRB professionals, IRB members, and research participants will need to work together to make this possible. Thoughtful application of the present regulations will answer the majority of the concerns of those who question the quality and efficiency of our human subject protection system. But in order to assure the public and future research volunteers that the system is providing adequate safeguards, there is a need for added transparency, national data and performance measures. This level of public accountability is both appropriate and necessary in order to regain the trust of the American people in the clinical research enterprise.

DISCUSSION

Rubenstein, Philadelphia: I am very interested in the subject, but I'm really challenged by the idea that the federal government can oversee this and enhance it in the way you say. With all the politicization of the situation at the moment, and the conflicts going on, it would seem unreasonable to expect that the Federal Government should lead this initiative. Rather, I wonder whether the National Academy of Sciences, IOM, or some similar organization could do it, as we have to get this out of politics and organize it in a way that's best for our subjects.

Fleischman, New York: I think, Dr. Rubenstein, your point is well taken. The problem, of course, is we have not stepped up to the bat. Although I see Dr. Cohen coming to the microphone. We need to step up to the bat. We need to be serious about voluntary solutions to hard problems. We need to not allow the scandals to divert our attention, and we cannot allow bad regulation to be the response to scandal. We need a concerted effort if the private sector is going to show the federal government that we don't need their help. I'm not optimistic we are ready to do that.

Cohen, Washington: Thank you Alan. I think it was a very good summary of a very serious problem. And I certainly agree with Arthur that we need to find ways for the private sector to grapple with this issue. In that regard, as you know, the Association for Accreditation of Human Subject Protections Programs, AHARP has an accreditation mechanism for looking at these programs with very tough standards, at least in my view. I am curious to know whether you think that mechanism is likely to be sufficient to forestall federal intervention. I am concerned about this because we have had the opportunity now for almost three years to participate in this voluntary accreditation program and the uptake by our community has been disappointingly slow. Admittedly there's been a lot of distractions; HIPAA and many other things have intervened in the process, and it's also an expensive thing to get these programs up to snuff and no institution wants to be subject to accreditation if they don't think they are going to have an easy time passing. So I understand the reluctance of institutions to participate, but nevertheless I think, unless we get some more momentum behind the voluntary process, it's not going to suffice.

Fleischman: Thank you for that comment and question Jordy. The process of accreditation as it exists today is clearly necessary but far from sufficient. We still don't have good performance measures. What accreditation measures is process measures and paper compliance. We do not have agreement, even after multiple IOM reports, on what it would look like if we had a system that was actually protecting human subjects. We don't know how to measure adverse events, aggregate those data and find out what it would mean in the real world. So, although accreditation is critically important, it's far from sufficient, and I think it'll take some wise heads in continuing dialogue to develop the appropriate outcome measures.

Berk, New York: I'd just like to put this in broader prospective which is, even, I think, more distressing than some of the implications which you have discussed. I will tell you a personal experience. I just moved from the Mt. Sinai School of Medicine to Columbia, which involved moving my apartment from the east side to the west side of Manhatten. That represented an enormous cultural change, and was one step short of requiring a visa. I had to transfer a medical practice, and several NIH grants, and in the course of doing that I ran up against human research protection regulations, animal protection regulations and HIPAA. It took my staff and me more than three months to deal with the HIPAA requirements forgetting the medical records of my patients transferred from institution to another despite the signatures of all the patients saying that they wanted this to occur. In the course of recertifying my grants at a new institution, I had to go through the human research approvals process from scratch and the twice as difficult animal approvals, so that the paper work for those two types of approval actually exceeded by a factor of two the scientific content of the grants that were being transferred. So without in any way wanting to trivialize the issues involved in appropriate regulation and protection of human subjects in research, the confidentially of medical records, and animal rights, they have become, collectively, part of an enormous system in which the regulators are out of control. My personal experience just highlights the fact that we are being subject to an enormous amount of regulation, much of which—as you have already indicated—is more focused on legalistic compliance than in really protecting our human subjects, their medical data, or the mice we work with. I think the profession needs to respond in a much more active way than it has been. We've been very passive in accepting regulations that have been handed down that don't achieve their goals. What I have said is not so much a question as an editorial, and I hope you'll forgive me for that.

Fleischman: I would like to respond, Dr. Berk. At the New York Academy of Medicine in New York under Jerry Barondess' leadership, we've brought together the academic community in New York. Let me tell you the problem is us. It's not the regulations. There is absolutely nothing in the regulations that requires those well-intended and high-paid administrators at Columbia to slowdown the efficient transfer of information concerning your research program to Columbia from Mt. Sinai. It is not in the regulatory structure. It is in the implementation at the local level. We at the New York Academy of Medicine do research. And we have an efficient IRB that puts research as a very high priority, and we think we do review ethically.