Lessons from the controversy over the SUPPORT study

The recent controversy over the SUPPORT study of oxygen therapy for premature babies came to a stunning denouement a few weeks ago. On June 4, the United States government Office of Human Research Protections (OHRP), which had previously warned the researchers that they had violated federal regulations regarding the protection of research subjects, withdrew their “compliance actions.”¹ They were probably aware of the content of two essays that would be published in the New England Journal of Medicine the next day. One of those publications was from the leaders of the NIH.² The other was from twenty-six leaders in bioethics and neonatology.³ Both were stinging criticisms of OHRPs decision about the SUPPORT study.

This is thus a disagreement at the highest levels between, on the one side, the federal agency charged with protecting research subjects and, on the other, both the federal agency charged with conducting research and many bioethicists and neonatologists who both helped develop the ethical guidelines for neonatal research and who serve on Institutional Review Boards that implement those guidelines. Clearly, the issues at stake go beyond the conduct of this particular study. OHRP acknowledged this by stating that they were not just putting this particular compliance action on hold but that they also “plan to take no further action in studies involving similar designs until the process of producing appropriate guidance is completed.” They promised to quickly hold an open public meeting and then develop draft guidelines that will be available for public review and comment before they are finalized and implemented.

The focus of the controversy is a particular type of clinical study, one designed evaluate the risks and benefits of treatments that are in widespread clinical use but that have not been rigorously evaluated in comparison to one another. This has been called “comparative effectiveness research.” It must be distinguished from studies in which one therapy is the prevailing standard of care and “a new agent or approach is evaluated, often in a large, randomized trial setting in which the new agent is compared with the previous best standard therapy.”⁴ Current research regulations were clearly designed with the latter type of study in mind. They focus on the obligation to communicate the risks of enrolling in research that involves a therapy that may have unrecognized and unforeseen risks. It is clear, in such studies, what the default therapy would be if a patient or parent does not consent to be in the study. They would then receive the standard therapy.

In the circumstances of prospective comparative effectiveness research, by contrast, there is no established standard therapy against which an innovative therapy is being compared. Instead, both therapies in the study are standard. It is unclear which would be provided if the patient chose not to enroll in the study.

A unique ethical element of comparative effectiveness research is that it is difficult to prospectively quantify the risks of being in the study. This is, in part, because there are two separable risk comparisons that must be made. The first is a comparison of the risks between the two arms of the study. Presumably, one of the therapies will prove better or worse than the other. But, presumably, experts disagree about which therapy will be better. If they did not, then there would be no reason to do the study. The other risk is the risk of being in the study at all compared to the risk of not being in the study. It is unclear, in comparative effectiveness research, whether it is safer or more risky to be in the study than to receive the same therapies off protocol.

Under current guidelines, the prevailing assumption is that it is riskier to be in a study than to not be in a study. The increased oversight of research, compared to standard therapy, reflects the view that research subjects need special protections from the risk of research. The controversy over the SUPPORT study and, by implication, other similar studies, is, in part, because there is powerful evidence that being in the study was safer than not being in the study. The nature of that evidence and its implications suggest two important lessons that we should learn from the SUPPORT study controversy.

The claim that study subjects were safer than comparable babies who were not in the study could only be plausibly made because the NIH had previously sponsored a registry of all babies born in the Neonatal Research Network. This registry, called the Generic Database (GDB) collects and publishes data on outcomes for all babies born at the participating centers who weighed less than 1000 grams at birth.⁵ As a result, it was possible to compare outcomes for babies in the SUPPORT study with a comparable group of babies treated at the same institutions at the same time. This was, then, a non-randomized concurrent control group. The data were striking. Babies in the “low oxygen” arm of the SUPPORT clinical trial had a mortality rate of 19.9%. The babies in the “high oxygen” arm of the study had a mortality rate of 16.2%. Comparable babies in the network overall had a mortality rate of 24%. For severe retinopathy, the numbers are 8.6% (low oxygen group), 17.9% (high oxygen group) and 24.1% (overall group). That data thus showed that babies who were enrolled in SUPPORT had better outcomes than those not enrolled.

So the first lesson from the SUPPORT controversy is that prospective randomized trials and registry data are complimentary in important ways. Registries like the GDB allow estimates of outcomes for patients whose treatment is provided with the practice variation that is common in the real world of clinical practice. Furthermore, advances in medical informatics and the widespread adoption of electronic health records make it easier to interrogate clinical records to analyze outcomes without the need to formally enter data into a specific database.

The second important lesson from the SUPPORT study is related to the first. Comparison of outcomes between, on the one hand, patients in research studies and, on the other, similar patients receiving conventional therapy will allow us to quantify the risks of research in ways that we never could before. We will be able to evaluate, as was possible for the premature babies in the SUPPORT trial, whether it is, in fact, riskier for patients to be enrolled in clinical trials than it is for them to receive treatment based only on the clinical judgment of their physician.

These comparisons could overturn the entire basis for heightened scrutiny of clinical research. That view, articulated by OHRP, is based on the idea that doctors in clinical practice are better positioned to protect their patients than are clinical investigators because the clinical investigators have divided loyalties. OHRP wrote, “Doctors are required, even in the face of uncertainty, to do what they view as being best for their patients. Researchers do not have the same obligation.”¹ They call that “a fundamental difference between the obligations of doctors and the obligations of researchers.” This fundamental difference, according to the prevailing view, makes research more risky.

Some people have always been skeptical. As early as 1979, Levine, who would later write one of the first comprehensive textbooks of research ethics, called the belief that “being a research subject is a highly risky undertaking” an “incorrect belief.”⁶ Chalmers argued, similarly, that “...clinical trials are the most ethical way to benefit patients whenever there is uncertainty about proper diagnosis and therapy.”⁷ There have been studies suggesting that it is often (but not always) safer to be in a clinical trial than to receive the same therapies off-protocol.⁸ In the past, however, the data on non-enrolled patients has been difficult to obtain. That is no longer true.

Advances in medical informatics may enable us to better understand the risks and benefits of research. This, in turn, may require a shift in the focus of research regulation. If there is evidence, as there was in the SUPPORT study, that it is safer to be in a study than to not be in a study, then the ethical requirements of informed consent will make it essential to inform potential study subjects of this fact. That would be a welcome but challenging notion to incorporate into the worldview of patients, clinicians, and researchers.