Burdensome Research Procedures in Trials: Why Less Is More

Abstract

A large volume of trials involve invasive, nontherapeutic research procedures, like organ biopsy or sham surgeries, that can pose risks comparable with the experimental treatment itself but that have no direct benefit for volunteers. Though such procedures can enhance the value of clinical investigations, recent studies suggest that many studies involving invasive, nontherapeutic research procedures are not well planned and reported; some studies suggest that their results are often not utilized in the planning of new investigations. This commentary offers recommendations for how investigators, sponsors, and ethics committees might improve evaluation and implementation of studies involving invasive nontherapeutic procedures. We conclude by urging more demanding scientific standards for the rationale, design, and reporting of burdensome, nontherapeutic research procedures—particularly where they involve risk of serious complications.

In most clinical trials, the treatment under investigation accounts for the vast majority of risk and burden. However, a growing number of trials are incorporating invasive research procedures, like organ or tumor biopsies (1,2), that can pose risks comparable with the experimental treatment itself. Because these procedures often do not benefit participants, they raise important issues for human protection that have not always been negotiated satisfactorily.

Several scientific trends have spurred the growing use of invasive research procedures in trials. Targeted drug development and predictive medicine (which uses molecular signatures to match patients to drugs) often provide strong motivations for collecting tissue for molecular analyses. Another factor is regenerative medicine. Because many regenerative medicine approaches entail surgical delivery, blinding in randomized trials may require “sham” procedures that mimic the experience of collecting or receiving cells.

Economic factors are also encouraging investigators to use invasive procedures. Drug development is costly, in part because most drugs that enter into clinical testing do not demonstrate efficacy in large randomized trials. Many drug developers are turning to enrichment designs (which select patients on the basis of a biomarker) to boost their success rate. Invasive procedures have also been used to assess drug response at a molecular level, although the scientific value of this approach is controversial (3).

The use of invasive research procedures is especially widespread in cancer, where the value of molecularly targeted drugs is well recognized. Twenty-four out of 30 drugs approved by the US Food and Drug Administration (FDA) in 2014–2015 for oncology indications were for novel molecularly targeted therapies (4). An increasing number of oncology trials require a research biopsy during therapy to guide treatment, or for secondary or exploratory correlative end points (1). Some investigators have argued that such studies are necessary to ascertain mechanism, or even to define an optimal dose prior to a randomized dose-ranging phase II study (5,6). Such practices are not limited to oncology. Trials in neurological disease like Alzheimer’s (7,8) or multiple sclerosis (9) sometimes perform lumbar puncture to assess biomarker response in cerebrospinal fluid. In cardiology, randomized trials of cell therapy have often involved sham bone marrow biopsies or cardiac catheterizations (10).

As their name would suggest, invasive research procedures can be burdensome and risky. In cancer, complications can include pneumothorax, bleeding, infection, and arrthymia (11). Some procedures, like liver or lung biopsy, involve risk of major complications. At least one participant in an oncology trial has died from bleeding due to a research biopsy (12). Several papers describe major complication rates in cancer biopsy approaching 0.8%, but reporting of adverse events from research procedures is limited (13,14). Many other procedures, like bone marrow aspiration, breast biopsy, or lumbar puncture, do not entail risk of major complications. They are nevertheless stressful and uncomfortable for volunteers.

Invasive research procedures performed in the context of correlative, exploratory, or sham comparators have no direct benefits for research subjects; their ethical justification rests on their scientific rationale (15). However, invasive research procedures may not receive an appropriate level of scrutiny. To our knowledge, drug regulators do not have an explicit mandate to evaluate research procedures; sometimes research procedures are performed in the context of studies that are not contained within the trial protocol and are hence invisible to drug regulators. Though the FDA does consider them when reviewing protocols, our experience leads us to believe drug regulators grant wide discretion for review of research procedures to institutional review boards (IRBs) and investigators. Though IRBs are authorized to evaluate the balance of risks and benefits, one might question whether they exert an appropriate influence over the inclusion of invasive procedures given the prevalence of issues related to their design and justification.

For instance, one study reported that only 26% of study protocols for biopsy for correlative studies in cancer contained a statistical analytic plan (13). Another study found that a minority of such studies used blinded histopathological assessment (16). This leaves analyses highly susceptible to risk of bias. Also troubling are patterns of outcome reporting. Only a third of studies report results in full (17), and in a survey 40% of authors of publications involving invasive research procedures in cancer described incomplete reporting as occurring “more often than not.” This is reinforced by the observation that most studies report pharmacodynamic experiments as positive (16). A deficit of negative results suggests either that researchers are not publishing negative findings or that researchers are using correlative studies to address questions for which answers are mostly already known at the outset of investigation. Finally, some studies suggest findings are not used often to plan subsequent investigations. For example, one study found that only a third of biopsy-derived pharmacodynamic studies in phase I cancer studies were cited in subsequent publications; fewer than half of “positive” analyses were cited in subsequent publications (3).

How should one approach the design, conduct, or ethical evaluation of invasive research procedures? One strategy would be to grant researchers wide discretion in judging risk/benefit and allow subjects to decide whether they want to consent to invasive procedures. Several studies suggest that many research subjects are very willing to accept research procedures in trials (18–20). However, relying on consent alone is problematic. First, invasive research procedures are often mandatory, especially when trials involve biopsies that determine eligibility or when they involve sham procedures. Second, scientific value is—by its very nature—an expert judgment. Few patients are equipped to assess the scientific impact of an investigation and to weigh this against procedure risks. Indeed, that poor design and reporting practices prevail suggests just how hard it is for experts themselves to assess their scientific value.

An alternative is for sponsors, scientific review committees, ethics committees, data monitoring committees, and investigators to work together to render risk/benefit judgments. In Box 1, we offer questions reviewers should consider when weighing risks and benefits. These extend recommendations previously offered by other commentators and place particular emphasis on the way design and reporting shape the justification of procedures (21,22). We further suggest that invasive research procedures that present risk of serious complications should be limited to those that are necessary for addressing a question of major clinical or biological significance (ie, minimized in number and burden); they would also demand sufficiency for delivering that clinical or biological significance (ie, proper design and reporting).

Box 1.

Evaluating invasive research procedures

Risk

- What is the probability of major adverse events, based on a reference class of patients similar to those enrolling in the study?

- Can the same knowledge be obtained with lesser burden (eg, use of clinical tissue samples, use of liquid tumor biopsy, one instead of two tissue collections, or a less invasive sham procedure)?

- Does the protocol adequately describe measures for monitoring and managing symptoms and risks of invasive research procedures?

Scientific value

- Is there evidence to suggest feasibility of the investigation (eg, ability to recruit the desired sample size; ability to manage samples in accordance with the protocol)?

- Are analytical plans and hypotheses for study components involving invasive research procedures prespecified?

- Are threats to internal validity adequately addressed (eg, are studies involving invasive research procedures adequately powered; will samples be assessed in a blinded fashion; do assays have analytical validity)?

- If sham procedures are used, what is the precise aim of the control? If it is only for masking, might subjects be masked using less invasive procedures?

- Have researchers declared a commitment to complete reporting of all results, regardless of whether they confirm hypotheses?

- Can researchers articulate a plausible and evidence-based argument for how the absence of research procedure results would stymie further drug development?

- If procedures present a nontrivial risk of serious complications (ie, resulting in hospitalization or additional procedures), do answers to the above meet a very high standard?

Consent

- Do consent discussions address the risks and burdens of the specific research procedures that are contemplated? To what extent do subjects have prior experience with the research procedures, and are subjects given an option of declining procedures that are not critical to achieving a study’s main objectives?

- Is the consent process structured to ensure subjects acknowledge and accept procedures (and are not distracted by a focus on drug risks)?

Invasive research procedures furnish many human investigations with scientific cachet. However, their rationale, conduct, and reporting have often failed to join this cachet to substance. An aura of scientific rigor is no substitute for a dispassionate and case-by-case assessment of a research procedure’s justification. Applying principles like those advocated here would almost certainly curtail the number of studies with invasive research procedures. However, those remaining will meet a higher standard of scientific quality and ethical justification.

Funding

JK was funded by a grant from Genome Canada (PACE-OMICS) and JP was funded by a grant from the Greenwall Foundation for Bioethics.

Notes

The funder had no role in the writing of this manuscript or the decision to submit it for publication. We thank the anonymous peer reviewers for their helpful suggestions.

JK, DR, and MR declare that they have no competing interests related to the manuscript. JP has received research funding from Pfizer, and his spouse is an employee of GlaxoSmithKline.

JK, DR, MR, and JP all discussed the concept of the manuscript. JK wrote the first draft. DR, MR, and JP revised and edited the draft.