Risk in Clinical Research: Size Matters!

There came a time when the risk to remain tight in the bud was more painful than the risk it took to blossom.

Anaïs Nin

Risk, failure, and success are all part of the same equation. We attempt to prevent risk, but we can never avoid it. If risk avoidance becomes the sole guiding principle, immobility results. The large number of clinical trials in gene therapy implemented over the past 20 years attests to the willingness of all stakeholders to confront potential risk and make gene therapy successful.

Despite increasing enthusiasm for gene therapy over the past decade, in this issue of Molecular Therapy, Deakin et al. ask whether clinical gene therapy has become too risk aversive—and, if it has, whether this is delaying progress.¹ The authors highlight a number of pertinent issues regarding risk assessment in gene therapy, underscoring the notion that clinical risk is successfully addressed through strong preclinical data, the involvement of patients and caregivers, and a detailed ethical discussion of the potential dangers when testing a new therapeutic.

The key issue addressed by Deakin et al. is who should define acceptable risk. It is surprising that it has taken gene therapists 20 years to raise this question, which has generally been considered answered by those involved in determining the safety, ethics, and informed-consent process and in overseeing the intricate process of initiating clinical trials in gene therapy.

The authors suggest that the status quo has steered clinical trials toward approaches promoted as safer, particularly with regard to patient populations that can be enrolled. They argue that the regulatory and ethical bias toward perceived safety has led to trials being performed on patients who may have little to benefit from a risky procedure, either because they have a mild, non-life-threatening form of the disease or because their disease may have advanced too far to be significantly ameliorated by a particular gene therapy approach.

When we estimate the risk-to-benefit ratio, as the numerator approaches zero, the denominator approaches infinity. If potential clinical benefit is compromised by a flawed consideration of disease physiopathology,^2,3 whether the procedure will be safe becomes clinically irrelevant. Deakin et al. argue cogently that a stronger voice should be given to patients, scientists, clinical investigators, and other stakeholders, to complement the authority currently bestowed on bioethicists and other nonstakeholders who oversee the administration of gene therapy.

In a Commentary in this issue, Kimmelman takes a different view.⁴ He finds plenty of evidence for risk taken in trials of hereditary eye diseases,^5,6 heart disease, and a much-discussed trial for Parkinson's disease,⁷ and feels uneasy with patients and their advocates, in Deakin and colleagues' words, “seeking to reclaim ownership of risk.” He agrees that “a sound preclinical evidence base is crucial,” but feels that, although patients and clinical investigators “might be willing to accept greater uncertainty and higher risk,” the “integrity and health of the broader research enterprise” may not be sufficiently protected.

The constructive disagreements of Kimmelman represent a positive contribution to a call to give a much greater role in the decision-making process to patients, investigators, and physicians. Nevertheless, the central issue challenging the effectiveness and safety of clinical trials is buried in Kimmelman's assumption of a “sound preclinical evidence base” because it remains unclear how sound evidence should be defined.⁸

A majority of large phase III clinical trials fail to reach statistical significance despite years of promising preclinical data and phase I and II trials. That many phase III clinical trials employ hundreds of patients in each arm only to find that a new anticancer drug improves the survival of patients by no more than 10 days has remained largely unchallenged.^9,10 It may be that current standards of “sound preclinical evidence” are not sufficiently robust to overcome the challenge of human clinical trials. Many novel therapies are tested in a single, simple disease model, with many experimental variables kept within a narrow window. In such experimental designs, an increase of a few days' survival may be enough to achieve statistical significance, but are such data clinically significant?^9,10,11 In many cases, the preclinical evidence is unlikely to be robust enough to proceed to clinical trials, even with P values below the magic 0.05 value.

As I have argued in detail elsewhere, we need a practical approach to determining preclinical robustness;^8,12 i.e., under what circumstances are we convinced that our preclinical data are strong enough to successfully compete in the challenging environment of a clinical trial in human patients, where there is little control over many experimental parameters? For example, to render preclinical data more robust, any new therapy should be tested in multiple models of a particular disease, and the magnitude of any therapeutic response must be considered along with statistical significance. A drug that increases survival in a rodent by a “statistically significant” 10 days is unlikely to cure cancer in patients.

Patients suffering from deadly diseases need to be offered increased life expectancy measured in years, not days or weeks. Increased treatment efficiency will not be achieved by increasing the precision by which we measure a small effect but rather through the development of more effective treatments and the capacity to predict with greater accuracy which preclinical treatments might benefit patients. This will be achieved when our preclinical treatment models are robust and the therapeutic effects truly large and therefore clinically significant. Statistical significance is necessary, but never sufficient.¹¹ In gene therapy, as in all of translational medicine, the size of your effect matters.