RANDOMIZED CLINICAL TRIALS AND PERSONALIZED MEDICINE

Abstract

Randomized Clinical Trials (RCTs) have had a long, and often illustrious, history in the biomedical and clinical sciences. However, as Deaton and Cartwright make clear, population-based RCTs are assumption-laden and not necessarily the most appropriate or compelling strategies to use to assess an intervention in many settings. For example, the emergence of ‘personalized’ medicine, in which interventions are chosen for an individual patient based on that patient’s nuanced and possibly unique genetic or biochemical profile, has called into question the value of population-based RCTs. Although many researchers have proposed trial designs appropriate for advancing personalized medicine, this commentary focuses on the motivation and basic elements of three extensions and alternatives to traditional RCTs appropriate for personalized medicine. These extensions and alternatives to RCTs leverage modern developments in biomedical assays and health technologies, ‘big data’ collections and artificial intelligence, and include isolated N-of-1 trials, aggregated N-of-1 trials, trial designs that test an algorithm for matching interventions to patient profiles rather than testing a single intervention, and clinical and health ‘rapid learning systems.’ Ultimately, it is argued that although some elements of the traditional RCT are not likely to go away, how these elements are exploited to advance personalized medicine will be radically different than the way they were leveraged in population-based RCTs of the past.

INTRODUCTION

Randomized clinical trials (RCTs) have been the study design of choice for drawing inferences about a potential causal relationship between an intervention and patient outcomes. Unfortunately, RCTs suffer from a number of problems, many of which are well known to the scientific community but have yet to motivate sharp enough and consensus criticisms to lead to serious and pragmatic deliberations about alternatives, as pointed out by Deaton and Cartwright. In fact, it is arguable, as Deaton and Cartwright make clear, that in many settings traditional population-based RCTs are simply inappropriate. For example, in biomedical and clinical practice settings, the contemporary concept of ‘personalized’ medicine (otherwise known as ‘individualized’ or ‘precision’ medicine) – in which interventions are tailored to patients based on those patients’ unique molecular, physiologic, environmental and behavioral profile(1, 2) – is in many ways antithetical to the suggestion that statistically significant differences in the average responses of others to an intervention relative to average responses to a comparator intervention is enough to compel the use of that drug in all patients going forward.

In this brief commentary on RCTs, two extensions of traditional RCTs and one alternative to RCTs for advancing personalized medicine are described. The two extensions of RCTs to be discussed are: 1. N-of-1 and aggregated N-of-1 trials(3–5); 2. trials designed to assess the merits of strategies for matching one (or a few) among many interventions to patient profiles rather than assessing the merits of each individual intervention.(6) The alternative to RCTs for advancing personalized medicine to be discussed is rooted in the concept of a data-based ‘learning system.’(7, 8) Limitations of these extensions and alternative approaches to RCTs are also discussed, with the hope of sparking further discussion about the need for better strategies for vetting personalized medicine and the role, if any, of RCTs.

A BRIEF SYNOPSIS OF PERSONALIZED MEDICINE

The application of modern high-throughout, data-intensive biomedical assays and technologies, such as DNA sequencing, proteomics, imaging protocols, and wireless health monitoring devices have revealed a great deal of inter-individual variation with respect to mechanisms and factors that influence disease. As noted, this has led to the belief that interventions must be tailored (i.e., ‘personalized’) to the features each patient possesses. For example, it has been shown that patients with a specific cancer (chronic myelogenous leukemia or CML) whose tumors harbor a certain genetic alteration (known as the ‘Philadelphia chromosome’) can often be successfully treated with a certain drug, known as Gleevec, because Gleevec targets a specific enzyme (tyrosine kinase) that the Philadelphia chromosome causes to dysfunction and increase in a detrimental way.(9) As another example, individuals with cystic fibrosis caused by very specific pathogenic mutations that damage the CFTR gene can benefit from a drug known as Kalydeco, since Kalydeco stimulates the CFTR gene function despite the presence of those specific mutations.(10) A final example involves identifying potentially unique perturbations in a cancer patient’s tumor cells, known as ‘neo-antigens,’ harvesting cells from that patient that mediate the immune reactions, modifying those cells to specifically target the neo-antigens, and then putting the modified cells back in the patient’s body so these cells can attack the tumor cells. Such ‘immunotherapies’ have had notable successes, but can be very patient-specific.(11)

Given the growing number of drugs and intervention strategies that uniquely correct or overcome defects induced by very specific molecular perturbations that relatively small numbers of diseased individuals possess, a question of how to test those drugs and interventions for their efficacy arises. This is due, in part, to at least three different issues: 1. the limited number of patients that may possess disease-causing perturbations targeted by specific drugs might complicate recruitment into relevant trials and ultimately compromise the power of the studies to assess their efficacy; 2. a way of identifying patients with the relevant targeted perturbation must be leveraged in order to conduct appropriate trials, which may be logistically complicated and costly (e.g., it may be expensive to determine if a patient has an appropriate perturbation, so literally testing hundreds of people to identify the few with the perturbation could be challenging financially); and 3. there may be ethical issues surrounding the conduct of comparative trials testing personalized medicines since, in certain settings, if there is any reason to believe that a patient may benefit more from a personalized therapy given its formulation and mechanism of action (i.e., there is no ‘equipoise’ between interventions in that there is a reason to believe that one intervention is more likely to benefit the patient), providing what is likely to be an inferior treatment to that patient merely to have them act as a control for other patients could violate ethical standards and compromise peoples’ willingness and consent to participate in a trial.

N-OF-1 AND AGGREGATED N-OF-1 CLINICAL TRIALS

In the context personalized medicine, if is not known a priori what intervention might ‘match’ a patient’s profile (i.e., there is equipoise among interventions), then it becomes an empirical question as to which intervention might be most appropriate for that patient. N-of-1 or single subject trials are meant to objectively assess an individual patient’s response to an intervention by collecting enough data, under an appropriate study design, on that patient to enable valid statistical inferences to be drawn about the effect of that intervention. N-of-1 studies often leverage cross-over designs (repeated crossover designs, in particular, such as ‘ABABAB’ designs) to allow comparisons of a test intervention with a comparator (or placebo) intervention. Randomization can be exploited in such studies (e.g., randomizing the order in which a test intervention and a comparative intervention are provided) as can blinding, washout periods, multiple endpoints, and many other techniques often exploited in standard population-based RCTs.(3, 5)

N-of-1 studies are ultimately less concerned with how individuals in the population as a whole respond to a drug, but rather are designed to address how an individual patient responds. In this sense average treatment responses do not matter, except possibly in helping to prioritize drugs that might be assessed with an individual patient if there is equipoise and little reason to believe one or another intervention may benefit that patient to a greater degree. Importantly, N-of-1 studies can be aggregated and patterns across, e.g., unequivocal responders and non-responders to an intervention identified from the N-of-1 trials can be observed (e.g., maybe because they share a genotype). N-of-1 studies can suffer from a few issues such as serial correlation between the observations and carry-over effects but can be overcome with appropriate analytical methods and an appropriate design.(3) In addition, cross-over based N-of-1 trials are unethical and impractical for individuals with acute and/or life-threatening conditions, such as some forms of cancer of infectious disease, although sequential sampling, personal threshold-guided trials can be used in these settings.(5) Interestingly, it has been shown that, in a very limited number of settings, the conduct of N-of-1 trials on patients meant to objectively determine interventions that work best leads to reduced costs in the management of those patients going forward – suggesting that an initial investment in conducting such trials can lead to savings in the long run due to, e.g., fewer patient follow-up visits or fewer dissatisfied patients with a prescribed intervention.(12)

MATCHING INTERVENTIONS TO PATIENT PROFILES

Personalized medicine, as noted, is rooted in the belief that patients with different profiles may require different interventions. Therefore, the number of patients for which a particular intervention may work best might be small (or even only positively impact a single individual patient!). This complicates vetting relevant individual interventions using standard RCTs since the number of individuals appropriate to recruit for the trial (i.e., those with the right profile) might be limited and identifying them via profiling might be expensive. This is certainly the case for cancer treatments, where profiling patients’ tumors can lead to insights into how best to treat that patient, but the profiling can be expensive and the number of patients with appropriate profiles might be small.(13) Strategies for vetting personalized medicines have been developed however, in cancer contexts, and include ‘basket,’ ‘umbrella’ and adaptive trials.(6, 13)

The basic concept of these trials can be provided in a brief description of a basket trial, with an understanding that umbrella and adaptive trials extend this basic concept. Essentially, multiple patients are enrolled in a basket trial without regard to their tumor type. Each patient’s tumor is profiled, usually via DNA sequencing, leading to the identification of ‘driver’ perturbations in the tumor, such as mutations, that are likely pathogenic and cause the tumor to grow by disrupting the activity of particular genes. Based on an understanding of the mechanism of action of a number of drugs (i.e., what genes they affect), the patients are steered towards one of the treatment ‘baskets’ associated with one of the drugs. In other words, patients are simply matched to one of the available drugs based on what is found in their tumor profile. The trial then proceeds by comparing the outcomes of the patients provided drugs that are guided by the profiling (i.e., steered towards a treatment basket) vs. those provided drugs without the benefit of the profiling. Many basket trials have been pursued.(13)

One feature of basket trials is that if the trial fails, i.e., the patients receiving drugs that match their profiles do no better than those patients receiving drugs without the benefit of the profiling, then this would not necessarily suggest that the concept of personalized medicine is flawed, nor does it suggest that the drugs have no value, nor does it further suggest that basket trials are inappropriate. Rather, it could be that the scheme for matching drugs to patient profiles was flawed, or that the justification of the match of one of the drugs to a specific patient profile was flawed and more people were steered towards the treatment basket associated with that drug than others, reducing the effect of the matching strategy overall. This is fundamentally different than suggesting that an individual drug does not work as a result of its failure to show efficacy in a focused RCT on that drug.

Two additional features of such trials in the context of traditional RCTs are worth noting. First, some basket trials have been pursued that have a single basket and no comparison group, but rely on determining which patient profiles appeared to be associated with better outcomes to the intervention of interest in a series of post-hoc analyses.(14) Second, randomization in basket trials can border on being unethical if randomization is pursued after a patient has been profiled: if a patient is found to have a perturbation that is likely or known to be targeted by a specific therapy, then allowing that patient to potentially be put on a drug not known to target that perturbation (i.e., randomized to a treatment not known to match that patient’s tumor profile) could mean allowing that patient to be put on an inferior therapy for the sake of the trial, which is unethical. A counter to this argument is that if it has not been unequivocally shown that the drug supposedly matching the patient’s profile leads to better outcomes, then for all intents and purposes equipoise exists and randomization should be pursued for the sake of obtaining insight about the efficacy of the drug for patients with that profile.

HEALTH LEARNING SYSTEMS

As pointed out by Deaton and Cartwright, despite the belief that randomization will, on average, balance out the number of individuals with certain features that could influence responses to the interventions being tested between, e.g., an active and placebo arm of an RCT, there are often attempts to identify covariates that are associated with responses within an RCT by pursuing post-hoc analyses. The motivation of the analysis is to determine which patients may benefit most from the interventions of interest. Clinical trials may not be the most appropriate settings for this kind of analysis. In fact, in the context of traditional regulatory processes for approving the use of a medical intervention in the population at large, an argument could be made that if a medical intervention is found to be non-toxic and safe via traditional Phase I trials, and shown to be efficacious, at least to some degree, in Phase II trials, then the intervention should be approved and its empirical, every day practical use in clinical settings should provide indications of which individuals benefit most from that intervention. This would therefore question the utility (and costs!) of traditional, often RCT-based, Phase III and Phase IV clinical trials assessing the utility of the intervention in the population at large.

Eliminating Phase III and Phase IV trials on this basis would be, without question, controversial, but would be less controversial if appropriate infrastructure was in place to collect data on patients (e.g., demographic information, genetic and biochemical profiles, outcomes, etc.) in order to facilitate analyses meant to identify patient subgroups that benefit most from different interventions. In this way the clinical community would ‘learn’ about which patients should be provided interventions going forward. ‘Learning systems’ of this sort have been proposed by many researchers and clinicians and could leverage emerging machine learning, artificial intelligence and ‘big data’ analysis paradigms.(7, 8, 15) However, such systems would require pronounced attention to the quality, validity, harmonization and potential standardization of the data collected.(16, 17) In addition, such systems could be a vehicle for conducting trials comparing interventions, possibly without randomization, using, e.g., propensity scores and other statistical methods for ensuring appropriate and valid inferences can be drawn.(18)

CONCLUSION

The emergence of personalized medicine and sophisticated high-throughput, data-intensive biomedical assays have radically changed the way the clinical science is perceived and will be pursued. As a result, attention must be given to how the community will vet or prove the worth of new interventions and health technologies that are likely to be beneficial for possibly small subsets of the population or even individual patients. Traditional, population-based, RCTs rooted in the comparison of average responses between treatment arms, will undoubtedly have a role in this data-intensive, personalized medicine era (e.g., comparing personalized and non-personalized interventions to assess their overall benefits, and possibly cost-benefit tradeoffs, in the population at large), but a radically diminished one. It is arguable that the problems and limitations of traditional RCTs noted by Deaton and Cartwright, as well a watch-dogs of clinical research such as Ioannides and colleagues,(19–21) will become more pronounced, possibly leading to non-randomized trials,(22, 23) extensions of RCTs, such as N-of-1 trials and intervention-matching trials, and large-scale, data-based systems designed to learn as quickly as possible which patients appear to benefit most from specific interventions.