Testing Cardiovascular Drug Safety and Efficacy in Randomized Trials

Abstract

Randomized trials provide the gold standard evidence on which rests the decision to approve novel therapeutics for clinical use. They are large and expensive and provide average, but unbiased estimates of efficacy and risk. Concern has been expressed about how “unrepresentative” populations and conditions that pertain in randomized trials might be of the “real world”, including concerns about the homogeneity of the biomedical and adherence characteristics of volunteers entered into such trials, the dose and constancy of drug administration and the mixture of additional medications that are restricted in such trials but might influence outcome in practice. A distinction has been drawn between trials which establish “efficacy” and those that demonstrate “effectiveness” - drugs that patients actually consume in the “real world” for clinical benefit1. However, randomized controlled trials remain the gold standard for establishing efficacy and the testing of “effectiveness” with less rigorous approaches is a secondary, albeit important consideration. Despite this, there is an appreciation that “average” results may conceal considerable inter-individual variation in drug response, leading to a failure to appreciate clinical value or risk in subsets of patients2,3Thus, attempts are now being made to individualize risk estimates by modulating those derived from large randomized trials with the individual baseline risk estimates based on demographic and biological criteria - the individual Numbers Needed to Treat to obtain a benefit, such as a life saved4. Here, I will consider some reasons why large phase 3 trials - by far the most expensive element of drug development - may fail to address the “unmet medical needs” which should justify such effort and investment.

Introduction

Randomized trials provide the gold standard evidence on which rests the decision to approve novel therapeutics for clinical use. They are large and expensive and provide average, but unbiased estimates of efficacy and risk. Concern has been expressed about how “unrepresentative” populations and conditions that pertain in randomized trials might be of the “real world”, including concerns about the homogeneity of the biomedical and adherence characteristics of volunteers entered into such trials, the dose and constancy of drug administration and the mixture of additional medications that are restricted in such trials but might influence outcome in practice. A distinction has been drawn between trials which establish “efficacy” and those that demonstrate “effectiveness” – drugs that patients actually consume in the “real world” for clinical benefit¹. However, randomized controlled trials remain the gold standard for establishing efficacy and the testing of “effectiveness” with less rigorous approaches is a secondary, albeit important consideration. Despite this, there is an appreciation that “average” results may conceal considerable inter-individual variation in drug response, leading to a failure to appreciate clinical value or risk in subsets of patients^2,3. Thus, attempts are being made to individualize risk estimates by modulating those derived from large randomized trials with individual baseline risk estimates based on demographic and biological criteria – the individual Numbers Needed to Treat to obtain a benefit, such as a life saved⁴. Here, I will consider some reasons why large phase 3 randomized trials – by far the most expensive element of drug development – may fail to address the “unmet medical needs” which should justify such effort and investment.

Whose need is unmet?

Those best positioned to address whether a need is unmet are the relevant patients and this has most effectively been established by those with AIDS. However, more commonly the definition of which need is unmet is made by pharmaceutical companies which largely foot the bill for drug development and often do so exclusively for the large scale randomized trials necessary for their drug’s approval by the FDA.

In the case of nonsteroidal anti-inflammatory drugs (NSAIDs), the discovery of cyclooxygenase (COX) – 2 prompted the hypothesis that NSAIDs selective for inhibition of COX-2 would be less likely to result in serious gastrointestinal (GI) complications (that were attributed to inhibiting COX-1 in platelets and gastro-duodenal epithelium) than older drugs that inhibited both COX enzymes. Indeed, this hypothesis was not rejected by the outcomes of two randomized trials that showed the selective inhibitors, rofecoxib⁵ and lumiracoxib⁶ were less likely to cause such complications than their comparators. Although these trials – and those of celecoxib⁷ and valdecoxib⁸ showing that surrogates of such GI endpoints were less likely to occur than with mixed inhibitors - were designed to obtain drug approval based on an improved GI safety profile, they left unaddressed two fundamental issues relevant to clinical practice. The first was whether there was any difference in analgesic or anti-inflammatory efficacy between selective and non-selective NSAIDs. Although preclinical⁹ and clinical research¹⁰ was consistent with COX-2 being the major source of prostaglandins mediating pain and inflammation, there is evidence for a variable contribution from COX-1; indeed increased expression of both enzymes is evident in inflammatory tissues, such as synovia from the joints of patients with rheumatoid arthritis¹¹. No formal assessments of comparative efficacy were included in the large scale trials of NSAIDs.

The second issue was evidence that emerged as these trials began, consistent with a cardiovascular hazard from COX-2 selective NSAIDs¹² and of a pharmacodynamic interaction between NSAIDs that inhibited COX-1 (and COX-2) and low dose aspirin¹³. Such competitive, active site inhibitors of COX-1 could impede access by aspirin to its target serine residue in the platelet enzyme. This substituted sustained inhibition of thromboxane dependent platelet function due to covalent modification of COX-1 with transient inhibition during a dosing interval, thereby undermining the cardioprotective effect of low dose aspirin. This interaction would not be expected with drugs that were selective for inhibition of COX-2, as this form of the enzyme is not extant in mature human platelets.

As patients with chronic arthritis often have concomitant heart disease, they are commonly receiving low dose aspirin for cardioprotection. Even low doses of aspirin can confer risk of GI adverse events and this risk adds to that of nonselective NSAIDs¹⁴. However, the recognition of a cardiovascular risk from selective inhibitors of COX-2 due to suppression of prostacyclin (PGI₂) formation¹² left practitioners with an unmet medical need – how best to treat their patients with both heart disease and arthritis? No clinical trials designed to gain approval of COX-2 inhibitors addressed that question; either patients who needed aspirin were excluded⁵ or the trials were not powered to address comparative drug effect in patients who were vs those who were not on concomitant aspirin⁷.

Just as trials may be designed primarily to gain drug approval rather than address comprehensively an unmet medical need, they can also be designed in a way that protects a franchise. Celecoxib was predicted to confer a similar cardiovascular hazard as rofecoxib¹⁵. Three randomized trials¹² provide evidence that it does confer such a hazard and this risk was indistinguishable from that of rofecoxib in a recent overview analysis of almost 700 randomized trials of NSAIDs¹⁶. Despite this, rofecoxib was withdrawn from the market by its sponsor, while celecoxib is available, albeit with a “black box” warning. Shortly after the FDA advisory committee voted (25-0) in 2005 that celecoxib did indeed confer a cardiovascular hazard, the sponsor announced the Prospective Randomized Evaluation Of Celecoxib Integrated Safety Vs Ibuprofen Or Naproxen (PRECISION) trial as a statement of their confidence in the comparative cardiovascular safety of celecoxib in patients with or at high risk of cardiovascular disease. This is a noninferiority trial with a power now adjusted to only 80% and an upper bound of confidence of 1.4, thus arguably biased towards the null. Aside from this limitation, this trial was not powered or designed a priori to compare patients randomized to the three comparators on and off aspirin. This fundamentally undermines the ability to interpret the results of the trial as both ibuprofen and naproxen, but not celecoxib, pharmacodynamically interact with aspirin in the platelet; a “win” for celecoxib could reflect no more than an asymmetric impairment of cardioprotection by low dose aspirin in the other two groups. An attempt to circumvent this limitation by urging patients to take their morning dose of aspirin before the NSAID stems from a mis-interpretation of the original paper on the drug interaction; sufficient ibuprofen remains after the evening dose to sustain the interaction even if aspirin is given first in the morning¹³. These design flaws were highlighted before the trial was initiated and it was suggested that such an event driven trial would last much longer than the few years predicted by its leadership as doctors were unlikely to enlist patients with a high risk of heart disease into a trial including a drug with such an established risk. Indeed, the European Medicines Agency refused to approve the trial for performance in the European Union and only ~50% of patients currently enrolled in PRECISION are said to be taking aspirin. Such a prolonged trial might be seen to defend the franchise by deferring an “answer” while the patent for celecoxib remained extant and it was the only one of the three comparators that stood to benefit from direct to consumer advertising for arthritis. Similar design defects apply to the Standard Care versus celecoxib Outcome Trial (SCOT)¹⁷. This noninferiority study, purposefully targeting patients at low cardiovascular risk, also failed to account for the possibility of aspirin – NSAID interactions and had originally included an open label phase of celecoxib administration to determine tolerability prior to randomization. This was omitted after the first two years (2008–2010) as it was deemed not to influence adherence rates and at this time power was adjusted, like PRECISION, to 80% with an upper bound of 1.4. Meantime, back in the US, celecoxib, sustained by the longest direct to consumer advertisements for an approved drug on TV, remains roughly a $2Bn/ year product ¹⁵. As it approaches the end of its patent life in 2014, the estimated completion date for PRECISION is September 2015¹⁸.

There are other examples where unanticipated prolongation of recruitment into RCTs has helped defend the franchise. The Improved Reduction of Outcomes: Vytorin Efficacy International Trial (IMPROVE-IT) has sustained interest in the potential cardiovascular efficacy of ezetimibe despite unimpressive data from smaller studies and its potential association with an excess of cancer deaths ¹⁹. Here, the question is whether patients with relatively low LDL cholesterol on statin therapies might benefit from further reduction in cholesterol by addition of ezetemibe²⁰.

Where is the rationale?

A recent analysis of drug development drawing information from more than 800 commercial entities estimates that the likelihood of approval (LOA) of a drug entering phase 1 is on the order of 10%. The LOA for cardiovascular disease is even lower, around 6%; here approval is often configured on death rates and large trials are required. While most failures occur at the transition from phase 2 to phase 3, the failure rates in phase 3 cardiovascular trials approximate 50% which, given their cost – roughly 60% of the cost of drug development - presents a particular challenge to industry²¹.

Phase 2 classically represents an opportunity to elucidate the mechanism of drug action and to clarify how that might be of therapeutic relevance to a particular clinical condition. However, this requires the integration of preclinical knowledge, a sophisticated approach to deep phenotyping of drug response in small numbers of patients and a comprehensive understanding of clinical medicine and trials methodology. Investigators with such an integrated interdisciplinary background in basic and clinical pharmacology are in short supply, either as scientists within industry or as advisors from the academic sector²². An example of a large phase three trial performed on the basis of a very limited a priori rationale is the PReVeNTIoN oF CeReBRoVAsCulAR and cardiovascular Events of ischaemic origin with teRutroban in patients with a history oF ischaemic strOke or tRansient ischaeMic attack (PERFORM) study ²³– a comparison of a thromboxane receptor antagonist (terutroban 30mg/day) and low dose (100mg/day) aspirin. Here roughly 9500 patients who had had an ischemic stroke in the previous 3 months or a TIA in the previous 8 days were randomized between the treatments.

The first mechanistic rationale for this study was that the antagonist would not suppress PGI₂ while blocking the platelet aggregatory and vascoconstrictive effects of TxA₂. A second was that the antagonist would block the effects of incidental ligands at the TxA₂ receptor, such as free radical catalyzed isoprostanes²⁴ which would be unaffected by COX-1 blockade by low dose aspirin. Suppression of PGI₂ formation does have cardiovascular consequences, as we now know from the COX-2 inhibitor saga¹². However, unlike such NSAIDs, which suppress biosynthesis by 60–80%, low dose aspirin is relatively selective for COX-1 and suppresses prostacyclin by only about 20%^12,25. It might take a very large trial indeed to reveal the discriminant benefit of avoiding this modest suppression. As for isoprostanes, formation, these makers of lipid peroxidation are markedly elevated during reperfusion after a period of tissue ischemia²⁶. However, their levels are barely altered in patients such as those admitted to PERFORM who have not been subjected to therapeutic reperfusion²⁷. Thus, the mechanistic basis for the trial design was highly questionable from the outset and it was perhaps unsurprising that the near $1Bn study was stopped prematurely for futility on the basis of the recommendation of the Data Monitoring Committee.

Biomarker Beliefs and Magical Thinking

One reason the failure rate in phase 3 trials of cardiovascular disease is so high is that biomarkers measured in phase 2 poorly predict the outcome measured in phase 3, most commonly survival. However, there is also the issue of wishful thinking. A nice example here is the development of torcetrapib, an inhibitor of cholesterol ester transfer protein (CETP). Here studies in phase 2 revealed the desired dose dependent elevation of high density lipoprotein (HDL), an apparent biomarker of cardiovascular benefit. However, much confusion has surrounded the therapeutic value of raising HDL. Epidemiological studies have long associated HDL inversely with cardiovascular risk. but these observations are confounded by many aspects of a healthy lifestyle that may co-segregate with high HDL. Perhaps the most convincing evidence to the contrary is the failure of genetic variance in HDL levels to relate to cardiovascular risk ²⁸. Unfortunately the same studies of torcetrapib revealed a dose related increase in a much more established biomarker, albeit of risk not benefit; blood pressure. Vesting belief in validation of the underlying hypothesis for drug development – an elevation in HDL - rather than in the unexpected increase in the more validated end point - blood pressure - resulted in the close to $1 billion Investigation of Lipid Level Management to Understand its Impact in Atherosclerotic Events (ILLUMINATE) trial. This demonstrated that torcetrapib actually increased cardiovascular event rates and prompted its abandonment ²⁹. An immediate question for sponsors of trials of other CETP inhibitors was whether this represented a consequence of CETP inhibition or an off target effect of torcetrapib. Evidence emerged that torcetrapib might have conferred a cardiovascular risk not only via an increase in blood pressure but also, and perhaps distinctly, from an increase in plasma aldosterone by upregulation of CYP11B2 (aldosterone synthase). Importantly, torcetrapib elevated blood pressure in rodents, which lack CETP³⁰. This was not a feature of the other three CETP inhibitors in development. However, a phase 3 trial in almost 16,000 patients of another CETP inhibitor, dalcetrapib, failed to reveal a cardiovascular benefit, although unlike torcetrapib, it did not increase risk. Phase 3 trials of two other inhibitors, anacetrapib and evacetrapib continue. The absence of a clear impact of loss of function mutations of CETP – which elevate HDL cholesterol and lower LDL cholesterol - on cardiovascular outcomes and the failure of other genetic variants in HDL alone to influence events heighten the sense of uncertainty in this gamble. The residual hope is that salutary effects of CETP inhibitors independent of their impact on HDL, such as a reduction in LDL cholesterol, lipoprotein (a) and apoB, might deliver the desired outcome. Alternatively, as Rader and DeGoma suggest, this may be the first pharmacological mechanism to see four different molecules fail in large-scale, >$1 billion phase III trials³⁰.

Another example of magical thinking in phase 2 is the response to results of the Integrated Biomarker and Imaging Study (IBIS)-2 study of the lipoprotein – associated phospholipase A₂ inhibitor, darapladib³¹. Here, two primary endpoints were specified based on intravascular ultrasound palpography; C-reactive protein (CRP) was amongst the many secondary endpoints. Despite darapladib failing to influence either primary endpoint (or CRP), the decision was taken to proceed to phase 3. This was based on the attenuation by darapladib of one of the secondary endpoints in IBIS-2, the increase with time in lipid necrotic core, as assessed by virtual histology. This decision was despite the recognition that (i) the role of lipoprotein – associated phospholipase A₂ in cardiovascular biology is controversial (it also degrades platelet activating factor, suggesting its inhibition might not be desirable) and (ii) failure of a loss of function mutation to influence cardiovascular outcome in 26,000 individuals. Meantime, a phase 3 trial of varespladib, an inhibitor of a secretory phospholipase A₂ was terminated for lack of efficacy.

How useful is the answer?

Another drug long known to elevate HDL is niacin and trials that failed to show its cardiovascular benefit have been criticized as underpowered or for limitations of trial design. The latter was true of prematurely terminated Atherothrombosis Intervention in Metabolic Syndrome with Low HDL/High Triglycerides: Impact on Global Health (AIM-HIGH) which involved administration of lower doses of niacin to the control group to mask the effect of niacin induced flushing ³². The HPS2-THRIVE (Treatment of HDL to Reduce the Incidence of Vascular Events) trial was an opportunity to answer these outstanding questions. It was much larger than the antecedent clinical trials (25,673 patients) and suitably powered to detect a reasonable expectation of benefit. However, this was a trial to assess the usefulness of a drug combination, extended-release niacin (ERN) (2 g) and the prostaglandin D₂ receptor (DP1) antagonist, laropiprant (LRPT) (40 mg), in the reduction of major vascular events (composite of nonfatal myocardial infarction, coronary heart disease death, stroke, or arterial revascularization)³³. The study was not designed to answer the questions of how either drug alone acted or how they compared with the combination. ERN/LRPT treatment reduced LDL cholesterol by an average 10 mg/dl, and increased HDL cholesterol by 6 mg/dl. The primary endpoint was not different between the two groups. This study provided a clear cut answer for the drug combination and the sponsor ceased its further development. However, it is impossible to parse with confidence the differential impact of the two elements of this combination and to know how the results might extend to the common use of niacin. The results of ongoing trials of two remaining CETP inhibitors and inhibitors of PCSK9 will influence the HDL discussion. Will the performance of a sufficiently large and well-controlled study of niacin in more defined populations, say those relatively resistant to benefit from statins, ever seem like a cost-effective initiative to a commercial sponsor? The answer will likely drive the progressive abandonment of niacin, a drug that has long been a mainstay of cardiovascular therapy, while we still poorly understand its many potentially relevant mechanisms of action and have an incomplete picture of its clinical utility.

Subversion and disruption

The rosiglitazone saga began with an overview analysis in 2007 of heterogeneous trials suggested the possibility of a cardiovascular hazard from the anti-diabetic drug rosiglitazone³⁴. In contrast to the experience with COX-2 inhibitors, the data which were the basis for pressing the FDA to restrict access to this drug did not derive from placebo controlled trials and the limitations of the analysis were much debated. Again, in contrast to the COX-2 story, there was no experimental basis to explain the mechanism of the hazard, never mind why this risk should attach to rosiglitazone more than other members of the class. Considerable political pressure, not least abetted by evidence of bad behavior on the part of the sponsor, resulted in the FDA attaching severe restrictions to the use of rosiglitazone in 2010. Meantime sales of the drug and, more importantly, recruitment into a randomized comparison of rosiglitazone with pioglitazone and other diabetes drugs began to drop precipitously. Subsequently the FDA turned its attention to the RECORD (Rosiglitazone Evaluated for Cardiovascular Outcomes and Regulation of Glycemia in Diabetes) study, designed to assess the effects of rosiglitazone on cardiovascular outcomes in a randomized trial and commissioned an analysis by Duke University independent of the commercial sponsor. This failed to support the original hypothesis advanced by the 2007 meta- analysis of a cardiovascular hazard³⁵. While the FDA then removed language concerning an asymmetric cardiovascular risk from the label, the damage was done: rosiglitazone is highly unlikely ever to recover a substantial market share. There were few heroes in this story, replete as it was with data concealment, and allegations of intimidation and conflict of interest. However, it illustrates the necessity of obtaining evidence from randomized trials as a basis for drug approval and the limitations of other types of evidence, such as observational studies, pre-clinical studies in model systems, human genetics, and clinical pharmacology or, as in this case, a meta-analysis of heterogeneous trials. These approaches can raise a question or indeed, provide an explanation for the outcome of randomized trials, but not substitute for them in establishing risk or benefit.

Conclusions

Randomized trials remain the basis for drug approval, yet their high failure rate imposes an enormous cost on the biomedical enterprise. Attention has been drawn recently to the limitations of pre-clinical data as predictors of likely clinical response. In particular, the poor reproducibility of some data has led to calls for more rigorous standards in pre-clinical drug studies in model systems, including randomization, blinding and public provision of primary data³⁶ In some cases, carefully collected preclinical data fail dramatically to predict human biology ³⁷, in others they predict it quite efficiently ^12,15. Aside from the rigor with which preclinical studies are performed, it is worth remembering that a model is no more than a model and while the same “answer” drawn from more than one species may enhance the predictive power of such data, there is no species in which drug action unfailingly predicts what happens in humans. Interestingly, while studies in humans supported by industry are biased in favor of drug efficacy compared to those performed without such support, precisely the opposite bias appears to pertain in preclinical studies³⁸. Perhaps such pre-clinical studies are performed more carefully within industry as they will be influence directly investment decisions, but once the drug moves into clinical development, there are pressures from within the company to reinforce that decision.

There has been a call to perform phase 1 studies under randomized, double blind conditions, so better to provide interpretable data on which to base the decision to proceed³⁶. Other advances – such as microdosing to derive more safely estimates of both pharmacokinetic and pharmacodynamic responses to new chemical entities and genomic and metabolomic information to identify more accurately populations susceptible to drug action – may also improve the efficiency of the process.

However, the examples in this review focus on problems further downstream. These include how we decide what questions to address in clinical trials, how we interpret the rationale for such decisions, how vested interest in the evolution or cessation of a program in drug development can cloud judgment, particularly when it comes to the interpretation of suggestive, but inconclusive results in phase 2.

The opportunity and the challenge is to rely on a more thorough understanding of mechanism of drug action, dependent on reliable biomarkers and deep phenotyping approaches in phase 2 and to design more targeted, smaller and faster trials in phase 3 that are more likely to succeed. An example of such a strategy is the success of ivacaftor in improving forced expiratory volume by ~10% in patients with cystic fibrosis (CF) with the G551D mutation in the CF gene³⁹. Similarly, such a strategy can lead to a faster failure or the redesign of a trial based on such a failure. An example is the failure of ivacaftor in a subsequent study of 69 patients with the R117H mutation⁴⁰. Here, a pre-specified analysis of the 50 patients aged 18 years or older suggested benefit, a possibility that can be addressed directly in a further study.

Several further initiatives may serve to strengthen the trials process. These include (i) more vigorous engagement with patients in the determination of both unmet clinical need and the effectiveness of efficacious therapies⁴¹; (ii) provision of resource to perform trials that are in the interest of the public health, but not of a commercial sponsor; (iii) a safe haven, provided by the FDA, for systems pharmacology and incentives to explore of the full breath of drug action – not just what is thought to relate to the primary indication being pursued - early in human development⁴² and (iv) a strategic expansion of investigators trained in translational medicine and therapeutics – spanning the preclinical to clinical divide²².

These initiatives should be coupled with an integrative approach to mechanistic data – drawn from preclinical models and humans – with pharmacoepidemiology to develop hypotheses relating to adverse effects as randomized trials are typically underpowered to detect them. Examples of where such an integrated approach together with refined trial design has been helpful is in detection and elucidation of the cardiovascular hazard from NSAIDs and in probing the cardiovascular adverse effects of erythropoietin⁴³ and its potential to undermine efficacy in the treatment of cancer⁴⁴ Finally, the process would benefit from a more detailed and accessible provision to the public of incentives provided by industry to academia – both to investigators and to their institutions via indirect costs - than is presently extant and a recognition that conflict of interest can relate to fame as well as fortune.