Rethinking Phase II Clinical Trial Design in Heart Failure

Abstract

The incidence and economic burden of heart failure continue to rise worldwide, despite implementation of a number of effective heart failure therapies. Although there have been a number phase I–II studies of potential novel heart failure therapies over the past decade, none of these new compounds have been successful in phase III clinical trials. While there are likely a number of reasons for this failure, one of the problems that has become increasingly apparent is the inability of phase II trials to correctly identify novel therapies that will be successful in phase III clinical trials. In the following review, we will discuss the some of the problems inherent with current phase II heart failure clinical trials, as well as discuss possible ways to rethink phase II development of new therapies for heart failure.

Introduction

Heart failure is a burgeoning health care problem worldwide, and is the leading cause of heart failure admissions in the industrial world. Despite implementation of effective heart failure therapies and improved clinical outcomes, both the incidence of heart failure and the burden of disease continue to climb. Following the introduction of angiotensin converting enzyme inhibitors (ACE), angiotensin receptor antagonists, β-adrenergic blocking agents, and aldosterone antagonists there have been very view new effective pharmaceuticals approved for the treatment of heart failure. Indeed, with the exception of Ivabradine, which is only currently approved in Europe, there have been no new drugs approved for heart failure since Bidil^™ (fixed dose hydralazine isosorbide) was approved in 2005 by the Food and Drug Administration (FDA) for a very narrow heart failure indication.

Despite a wealth of basic science reports and early preliminary phase I–II studies describing potential novel therapies, none of these compounds have been successful in phase III clinical trials. Whether this disconnect between phase II and III studies signifies a lack of efficacy for the proposed therapeutic strategies, or alternatively, a failure to effectively design clinical studies is a topic of intense debate. It bears emphasis that the problem of developing new therapies for heart failure reflects the more general problem of developing novel therapies. Indeed, in 2004 the FDA’s Critical Path Initiative identified “the increasing difficulty and unpredictability of medical product development “[1]. A 2006 update announced that “[the FDA’s] outreach efforts uncovered a consensus that the two most important areas for improving medical product development are biomarker development and streamlining clinical trials” [2].

In the following review, we will discuss the some of the problems inherent with phase II clinical trial designs for the development of novel heart failure therapeutics, as well as discuss possible ways to rethink phase II development of new therapies for heart failure.

Why Heart Failure Drugs Fail

Although the discovery of ACE inhibitors, β-adrenergic blockers, and aldosterone antagonists has resulted in dramatic improvement in the care of heart failure patients, it is sobering to realize that these therapies were largely based on observations from small clinical studies, rather than on carefully designed pre-clinical based on novel target identification in the laboratory. Indeed, attempts to develop new heart failure therapies based on the rational identification of drug targets has resulted in disappointing results in phase III trials. One obvious explanation is that the low hanging fruit may have already been picked, and that demonstrating the benefit of additional add-on therapies on top of conventional triple therapy (ACE, β-blocker, aldosterone antagonist) is exceedingly difficult because annual mortality is ~ 5–7% for patients with moderate heart failure. Another explanation is that our current approach to treating heart failure has primarily focused on targeting cell-surface receptors or intracellular receptors [3]. While this approach has worked extremely well for antagonizing various components of the adrenergic and the renin-angiotensin-aldosterone systems, this type of “reductionist” approach has not worked well for antagonizing other systems (e.g. endothelin, adenosine, tumor necrosis factor). As a case in point, Ivabradine, which the most recently approved therapy for heart failure in Europe, blocks the I_f ion current channel, which is highly expressed in the sinoatrial node. Another potential reason for failure to develop new heart failure therapies is our inability to identify effective therapies in phase II studies, which abound with false positive results. Equally concerning is the potential for false negative results that result in cessation of a promising new therapy in phase II. Lastly, there is a growing potential for “false neutral” results, which refers to therapies that are as yet untested in the clinic because the development path forward is not at all clear.

Traditional Clinical Trial Design

The initial steps in the clinical development process for a new heart failure therapy are to provide a bridge between the basic science that originally suggested that a disease causing pathway could be targeted therapeutically, and the definitive studies that convince regulatory agencies that the therapy can beneficially influence outcomes in a patient population. This transition from “bench-to-bedside” is complicated by numerous factors, but it ultimately requires a “proof of concept” confirmation that the therapy performs in humans in its intended manner and a determination of appropriate doses to allow more widespread testing of hypotheses (i.e. dose selection). For newer targets that have not yet been tested in humans, it is often difficult to pick relevant clinical end points that will demonstrate proof of concept. Table 1 summarizes a variety of approaches to these critical steps in the evaluation of new therapies [4].

Table 1.

Approaches to the Design of Phase II Trials

Approach	Method	Advantages	Disadvantages
Intuitive	Provide drug to select group of investigators who observe effects in open-label, unblinded studies and make recommendations	Rapid, cheap and ease to conduct	Investigators cannot reliably discern clinical benefit Purely subjective
Mechanistic	Administer drug in multiple doses and select dose that has optimal biologic effect thought to best reflect mechanism of action of drug	Often quantitative Testable hypothesis Rational	Limited ability of pre-clinical research to reliably identify relationship of importance of mechanism in animals to human Biologic effect of drug may be impossible to measure in patients Drug may have multiple effects that may supercede its intended actions Difficult to determine the degree of the biologic effect required to demonstrate efficacy Short-term biologic effects may not be maintained long-term No clear relation between mechanistic efficacy and effect on clinical outcomes
Efficacy-Pilot	Administer drug in multiple doses and select dose that has optimal effect on an arrary of clinical end points	Clinical relevance	Usually no single end point selected, so determines if “things are generally going in the right direction” Differences are typically small and trends are often conflicting Does not give benefit: risk information
Safety-Pilot	Administer drug in multiple doses and select dose that has optimal safety profile	Clinical relevance	Assumes that mechanism of action and clinical efficacy is established Difficult to determine the correct dose of a new drug based on its safety profile Unlikely that safety can effectively be assessed in intermediate sized trial (<500 patients) Does not give benefit: risk information

(Adapted with permission from Packer M, Current perspectives on the design of phase II trials of new drugs for the treatment of heart failure. Am Heart J 2000; 139: S202–S206)

Clinical development programs of new heart failure therapies have traditionally been divided into 3 phases, each with distinct objectives and potential issues in trial design. The purpose of the Phase I studies is to assess the safety of a new therapy in humans, with the specific objectives to characterize the metabolism, pharmacokinetics, pharmacodynamics, dose-response, tolerability, and possible dose-limiting side effects of the therapy prior to further investigation. These studies vary greatly in size, but generally range from 20–80 patients. Phase II studies evaluate the dose and effectiveness of the therapy for a specific indication(s) in patients with the condition of interest and determine the common short-term side effects and risks associated with the drug. In pragmatic terms, Phase II studies are critical in defining the dose to be used in the larger Phase III studies and to confirm the proof of concept of the therapy being tested, as well as providing information on the anticipated magnitude of effect to be studied. Due to the limited time and financial resources that are available during the early phases of development, early studies may evaluate a limited number and range of doses and employ surrogate end points for clinical outcomes. Phase III trials are large studies which are designed to convincingly demonstrate the efficacy of the therapy, to provide safety information for a more complete evaluation of its benefit-to-risk characteristics, and to ultimately define how the therapy should be used. These trials often range from several hundred to thousands of patients. Recently, the distinction between Phase II and Phase III trials has become progressively blurred and less useful, and new study designs have been developed that transition rapidly between Phase II and Phase III studies while retaining much of the study architecture [5].

Types of phase II trials

Phase II clinical trials can be grouped into a single arm study, versus non-randomized trials vs. randomized trials that include a placebo arm (Figure 1). Given that the pathophysiology of heart failure is exceedingly complex, and that the responses to a given therapy are highly variable as a result of the heterogeneity of heart failure patients and presence of multiple medical combordities that may mute the effects of an effective new therapeutic agent, a single arm phase II trial comparing the clinical status of heart failure patients at baseline and after treatment is not likely to be inadequate for identifying compounds that will be successful in phase III. The next question in phase II development of a two arm trial is whether randomization is necessary to identify the effectiveness of the novel agent. Randomized trials have many advantages over studies with nonrandomized concurrent or historical controls, including the elimination of bias in the assignment of treatments and the achieving balance of the known and unknown baseline covariates that may influence response. However, randomization increases the size (and hence the cost) and duration of the study, which can be problematic in phase II.

Figure 1. Types of phase II trial designs.

(Modified and reproduced with permission Seymor L et al., Trial design task force of the national cancer institute investigational drug steering committee. Cancer Res 2010: 16;1764–1769).

One possibility that has been employed recently in the development of circulatory assist devices for heart failure patients is the use of a historical (i.e. a registry) or contemporaneous control group that permits a standard protocol to be used for defining inclusion and exclusion criteria. For example, the ADVANCE trial demonstrated that a new circulatory assist device was non-inferior with respect to the primary end point (survival on the original device, transplant or explant for ventricular recovery at 180 days) when compared to a contemporaneous group of patients with commercially available pump implanted contemporaneously, who were enrolled in the Interagency Registry for Mechanically Assisted Circulatory Support (INTERMACS) [6]. The most significant limitation of this type of trial design is that the comparability of patients in the historical group and treatment group cannot be ensured completely.

A popular approach that has been used to deal with baseline imbalances in historical control groups is the propensity score [7]. With a propensity score approach, baseline covariates are used to estimate the probability of using one treatment (e.g., a new therapy) or the other (e.g., control group). These probabilities can then be used to stratify patients. Treatment and control outcomes can be compared within strata or adjusted for their propensity score. The limitation of this approach for phase II heart failure studies is that it assumes that all of the clinically important differences between the treatment and control groups have been accounted for, and that there are no important, but unobserved differences between the two groups.

Based on the aforementioned limitations, randomization is usually the preferred trial design for phase II trials in heart failure. Given the inherent variability in responses in heart failure, blinding of the agents against placebo, or other doses of the same agent, or other active agents should is preferable wherever possible. Unlike many cancer trials, most phase II heart failure trials proceed directly to either an open label or blinded randomization scheme, rather than employing some of the adaptive steps illustrated in Figure 1.

Phase II trials in Heart Failure

Typically phase II trials in heart failure enroll a modest number of patients (ranging from 50–200), with randomization schemes weighted towards the experimental compound. The inclusion of several study arms leads to reductions in statistical power and/or leads to a smaller number of patients in the control arm, in order to account for the increased number of enrolled subjects in the treatment arms. However, the collective experience over the past 20 years has shown consistently that there is a price to be paid in heart failure trials when the control group in phase II or phase III does not contain a sufficient number of patients. For example, if the event rates in a small control group are spuriously high, and not reflective of the event rates for the entire cohort, it may give the false impression that the experimental compound is effective when compared to placebo. Another problem with this approach is that it may lead to a power calculation that overestimates the event rates, which can lead to phase III trial designs that are underpowered. With either scenario, the likelihood of replicating the results of phase II in phase III is substantially less.

There has been considerable debate regarding appropriate clinical end points for phase II trials. Given that it is not practical to use “hard” end-points such as mortality or mortality plus hospital admissions, most phase II studies have employed a variety of different surrogates such as functional capacity, quality of life, heart failure symptomatology (measured as the New York Heart Association class), left ventricular remodeling and/or ejection fraction, biomarkers, or various different clinical composites. A second potential issue in the early phases of development of a novel therapy is there are often limited clinical data available to help guide selection of relevant end points. Often times this results in “rounding up the usual suspects (vide supra),” rather than trying to identifying target specific end points. Alternatively, when it is not possible to clearly identify a relevant clinical end point, phase II trial designs will often employ a panoply of surrogate end points, in order maximize the chance of finding a positive signal in phase II. As a result of these problems, many phase II study designs for novel compounds often do not test relevant hypotheses with respect to how a new agent might positively impact patients with heart failure, and/or are unnecessarily expensive.

Several potential explanations have been advanced to explain the why the surrogate end points used in phase II trials do not predict therapeutic effects on mortality and/or morbidity in phase III heart failure trials. The explanation that has received the widest acceptance, particularly by regulatory agencies, is that the surrogate does not reliably predict the overall effect on the clinical outcome [8]. For example, heart failure progression could proceed through several causal pathways that are not mediated through the surrogate. Thus, although the intervention may have a significant effect on the surrogate, it may have no effect on the relevant disease-causing pathway. One of the classic examples of this scenario in heart failure trials is the use of exercise capacity to assess changes in functional capacity as a surrogate for disease progression. Although Pimobendan and Floseqinan improved exercise capacity in heart failure patients, both of these drugs substantially increased patient mortality when studied in phase III trials [9,10].

Another possibility is that the intervention might also indirectly affect the clinical outcome by unintended, unanticipated, and unrecognized mechanisms of action that operate independently of heart failure. This is seen in heart failure trials with drugs that improve the quality of life in heart failure patients, yet increase mortality in phase III studies [9,10]. Thus, surrogates have a relatively poor track record of predicting success in large randomized controlled heart failure trials. The notable exception to this statement is the assessment of ventricular remodeling, which has been shown to reliably predict clinical outcomes in clinical trials [11]. Unfortunately, this surrogate end point has not yet gained traction with regulatory agencies, despite overwhelming pre-clinical and clinical evidence that changes in remodeling are involved in the progression of heart failure.

Heterogeneous patient populations with significant variation in co-morbidities and disease severity, and short follow-up periods are also problematic in phase II clinical trial designs. Although a new experimental compound may be efficacious with respect to a surrogate end-point that lies within the disease causing pathway, the new compound may ultimately fail in phase III because the burden of noncardiac comorbidities often dominate the downstream even rates such as hospitalization and death [12]. While this type of problem can often be accounted for in phase III trials by focusing the primary end point on cardiovascular hospitalizations and/or cardiovascular death, in many instances the comorbidity may be inextricably linked to the cardiovascular outcomes (e.g. chronic kidney disease). Indeed this issue may be one of the root causes for our inability to develop drugs for patients with heart failure with preserved ejection fraction.

Differences in Experimental Methodology Between Basic and Clinical Science

A key difference between basic and clinical science methodology is the ability of basic scientist to continuously define and refine the hypothesis in real time based on the results of completed experiments. In sharp contrast, clinical study designs employing frequentist statistics require that the hypotheses, drug dose, sample populations, and relevant end points are stated precisely prior to trial initiation, in order to maintain the statistical integrity of the trial. As noted above prior to the initiation of phase II studies, they may be little information available with respect to which end points and patient populations an experimental compound may provide the most clinical benefit. In some sense there can be insufficient information to allow one to construct a meaningful clinical study. In this setting, studies are designed using the generic end points discussed above, and/or investigators attempt to extrapolate end points obtained in previous experimental studies in animal models. Although this type of strategy worked extremely well for the development of ACE inhibitors in SAVE (Survival and Ventricular Enlargement Study)[13], this type of approach has not worked well for other targets insofar as small animal models do not precisely recapitulate human disease. Thus, the rigidity of current clinical trial designs often leaves little room for adjustments to be made during the study period as clinical experience with the new agent or device is obtained.

Rethinking Phase II Clinical Trial Design in Heart Failure

How can phase II trials be redesigned so that they can fulfill their intended purpose of exploring mechanisms of benefit and predicting the efficacy of novel therapeutics in larger phase III heart failure trials? As discussed above our current phase II trial designs in heart failure are excessively rigid in an order to preserve statistical integrity and maximize statistical power. In this regard numerous new trial designs have been proposed in oncology trials, including randomized selection designs (pick-the-winner), adaptive designs [14], and randomized discontinuation designs [15], and other randomized designs [5]. Prospectively specified adaptive designs are of particular interest in the context of phase II heart failure studies of new agents wherein very little is known with respect to the appropriate patient population, or the optimal doses and/or schedules are not clearly known at the trial outset. Adaptive designs in such settings should be efficient and may result in improved precision in terms of predicting success in phase III. Despite the multiplicity of new designs that have been proposed, their inclusion in new heart failure trials has been notably absent.

Adaptive Trial Design

Adaptive trial designs may be divided into prospective designs, continuously adjusted (ad hoc) designs, or retrospective designs [16,17]. Prospective adaptive designs include studies where there is a pre-specified protocol to alter parameters such as sample size, follow up time, or clinical end points if a certain threshold is met. For example, based on interim analyses a study may be considered as futile and terminated prematurely. Alternatively, an interim analyses may suggest that a larger sample population or follow up time may be required to achieve sufficient power for the primary end points. Sequential trial designs represent prospectively pre-specified protocols that allow for interim analyses of the data. Continuous or ad hoc adaptive trial designs allow investigators the flexibility to alter several study parameters based on previously observed outcomes. For example, participant inclusion criteria, dosing regimens and end points may be modified based on interim analyses [17]. This approach provides investigators the opportunity to refine hypothesis based on observed results. Once such changes are made, subsequent conclusions are derived from combining retrospectively and prospectively collected data. While this approach appears attractive, it is has the possibility of introducing significant bias into the study. Specific statistical tools have been designed to minimize the effect of mid-study changes, which are discussed in more detail below. In general, the impact of potential bias is reduced by maximizing the amount of data that is collected prospectively (i.e., following protocol modifications) [17]. This is analogous to the use of discovery and validation cohorts.

Retrospective adaptive designs provide the greatest extent of flexibility for investigators. This trial design allows investigators to change the primary end point or statistical methodology after the trial has been closed, but prior to unblinding of the study. As one would expect, retrospective designs are the most vulnerable to the influence of bias, insofar as none of the data are collected in a prospective fashion following protocol alterations [16].

Given the limitations of bias, most adaptive trials belong to either the prospective or ongoing (ad hoc) categories. In either situation, the use of adaptive designs and appropriate statistical methodology are clearly spelled out in the study protocol. Specific adaptations involve modifications of several key aspects of clinical study designs, including refinements of participant inclusion and randomization, sample size, experimental drug dosing protocols, and relevant clinical end points. Several aspects of each of these strategies will be discussed below.

Adaptive methodology involving study subjects

There are several applications of adaptive trial designs involving study participants that may be well suited to heart failure trials. Given the complexity and heterogeneity of heart failure patients, it is difficult to be certain that the randomization process will always lead to equally balanced treatment and control arms in small phase II efficacy studies. Adaptive randomization is a prospective strategy that is employed to improve the efficiency of randomization [18]. In this approach, the probability that a subject may be assigned to a particular arm is dependent on the number of patients previously assigned that share particular characteristics. For instance, investigators may use adaptive randomization techniques to increase the probability that each study arm will include the same number of heart failure patients that have the same New York Heart Functional Class, background device use (e.g. bi-ventricular pacemakers) or comorbidities (diabetes, hypertension and coronary artery disease). This type of trial design technique should decrease heterogeneity, and thus increase the signal to noise ratio with respect to analyses of primary and secondary end point. An additional advantage of adaptive randomization is that it may improve the efficiency of subgroup analyses by assuring that subgroups of interest are equally allocated into each study arm.

Adaptive methodology to improve statistical power

Adaptive trial designs may also be utilized to improve study power if unanticipated efficacy signals are observed, or if the statistical power for the intended primary end point is insufficient based on event rates in the trial [19]. For example, if an investigational drug is being evaluated for its impact on LV remodeling in a phase II study and an interim analysis detects that this agent reduces the incidence of heart failure hospitalizations, the study may be modified to better evaluate this preliminary finding. This can be accomplished by enrolling additional subjects and/or extending the duration of follow-up durations employed to adequately power the study. This approach can be also be used to prematurely terminate a study for futility if there is no efficacy signal after the trial has collected sufficient events.

Adaptive methodology to enhance the study population

It is also possible to alter the study population during an ongoing trial in an effort to refine the hypothesis that is being tested. For example, if a novel agent is under evaluation is designed to improve LV function and reduce heart failure symptoms, and an interim analysis suggests that only patients with ischemic cardiomyopathy receive benefit, then enrollment criteria can be modified to restrict enrollment to patients with ischemic cardiomyopathy, and the sample size of the trial adjusted based on the anticipated power required to demonstrate a statistical difference in the cohort of patients with ischemic heart disease. Using this methodology, it is possible to explore if there is a particular patient population that a novel therapy may be best suited for. This approach may also be used to remove patient populations that either fail to show significant efficacy or are harmed by the tested therapy. An example of this strategy would include terminating enrollment of patients with moderate or severe chronic kidney disease, if an interim analysis demonstrated lack of efficacy or harm in this subgroup. An important caveat of this approach is the possibility of falsely identifying a positive signal for efficacy or harm in a particular patient subgroup. Alternatively, there is a similar risk of prematurely concluding that a patient population is receiving inadequate benefit at the time of an interim analysis due to insufficient power.

Adaptive methodology to compare drug dosing regimens

Phase II efficacy studies are traditionally challenged by the small sample sizes that accompany the multiple study arms required to explore the range of possible doses of a new therapeutic agent that might be appropriate to take forward into a phase III clinical trial. As a result of these limitations, typically only two or three doses can be effectively evaluated in a single study. The introduction of adaptive study designs has helped streamline assessment of multiple drug dosing strategies. The most common method used is termed “drop-the-loser” and has been successfully utilized in oncology studies (Figure 2) [20,21]. In this study design, patients are randomly assigned to one of several different dosing arms or a placebo (standard of care) arm. Using a common set of end points, experimental arms that do not meet specific efficacy criteria are successively dropped until a single dosing regimen is selected. Adaptive allocation strategies are employed to selectively assign patients into the placebo or better performing experimental arms. The end point used is typically a composite of efficacy and safety measures. Of note, this study design is not capable of generating traditional dose response curves.

Figure 2. Adaptive trail design for picking the appropriate therapy.

In the example trial, single agent drug B is selected in the phase II part of the trial and continues into phase III. The number of patients in phase II is chosen adaptively. The randomization in the phase II part can also be adaptive, as indicated in the figure. In the phase III part (confirmatory stage) the sample size depends on the results of phase II. Phase III might have interim analyses for stopping accrual early, for either expected success or futility. The drug B versus control element during phase II may (inferentially seamless) or may not (operationally seamless) be counted in the phase III comparison. Controlling the type I error rate in the former case requires simulating the entire trial. (Modified and reprinted with permission Berry, D. A. Adaptive clinical trials in oncology Nat. Rev. Clin. Oncol. 2011; 9: 199–207)

“Drop-the-loser” designs are well suited for combined phase II and III studies. Selection of optimal dosing is identified in the phase II component through the strategies described above. Once a dose is selected, it can then be brought forward into a larger phase III trial (Figure 2).

Adaptive Methodology to Test Multiple Hypotheses

Adaptive methodologies can be employed to test multiple hypotheses within a single trial. Analogous to the use of adaptive designs to identify patient subgroups with superior outcomes, adaptive strategies can also be utilized to select appropriate clinical end points from a series of different possible end pints. This is a critical design issue, insofar as the ideal clinical efficacy end point may not be immediately obvious at the onset of the trial. Of equal importance, this type of strategy may avoid selecting the wrong primary outcome variable, which may lead to termination of the drug development program. This is an especially important issue for phase II heart failure trials, as discussed above. Adaptive study designs have the flexibility to pre-specify a list of plausible clinical outcome variables in the trial design, and then select the most robust end point through the use of serial interim analysis. Once a primary or small number of primary efficacy end points are identified, the study proceeds in using the pre-specified end points defined in the earlier phases of the trial, thus preserving the statistical integrity of the trial. Multiple methodologies may be employed to select from an array of plausible clinical outcomes, ranging from a single interim analysis to multiple interim analyses analogous to the “drop the loser” schemes mentioned previously. The advantage of using these approaches is that investigators do not need to know or presume to know the ideal clinical outcome variable that needs to be evaluated. Instead, the selection of primary efficacy end points is driven by observations that are made in the early phases of the trial. It is possible that adaptive designs may not only aid in identifying the most relevant clinical end points to utilize in larger phase III studies, but also, help define the mechanism(s) by which new therapeutic agents impact on heart failure patients. One disadvantage of this strategy is that a modest set of plausible clinical outcomes must be pre-specified prior to initiation of the clinical study. Including too many outcomes may dilute the statistical power, whereas excluding potential outcomes of interest may limit the information that can be obtained from these types of flexible designs.

Statistical considerations for Adaptive Trial Designs

Adaptive trial designs provide an attractive degree of flexibility when compared to traditional frequentist trial designs that employ fixed end points. However, this flexibility comes at the expense of the statistical framework of the trial, which often becomes exceedingly complex in order to preserve the statistical integrity of the trial. Bayesian approaches, which focus on the probability of occurrences, have provided the much needed technical advances in this field. Details of Bayesian statistics are beyond the scope of this discussion and are reviewed elsewhere [22]. Another method that investigators have used to handle the dilemma of flexible trial design is through the use of interim analysis to evaluate emerging results. This methodological process is referred to as a “group sequential design.” However, given that repeated analysis of accumulating data with conventional statistical testing can lead to substantially increased false positive (Type I) error rates, it is often necessary to employ an alpha spending function to control the Type I error in order to allow the investigators to evaluate the data as needed. The down side of this is that it requires that a more stringent significance level is used for the primary end point. This obviously poses problems for modest size phase II trials, which are generally underpowered statistically.

Emerging Methodologies For Evaluating Clinical End Points

Composite end points are frequently used in heart failure trials studies in order to capture more clinical events and thereby increase statistical power. However, there are disadvantages to selecting a composite end point as the primary outcome in a clinical trial. As currently employed, each component of primary end point contributes equally to the composite end point, even though each end point may carry a different clinical significance. For example if the primary end point is a composite of mortality, heart failure hospitalizations, and worsening heart failure symptoms each of these components is weighted equally. Obviously death is the most important component of the composite, reducing mortality but may not be achievable in a small phase II/III trial. Moreover, it is possible that including mortality as an end point in the composite may decrease the statistical power of the primary end point to detect a benefit in the other components of the composite. This was the case in the recently completed WARCEF (Warfarin Versus Aspirin for Reduced Cardiac Ejection Fraction) trial, which showed a significant (HR 0.5 95% CI 0.33–0.82; p < 0.005) reduction in ischemic stroke, but was negative for the primary end point of the trial, which was a composite of ischemic stroke, intracranial hemorrhage and all cause death (p = 0.40) [23].

How can the process of analyzing composite end points be altered to reflect the relative importance of each component? There have been several distinct approaches proposed to deal with this trial design issue. One strategy that has been employed recently is the global rank scoring system [24], which is based on a hierarchical analysis system, and allows the investigator to appropriately weight components of a composite end point using a global rank scoring system. In the global rank test study participants are ranked according to a pre-specified scheme that weighs clinical outcomes in a hierarchical manner. An advantage of this method is that it can incorporate both clinical events and continuous variables, which allows for the use of biomarkers as a primary outcome variable. For example, consider a trial investigating the effect of a novel therapy on hospitalized patients with decompensated heart failure. The outcomes measured include 30-day mortality, repeat heart failures hospitalization, improvement in heart failure severity, and changes in the level of B type natriuretic factor (BNP). At the conclusion of the trial patients are placed into four groups: 1) subjects who died within 30 days, 2) subjects with repeated admissions for heart failure, 3) patients who were not readmitted with worsening or no change in heart failure symptoms, and 4) patients who were not readmitted with improved heart failure symptoms. Within each group, the patients are ranked by either time to event, or heart failure symptomology and BNP level. Summation of the global rank score provides an ordered analysis of the composite end point of 30-day mortality, heart failure readmission, heart failure symptoms, and BNP level (Figure 3). This analysis maximizes the power of using a composite end point without being subjected to some of the above mentioned pitfalls of traditional approaches.

Figure 3. Schematic of the global rank score.

In this example, patients are randomized to receive either a control or experimental therapy. Using the global rank score, a composite outcome of 30-day mortality, HF hospitalization, heart failure symptoms, and BNP level is employed. Patients are sorted into 4 groups using a hierarchical approach: A) experienced mortality within 30 days, B) alive but hospitalized for HF, C) no hospitalizations but experienced worsened HF symptoms, and D) patients without hospitalizations or worsening HF symptoms. Patients in the first two groups (A–B) are ranked by time to event. The latter two groups (C–D) are ranked by HF severity score and BNP level. A global rank score is obtained for each treatment group by adding the respective patients ranks. A higher score indicates improved outcomes.

A second method that can be used to test composite end points in a prioritized manner is the “win ratio” [25]. This approach is designed to combat two fundamental difficulties that may be present in typical efficacy studies, study population heterogeneity and important censored events. The latter is most evident in time to event analyses of composite end points, where only the first event is captured and subsequent events are censored. If the first event is less important than the second event, significant information may be lost using current analysis schemes. This is apparent in the above example where the first event was hospitalization and the second event was death. The basis of this methodology is that it forms “pairs” of patients from the treatment and control arms who are matched on pre-specified comorbidities, characteristics of disease severity, and duration of follow up, analogous to the technique of propensity matching. The outcomes of each matched pair are then compared for multiple end points that are ordered in a hierarchical fashion. For example, a representative order might be mortality, heart failure hospitalization, and heart failure symptoms. Using a pre-defined algorithm, matched subjects from the control and treatment arms are designated as a winner or loser (Figure 4) for each of the pre-specified end points. If no winner can be determined, then the pair is considered to be tied. For example, consider a matched pair where subject A died at 3 months and subject B died at 9 months. In this scenario subject B would be the winner. Alternatively, consider another matched pair where subject A was hospitalized at 2 months, and subject B died at 8 months. Here, even though subject A experienced the first event, death is considered a more important outcome and subject A would be the winner. If neither subject A and nor subject B died or were hospitalized, the winner would be determined by heart failure symptom index score. The total number of wins and losses for each study arm is summed and a win ratio is calculated. The win ratio is defined as the number of wins divided by the number of losses. A p value and 95% confidence interval can be generated for this type of analysis. This innovative approach was recently proposed and applied retrospectively to several completed randomized controlled trials heart failure trials, including the EMPHASIS-HF (Eplerenone in Mild Patients Hospitalization And SurvIval Study in Heart Failure) [26], and CHARM (Candesartan in Heart Failure assessment of Mortality and Morbidity) studies [27]. When the win ratio was applied to each of these clinical trials, the reported results were successfully reproduced using both the matched and unmatched analysis. Although the global rank test and the win ratio have tremendous appeal in terms of phase II clinical trial design, in that they allow one to evaluate hard clinical end points in phase II studies, it bears emphasis that these approaches have not been validated in terms of the ability to predict success in phase III.

Figure 4. Schematic of the win ratio.

The win ratio is best performed using propensity matched patients pairs. For each pair a winner and loser is determined, or the pair is considered to be tied. The determination of a winner is made using a predefined hierarchical outcome scheme. In this example, mortality is considered the most important outcome followed by HF hospitalization. Possible scenarios resulting in a winner for mortality and HF hospitalization as well as no winner (or tied) are outlined. The length of each blue arrow indicates the duration of patient follow up. Arrows ending in a solid circle denote either incomplete or a shorter duration of follow up. The win ratio is calculated by adding the number of wins divided by the number of losses for the experimental group. A statistically significant win ratio that is greater than 1 indicates a positive outcome. (Modified from and reprinted with permission Pocock SJ, et al The win ratio: a new approach to the analysis of composite end points in clinical trials based on clinical priorities. Eur Heart J 2012; 33:176–82)

Future Perspective

The significant time and cost that must be invested to develop a novel therapeutic using current clinical study designs has become overwhelming from a fiscal perspective, and threatens to dampen enthusiasm for developing novel therapeutics for heart failure, as well as foster the continued development of “me too” heart failure drugs that target the same pathway in different ways. Phase II trials represent the critical “portal” that new compounds must pass through prior to entering more definitive phase III trials. As noted in the current review, our recent experience has taught us that phase II heart failure studies in their current form do not predict future success in large phase III trials. Here, we have reviewed multiple different adaptive approaches that could be employed to improve the efficiency of phase II studies for indentifying novel therapies for developing heart failure drugs. Although the adaptive trial design approach has been embraced in early phase oncology trials [5,28], adaptive trial designs have had less uptake by the heart failure community. While the exact path forward is not at all clear at the time of this writing, there is hope that adaptive trial designs may provide clinical investigators with the flexibility to evolve hypotheses and dosing regimens for novel therapies as they emerge from the laboratory and undergo clinical testing in humans. If clinical investigators are afforded similar luxuries as basic scientists to refine ongoing studies based on prior observations, it is possible that the efficiency and predictive value of phase II studies may improve remarkably.

Executive Summary.

• The incidence and economic burden of heart failure continue to rise worldwide, despite implementation of a number of effective heart failure therapies.

• There has been a dearth of approved new therapies for heart failure over the past decade

• Current phase II heart failure trial designs do not correctly identify novel therapies that will be successful in phase III clinical trials.

• Adaptive trial designs offer the promise of improved efficiency and predictive accuracy for developing novel heart failure therapies that are successful in phase III clinical trials.