Progress in the Fight Against Multidrug-Resistant Bacteria 2005–2016: Modern Noninferiority Trial Designs Enable Antibiotic Development in Advance of Epidemic Bacterial Resistance

Summary

From a public health perspective, new antibacterial agents should be developed before widespread resistance to existing agents emerges. Anticipatory drug development based on noninferiority trials offers a path to making safe and efficacious antibiotics available ahead of epidemic resistance.

Abstract

From a public health perspective, new antibacterial agents should be evaluated and approved for use before widespread resistance to existing agents emerges. However, for multidrug-resistant pathogens, demonstration of superior efficacy of a new agent over a current standard-of-care agent is routinely feasible only when epidemic spread of these dangerous organisms has already occurred. One solution to enable proactive drug development is to evaluate new antibiotics with improved in vitro activity against MDR pathogens using recently updated guidelines for active control, noninferiority trials of selected severe infections caused by more susceptible pathogens. Such trials are feasible because they enroll patients with infections due to pathogens with a “usual drug resistance” phenotype that will be responsive to widely registered standard-of-care comparator antibiotics. Such anticipatory drug development has constructively reshaped the antibiotic pipeline and offers the best chance of making safe and efficacious antibiotics available to the public ahead of epidemic resistance.

Progressive emergence of bacterial resistance is a major threat to the public, as well as to medical progress [1]. From a public health perspective, new antibacterial agents (hereafter, “antibiotics”) with an improved microbiologic spectrum of activity should be developed before widespread bacterial resistance emerges. This desire leads immediately, however, to a paradox: although it is easy to demonstrate that a novel test agent has an improved spectrum both in vitro and in preclinical animal infection models, rigorous demonstrations of the test agent’s superior efficacy over existing agents both are and should be difficult to implement on a routine basis in clinical trials of human infections:

• As inadequately treated acute bacterial infections can be rapidly fatal and enrollment into a trial must often be undertaken empirically before culture results are known, it is obviously desirable that the control arm be predicted to be efficacious in all studies, including studies of potentially superior new agents.

• Indeed, if resistance is known or suspected to the control arm, then the control arm should always be adapted to offer some form of best available therapy that is predicted to be efficacious. In short, it is important from an ethical viewpoint that the trial make every attempt to use an efficacious control and not anticipate showing superiority relative to an ineffective or substandard control.

• The only exception to this ethical imperative would be if there were either no efficacious options whatsoever for the infecting strain (or if all forms of best available therapy were meaningfully suboptimal). It is of course obvious that such a situation would imply a situation with grim public health implications.

• Finally, the window of opportunity to reliably design trials to show superiority because of a complete lack of therapeutic options would close with emergence of a new efficacious therapy.

The frustratingly circular nature of this paradoxical problem is made more confusing because active-control trials (including superiority trials) of new antibiotics are so easily designed on paper [2–4]. The heart of the challenge is that good infection prevention and hygiene should mean that trials focused on enrolling patients infected with multidrug-resistant (MDR) or extensively drug-resistant (XDR) pathogens will always be difficult to complete.

There is, however, a path forward for development of the needed array of novel antibacterial therapies. Noninferiority design trials focused on enrolling infections due to usual drug resistance (UDR, defined as any resistance profile to established drugs that is not MDR or XDR) pathogens are readily implemented, as by definition UDR pathogens are the most common. Furthermore, it is easy to identify a globally acceptable, standard-of-care comparator with high activity against such pathogens [5, 6].

In this manuscript, we review the data showing that antibiotic development programs based on strong preclinical evidence and progressed using rigorously designed and implemented pivotal noninferiority trials provide clear, informative data for both regulatory decision-making and clinical practice [7–12]. We will also show how use of such trials has reshaped the global development pipeline in a very constructive fashion. Use of noninferiority trial designs in antibiotic drug development is and will remain critically important from a public health perspective to support anticipatory availability of new therapies for treatment of rare, but emerging, problem pathogens.

APPROACHES TO DESIGN OF ACTIVE-CONTROL TRIALS FOR ACUTE INFECTIONS

Active-controlled treatment trials compare the efficacy of a new treatment to that of an existing treatment using modern statistical techniques. Such trials can attempt to demonstrate either (1) the statistical superiority of a new agent vs an older agent or (2) the clinically relevant, and statistically defined, degree of comparability of the new therapy to an existing agent.

Superiority Trials

Trials designed to demonstrate the superiority of a new agent over an old one are arguably highly desirable. When feasible, they provide extremely convincing evidence of the efficacy advantages of a new agent over an existing therapy. They are unlikely to reach erroneous conclusions due to poor study design and/or conduct, including, for example, use of an endpoint that fails to detect treatment differences or the inclusion of patients who do not have the disease entity of interest (eg, one does not wish the study population of patients with an acute bacterial pneumonia to be diluted by a sizeable subgroup with viral pneumonia). In short, superiority trials are the preferred design for drug development and have been used successfully in many therapeutic areas.

However, challenges arise in the use of superiority designs in the study of acute, severe infections [4]. In contrast to the situation with chronic infection such as tuberculosis, human immunodeficiency virus, or hepatitis C virus infection where the pace of disease progression permits brief periods of placebo therapy without risk to the patient, acute, severe bacterial infections such as pneumonia can progress rapidly with complications including death. Hence, ethically designed clinical trials of such acute infections must generally enroll before culture results are known and make every effort to offer efficacious initial therapy despite this limitation. Because patients cannot be randomized to placebo or a therapy predicted to be ineffective via susceptibility testing, ethical trial designs require empirical inclusion only of patients infected with a pathogen expected to be susceptible to both the test agent and the comparator antibiotic. The feasibility of demonstrating superiority is further reduced by optimized dosing of antibiotics, as determined by preclinical microbiological data and pharmacokinetic modeling approaches [4, 7, 13, 14]. Today’s standard-of-care antibiotics are dosed to achieve a high probability that individual patients will achieve an efficacious exposure for susceptible pathogens.

Other challenges to use of superiority design trials are logistical. Patients with an acute, severe infection must be identified by the clinical investigator within hours of presentation for care; prolonged screening and enrollment procedures severely constrain patient accrual. Furthermore, unless point-of-care rapid diagnostics are readily available, attempts to enroll patients with an infection caused by a specific resistant pathogen must be empirical and anticipatory, meaning that many-fold the final required sample size of patients must be enrolled to find a few with the targeted resistant organism. But, and as noted above, detection of a resistant pathogen would mandate use of an alternative comparator regimen predicted to be active against the infecting pathogen.

Finally, if a superiority standard is required, demonstrating superiority with one new agent resets the bar, effectively terminating development of other potential promising agents until resistance to the new agent also becomes widespread and thereby depriving patients and physicians of other potential benefits of new agents (eg, improved safety or tolerability; more convenient dosing regimens) [4, 7].

Although the focus of antibiotic development could be on infections so trivial that placebo treatment is acceptable, this approach limits study to minor skin, urinary, or respiratory tract infections and would not show an advantage over available treatment. Furthermore, pausing development of new agents until superiority trials were feasible due to the complete absence of therapeutic alternatives would place individuals and society at risk.

In summary, superiority trials are best seen as an adjunct to the noninferiority designs discussed in the next section. A white paper published in 2012 by the Infectious Diseases Society of America [4] focused on this point and provides the interested reader with examples of ways superiority trials could be implemented if highly resistant pathogens were sufficiently frequent. As discussed in that paper, such approaches include hierarchical noninferiority–superiority trials, monotherapy superiority trials using either empirical or culture-confirmed randomization, nested superiority–noninferiority trials, combination superiority trials, historically controlled superiority trials, and organism-specific superiority trials. These are all sound design options, but it is our collective experience that implementing such programs is difficult in practice for the reasons discussed above. And although it is always possible to observe a superiority result from subsets within a noninferiority trial, it is our hope that this will be uncommon, and registration of new agents should not be dependent on rare events! By analogy with agents active against bioterror pathogens, anticipatory creation of an antibiotic drug pipeline is critical [15]. Therefore, trial designs that facilitate the development of new antibiotics before epidemic spread of bacterial resistance are highly desirable. Noninferiority trials are central to achieving this goal with trials that are feasible whether resistance is widespread or not [7, 13]. Use of innovative statistical methods (eg, Bayesian methods) to support both superiority and noninferiority trial design is also worthy of further exploration [4].

Noninferiority Trials

In a noninferiority trial, active treatments are compared with the knowledge that sufficiently similar response rates will permit a conclusion that clinically relevant differences in efficacy are very unlikely. Critical design elements for robust noninferiority trials include (1) disease definitions that identify patients with an acute, severe infection that requires antibiotic therapy, (2) a reliable and reproducible endpoint, and (3) data showing a benefit of active antibiotic therapy over placebo [7]. Recently, these elements have been refined for 6 well-characterized acute, severe bacterial infections (acute bacterial skin and skin structure infection, community-acquired bacterial pneumonia, hospital-acquired bacterial pneumonia, ventilator-associated bacterial pneumonia, complicated intra-abdominal infection, and complicated urinary tract infection (Table 1) [10, 16–22].

Table 1.

The 6 Types of Severe Infection Recommended for General Antibacterial Development

Infection	Enrollment Criteria	Endpoint	Untreated/Delayed Therapy Response Rate	Active Therapy Response Rate	Treatment Effect Size (M1)a	Recommended NI Margin (M2)
Acute bacterial skin and skin structure infection [16]	75-cm2 area of erythema (approximately the size of a dinner plate)	20% reduction in size of area of erythema at 48 h of therapy	73%–77%	98%–99%	18%	10%
Community-acquired bacterial pneumonia [17]	A specific set of pulmonary symptoms plus a high level of severity	Improvement at days 3–5 in baseline symptoms based on a specific scale	5%–40%	>70%	>20%	12.5%
Hospital-acquired and ventilator- associated bacterial pneumonia [18]	A specific set of pulmonary symptoms	28-d all-cause mortality	62% (here, higher is worse, unlike the other situations)	20%	20%	10%
Complicated intra-abdominal infections [19]	Operative diagnosis	Resolution of baseline symptoms	39%	82%	14%	10%
Complicated urinary tract infection [20]	Risk factors plus symptoms plus evidence of pyuria	Resolution of symptoms and sterilization of urine	33%	69%	36%	10%

Other important types of infection (eg, gonorrhea) can also be studied, but these 6 infections (note that hospital-acquired and ventilator-associated bacterial pneumonia are often studied together but count as two different types of pneumonia) provide pathways that will be relevant for most antibacterial agents.

Abbreviation: NI, noninferiority.

^aThe estimate of treatment effect size (M1) is not just the mathematical difference between the response rates for untreated/delayed therapy and response rates for active therapy but is conservatively estimated as the distance between the upper bound of the 95% confidence interval (CI) around the known or estimated placebo response rate (which can be greater than zero) and the lower bound of the 95% CI around the active treatment response rate [10]. After estimating M1, a decision must be made about the maximum clinically acceptable potential difference between the investigational antibiotic and the comparator antibiotic [10, 21]. This maximum difference is the noninferiority margin, also known as M2, and must be smaller than M1. Selection of M2 should consider differences between the historical and current trials, the potential loss of efficacy deemed clinically important, the feasibility of generation of clinical data, and the magnitude of unmet medical need [21, 22], and it is these factors that lead to the range of selected values of M2. For more details, the interested reader is referred to the detailed methodology for computing M1 and M2 discussed in the 2016 publication on this topic from the US Food and Drug Administration [10].

The focus on these specific infections is an important evidence-based element underpinning use of noninferiority designs for new agent registration. Although some have noted that (for example) skin infection is not the most important potential use for a new antistaphylococcal agent or that inclusion of outpatients suggests that patients lacked a life-threatening infection [23], such arguments fail to recognize that (1) patients meeting the definitions of new regulatory guidance have a severe infection that will have a poor outcome without efficacious antibiotic therapy and (2) initial registration based on one of these frequent infections enables studies of other infections. Studies of important but less frequent infections such as endocarditis are desirable, but enroll slowly and, among other hurdles, may be impossible to blind [24].

Points of Concern With Noninferiority Trials

Because success for noninferiority designs follows when a difference is not found, such trials are at risk of generating invalid conclusions due to experimental noise from patient heterogeneity, inclusion of patients lacking the target disease, poor adherence to study procedures, and/or use of ill-defined, nonstandardized endpoints [23, 25, 26]. Such risks can largely be mitigated with careful trial design and implementation, which have been the focus of recent regulatory guidance. Further, efficacy analyses performed on a prespecified smaller population of treated patients focused on those with the prespecified pathogens of interest and without treatment confounders (such as excessive use of potentially efficacious prior antibiotics) also help to reduce the influence of experimental noise.

Although the term noninferiority might suggest the possibility of accepting inferior efficacy relative to the control, this risk is small when the design features previously outlined have been addressed. Uncertainty can be further reduced by demonstrating noninferiority in a second trial or by providing strong data on the pharmacological basis of activity [27].

One additional prominent concern has been the theoretical risk of “biocreep,” wherein use of successively less efficacious comparator agents results in a sequential degradation of the acceptable efficacy of a new antibiotic [28]. To preclude unwarranted complacency, this issue must remain front and center for drug development. It can be mitigated by ensuring there is always a review of the efficacy of the comparator for a trial as well as use of a comparator judged to be comparable to the best available agents for the given syndrome, and with a demonstrable benefit over no treatment or placebo. It is noteworthy that a number of recent noninferiority trials have in fact shown that inferior or reduced activity is detected by modern noninferiority trial designs [29–31].

MODERN NONINFERIORITY TRIAL DESIGNS FOR ANTIBACTERIAL AGENTS

As noted above, noninferiority trials are not usually focused on the study of the resistant organisms for which the need for new therapeutic options is arguably greatest. Rather, the trials seek to demonstrate efficacy and safety in well-characterized, severe infections due to UDR pathogens. When combined with preclinical data and pharmacokinetic/pharmacodynamic data from human infections, the trial data will predict the response of specific MDR and/or XDR, not just UDR, pathogens. That is, the new agent’s potential utility is defined by its in vitro activity against emerging drug-resistant pathogens of public health concern rather than by the lack of activity of other agents against those pathogens [5, 6]. Especially when the mechanism of action of an investigational antibiotic is novel, its activity against pathogenic microorganisms will generally be independent of the activity of existing agents.

Importantly, it is also possible to gain in parallel some experience with highly resistant pathogens, including those resistant to the comparator agent used in the noninferiority trial. First, a salvage study of either open-label treatment with the novel agent or randomization vs a “best available therapy” control selected on a per-patient basis both can and should be run alongside the noninferiority trial. Second, and because patients must be enrolled and randomized before culture results are known, it is possible that at least a few patients in a sizable noninferiority study will be found in retrospect to have had a pathogen resistant to the control therapy, thus allowing for some direct insight into the activity of test drug in these patients. If care is taken to avoid unblinding, it might even be possible to move such patients into the parallel salvage study. Finally, collection of data in a real-world setting might be used postapproval to support further generation of insight into the drug’s utility.

Concerns about poorly implemented noninferiority trials have been extensively studied over the past decade. For antibacterial agents, each challenge has been addressed with the high-quality noninferiority designs presented in guidance documents published by the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA) [13, 16–20]:

• Exhaustive in vitro and animal in vivo exposure–response relationship data for the novel agent are required. Because exposure–response predictions of efficacy in antibacterial drug development are robust, these data provide a high a priori likelihood of efficacy when adequate drug exposures are achieved.

• Six types of severe bacterial infections are recommended for routine study (see Table 1):

  • The characteristics of patients with sufficient severity of illness have been defined,

  • Reproducible endpoint measures have been defined [9],

  • The placebo and active response rates are estimated by exhaustive literature reviews, and

  • A conservative, clinically relevant noninferiority margin has been proposed [21].

As also mentioned above, the focus on a few specific types of infection is noteworthy. The FDA and the EMA have chosen well-characterized infections to enable scientifically sound, feasible pathways to accumulate robust data on the efficacy of new agents: Patients are readily found, readily proven to have severe infection, and known to have a predictable clinical course absent efficacious antibiotic therapy. A great advantage of the 6 infection types recommended by EMA and FDA for the routine study of novel antibacterial agents is that these infections are sufficiently frequent that randomized, blinded data can be produced in a reasonable timeframe to support a well-reasoned regulatory decision.

NONINFERIORITY STUDIES ARE CRITICAL TO PROTECTING THE PUBLIC HEALTH

Updated FDA guidances have now enabled registration of several new antibiotics. Results to date also suggest that noninferiority trials conducted rigorously can detect both deficiencies and advantages of novel agents.

Some have suggested that noninferiority trials are unethical because failure to seek superiority favors commercial over patient interests [25]. We believe this viewpoint overlooks the fact that it would be detrimental to the public health to require that all future drugs be superior to existing drugs based on clinical efficacy measures [3, 26]. Specifically, noninferiority trials address the problem that one size does not fit all: By means of these trials, agents with superiority grounded in other features (eg, a different spectrum of activity, a novel mode of action, a better adverse event profile, and/or more convenient administration) can be made available for the benefit of specific patients.

A recent drug development vignette highlights the full array of challenges reviewed in this paper. The developers of plazomicin sought initially to implement a pivotal program focused entirely on enrolling patients infected with highly resistant pathogens for which a standard-of-care colistin-based therapy (the only then-current alternative) was thought likely to be either toxic or limited in efficacy. When it became apparent that substantial enrollment in such a program was infeasible, a pivotal trial focused on UDR pathogens was added and the study in the setting of MDR/XDR pathogens made supplemental [32, 33]. This trial program ultimately demonstrated a mortality benefit of the new agent over colistin-based therapy [34]. It is of course good news that a new agent has progressed, but it should be noted that (1) this occurred because patients in the colistin-based therapy arm died due to lack of adequate therapy and (2) the emergence of this drug plus other recently licensed agents will progressively make it unethical to use such an inferior colistin-based treatment arm as a control in this setting.

The insights discussed in this article into the regulatory science of noninferiority trial design have profoundly reshaped the antibiotic pipeline (Figure 1). The most recent approval of an oral administration–only antibiotic was in 2003, and the last approval of an antibiotic for milder outpatient skin or upper respiratory infections was in 2001. Conversely, all initial antibiotic registrations since 2009 have included an intravenous route of administration and have been for one of the infections recommended for routine study (Table 1). Table 1 also highlights the resurgence of antibiotic development since the creation circa 2009–2010 of updated regulatory guidance—although much remains to be done, these are encouraging signs that new agents will be available in the future.

Figure 1.

Initial antibacterial approvals by route and indication, 1995–2016. Initial approvals of antibacterial agents from 1995 to 2016 were retrieved from CDERWatch and Drugs@FDA and are shown by approved route and number of initially approved indications (some agents were approved for >1 indication). As needed due to evolution of indication terminology, indications were grouped. The 2016 approval of bezlotoxumab to reduce recurrence of Clostridium difficile infection is not shown. Abbreviations: CABP, community-acquired bacterial pneumonia; cIAI, complicated intra-abdominal infection; cSSTI, complicated skin and skin structure infection; cUTI, complicated urinary tract infection; Genital, uncomplicated gonorrhea, prostatitis, nongonococcal urethritis, and chlamydia; IV, intravenous; Meningitis, bacterial meningitis; NP, nosocomial pneumonia including hospital- and ventilator-associated pneumonia; Mild RTI, acute otitis media, acute bacterial exacerbation of chronic bronchitis, acute sinusitis, and pharyngitis/tonsillitis; uSSTI, uncomplicated skin and skin structure infection; uUTI, uncomplicated urinary tract infection.

In conclusion, noninferiority trial designs are a necessary and essential part of antibiotic drug development. Without them, ensuring availability of efficacious and safe, novel antibiotics in advance of epidemic spread of resistant bacterial strains is impossible, to the obvious detriment of individual and public health.

Notes

Author contributions. All authors contributed to all phases of manuscript generation, from concept to writing.

Disclaimer. The views expressed are those of the authors and not necessarily those of their respective employers.

Potential conflicts of interest. J. H. R. is Chief Medical Officer and Director of F2G, Ltd; Chief Strategy Officer, CARB-X; Non-Executive Director and Consultant, Adenium Biotech ApS; Operating Partner and Consultant, Advent Life Sciences; Expert-in-Residence, Wellcome Trust; serves on the Scientific Advisory Boards of Macrolide Pharmaceuticals; Spero Therapeutics; and Bugworks Research, Inc; is a shareholder in AstraZeneca Pharmaceuticals, F2G, Ltd, Adenium Biotech ApS, Advent Life Sciences, Macrolide Pharmaceuticals, and Bugworks Research, Inc; and has received consulting fees from Phico Therapeutics, ABAC Therapeutics, Polyphor, Ltd, Heptares Therapeutics, Ltd, and Gangagen, Ltd. G. H. T. has received board compensation and/or consultancy fees from Achaogen, Actelion, Adynxx, Anacor, Durata, Cubist, Meiji, Nabriva, and Zavante; and is a shareholder in AN2, Calixa, Durata, Kalyra, Mpex (Tripex), and Nabriva. M. J. G. is previously an employee of AbbVie and now an independent consultant. He has received consulting fees from AbbVie, Arsanis, Astellas, AtoxBio, Axar, Cempra, Cubist, Denovo, Forest/Actavis, GLG, Melinta, NDA Partners, RedHill, Spero, TenNor, Tetralogic, and VenatoRx; is a shareholder in Abbott, AbbVie, Celgene, and Johnson & Johnson. B. I. E. is Scientific Advisory Board Chair for CARB-X; a former employee of Cubist Inc and Merck & Co, Inc; and holds stock in Eli Lilly. J. F. T. is a previous employee and shareholder of GlaxoSmithKline Pharmaceuticals and is now an employee of Spero Therapeutics. R. M. E. is a consultant to Shionogi Inc, Summit Plc, Clinapace Worldwide, Fujifilm Pharmaceuticals USA, Inc, KBP Biosciences, and Antabio SAS; and is a shareholder of Pfizer, Amgen, and Johnson & Johnson. M. N. D. is an employee and shareholder in The Medicines Company and holds equity in Rempex and Tripex Pharmaceuticals. A. D. was previously an employee of AstraZeneca Pharmaceuticals and has received consulting fees and share options from several pharmaceutical and biotechnology companies actively developing antibiotics. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.