Optimizing the Design and Analysis of Clinical Trials for Antibacterials Against Multidrug-resistant Organisms: A White Paper From COMBACTE’s STAT-Net

Abstract

Innovations are urgently required for clinical development of antibacterials against multidrug-resistant organisms. Therefore, a European, public-private working group (STAT-Net; part of Combatting Bacterial Resistance in Europe [COMBACTE]), has reviewed and tested several innovative trials designs and analytical methods for randomized clinical trials, which has resulted in 8 recommendations. The first 3 focus on pharmacokinetic and pharmacodynamic modeling, emphasizing the pertinence of population-based pharmacokinetic models, regulatory procedures for the reassessment of old antibiotics, and rigorous quality improvement. Recommendations 4 and 5 address the need for more sensitive primary end points through the use of rank-based or time-dependent composite end points. Recommendation 6 relates to the applicability of hierarchical nested-trial designs, and the last 2 recommendations propose the incorporation of historical or concomitant trial data through Bayesian methods and/or platform trials. Although not all of these recommendations are directly applicable, they provide a solid, evidence-based approach to develop new, and established, antibacterials and address this public health challenge.

Innovations are urgently required for clinical development of antibiotics against multidrug-resistant organisms. COMBACTE-STAT-Net provides recommendations for improved pharmacokinetic-pharmacodynamic modeling, more sensitive primary end points, hierarchical nested-trial designs, and use of historical/concomitant trial data through Bayesian methods and/or platform trials.

There is a gap between the number of new antibacterials in research and development (R&D) and the medical need caused by the increasing prevalence of infections by multidrug-resistant organisms (MDROs) [1]. In a 2014 survey [2], pharmaceutical industry professionals provided their opinion on the main challenges underlying this discrepancy. Most frequently, they indicated the low return on investment for antibacterials, followed by the lack of new regulatory pathways for antibacterial medicines that address a high unmet medical need, such as novel treatment options against MDROs. Importantly, innovative trial designs were seen as an important tool to promote R&D efforts for new antibacterials. Since this survey was conducted, political awareness of the need to encourage R&D for new antibacterials has risen enormously, with regulatory guidance being updated and international harmonization efforts underway [3–5]. However, critical to these processes is the need to advance and optimize trial design and make more effective use of the available data, so as to accelerate antibacterial approval and ensure appropriate use of established antibiotics active against MDROs.

Indeed, in the area of MDRO therapeutics, where the number of patients with resistant infections for each specific indication is generally still small, large studies are impossible or impractical [6]. Within traditional randomized controlled trials (RCTs), outcomes for patients with susceptible and resistant infections are combined, and although the subset of infections with resistant pathogens is presented separately, sample size is often insufficient. As a consequence, assessment of safety and effectiveness of new agents against MDROs is challenging, and without novel trial designs, which make better use of the available data, progress is difficult. Rex et al [7] have proposed a “totality-of-evidence” approach to resolve the impasse; instead of 2–3 large phase II and III trials, multiple sources of data could contribute to the evidence base of the clinical benefit of a new drug, including smaller RCTs. However, this approach needs to be supported by robust pharmacokinetic and pharmacodynamic (PK/PD) data on optimal dosing of patients and the ability to design, analyze, and interpret clinical trials as efficiently as possible.

In 2013, the Innovative Medicines Initiative (IMI; www.imi.europa.eu) established the Combatting Bacterial Resistance in Europe consortium (COMBACTE; www.combacte.com) to conduct prospective clinical trials and refine clinical trial design for new treatments against MDROs [8]. The specific objective for STAT-Net (Workpackage 4) was to deliver strategies that may yield more efficient phase II and III clinical trial programs, and to focus on 3 pillars: improved PK/PD modeling, enhanced end points, and innovative trial designs. In this white paper, we propose several innovative design and analysis strategies for regulatory and pragmatic clinical trials to support the evaluation of new and established antibacterials against MDRO infections, and discuss their scientific robustness and feasibility.

METHODS

STAT-Net was initiated in response to the sixth call for proposals issued by IMI in 2012 [9]. Subtopic 1A, Workpackage 4, specified 3 research pillars, as described above, which were aligned with regulatory and scientific challenges at the time of writing. One of us (S. H.) assembled, through open invitation, a group of experts with strong track records in PK/PD, biostatistics, Bayesian statistics, infectious diseases, intensive care medicine, epidemiology, and clinical development. Aligned with the IMI public-private partnership, pharmaceutical company partners joined to provide additional expertise. This group proposed a description of work in line with the original call, which was granted by IMI in autumn 2012.

Between January 2013 and July 2017, systematic reviews, reanalyses of existing clinical trial data, and simulations have been carried out. Some of this work has already been peer reviewed and published [10–19]. The final recommendations presented here are based on this work, extensive discussions via teleconferences, and STAT-Net–specific and COMBACTE-wide meetings, and they have been approved by all authors. We created an objective scoring system for each recommendation, which was approved by all partners (Table 1), and was used to score (1) alignment with the current regulatory framework; (2) feasibility of technical implementation, like the need for specific biostatistical or PK/PD skills; (3) ease of data interpretation; (4) ease of practical implementation at the clinical site; and (5) the strength of the evidence base for each recommendation (Table 2). Whenever disagreement concerning a recommendation arose, recommendations and scoring were adapted until consensus was reached. Based on the average score, recommendations are presented as “strongly recommend,” “recommend,” “strongly suggest,” or “suggest.”

Table 1.

Classification Table for the Recommendations

Classification	Alignment With Current Regulatory Framework	Ease of Technical Implementation	Ease of Interpretation	Ease of Practical Implementation	Evidence Base	Formulation
+	Adaptations in design AND analytical methods needed, currently not supported by regulatory authorities	Only by statistical or PK/PD experts familiar with the method	Very complex; requires statistical expertise and experience with the method	Design and analytical methods require significant adaptation of standard clinical trial protocols, AND extra data are required	Based on expert opinion, and/or external panel consensus.	“We suggest ..”
++	Adaptations in design OR analytical methods needed, currently not supported by regulatory authorities	Only applicable by statisticians or PK/ PD experts	Moderately complex; statistical expertise required	Design and analytical methods require moderate adaptations of standard clinical trial protocols OR extra data are required	Based on encouraging results from simulations	“We strongly suggest…”
+++	Adaptations in end points, consultation with regulatory authorities required	Applicable by those experienced in applied statistical or PK/PD analysis	Moderately easy; clinical trial background required but no need for a statistical background	Design and analytical methods require small adaptations to specific parts of a standard clinical trial protocol, and no extra data are required	There is fair research- based evidence to support the recommendation; reanalysis of clinical trial data has provided encouraging results	“We recommend…”
++++	No obstacles for implementation identified, in line with current regulatory guidelines	Only basic statistical or PK/PD expertise required	Easy; no need for a statistical or clinical trial background	Standard clinical trial protocols can be applied, and no extra data are required	There is good research- based evidence to support the recommendation; it has already been applied and tested in clinical trials	“We strongly recommend…”

Abbreviation: PK/PD, pharmacokinetic and pharmacodynamic.

Table 2.

Recommendations and Their Classification Regarding Current Regulations, Implementation, Interpretation, and Evidence Base^a

Recommendations	Alignment With Current Regulatory Framework	Ease of Technical Implementation	Ease of Interpretation	Ease of Practical Implementation	Evidence Base
Innovative biostatistical methods for PK/PD modeling
1. We recommend that phase II and III clinical trials of new antibiotics, particularly those active against MDROs, always apply population PK models to describe and explain PK variability, optimize dose finding, and evaluate outcome data relative to exposure.	++++	+++	++	+++	++++
2. We recommend an EU-coordinated regulatory procedure for reassessment of old antibiotics and their licensing, particularly those active against MDROs, which addresses justification of dosing regimens and exposure-response data according to modern PK/PD principles. This should include description of PK/PD targets, development of population PK models, and reassessment of antibacterial spectra.	++	+++	++	+++	+++
3. We recommend that future clinical PK/ PD studies provide more robust results by a priori determination of the sample size, adjustment for known confounders of the exposure-response relationship, assessment of both microbiological and patient-oriented outcomes, and application of appropriate statistical techniques.	++++	+++	+++	+++	+
Selection of novel and more sensitive primary outcomes for clinical trials
4. We recommend using rank-based composite end points combining patient-oriented and disease-related end points to assess new antibacterial therapies against MDROs.	+++	+++	+++	++	++
5a. We strongly suggest, in trials dealing with MDROs, applying multistate models to examine a range of time-dependent clinical outcomes in the primary analysis to better characterize the patient’s disease course.	++	+	++	++	+++
5b. We strongly suggest, when applying multistate models, performing statistical significance testing for the probability of being cured and alive over the entire treatment process rather than at the end of follow-up.	+	+	++	+	+++
Innovative trial design in the absence of rapid diagnostics
6. We strongly suggest that trials aiming to assess the clinical benefit of a new therapy against MDRO pathogens should apply a hierarchical nested-trial design if a priori power calculations indicate feasibility.	++	++	++	++	++
Methods to incorporate historical clinical trial data
7. We strongly suggest that clinical trial investigators make use of the multitude of historical clinical trial data in the design and analysis of novel MDRO treatment trials.	++	++	+++	++	+++
8. We suggest the use of platform trials to study new antibacterial treatments against MDROs.	+	+	++	++	+

Abbreviations: EU, European Union; MDROs, multidrug-resistant organisms; PD, pharmacodynamic; PK, pharmacokinetic.

^aSee Table 1 for definitions of symbols +, ++, +++, and ++++.

RESULTS

Innovative Biostatistical Methods for PK/PD Modeling

Recommendation 1 (Study Design)

We recommend that phase II and III clinical trials of new antibiotics, particularly those active against MDROs, always apply population PK (popPK) models to describe and explain PK variability, optimize dose finding, and evaluate outcome data relative to exposure.

Antibiotic efficacy, which determines microbiological cure, depends on the in vitro potency of the drug (usually expressed as the minimum inhibitory concentration [MIC]) and the in vivo exposure of the microorganism to the drug, which relies on the concentration-time profile (pharmacokinetics) and the dose. Owing to MIC and PK variability, dosing optimization for certain patient subgroups is essential to avoid poor outcomes from ineffective treatment or resistance selection with too low doses, or adverse events with too high doses [20].

Identification of efficacious dosing regimens based on preclinical PK/PD analyses and phase I and II studies support the dosing rationale in phase II and III clinical studies [21]. An important aspect of preclinical PK/PD analyses is the identification of a dominant drug-specific PK/PD index, which describes the exposure-response relationship (eg, the percentage of the dosing interval that the free drug concentration (fT) remains above the MIC, fT > MIC) and the minimal index value that ensures a high probability of efficacy (the PK/PD target, eg, 50% fT > MIC). In phase I studies, the PK of several dosing regimens is explored in healthy volunteers. Subsequently, in phase II and III studies, popPK models can be further developed to describe and explain PK variability in the heterogeneous target population [21]. Simulations based on popPK models can be used to determine the probability of target attainment of various dosing regimens in specific subpopulations, and thereby improve the efficiency of phase II and III studies. PopPK data, combined with MICs and outcome data, can also be used to refine preclinical exposure-response relationships and identify clinically relevant PK/PD indices and targets.

This recommendation is highly evidence based and in alignment with the current regulatory framework (Table 2) [21, 22]; however, it is not yet fully applied for all recently approved antibiotics. Another drawback is that evaluation of data is often restricted to blood levels; data for other relevant body sites are also important, but interpretation for these body sites still remains uncertain. This recommendation is applicable for all infections, including those caused by MDROs (Table 3).

Table 3.

Applications and Possible Benefits and Disadvantages for Each Recommendation

Recommendation	Type of Trial	Indications	Population	Benefits	Disadvantages
Innovative biostatistical methods for PK/PD modeling
1. We recommend that phase II and III clinical trials of new antibiotics, particularly those active against MDROs, always apply population PK models to describe and explain PK variability, optimize dose finding, and evaluate outcome data relative to exposure.	PK/PD analyses with PK and PK/PD data from phase I, II, and III studies; PK/PD target based on preclinical and clinical data	All infections	All	Optimized dosing, increasing the likelihood of detecting true efficacy in RCTs, decreasing the likelihood of emergence of resistance	Additional patient sampling required; extra costs for RCT sponsors
2. We recommend an EU-coordinated regulatory procedure for reassessment of old antibiotics and their licensing, particularly those active against MDROs, which addresses justification of dosing regimens and exposure-response data according to modern PK/PD principles. This should include description of PK/PD targets, development of population PK models, and reassessment of antibacterial spectra.	PK/PD analyses with PK and PK/PD data from phase I, II, and III studies; PK/PD target based on preclinical and clinical data	All infections	All	Optimized dosing and indications of old antibiotics, increasing the likelihood of detecting true efficacy in RCTs, decreasing the likelihood of emergence of resistance	Need for alignment with regulatory authorities; new licensing required; public funding for reassessment studies required
3. We recommend that future clinical PK/PD studies provide more robust results by a priori determination of the sample size, adjustment for known confounders of the exposure- response relationship, assessment of both microbiological and patient- oriented outcomes, and application of appropriate statistical techniques.	PK/PD analyses with PK and PK/PD data from phase I, II, and III studies. PK/PD target based on preclinical and clinical data.	All infections	All	More reliable PK/PD data, optimized dosing, increasing the likelihood of detecting true efficacy in RCTs, decreasing the likelihood of emergence of resistance	Additional patient sampling required; extra costs for RCT sponsors
Selection of novel and more sensitive primary outcomes for clinical trials
4. We recommend using rank-based composite end points combining patient-oriented and disease-related end points to assess new therapies against MDROs.	All	All infections, especially those with low mortality	All	More meaningful, and sensitive end points, increasing the likelihood of true positive findings in RCTs	End points could become more subjective; end points may be more difficult to interpret; it can be difficult to establish an acceptable NI or superiority margin
5a. We strongly suggest, in trials dealing with MDROs, applying multistate models to examine a range of time- dependent clinical outcomes in the primary analysis to better characterize the patient’s disease coursea.	All	Those with moderate to high mortality rates	Populations with moderate to high mortality rates	More meaningful, and sensitive end points, increasing the likelihood of true-positive findings in RCTs	Composite end points may be more difficult to interpret; it can be difficult to establish an acceptable NI or superiority margin
Innovative trial design in the absence of rapid diagnostics
6. We strongly suggest that trials aiming to assess the clinical benefit of a new therapy against MDRO pathogens should apply a hierarchical nested-trial design if a priori power calculations indicate feasibility.	NI trials	All	MDROs	Statistically sound results for treatment efficacy in the MDRO subgroup without the need for rapid diagnostics	Large sample of non-MDRO patients required
Methods to incorporate historical clinical trial data
7. We strongly suggest that clinical trial investigators make use of the multitude of available historical clinical trial data in the design and analysis of novel MDRO treatment trials.	All	All	All	RCTs can include a lower number of patients, but remain powered, and existing evidence is efficiently used	Increased type I error; need for historical RCT data, which may be difficult to obtain
8. We suggest the use of platform trials to study new antibacterial treatments against MDROs.	All	All	All	Increasing the efficiency of RCTs by using the same patients for multiple RCTs	Large database and high workload, which may not be used eventually; potential conflicts between different study sponsors and/or companies

Abbreviations: EU, European Union; MDRO, multidrug-resistant organism; NI, noninferiority; PD, pharmacodynamic; PK, pharmacokinetic; RCT, randomized controlled trial.

aThis also holds for recommendation 5b, as it is an extension of recommendation 5a.

Recommendation 2 (Regulatory Procedures)

We recommend a European Union (EU)–coordinated regulatory procedure for reassessment of old antibiotics and their licensing, particularly those active against MDROs, which addresses justification of dosing regimens and exposure-response data according to modern PK/PD principles. This should include description of PK/PD targets, development of popPK models, and reassessment of antibacterial spectra.

Many currently used antibiotics have been available clinically for many years, and long before current PK/PD principles and popPK modeling programs were used in drug evaluation [20]. PK analyses were based on noncompartmental or simple compartmental methods without covariate investigation. PK/PD targets were mostly unknown, and probability of target attainment simulations were not performed. The optimal dosing regimens of many old antibiotics therefore remain unknown, which makes the probability of efficacy attainment uncertain. Moreover, the antibacterial spectrum of old antibiotics is often poorly defined, as changes in resistance epidemiology have not been systematically studied.

Reanalysis of old PK data may support dosing optimization of old antibiotics, provided the PD target is or will be established. For example, within the framework of STAT-Net, a popPK model using 45-year-old amoxicillin drug concentration data was developed, and new dosing recommendations were published based on probability of target attainment simulations [10]. Unfortunately, the retrieval of such old data can be cumbersome. Alternatively, new PK studies of old antibiotics can be performed, for example, in the EU project AIDA (www.aida-project.eu). Unfortunately, even though new popPK models are being developed, PK/PD targets are still lacking for many old antibiotics [20]. Thus, developing rational dosing recommendations for all old antibiotics can be complex.

This recommendation is evidence based (Table 2), but it does not align with current regulatory approaches, because a coordinated redevelopment procedure including updating of summary of product characteristics is currently unavailable [20]. However, some studies are performed in EU and National Institutes of Health projects (COMBACTE, AIDA, National Institutes of Health; www.nih.gov/news-events/news-releases/nih-funds-four-clinical-trials-fight-antibiotic-resistance), which may provide a suitable basis. This recommendation is applicable for all infections, including those caused by MDROs (Table 3).

Recommendation 3 (Study Design)

We recommend that future clinical PK/PD studies provide more robust results by a priori determination of the sample size, adjustment for known confounders of the exposure-response relationship, assessment of both microbiological and patient-oriented outcomes, and application of appropriate statistical techniques.

Although clinical PK/PD studies of antibiotics are always performed, their design and analysis have had limitations. Consequently, guidance on dosing regimens does not always have a strong evidence base, and to maximize the possibility of successful clinical outcomes and pathogen eradication, especially in the case of MDROs, more robust data are needed.

We conducted a systematic review of clinical PK/PD studies published since 1980, which related a calculated PK/PD index to the probability of a clinical or microbiological response [18]. After deduplication, 6096 records were reviewed, resulting in the final inclusions of 85 articles containing 97 PK/PD analyses. Only 3 of 97 studies included a sample size calculation, and as such it cannot be determined whether clinical meaningful results would have been detected if present. Less than half of the studies included adjusted analyses for known confounders, including physiological patient characteristics, infection characteristics, and concurrent treatments. About half of the studies focused on clinical response, and the other half reported bacteriological response, but in most cases both would be important. Clinical response is the most important outcome to the patient, but bacteriological response measures the direct effect of the antibiotic.

From an analytical perspective, only 61 of 97 studies reported some form of regression, performed time-to-event analysis, or used the Hill (E_max, maximum drug effect) equation to look at the association between PK/PD index and outcome. To prevent inappropriate interpretation and multiplicity errors caused by analyzing a number of factors in a data-dependent way, we recommend that preapproved analysis plans should be implemented. These should ideally be based on the recently published regulatory guidance document EMA/CHMP/594085/2015 [21], which took into consideration the above-described work and emphasizes application of the most appropriate methods. Furthermore, presentation of confidence intervals around stated effects will guard against inappropriate interpretation of underpowered analyses. This recommendation is based on expert opinion, is moderately easy to apply (Table 2), and is applicable for all infections including those caused by MDROs (Table 3).

Selection of Novel and More Sensitive Primary Outcomes for Clinical Trials

Recommendation 4 (Study Design/Analysis)

We recommend using rank-based composite end points combining patient-oriented and disease-related end points to assess new therapies against MDROs.

At present, an ideal end point that would allow assessment of the efficacy of new therapies against MDRO is still lacking [16]. Patient-oriented end points, such as mortality or quality of life, are robust and matter directly to patients [23]. However, they also rely on several other noninfectious, confounding factors. Moreover, they require large sample sizes for noninferiority (NI) testing, with clinically unacceptable NI margins [16]. On the other hand, disease-oriented primary end points, such as clinical cure or organ failure–free days, are more sensitive and require smaller samples, but these are not always unequivocally linked to true patient benefit [16].

In this context, composite end points seem to be worthy of further study [16], especially if they are easy to use and give appropriate priority to the more clinically important events (eg, mortality). New methods for analyzing composite end points have already been reported [24, 25], and applied [26, 27]. In a recent Delphi process, 26 experts in the field of severe nosocomial pneumonia confirmed that such a composite end point, combining patient-oriented and disease-related end points, was expected to be the best method for assessing antibacterial efficacy in future clinical trials. This could be extended by applying multistate models (recommendation 5a), or a hierarchical nested-trial design (HNTD) (recommendation 6). The development of a rank-based composite end point needs to be planned before RCTs start and discussed with regulatory authorities. It is technically feasible to apply, but interpretation is complex (Tables 2 and 3).

Recommendation 5a (Study Design/Analysis)

We strongly suggest, in RCTs dealing with MDROs, applying multistate models to examine a range of time-dependent clinical outcomes in the primary analysis to better characterize patients’ disease course.

Food and Drug Administration and European Medicines Agency guidelines have suggested the use of different primary end points in clinical trials evaluating treatment for patients with hospital- or ventilator-associated pneumonia, end points that include either a clinical outcome at the test of cure (TOC) visit or all-cause mortality at a specific point in time [28]. As discussed above, composite end points combining these clinical important events could be more informative. On top of that, validity could be improved by considering the occurrence of events over time, instead of assessing them at a predefined time interval.

If cure and death end points are of particular interest, both measures of clinical benefit can be simultaneously accounted for in a multistate framework using the coprimary end point “get cured and stay alive over time.” The application of this type of analysis has been illustrated by using data from a recently published trial on patients with hospital- or ventilator-associated pneumonia [15] and can be adapted to more complex disease histories, for example in patients with Clostridium difficile infection [14]. The application of such multistate models has the advantage of avoiding common survival biases (which occur if one deletes or censors death outcomes when studying cure), because ignoring time dependency may lead to overestimated or underestimated efficacy. Furthermore, it provides patient-relevant information about being cured and staying alive, instead of providing cure rates and mortality rates separately. This recommendation requires specific statistical expertise but can be applied to many potential MDRO treatment indications in both the design and analytical phases of RCTs (Tables 2 and 3).

Recommendation 5b (Study Design/Analysis)

We strongly suggest, when applying multistate models, performance of statistical significance testing for the probability of being cured and alive over the entire treatment process rather than at the end of follow-up.

Traditionally, the hypothesis tested in RCTs uses the data obtained at the end of follow-up (eg, at the TOC visit). So far, no method has been validated that statistically tests NI or superiority for a multistate end point demonstrating a treatment effect over the complete treatment process instead of merely at the end of follow-up.

We applied an innovative resampling technique to construct 1-sided simultaneous confidence bands to test the difference in probabilities of being cured and alive between study arms [29, 30], which performed well [13]. This provides a comprehensive picture of the time-dynamic effect of the entire treatment process while preserving the desired α-level for statistical testing, resulting in a much stronger NI or superiority statement.

This recommendation is promising for the analysis of future RCTs, although it involves statistical expertise for implementation. Moreover, NI margins are difficult to establish, given their reliance on historical data of the effects without treatment. Discussions with regulatory authorities would be required to agree on an acceptable NI margin for this novel outcome measure (Table 2).

Innovative Trial Design in the Absence of Rapid Diagnostic Tests

Recommendation 6 (Study Design/Analysis)

We strongly suggest that trials aiming to assess the clinical benefit of a new therapy against MDRO pathogens should apply an HNTD if a priori power calculations indicate feasibility.

Superiority trials for new antibacterials targeting MDROs are, in general, considered infeasible [6]. It is usually impossible to select a MDRO subgroup at the time of randomization, because this usually occurs before standard organism susceptibility testing is available. Rapid diagnostic testing would be very useful in this respect, but unfortunately rapid antibiotic susceptibility testing has not yet developed to a level that would make application for RCTs feasible. If a new drug against MDROs were to be tested among a mixed patient group, with susceptible and MDRO infections, the chances of showing superiority is probably low, especially if the proportion of patients infected with MDROs is low. Therefore, NI trials have become the standard in this area, with limited data about clinical benefit for the MDRO patient subgroup.

The HNTD [31] is an innovative approach of addressing clinical benefit for patients with susceptible infections and patients with MDRO infections within a single RCT. The HNTD originally suggests power calculations to be aligned with inference hierarchy, and thus sample size calculations will be based on the overall NI testing. In our simulations, we observed that demonstration of superiority in the MDRO subgroup can become practically infeasible if the sample size of this subgroup is small. The power implications of designing the trial on the basis of the superiority test in the MDRO subgroup should therefore be explored in advance.

This is especially important from an ethical point of view, because it will indicate the likelihood of success of a RCT for the targeted MDRO subgroup, which should be a criterion for patient participation, just like expected beneficence for the subgroup of patients with susceptible infection. Nevertheless, hierarchical approaches should be considered whenever feasible, because they can provide valuable information for both the susceptible and MDRO subgroups. Possibly, a combination with rank-based composite end points (recommendation 4) could make this approach more powerful. Application of this recommendation is moderately complex, should be discussed with regulatory authorities before application, and requires a large sample of MDRO patients (Tables 2 and 3).

Methods to Incorporate Historical Clinical Trial Data

Recommendation 7 (Study Design/Analysis)

We strongly suggest that clinical trial investigators make use of the multitude of available historical clinical trial data in the design and analysis of novel MDRO treatment trials.

As discussed earlier, in most trials it is difficult to prospectively identify a large number of patients with MDRO infections and thus adequately power RCTs. Historical data from previous studies could be used more effectively to make the preparation and conduct of clinical trials more feasible and efficient.

Historical data are already used to help justify NI margins, but they can also be included in a Bayesian approach by using preexisting data to incorporate knowledge about possible trial results. Where several trials have been conducted with a similar or identical treatment to the control arm of the new trial, a meta-analysis based on these data can be performed, and a mathematical expression of the prior knowledge regarding control efficacy can be constructed. These priors can be interpreted as historical control patients, and when used alongside clinical trial patients can reduce the number of patients needed in the RCT, or increase the power of a given comparison [32, 33]. The new trial can be conducted using a traditional design [34, 35] or an adaptive design, where the sample size or randomization ratio can be adjusted based on interim analyses to optimize power [33, 36].

This method highly depends on the quality of historical data, detail of publicly available data, and possible time-dependent changes in medical care. It also requires extensive communication between the sponsor and regulatory agency to agree on the prior distribution and acceptable type I and II error rates in the context of the efficiencies gained with such a design. This recommended method is not limited to specific end points or infection types, so it can be applied in any trial where relevant data are available (Tables 2 and 3).

Recommendation 8 (Study Design/Analysis)

We suggest the use of platform trials to study new antibacterial treatments against MDROs.

If no historical data exist (recommendation 7), there are other, more efficient ways to improve sample size. In a platform trial, investigators focus on the disease rather than any particular experimental therapy, and they can simultaneously, or subsequently, investigate multiple experimental and control treatments, as a way to handle patient involvement as effectively as possible.

A key recommendation in the report issued by the President’s Council of Advisors on Science and Technology in the United States [37] is the establishment of a clinical trials infrastructure that would in turn support a “platform trial” for antibacterial agents [38–40]. Although there are notable operational hurdles (for example, finding/identifying patients), many of them are germane to the larger complexities of implementing clinical trials for antibacterials [37]. Initiating a single platform trial in this setting could therefore aid in managing some of these barriers, similar to what has already been accomplished in therapy areas such as oncology and Alzheimer disease [41].

The efficiency of such a platform trial is driven by key design features. These include use of a shared control and incorporation of Bayesian methods to allow use of information across sites of infection and/or from historical data (recommendation 7) in a single analysis, bearing in mind the general prerequisites for using Bayesian approaches, such as assuring similarity of historical to contemporary data [38, 42, 43]. This recommendation requires specific statistical expertise, and there is no real-world experience yet for antibacterial development, but based on experiences in other medical fields it seems a promising alternative (Tables 2 and 3).

DISCUSSION

Innovations are urgently required in the field of antibacterial development, especially for treatments against MDROs. The recommendations provided here could be instrumental to advances in this field. Although the proposed recommendations would not always be applicable within the same trial, not all of them align with current regulatory guidelines, and they differ regarding ease of implementation or interpretation and evidence base, they are all relevant to the debate supporting change.

In Table 2, the different recommendations were scored, showing that, in general, recommendations for PK/PD studies have the strongest evidence base, and our view is that they should be implemented as soon as possible to improve drug dosing in RCTs. Bayesian methods, incorporating data from historical controls in new RCTs, can be successfully used to reduce the number of patients required for RCTs. They have already been applied to reanalyze clinical trial data, and regulatory applicability is promising, provided that the historical and contemporary data can be shown to be comparable. Rank-based composite, or time-dependent, end points are another way to improve statistical efficiency and provide more meaningful data at the same time. However, technical and practical implementation still present some challenges, and a regulatory framework to support this approach is still lacking.

Our recommendations related to HNTDs and platform trials are the least pertinent. Although the HNTD approach has clear merit, it generally faces the huge challenge of recruiting sufficient patients with MDROs to support a meaningful superiority assessment. Platform trials could efficiently provide data for multiple, concurrent or subsequent, control arms, by establishing a common clinical trials infrastructure, but practical implementation is challenging, particularly gaining commitment at the initiation of the platform trial. By increasing RCT efficiency, superiority trials could become more feasible, which would be preferable, considering the ethical issues associated with NI trials [44].

STAT-Net partners are already working on the evaluation and refinement of some of the proposed solutions. First, as an extension of recommendation 6, a novel HNTD is being evaluated, in which an alternative, more sensitive end point (statistically) is introduced for the subgroup of patients infected by resistant pathogens. Such an approach could curtail the required sample size for superiority testing in the subgroup. To maintain confidence in the clinical benefit, the point estimate for the clinical end point of interest, as used in the NI assessment for the whole sample (ie, cure rate), should be similar as well. An example of a more statistically sensitive end point could be the regulatory approved end point of absolute reduction in skin lesions in skin and soft-tissue infections. This approach could be especially valuable for very rare MDROs.

Second, the application of multistate models (recommendation 5) is being studied for combined end points other than cured and alive, for example, to be alive and not receiving mechanical ventilation in patients with acute respiratory distress. Study planning for more specific multistate end points will typically be simulation based [45]. Previous studies can be used to define the assumptions needed for these types of simulation studies, possibly using only published graphs of outcome probability over time for control groups, as explained by Allignol et al [46].

In addition, novel statistical methods will be tested that allow historical control data from a single study (as compared with multiple studies, recommendation 7) to be used in future RCTs [12]. Outcome estimates from a single previous trial can be weighted depending on similarity to the outcome in the new trial, thereby increasing sample size and power with only a limited increase in type I error. This may help demonstrate the totality of evidence to reviewers, by including the relevant historical data in a secondary analysis. This new approach can also be extended to multiple historical studies [11] and as such is a promising alternative to the methods listed in recommendation 7, because it is more flexible and does not require the strong assumptions about heterogeneity and exchangeability.

Finally, efforts to initiate an antibacterial platform trial (recommendation 8) are underway in both the EU, through COMBACTE-NET and PREPARE, and the United States, through the Antibiotics Resistance Leadership Group (www.arlg.org), and also globally, through the Wellcome Trust. Additional simulation work to understand operating characteristics and further discussions with regulatory agencies on a case-by-case basis will be needed to continue to progress and embed these innovative trials.

The work presented here was bound by the IMI call text and the subsequent description of work. Since 2012, the landscape has changed, and a multitude of methodological approaches for accelerated antibacterial development have been proposed. Rex et al [7] have discussed the 4-tiered approach to registration, whereby required strength of the evidence depends on the severity of the unmet medical need, ranging from disease focused double phase III RCTs to pathogen-focused observational studies. Although the current regulatory framework seems more open for these alternative routes, acceptance still requires alignment and assessment of unmet medical need.

For PK/PD data, the European Medicines Agency has recently updated their guidance document [21], and sponsors are now encouraged to include and use PK/PD data in their application for regulatory approval of new antibiotics, although this is, unfortunately, not yet fully adopted in all current application dossiers. In addition, a proper framework for the reassessment of old antibiotics is urgently required. Multiple Bayesian applications have been proposed as well, of which some would fit logically within the current regulatory framework; Bayesian-based meta-analyses could be used to determine more appropriate NI margins, while balancing the degree of unmet need and the feasibility of the RCT [47].

From an ethical point of view, the possible benefit for society should be subordinate to the individual risks of participation in research, and should include assessment of operational risks of RCTs (eg, too-complex trials stopped due to poor recruitment), and scientific rigor (eg, data validity and power issues). To quote Ruberg, “Our professional challenge is to implement adaptive approaches while maintaining sufficient rigor in the design and analysis of clinical development trials and programs without inhibiting innovation or delaying the access of needed medications to patients who are waiting.” [48]. Hopefully, these recommendations and their continued evaluation and evolution will accelerate antibacterial approval and ensure appropriate use of established antibiotics to help those in need as soon and best as possible.

Notes

Disclaimer. The funders had no role in study design, conduct, analysis, and interpretation of the data; drafting of the manuscript; and the decision to submit the manuscript for publication.

Author contributions. All authors read and met the ICMJE criteria for authorship, participated in regular conference calls, and agree with the recommendations. M. E. A. d. K., E. B., and S. H. drafted the manuscript. M. E. A. d. K., H. S., F. d. V., I. G., E. W., A. M., S. N., O. A., T. A., J. B., L. H., A. M. L., A. P. M., J. W. M., J. F. T., M. W., E. B., and S.H. wrote the first version of the paper. D. W. and A. D. contributed to the writing of the manuscript.

Acknowledgments. We acknowledge Eric Burroughs, René Eijkemans, Fabricio Da Liberdade Jantarada, Esther Oomen–de Hoop, Kit Roes, Nicola Little, Martin Schumacher, and Jeffrey Wetheringthon for their valuable contributions to the work of STAT-Net.

Financial support. This work was supported by the IMI Joint Undertaking (JU) (grant 115523), Combatting Bacterial Resistance in Europe, with resources including financial contribution from the EU’s Seventh Framework Programme and in-kind contributions from companies in the European Federation of Pharmaceutical Industries and Associations (EFPIA), and by the German Research Foundation (Deutsche Forschungsgemeinschaft) (grant WO 1746/1–2 to M. W.). O. A., T. A., D. W., and A. D. (formerly) work(ed) for an EFPIA member company in the IMI JU, and costs related to their part in the research were carried by the respective company as in-kind contribution under the IMI JU arrangement.

Potential conflicts of interest. H. S. is now an employee of Roche Pharma AG. O. A. and T. A. are employees of GlaxoSmithKline. A. P. M. has received funding from Merck Pharmaceuticals, Achaogen Pharmaceuticals, Paratek, VenatoRx, and Bayer Healthcare. J. M. has received funding from Adenium, AstraZeneca, Basilea, Cubist, Polyphor, Roche, Eumedica, Venatorx, Aicuris, and Wockhardt. J. F. T. has received funding and personal fees from 3M. D. W. is a shareholder and a former employee of AstraZeneca. A. D. has worked as a consultant for Achaogen, Allecra, Cidara, ContraFect, Davolterra, F2G, Geom, GSK, Nabriva, Phico, Roche, Spero, TenNor, VenatoRx, and Zavante. S. H. has received personal fees from Sandoz, DNA Electronics, Bayer, and Takeda and has received grants from Pfizer. All other authors report no potential conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.