Comparing optimization criteria in antibiotic allocation protocols

Abstract

Clinicians prescribing antibiotics in a hospital context follow one of several possible ‘treatment protocols’—heuristic rules designed to balance the immediate needs of patients against the long-term threat posed by the evolution of antibiotic resistance and multi-resistant bacteria. Several criteria have been proposed for assessing these protocols; unfortunately, these criteria frequently conflict with one another, each providing a different recommendation as to which treatment protocol is best. Here, we review and compare these optimization criteria. We are able to demonstrate that criteria focused primarily on slowing evolution of resistance are directly antagonistic to patient health both in the short and long term. We provide a new optimization criteria of our own, intended to more meaningfully balance the needs of the future and present. Asymptotic methods allow us to evaluate this criteria and provide insights not readily available through the numerical methods used previously in the literature. When cycling antibiotics, we find an antibiotic switching time which proves close to optimal across a wide range of modelling assumptions.

1. Introduction

Throughout the twentieth century, a small number of discoveries have fundamentally reshaped our world. Transistors have created a world of computation and communication [1–3], the Haber–Bosch process for nitrogen fixation has massively increased our ability to produce crops [4,5], and the development of antibiotics [6,7] has not only redefined our battle against disease but also acted as a key enabling technology in surgery and intensive care [8–10]. Antibiotics have helped lift life expectancy from 47 at the beginning of the twentieth century, to the 79-year average we see today [11].

At the same time, evolution, that same process that gave birth to penicillin, ensures that antibiotics are a limited resource; every time a given antibiotic is used to save a life or prevent an infection we inevitably select those bacteria most able to resist for survival. Over weeks and years this sustained selective pressure has given rise to a variety of multiresistant ‘superbugs’ [12]. Evolution of multiresistance is further accelerated by horizontal gene transfer (HGT) [13,14], a process by which one bacterial lineage may share genetic material with another. Bacteria have, over time, developed many forms of antibiotic resistance (ABR), first to penicillin, and then to erythromycin, methicillin, vancomycin and carbapenems [15,16].

While a large number of antibiotics have been developed over the past century, the vast majority act through only a small number of essential mechanisms; as an example amoxicillin, cephalexin, doripenem, meropenem, aztreonam, ceftolozone and many others all act through the same β-lactam group as penicillin [17]. Bacteria that are resistant to one of these compounds will quickly develop enhanced resistance to the others [18]. In the past 30 years, only a few truly novel antibiotic compounds have been developed (for example, complestatin and corbomycin, discovered 2020, [19]), and of those compounds discovered, even fewer have made it to market (see table II of Coates et al. [20] for a list of antibiotics discovered in the past decades, and where they are in the process of clinical trials).

These practical difficulties are further exacerbated by the fact that antibiotic research is unprofitable [21,22]; development of new antiobiotics is hugely costly, yet any new drug is liable to be kept on hospital shelves as a ‘reserve’ antibiotic, and if it is used, will cure a patient with days or weeks, as opposed to the years or lifetime for drugs designed to treat chronic illnesses, such as cancer, depression or high blood pressure.

Given the substantial human and economic costs caused by resistant and multiresistant bacteria in clinical settings [23], currently including billions of dollars, and tens of thousands of lives, there is substantial interest in understanding how best to slow the evolution and spread of multiresistant genotypes. While there is substantial interest in developing ‘evolution proof drugs’, targeting bacterial virulence [24], using chemical mechanisms to inhibit resistance mechanisms directly [25], or pairing antibiotics where resistance to one drug implies sensitivity to another [26], for the time being, our primary tool in minimizing ABR is simple antibiotic stewardship: selecting the time and place of antibiotic use so as to minimize selection for resistance.

Antibiotic stewardship has been studied from both a clinical point of view (see the review articles by Dik et al. [27] and Chatzopoulou et al. [28] as well as [29–31]) and from a theoretical standpoint (see the literature described below).

The archetypical question asked in this context is ‘when and how should clinicians prescribe antibiotics?’ Is a hospital better off prescribing a variety of different antibiotics or applying a monoculture, switching the drug of choice every few weeks? Should all patients be prescribed multiple antibiotics (combination therapy) so as to ensure recovery, or is a lighter touch better in the long run? And how, precisely, do we define ‘better’ anyway? Is it more important to ensure optimal patient outcomes in the present, or slow the development of multiresistant bacteria over an evolutionary timescale?

A first inquiry into these conundrums was made in a simulation study by Bonhoeffer et al. [32]. Subsequent models have built on Bonhoeffer’s approach, exploring horizontal gene transfer [33,34], extensions to a larger number of antibiotic variants [35,36] and the effects of stochasticity [37]. Ibargüen-Mondragón et al. use control theory to consider both the effects of antibiotic treatment and immune response [38], while Houy et al. consider the question from the point of view of ‘what information do decision makers have access to?’ [39]. McCloud et al. explore the importance of linkage disequilibrium, the tendency of two resistance genes to be found together or apart, and model evolution on the scale of multiple inter-connected populations (for the sake of our discussion, one might imagine patients in multiple different hospitals) [40].

Concerningly, among these (and many other) investigations, no consensus on optimal treatment protocols has been reached. Uecker & Bonhoeffer [41] explain these contradictions in their clear, concise and comprehensive review article, in which they provide an overview of the subtle modeling decisions and mismatched optimization criteria that play into various contradictory rankings of antibiotic deployment strategy.

The focus of this article will be following up on a number of questions raised in Uecker & Bonhoeffer’s review. We compare the strengths and weaknesses of the various proposed optimization criteria in the literature. We propose a new, more physically justifiable optimization criteria, incorporating both patient health and evolutionary risk, a middle path between the various criteria considered previously. In addition, we find a number of analytic results that will hopefully reduce the need for time-consuming numeric exploration of parameter space, and give somewhat clearer insight into how parameter values combine to determine patient outcomes. For the important case of cyclic treatment protocols, we use our asymptotic results to identify a ‘saturation time’ t_sat which gives ‘almost optimal’ cycle time across a wide range of assumptions. We observe that optimization criteria focused on delaying multiresistance often harm patient health in both the short and long term: they are often maximized by selecting treatment protocols with patient outcomes worse than the long-term effects of antibioitic resistance.

Our goal in what follows is not to question the orthodoxy of aggresive antibiotic treatment [42]. Instead our sole goal is to examine the methods by which we evaluate antibiotic management protocols, and explain the root cause of a number of conflicting and contradictory results previously published.

In §2, we introduce the basic model and current antibiotic protocols as described in the literature. Section 2.2 introduces four possible optimization criteria, both novel and historic. In §3, we examine the mean number of uninfected patients for a variety of antibiotic protocols and gather several novel analytic approximations for patient health. In §4, we explore the behaviour of four different optimality criteria, and how ‘optimal’ results change (or don’t) depending on which criteria is used and how multiresistance arises. We demonstrate how a number of optimization criteria are directly antagonistic to one another, and indeed, to patient health. These results are summarized in §5. We find combination therapy to be appropriate over a wide range of contexts, and also identify t_sat, a cycle time that is ‘close to optimal’ over a wide range of modelling assumptions. It is our hope that by identifying why different papers have reached such contradictory conclusions, we can guide medical practitioners and future modellers towards better criteria, or at the very least, give them the tools to evaluate which criteria are most relevant.

2. Model

Let us begin by introducing a small amount of biology and defining the two models of in-hospital ABR spread that we will be building upon. Primarily, we make use of Bonhoeffer et al.’s original model from 1997 [32]. This model has acted as the basis of many of the papers that came after, and hence acts as an excellent testing space for comparing and understanding the various optimization criteria found in the literature. The second model we considered was proposed more recently by Uecker & Bonhoeffer [43] and is designed to overcome one particular simplification in the original model, which will be relevant in our future discussions. This simplifying assumption, and when it is and is not appropriate, will be discussed later.

These models are selected primarily for their analytic tractability, clarity, and foundational position in the literature. Our goal in this paper is to compare differing optimization criteria and resolve conflicting recommendations as cleanly as possible, Bonhoeffer’s model is perfect for this. With that said, future researchers may find that more detailed models incorporating more recent clinical understanding of effects, such as horizontal gene transfer [44,45] and bystander selection [46], may be more appropriate when making treatment recommendations. With that said, let us start by describing Bonhoeffer et al.’s 1997 model [32].

Imagine a hospital or hospital ward containing a number of patients. There exists some bacterial infection (for example, Streptococcus pneumoniae or Staphylococcus aureus) that spreads through the patient population and can be treated using one of two frontline antibiotic drugs (such as linezolid or telavancin). For the sake of generality, we refer to these drugs as A and B. Bacterial infections will either be susceptible to treatment, resistant to one or other of the available antibiotics, or doubly resistant. Denote the population of patients infected with each of these classes of infection as S, R_A, R_B and R_AB, respectively. The hospital also includes a population of uninfected patients, X, who are nonetheless ‘exposed’ to infection. These patients represent patients recovering from surgery (or similar medical conditions unrelated to infection). While infection may provide an additional load, uninfected status does not correspond to a clean bill of health, and conversely, an infected patient may still be healthy enough for discharge if they have recovered from surgery, chemotherapy, or whatever was their primary cause for entering the hospital. It is also important to note that here ‘susceptible’ and ‘exposed’ are not used in the same manner as standard epidemiological SEIR models. Here ‘susceptible’ refers to the infecting bacteria’s susceptibility to antibiotic treatment, and ‘exposed’ refers to a patient's proximity to possible infection, as opposed to the presence of bacterial infection that has not progressed to the ‘infectious’ stage, as implied in SEIR models.

With these five classes of patients, we now construct a compartment type model [47]. Patients switch from one infected status to another via a variety of processes (figure 1). Patients arrive at the hospital at some rate m_X, m_S, m_A, m_B and m_AB; immigration rate for each class will depend on the prevalence of ABR in the community. Patients in all compartments are discharged at some rate μ; this rate is assumed to be equal across all compartments. In appendix A, we relax this simplifying assumption and consider a model that explicitly differentiates between discharge due to death or recovery. The exact details of discharge have minimal impact on all major results, and hence we assume discharge rate μ is independent of infection status, as is traditional in the literature.

Figure 1.

Schematic diagram of the compartment model, as defined in equations (2.1a)–(2.1e). The diagram shows the five model compartments and the flow of individuals between them. Mutation events are marked with dashed lines and are treated as being rare stochastic events. More common infection, recovery, immigration and emigration events are indicated by continuous lines; differing line thickness is used to indicate that some events are more common than others (for example, β_S > β_AB, τ + γ > γ). Line thickness is not to scale. Depending on the context, the R_AB compartment of the above model will, or will not, be included in the analysis.

Exposed individuals become infected according to mass action kinetics at rates β_S S X, β_A R_A X, β_B R_B X and β_AB R_AB X. Differences in infection rate β represent the metabolic cost a bacteria must pay in order to maintain resistance; more resistance comes at a higher cost, hence β_S ≥ β_A, β_B ≥ β_AB. All β are assumed to be fairly similar, differing by only 1 or 2%, as opposed to an order of magnitude. Hospital policies that involve isolating ABR infected patients may be used to artificially reduce β_AB further [48]. While these strategies can be very effective, they will not be our focus here.

Infected individuals recover naturally at some rate γ, and recover at some higher rate τ + γ if administered suitable antibiotic treatment (hence, administering drug A leads to recovery rate τ + γ in the S and R_B population and recovery rate γ in the R_A and R_AB population.) We model which antibiotics are currently being administered via the indicator functions χ_A(t) and χ_B(t). These functions take the value zero or one, depending on whether or not the antibiotic in question is being administered at time t.

Single resistance strains are assumed to be prevalent enough in the community that de novo evolution of single resistance can be treated as negligible. The validity of this assumption will depend on the population under study.

Finally, multiresistance can be introduced to a hospital either via de novo mutations, horizontal gene transfer or importation from the wider community. Each of these mechanisms leads to different predictions and recommendations, which will be discussed in more detail on §4. For the time being, we consider two separate cases: the behaviour of the system either before multiresistance (R_AB(t) = 0 = m_AB), and the behaviour of the system after multiresistance(R_AB(t) > 0). The transition from R_AB = 0 to R_AB > 0 is discussed in §4.

Taken together, these processes lead to the following five compartment model:

$$\dot{S}=m_S-\mu S+{\beta }_{S}SX-(\gamma +\tau max({\chi }_A,{\chi }_B))S,$$ 2.1a

$${\dot{R}}_A=m_A-\mu R_A+{\beta }_A{R}_{A}X-(\gamma +\tau {\chi }_B)R_A,$$ 2.1b

$${\dot{R}}_B=m_B-\mu R_B+{\beta }_B{R}_{B}X-(\gamma +\tau {\chi }_A)R_B,$$ 2.1c

$${\dot{R}}_{AB}=m_{AB}-\mu R_{AB}+{\beta }_{AB}{R}_{AB}X-\gamma R_{AB},$$ 2.1d

$$\begin{array}{rl}\mathrm{and}\dot{X}& =m_X-\mu X-{\beta }_{S}SX+(\gamma +\tau max({\chi }_A,{\chi }_B))S-{\beta }_A{R}_{A}X+(\gamma +\tau {\chi }_B)R_A\\ & -{\beta }_B{R}_{B}X+(\gamma +\tau {\chi }_A)R_B-{\beta }_{AB}{R}_{AB}X+\gamma R_{AB}.\end{array}$$ 2.1e

The above system of equations assumes that the same antibiotic regime is applied to all patients: χ_A(t), χ_B(t) take the values zero or one. These assumptions are inappropriate however when administering different drugs to different patients. The most obvious way to represent intermediate prescription values in the above model would be to select χ_A(t) = 0.5 = χ_B(t); unfortunately, this would correspond to assigning all infectious patients both drugs half the time (for example drug A in the morning, and drug B at night). In clinical practice, this does not happen, and instead ‘mixed’ drug regimes refer to the practice of assigning half of the patients drug A across the entire course of their treatment, and the remaining patients drug B.

In order to properly model this, we use the seven compartment model of Uecker et al. [43]). In this model, we track both the resistance status of infections and the prescription status of the corresponding patients, splitting compartment R_A into $R_A^A,R_A^B$ and R_B into $R_B^A,R_B^B$. Here, subscripts represent the resistance profile of the infection, while superscripts represent the drug currently prescribed. Hence, $R_A^B$ and $R_B^A$ represent effective treatments, while $R_A^A$ and $R_B^B$ represent ineffective treatments. Treatments are ‘corrected’ at a rate q, transferring patients from $R_A^A\to R_A^B$ and $R_B^B\to R_B^A$. Because all prescriptions are assumed to be equally effective against susceptible bacteria, there is no need to split the S compartment, similarly with doubly resistant infections. χ_A(t) and its complement χ_B(t) no longer represent the probabilities of receiving a particular drug in the present, but instead the probabilities of being referred to a particular treatment group. The governing equations for $R_A^B$ and $R_A^A$ are

$$R_A=R_A^B+R_A^A$$ 2.2a

$${\dot{R}}_A^B={\chi }_B{m}_A-\mu R_A^B+{\chi }_B{\beta }_A{R}_{A}X-(\gamma +\tau )R_A^B+q{R}_A^A$$ 2.2b

$$\mathrm{and}{\dot{R}}_A^A={\chi }_A{m}_A-\mu R_A^A+{\chi }_A{\beta }_A{R}_{A}X-\gamma R_A^A-q{R}_A^A,$$ 2.2c

with similar equations governing $R_B^A$ and $R_B^B$. $\dot{S}$ and $\dot{X}$ are as defined in equations (2.1a) and (2.1e). We also consider an antibiotic switching rate q; this corresponds to the rate at which ineffective antibiotics are replaced by effective antibiotics in the case of single resistance. In the limit q = 0, patients are kept on their initial prescription indefinitely no matter what. In the limit q → ∞, patients are shifted to optimal treatment immediately. Parameter q can be thought of as a proxy for the intensity of testing for ABR within a given hospital system. A schematic of this behaviour is given in figure 2.

Figure 2.

Schematic diagram of Uecker’s 7-box model. In this model, we track the drug being applied to individual patients: R_A is split into $R_A^A$ and $R_A^B$ (A resistant bacteria being treated with A or B, respectively). Although S bacteria are treated with either one drug or the other, there is no need to track which (assuming similar recovery rates under treatment). Similarly with doubly resistant infections. Ineffectual treatment combinations ($R_A^A$ or $R_B^B$) are replaced by effective treatment options ($R_A^B$ or $R_B^A$) at some rate q (drug switching, turquoise arrow). Differences in line thickness are indicative of differences in the corresponding rate constants (recovery from $R_B^A$ is faster than recovery from $R_B^B$, for example).

In what follows, Uecker’s 7-box model is used when 0 < χ < 1 and Bonhoeffer’s 5-box model is used when χ ∈ {0, 1}. See table 1 for a list of parameters, their descriptions and (where appropriate) default values.

Table 1.

Description of parameters, their standard values (where appropriate) and a summary of optimization criteria used throughout the main text.

description	symbols	default value
number of patients on the ward but not currently infected	X(t)	—
patients infected with AB susceptible infections	S(t)	—
patients infected with drug A resistant strain	RA(t)	—
patients infected with drug B resistant strain	RB(t)	—
patients infected with multiresistant strain	RAB(t)	—
drug A resistant patients treated with drug A	$R_A^A(t)$	—
drug A resistant patients treated with drug B	$R_A^B(t)$	—
drug B resistant patients treated with drug A	$R_B^A(t)$	—
drug B resistant patients treated with drug B	$R_B^B(t)$	—
infection rate of susceptible strain	βS	1
infection rate of drug A resistant strain	βA	0.99
infection rate of drug B resistant strain	βB	0.99
infection rate of multiresistant strain	βAB	0.98
arrival rate of uninfected patients at hospital	mX	∼2/45 days
arrival rate of patients with non-resistant infections	mS	0.5 mX
arrival rate of drug A resistant patients at hospital	mA	0.002 mX
arrival rate of drug B resistant patients at hospital	mB	0.002 mX or 0.004 mX
arrival rate of patients with multiresistant infections	mAB	0
hospital discharge rate (μ = mX + mS + mA + mB)	μ	1/15 days
baseline recovery rate	γ	1/10 days
recovery rate with suitable treatment	γ + τ	1/2.5
rate at which inappropriate drugs are replaced by appropriate drugs (seven box model only)	q	0
cycle time for drug cycling	T	varies
proportion of patients receiving drug Aa	χA(t)	varies
proportion of patients receiving drug Ba	χB(t)	varies
Rate of mutation. Multiplied by various different things depending how RAB is assumed to arrive.	ν	1 or 0.1
total uninfected ‘person days’ in first year	X365	${\int }_0^{365}X(t)\hspace{0.17em}\mathrm{d}t$
time until multiresistant arrival	Tε	—
time until half of all infections are multiresistant	T1/2	—
total uninfected ‘person days’ until T1/2	XT	${\int }_0^{T_{1/2}}X(t)\hspace{0.17em}\mathrm{d}t$
long-term average X(t) value, prior to RAB arrival	$\overline{X}$	${\int }_{t_0}^{t_0+T}\frac{X(t)}{T}\hspace{0.17em}\mathrm{d}t$
long-term average X(t) value, after RAB arrival	${\overline{X}}_{AB}$	${\int }_{t_0}^{t_0+T}\frac{X(t)}{T}\hspace{0.17em}\mathrm{d}t$
smoothed ‘gain in X’ up until Tε	XT*	$(\overline{X}-{\overline{X}}_{AB})T_{ϵ}$

^aInterpretation of these parameter differs between the 5-box and 7-box model.

2.1. Common antibiotic management protocols

In this article, we will compare three main antibiotic management protocols frequently discussed in the literature: combination therapy, mixing and cycling (mixing and combination therapy are most common in clinical practice). These protocols define which antibiotic is initially prescribed to a patient when they are admitted to the hospital, or after surgery. This prescription may later be changed based on either patient recovery (or lack thereof), or when ABR infection is detected (see, for example, the Dutch ‘search and destroy’ policy for ABR [49]).

The simplest of these protocols, combination therapy, prescribes both A and B to all infected patients at all times. This approach is intended both to improve patient outcomes and to prevent multiresistance from arising by eradicating single resistant strains as quickly as possible. Unfortunately, combination therapy leads to increased direct costs and potentially heavier side effects. It also increases total drug prevalence, leading to concerns that it may encourage broad spectrum antibiotic resistance [50]. This duel action of increasing total antibiotic prevalence, but rapidly quashing single resistant strains, leads to some uncertainty with respect to the net effect of combination therapy on ABR. In this study, combination therapy is represented using Bonhoeffer’s 5-box model, with χ_A(t) = χ_B(t) = 1.

Mixing protocols assume that each patient is assigned either drug A or B with some probability, usually (but not always) χ_A = χ_B = 0.5. While many papers [33,36,51] have studied mixing using Bonhoeffer’s 5-box model (or similar), Uecker’s more detailed 7-box model is more faithful to clinical reality and will be the model used here whenever mixing is discussed.

Cycling protocols treat all patients with the same drug at any point in time, and switch back and forward between two (or more [35]) treatments every T days, preferentially treating with drug A for the first T days, and with drug B for the next. These time periods are generically (though not always [52]) assumed to be equal. Mathematically, cycling is represented by

$${\chi }_A(t)=\{\begin{array}{cc}1& \text{mod}(t,2T)<T\\ 0& \text{mod}(t,2T)\ge T\end{array}$$

with χ_B(t) = 1 − χ_A(t). Both the 5-box and 7-box model can be used to represent cycling; which is more realistic will depend on the exact implementation of cycling used in clinical practice. For the sake of analytic accessibility, we study cycling in the context of the 5-box model. Basic simulation experiments indicate that the difference between the two models is negligible, except in the case of short cycle times, where fast cycling behaves like mixing.

Other, more detailed, management protocols have been studied. Beardmore & Peña-Miller [53] make use of detailed control theory techniques in order to determine optimal aperiodic antibiotic rotation protocols, cutting through the more heuristic approaches used elsewhere in the literature. Kouyos et al.’s [37] numeric exploration expands the space of possible treatment protocols by considering ‘informed switching’ protocols adapted for the stochastic hospital environment. Consideration of these more complex protocols is beyond the scope of this paper, but we do recommend these past works to the interested reader.

2.2. Optimization criteria

Broadly speaking, antibiotic protocols seek to achieve two conflicting goals: to maximize patient health and minimize the rate at which resistant bacteria (especially multiresistant bacteria) arise and develop. While these goals are easy enough to understand in an intuitive sense, there have nonetheless been a number of different formulations mathematically; each with their own strengths and weaknesses. For the sake of brevity, we can not consider all possible optimization criteria ever considered in the literature; our goal here is to present a representative sample. Additional optimization criteria, such as exponential discounting (as used in, for example [54]) and Bonhoeffer’s G_1/2 are discussed in appendix D and B.4, respectively.

The most straightforward evaluation criterion is simply to maximize the number of ‘healthy’ patients over a given time frame (most often, 1 year):

$$X_{365}={\int }_0^{365}X(t)\hspace{0.17em}\mathrm{d}t.$$

This evaluation criterion is illustrated in figure 3a. It was used in Bonhoeffer’s initial exploration of antibiotic protocols [32] (albeit with a constant offset). Under reasonable assumptions, it can be shown that maximizing X is equivalent to minimizing patient fatalities and minimizing total hospitalization time (appendix A).

Figure 3.

Comparison of different optimization criteria. Here, we consider some hypothetical hospital for which the number of uninfected patients X(t) bounces around at high levels before the introduction of a small number of multiresistant infections at time $T_{ϵ}$. At this stage, multi-resistance increases to high prevalence (T_1/2), while the number of uninfected patients drops, oscillating around a low equilibrium value, ${\overline{X}}_{AB}$, for the rest of time. (a) the X₃₆₅ optimization criterion attempts to maximize the total number of uninfected patients (area under the curve) up until a given time (usually t = 365.) (b) The T_1/2 optimization criterion seeks to maximize the time taken until multiresistance takes over half the population, R_AB = 0.5. (c), X_T aims to maximize the number of uninfected patients up until T_1/2. (d) X_T* maximizes the gain in uninfected patients relative to the multiresistance equilibrium, prior to multiresistance. Rather than integrating over X(t) we instead calculate the X equilibrium both before ($\overline{X}$) and after (${\overline{X}}_{AB}$) the introduction of multiresistance. While this criterion may seem unusual, one step removed from a direct integration of X, unlike the other criteria it does not depend on initial conditions.

As an evaluation criterion, X₃₆₅ runs into two difficulties: firstly, it is explicitly 'blind’ to the time of multiresistance introduction. Secondly, selection of different time windows or initial conditions may lead to different results; long time horizons emphasize the eventual steady state, while shorter time horizons are informed by initial transient behaviour. It is not always clear which time window is most appropriate.

If our interest is primarily in the arrival dynamics of multi-resistant bacteria, then we may instead attempt to maximize T_1/2, the time at which half of all infections are doubly resistant (R_AB). While interesting and meaningful from an evolutionary standpoint, T_1/2 makes a poor optimization criterion; maximization of T_1/2 generically leads to withholding all antibiotic use, thus delaying the takeover of doubly resistant mutants indefinitely. This optimization criterion is illustrated in figure 3b.

One possible balance between these two conflicting goals is to instead maximize

$$X_T={\int }_0^{T_{1/2}}X(t)\hspace{0.17em}\mathrm{d}t.$$ 2.3

This is illustrated in figure 3c. While providing some balance between maximizing health and time, it is doubtful that X_T provides the correct balance between X₃₆₅ and T_1/2. In effect, X_T assumes that we will have no healthy patients from T_1/2 onwards. It seems unlikely this was the intent of authors using this criterion, but it is the implicit result.

Much like T_1/2, X_T is liable to recommend non-treatment, as integrating even small numbers over an infinite time window gives ‘optimal’ results. Also, as mentioned by Uecker & Bonhoeffer [41], the criterion completely ignores the effects of a particular epidemic protocol on the time after the emergence of R_AB. Because X_T depends on the exact time course of X(t), there is also the risk that it will give conflicting answers for systems with different initial conditions. Because the exact initial conditions for a particular hospital are not knowable in advance, it would be preferable to avoid such sensitivity.

All three of these difficulties can be avoided by instead considering the novel optimization criteria (as illustrated in figure 3d)

$$X_{T^{\ast }}={\int }_0^{T_{ϵ}}\overline{X}-{\overline{X}}_{AB}\hspace{0.17em}\mathrm{d}t=(\overline{X}-{\overline{X}}_{AB})T_{ϵ}.$$ 2.4

Here, $\overline{X}$ indicates the long-term average of X prior to multiresistance, ${\overline{X}}_{AB}$ is the long-term average after multiresistance occurs and $T_{ϵ}$ denotes the first time that the multiresistant population reaches some low level ε, potentially set so that ε represents a single multiresistant infection. The use of long-term averages removes the effects of initial conditions and makes X terms more analytically accessible. In the case of cycling, these averages are taken over the course of one entire cycle; in the case of ‘static’ protocols, these averages are the equilibrium values of X.

X_T* can be thought of as being ‘formally equivalent’ to optimization over the integral ${\int }_0^{\mathrm{\infty }}X\hspace{0.17em}\mathrm{d}t$, albeit with the ‘constant’ ${\int }_0^{\mathrm{\infty }}{\overline{X}}_{AB}\hspace{0.17em}\mathrm{d}t$ subtracted off so as to render the optimization criteria finite. In some sense, our goal is not to maximize the total number of healthy patients over some time window (which can be manipulated via manipulation of the time window), but instead the increase in health due to the use of antibiotics, over all time.

Subtracting off ${\overline{X}}_{AB}$ also serves to forbid certain degenerate strategies where $\overline{X}<{\overline{X}}_{AB}$. These strategies represent protocols which give worse patient outcomes than the threat of multiresistance itself (for example, never using antibiotics), but delay multiresistance indefinitely ($T_{ϵ},T_{1/2}\to \mathrm{\infty }$). Protocols of this kind tend to give ‘infinite value’ according to time focused criteria such as X_T and T_1/2. By contrast, X_T* assigns a negative score to such protocols ($\overline{X}-{\overline{X}}_{AB}<0$). We will discuss this detail more thoroughly in §4. See figures 7 and 8.

Figure 7.

(a,b) The expected values of X₃₆₅ for high and low mutation/importation rate ν. Each line indicates a different channel for introduction of multiresistance. (c,d) Expected values of T_1/2 and X_T. As might be expected, both optimality criteria tend to infinity as χ_A nears 1 or 0, indicating that for sufficiently extreme values, R_AB will never account for half the infected population. This applies regardless of the multiresistance introduction channel. (e) X_T* for a variety of χ_A values. (f) X_T* rescaled such that max (X_T*) = 1. This rescaling makes maxima more clearly identifiable, and is as mathematically valid as any other scaling, seeing as comparing ν between different multiresistance introduction methods is inherently meaningless in the context of the current model.

Figure 8.

The same as figure 7, except here we optimize for different cycle times as opposed to different mixing ratios. (c,d) Once again we see that T_1/2 and X_T are maximized for extreme values (in this case, infinity cycle time). In (f), we observe that regardless of the multiresistance arrival channel, the saturation time t_sat gives close to optimal results.

It is important to note that no assumption is made that $\overline{X}$ and ${\overline{X}}_{AB}$ are achieved using the same management protocols, hence (for example) it is possible that a cycling protocol may be used prior to multiresistance, and a mixing approach afterward. X_T* optimizes on the assumption that we pick the best possible protocol post multiresistance introduction.

$T_{ϵ}$ is chosen over T_1/2 as our stopping criterion for technical reasons. See appendix B.4 for details, as well as a comparison between X_T* and the G_1/2 optimization criterion put forward by Bonhoeffer [32].

See figure 3 for a schematic illustration of all optimization criteria. Optimization criteria are also summarized in table 1.

3. Mean X values

In order to make sense of X_T* (and other optimization criteria), it will prove helpful to calculate both $\overline{X}$ and ${\overline{X}}_{AB}$. Rather than calculate numeric integrals of X for a variety of initial conditions and parameter values (as has been done in previous papers [33,35,36]), our goal in what follows is to find asymptotic approximations for a variety of parameter regimes, making use of different simplifying assumptions in each case. See figure 4 for sample trajectories in each of these regimes.

Figure 4.

General behaviour of the system when using mixing (a), combination therapy (b), or cycling (c,d). For the two ‘static’ strategies, (a,b), random initial conditions rapidly approach equilibrium for S and X. In the mixing case (a), R_A and R_B approach equilibrium slowly; for the parameters considered here, they become equal in limit as t → ∞; differences are purely based on initial conditions. For cycling (c,d), background colour indicates the antibiotic currently in use. We consider two distinct regimes: ‘fast cycling’ (c), in which equilibrium is never reached, and ‘slow cycling’ (d), in which R_A, R_B approach equilibrium values and are stabilized by importation from the community before drug switching occurs.

Let us start by considering the case of combination therapy prior to the introduction of multi-resistant bacteria. In this case, assuming antibiotics are effective at keeping down infection (τ + μ) ≫ β_i X for each β_i, all infected compartments can be well approximated by the balance between immigration and recovery

$$S\approx \frac{m_S}{(\tau +\mu +\gamma )},$$ 3.1

$$R_A\approx \frac{m_A}{(\tau +\mu +\gamma )}$$ 3.2

$$\mathrm{and}{R}_B\approx \frac{m_B}{(\tau +\mu +\gamma )}.$$ 3.3

The total population of the ward is given by $\sigma =\sum m_i/\mu$, and hence the long-term equilibrium of X is well approximated by

$$\overline{X}\approx {\overline{X}}_{\mathrm{combo}}=\sum m_i/\mu -(m_s+m_A+m_B)/(\tau +\mu +\gamma ).$$ 3.4

In the case of mixing (prior to multi-resistance), we use the 7-box Uecker model (figure 2). In order to calculate the long-term equilibrium behaviour, we first calculate what fraction of cases are receiving effective treatment for each strain. Effective treatment ratios can be shown to be equal to

$$\frac{R_B^A}{R_B^B+R_B^A}={\rho }_B=\frac{(\mu +\gamma ){\chi }_A+q}{\tau {\chi }_B+\mu +\gamma +q}$$ 3.5

and

$$\frac{R_A^B}{R_A^A+R_A^B}={\rho }_A=\frac{(\mu +\gamma ){\chi }_B+q}{\tau {\chi }_B+\mu +\gamma +q}.$$ 3.6

This in turn leads to three different approximations of X at equilibrium, depending on which resistant strain is dominant

$${\overline{X}}_{\mathrm{mix}}^A=\frac{(\mu +\gamma +\tau {\rho }_A)}{bet{a}_A},$$ 3.7a

$${\overline{X}}_{\mathrm{mix}}^B=\frac{(\mu +\gamma +\tau {\rho }_B)}{{\beta }_B}$$ 3.7b

$$\mathrm{and}{\overline{X}}_{\mathrm{mix}}^S=\sum \frac{m_i}{\mu }-\sum (m_s+m_A+m_B)/(\tau +\mu +\gamma ).$$ 3.7c

The true equilibrium value for $\overline{X}$ is well approximated by the smallest of these three values. If ${\overline{X}}_{\mathrm{mix}}^A$ is smallest, this indicates that R_A is the dominant infection strain, the current limiting factor on improved health. If ${\overline{X}}_{\mathrm{mix}}^S$ gives the smallest value (usually for high treatment correction values, q), this indicates that infection is dominated by disease importation; treatment within the hospital is close to optimal and ABR within the hospital is dominated by importation from the community. This situation might arise for example in situations where we have high levels of testing for antibiotics (such as the Dutch ‘search and destroy’ ABR policy [49]), or in situations where community levels of ABR are very high.

Illustrations of these results are given in figure 5. Derivation of these results can be found in appendix B.2.

Figure 5.

Equilibrium values of uninfected individuals when using a mixing protocol. (a) As q (the drug correction rate) is increased, $\overline{X}$ increases approximately linearly (following the ${\overline{X}}_{\mathrm{mix}}^A$ equilibrium), before approaching a maximal value at the ${\overline{X}}_{\mathrm{mix}}^S$ equilibrium. (b) As drug selection probability χ_A is varied, the equilibrium population $\overline{X}$ increases to a maximum and then decreases, passing from a ${\overline{X}}_{\mathrm{mix}}^A$ limited regime to a ${\overline{X}}_{\mathrm{mix}}^B$ limited regime. Here, we select drastically different β_A, β_B, so as to illustrate the effects of asymmetric infection rates. In practice, these rates can be expected to be approximately equal, and χ_A = 0.5 gives close to optimal results. For both figures, the solid blue line represents numeric estimation of X at equilibrium, calculated using Newton’s method. The black dashed lines are the upper bounds, as given by ${\overline{X}}_{\mathrm{mix}}^A$, ${\overline{X}}_{\mathrm{mix}}^B$, ${\overline{X}}_{\mathrm{mix}}^S$ above. The black dotted line (just below the dashed line, left panel) indicates a somewhat tighter upper bound for ${\overline{X}}_{\mathrm{mix}}^S$; this improved approximation gives only modest improvements, and requires significantly more algebra, see appendix B.2 for details.

The next case to consider is the cycling case. We are interested in the long-term mean value of X averaged over precisely one cycle length: ${\overline{X}}_{\mathrm{cycle}}=\underset{t_0\to \mathrm{\infty }}{lim}{\int }_{t_0}^{T+t_0}X(t)/T\hspace{0.17em}\mathrm{d}t$, where T is the length of a single cycle (we consider t₀ → ∞ so as to ignore transient effects that might be caused by initial conditions. Alternatively, we might solve to find X(t) on the interval [0, T] with periodic boundary conditions). For the time being, we consider the symmetric case, in which A and B have identical migration and treatment parameters (m_A = m_B, β_A = β_B) and equal cycle time. Detailed calculations applicable to both the symmetric and asymmetric case are provided in appendix B.3.

For cycling, there exist two major parameter regimes: a ‘fast cycling’ regime in which the equilibrium is never reached and a ‘slow cycling’ regime in which the system approaches its long-term equilibrium with each cycle (figure 4c,d). For sufficiently fast cycling, it can be shown that

$${\overline{X}}_{\mathrm{cycle}}\approx {\overline{X}}_{\mathrm{cycle}}^{\mathrm{fast}}=\frac{\mu +\gamma +\tau /2}{{\beta }_A},$$ 3.8a

each antibiotic is used half the time, and hence, each applies at 50% effectiveness. For slow cycling, the long-term average approaches

$${\overline{X}}_{\mathrm{cycle}}\approx {\overline{X}}_{\mathrm{cycle}}^{\mathrm{slow}}=\check{X}+\frac{C}{T}.$$ 3.8b

Here, C represents the transient ‘spike’ in X immediately following the change in treatment, while resistance to the new drug is rare (figure 4). The effects of the spike are diluted across the length of a cycle. $\check{X}$ represents the equilibrium value of X approached over the course of an arbitrarily long cycle, once resistance to the new drug has taken hold. Each drug has less than 50% effectiveness, because slow cycling results in resistance becoming ubiquitous in the population with each cycle; each drug spends most of its time treating the bacterial strain it is least effective against. So long as resistance importation is rare (m_A, m_B ≪ R_A, R_B) both $\check{X}$ and C can be calculated

$$\check{X}=\frac{\mu +\gamma }{{\beta }_A},$$ 3.9a

$$C=\frac{[\log (\sigma -\check{X}-\check{S}-\check{R})-\log (\check{R})-1]}{{\beta }_A},$$ 3.9b

$$\sigma =\sum m/\mu ,\check{R}=\frac{m}{\tau }$$ 3.9c

$$\mathrm{and}\check{S}=\frac{m_S}{\tau +(1-{\beta }_S/\beta )(\mu +\gamma )}.$$ 3.9d

As previously, σ is the total population. $\check{R}$ and $\check{S}$ are the equilibrium population size for susceptible and resistant strains, assuming appropriate treatment (so, R_A being treated with B, or R_B being treated with A).

For all T, $\overline{X}$ is best approximated by the minimum of ${\overline{X}}_{\mathrm{cycle}}^{\mathrm{slow}}$, ${\overline{X}}_{\mathrm{cycle}}^{\mathrm{fast}}$, see figure 6. This approximation is generally fairly tight, except in the boundary region where ${\overline{X}}_{\mathrm{cycle}}^{\mathrm{slow}}\approx {\overline{X}}_{\mathrm{cycle}}^{\mathrm{fast}}$. We refer to the boundary between fast and slow cycling as the ‘saturation time’, denoted t_sat. Saturation time can be calculated by setting ${\overline{X}}_{\mathrm{cycle}}^{\mathrm{slow}}={\overline{X}}_{\mathrm{cycle}}^{\mathrm{fast}}$, and solving

$$t_{\mathrm{sat}}=2[\log (\sigma -\check{X}-\check{S}-\check{R})-\log (\check{R})-1]/\tau .$$ 3.10

Figure 6.

Comparison of $\overline{X}$ as calculated numerically, versus the approximation $\overline{X}\approx min(((\mu +\gamma +\tau /2)/\beta ),\check{X}+C/T)$. Analytic results provide an accurate approximation of $\overline{X}$ for most T values, except in a small window on either side of the ‘saturation’ time t_sat.

The final parameter regime to consider is the equilibrium in the presence of multiresistant bacteria, that is to say R_AB > 0. In this case, it is easy to show that R_AB must increase until

$$\overline{X}={\overline{X}}_{AB}=\frac{\mu +\gamma }{{\beta }_{AB}}.$$ 3.11

This result applies regardless of which treatment protocol is employed. Strategies that would receive higher $\overline{X}$ in the absence of multiresistant infections will instead approach an equilibrium or cycle with mean value close to ${\overline{X}}_{AB}$. Protocols with worst health outcomes than the best possible response to multiresistance ($\overline{X}<{\overline{X}}_{AB}$), such as withholding antibiotics entirely, will retain their low X value, and will result in the R_AB population decaying at a rate proportional to ${\overline{X}}_{AB}-X$ (see appendix B.4 for details).

4. Introduction of multiresistance

Analytic calculations of $\overline{X}$ before and after the introduction of the doubly resistant strain provide some measure of the relative ranking of different ABR protocols. In order to evaluate ${\int }_0^{T_{ϵ}}\overline{X}-{\overline{X}}_{AB}\hspace{0.17em}\mathrm{d}t$ however, we must also consider when double resistant bacteria are introduced, that is to say, we must estimate $T_{ϵ}$. Because X_T* is linear in $T_{ϵ}$, it is sufficient to estimate the expected value of $T_{ϵ}$.

Doubly resistant strains are introduced to a hospital via a number of different channels, each with its own unique rate constant. For example, if a doubly resistant strain is imported from outside the hospital at some constant rate M_import = m_AB, then the expected introduction time of doubly resistant infection is constant, and independent of the antibiotic management protocol currently in use in the focal hospital (though potentially dependent on other hospital or inter-hospital policies [55]).

Multiresistant strains can also arise via de novo mutations. Such mutations can be modelled as either dependent on selective pressure ($M_{\mathrm{select}}={\nu }_B^{\mathrm{select}}({\chi }_{AB}+{\chi }_B)R_A+{\nu }_A^{\mathrm{select}}({\chi }_{AB}+{\chi }_A)R_B$) [34] or entirely independent of selective pressure ($M_{\mathrm{base}}={\nu }_B^{\mathrm{base}}{R}_A+{\nu }_A^{\mathrm{base}}{R}_B$) (one of multiple possibilities originally considered by Bonhoeffer et al. [32]). Finally, multiresistance may also occur via horizontal gene transfer between existing strains; the case was considered by Bergstrom et al. [33], where the assumed multiresistance is produced at a rate proportional to M_HGT = ν^HGT(R_A × R_B). More detailed models of HGT, based on within host dynamics, have been studied (for example Roberts et al.'s recent work [56]). For the time being, and for the sake of simplicity, we follow in the footsteps of Bergstrom et al. The relative importance of each of these four channels can vary from pathogen to pathogen, and will also depend critically on antibiotic stewardship policy across the surrounding region; careful stewardship may mean that M_import is minimized and we are mainly interested in de novo mutations or gene transfer. In a region with high ABR prevalence M_import may dominate.

While each of the above are reasonable assumptions they are all, in some sense, ‘cartoon’ approximations of exceptionally complex processes. Future investigation into the various sources of multiresistant infection may well suggest improvements upon the above terms, or even introduce new terms representing previously ignored channels such as horizontal gene transfer from a patient’s commensule bacteria [57]. At the very least, it would be useful to determine the relative contribution of each channel in clinical practice. Such questions, however, are far beyond the remit of this simple mathematical analysis. For the time being, we treat each of the above M_i as given and assume that in any given circumstance one channel of multiresistance dominates all others.

Our goal in what follows is not to make any specific or universally applicable policy recommendations, but instead to examine the types of recommendation made by various optimization criteria under the influence of different M_i. We are in some sense evaluating the optimization criteria themselves, rather than the management protocols on which they act. Rather than attempt a full of exploration of the entire parameter space, we focus on two straightforward test cases in order to illustrate the types of behaviour generally observed.

4.1. Comparison of optimization criteria: mixing

In what follows, we make use of Uecker’s 7-box model, with parameter values β_S = 1, β_A = β_B = 0.99, β_AB = 0.98, q = 0. We are interested in the recommendations made by the four optimization criteria X₃₆₅, T_1/2, X_T and X_T* with respect to the mixing rate parameter χ_A. For the sake of breaking symmetry slightly we set m_B = 2 m_A. Results in the q > 0 case are explored in appendix D. A full description of all parameter values and simulation methods is made available via github [58], and is saved in the file ‘EvaluatingAllOptimalMetricsMixing.m’.

In all cases, we assume that R_AB(0) = 0 and that multiresistant mutants appear according to a Poisson process with rates proportional to M_import, M_base, M_select or M_HGT. Results for each of the optimization criteria are presented in figure 7. Both X₃₆₅ and X_T* have local maxima near χ_A = 1/2 for M_import, M_base, M_select. For M_HGT, these two optimization criteria have sharp local minima near χ_A = 1/2, with their maxima off centred (to differing degrees). This makes sense: if resistance is primarily formed via horizontal gene transfer, than the primary goal of any management protocol is to avoid regions of treatment space where R_A and R_B coexist. X₃₆₅ and X_T* differ in how far from χ_A = 1/2 one must move in order to reach optimal results.

By contrast, X_T and T_1/2 both recommend extreme values of χ_A, tending to infinity precisely in those areas where X_T* < 0. Once again, this matches expectation: if $\overline{X}<{\overline{X}}_{AB}$ then R_AB cannot increase in prevalence, and hence T_1/2 = ∞.

When dealing with the optimality criteria X_T*, X_T and T_1/2, ν acts primarily as a scaling constant (with a constant offset in the case of X_T and T_1/2, caused by the delay between T_1/2 and $T_{ϵ}$). In the case of X₃₆₅, by contrast, the value of ν acts to determine the competition between multiresistance arriving and the year ending, leading to qualitatively different results depending on the exact value of ν (figure 7a,b). Fortunately, these changes have at most modest effects on the optimal mixing ratio, even for X₃₆₅. Hence, precise knowledge of ν is not crucial.

The important lessons from figure 7 in terms of criteria comparison are that X_T and T_1/2 are virtually indistinguishable, and can be seen to optimize in almost precisely the opposite direction to X₃₆₅ and X_T*. X₃₆₅ and X_T* are broadly similar in their overall behaviours, though not always in their precise recommendations.

4.2. Comparison of optimization criteria: cycling

We next consider the implications of different optimization criteria in the context of antibiotic cycling. In this case, the 5-box model is used, with parameter values β_S = 1, β_A = β_B = 0.99, β_AB = 0.98. Rather than attempt to explore the entire parameter space of possible cycling protocols, we here assume symmetric cycling times, and equal importation rates (T_A = T_B, m_A = m_B), see appendix D for comparisons in the asymmetric case. We are interested in determining the expected value of X₃₆₅, T_1/2, X_T and X_T* for a variety of cycling times T. Expectation is taken over all possible introduction times of the multiresistant mutant, and also over the ‘initial phase’ (how far through the cycle the system is at t = 0). Initial phase is selected uniformly at random between 0 and 2T. Full code is available via github [58], and is saved in the file ‘EvaluatingAllOptimal_cycling5box.m’. See figure 8 for results.

Once again, X_T and X_T* give conflicting recommendations. Much like the mixing case, X_T and T_1/2 both recommend extreme parameter values: both metrics increase monotonically with cycle time T and tend to infinity as T → 3957.86; this is the smallest value for which $\overline{X}<{\overline{X}}_{AB}$. By contrast, both X₃₆₅ and X_T* are maximized for intermediate values of T. For X_T*, optimal cycle times for all multiresistant introduction channels cluster around t_sat. This makes sense—t_sat denotes the largest T value that can be used without suffering reductions in $\overline{X}$. Any M_i that is minimized by increasing cycle time T can be increased up to t_sat at no cost. Any increase beyond t_sat inevitably comes at the cost of a reduction in $\overline{X}$. Because the transition between ${\overline{X}}_{\mathrm{cycle}}^{\mathrm{slow}}$ and ${\overline{X}}_{\mathrm{cycle}}^{\mathrm{fast}}$ at t_sat is not sharp, the exact location of the maximum of X_T* varies slightly depending on the source of multiresistance. For M_base and M_import, X_T* is maximized just below t_sat, for M_HGT and M_select, the cost of switching antibiotic is higher, and X_T* is maximized for T slightly larger than t_sat. This clustering of optimal results around t_sat is stable to variations in parameter values; see appendix C.

Qualitatively, X₃₆₅ gives results that are similar to X_T* in some ways, but not identical. Overall, X₃₆₅ can be described as ‘lumpier’; this higher complexity results from the interaction between three different timescales: the timescale of cycling, the timescale of ABR introduction, and the timescale of the integral (one year). When ν values are very small, the probability of a mutation occurring within 365 days becomes small. In this case, X₃₆₅ approaches $365\overline{X}$. By contrast, when ν values are large, mutation occurs within one or two cycles. In this case, the approximation $M=\overline{M}$ is no longer appropriate and the exact phase of cycling at the start of the simulation has a significant impact: for example, if using M_HGT then multiresistance most commonly arises during the time of drug switching. Whether t = 0 occurs before or after a switch can have significant impact on the time until multiresistance arrival, and hence on the overall shape of X₃₆₅. The large ν regime for X₃₆₅ draws attention to transient effects caused by initial conditions. These time-scale effects are ignored by X_T*, which considers only long-term averages of M, and has no predefined ‘end point’ at 1 year.

4.3. Comparison of optimization criteria: combination therapy

The final antibiotic management protocol that we consider is combination therapy, in which both antibiotic treatment options are used simultaneously for all infections. Unlike the previous two cases, where we have a parameter value to vary over, combination therapy has no such parameter values: it is, in essence a single protocol, as opposed to a wide family of protocols. As such, there is no good way of comparing combination therapy to itself, and instead the protocol must be compared to the best results from the previous protocols. The results below are based on solving equation (3.4) and comparing to the optimal results for mixing and cycling. Full code for this work can be found on github [58] included as a special case in the file ‘EvaluatingAllOptimalMetricsMixing.m’. The parameter values assumed were μ = 1/5, β_S = 1, β_A = β_B = 0.99, β_AB = 0.98, γ = 1/10, γ + τ = 1/2.5, other import/export parameters can be found in the file itself, as needed.

Using either cycling or mixing, X_T and T_1/2 can be pushed towards infinity for suitably extreme parameter values (χ → 0 or 1 in the case of mixing, T → ∞ in the case of cycling). Combination therapy allows no such ‘infinite optimization’ for either of these protocols, and hence, for any protocol focused on the dominance time of multiresistance, must be considered strictly worse. This is in line with results by Obolski & Hadany [34], who rank protocols based on the emergence time of multiresistance, and conclude that cycling is preferable to combination therapy.

For X₃₆₅, with either large or small ν values, combination therapy is of the order of 2-3 times better than optimal mixing and optimal cycling for M_base, M_select and M_HGT (cycling beats mixing for M_base, M_import, but is inferior for M_HGT, though in most cases, differences are marginal). For M_import, we find that optimal cycling is superior to combination therapy, which is superior to optimal mixing, regardless of ν.

For X_T* however, we find that combination therapy dominates all other strategies, regardless of M. For M_import, combination therapy gives X_T* three times larger than both optimal cycling and optimal mixing. For M_select, we find a approximately 190-fold increase, for M_base, combination therapy is approximately 500 times better than optimal (symmetric) cycling, and approximately 700 times better than optimal mixing. In the case of M_HGT, combination therapy gives X_T* values more than 1000 times greater than optimal mixing, which is in turn more than 10 times greater than optimal symmetric cycling (this difference is reduced for asymmetric cycling, but in this case, the optimal cycling times tend to zero, indicating that mixing is the superior strategy, see appendix D for details). The large ratios involved here should be read with a degree of caution: we do not claim that combination therapy is hundreds (or thousands) of times ‘better’ than mixing or cycling strategies. By its nature X_T* takes values closer to 0 than many of the other evaluation criteria, hence exaggerating the ratio between different values. X₃₆₅ may be preferable for more physically meaningful comparisons between combination therapy and other protocols. That said, for models considered here it is clear that X_T* unequivocally favours combination therapy over all other protocols.

5. Discussion

Antibiotic resistance, and the proliferation of multi-resistant bacteria, pose significant challenges to modern healthcare systems, threatening to roll back the past century of antibiotic research [16]. Antibiotic management protocols are designed with the goal of improving patient outcomes while preventing (as much as possible) increases in resistance. The question of how best to represent ‘good outcomes’ mathematically runs into certain difficulties: individual optimization criteria often run at cross purposes and in many cases entirely contrary to one another.

Based on our explorations in §4, it seems likely that criteria intended primarily to delay the arrival of multi-resistance (T_1/2 and X_T) should be avoided in most circumstances. These criteria tend to infinity precisely in those regions where health outcomes are worse than the long-term impact of multiresistance itself. This may be appropriate in certain cases: when dealing with mild, short-term illness, antibiotic stewardship may be prioritized over immediate recovery [59]. This is not generically the case considered in this article however, where our focus has been antibiotic policy for preliminary antibiotic allocation for inpatients at a hospital (prior to more detailed ABR testing). With this in mind, it would appear that time maximizing optimality criteria are most often actively harmful to the patient population, both in the short and long term. It is precisely these time maximizing optimality criteria that recommend against combination therapy [36]. In all other cases, when combination therapy is considered, it is found to be superior to both cycling and mixing based protocols. Whether or not this rather strong result holds up when considering additional factors such as monetary cost or bystander selection is currently unknown.

In order to balance the value of delayed multiresistance with improved health outcomes, we construct the novel optimization criterion, $X_{T^{\ast }}={\int }_0^{T_c}\overline{X}-{\overline{X}}_{AB}\hspace{0.17em}\mathrm{d}t$; this is in some sense equivalent to optimizing ${\int }_0^{\mathrm{\infty }}X(t)\hspace{0.17em}\mathrm{d}t$, albeit with the ‘constant’ ${\int }_0^{\mathrm{\infty }}{\overline{X}}_{AB}\hspace{0.17em}\mathrm{d}t$ subtracted off so as to avoid infinities. By definition, the criterion only gives positive values for protocols that improve patient outcomes relative to a ward dominated by multiresistant bacteria.

Asymptotic arguments allow us to calculate the mean number of uninfected patients, $\overline{X}$, for a variety of cases, both pre- and post-multiresistance arrival (§3). ${\overline{X}}_{AB}$ can be shown to be independent of management protocol, while $\overline{X}$ is protocol dependent. In almost all approximations of $\overline{X}$, the infection rate β is a key parameter. While we have generally discussed differences in β as being metabolic costs of resistance, it is important to note that these costs can be imposed ‘artificially’ through targeted isolation of infected individuals. This is suggestive of the critical importance of such clinical measures as improved hygiene practices and rapid diagnostics and isolation of ABR cases [48,49,60].

In all cases, we find that our results are sensitive to ABR importation rate m_A, m_B and m_AB, that is to say, the prevalence of ABR in the community. When using cycling, we find that optimal cycle times for ${\overline{X}}_{T\ast }$ scale with t_sat, the so-called ‘saturation time’ above which which $\overline{X}$ rapidly decays (figure 8). The exact position of optimal cycling relative to t_sat depends on the source of multiresistant infection (horizontal gene transfer, spontaneous de novo mutation, or selective de novo mutation). Because t_sat depends on the prevalence of ABR in the community, knowledge of the local community may be crucial for selecting optimal cycle times.

Recommendations for optimal mixing depend on the means of multiresistance introduction: balanced 50:50 mixing gives good results when multiresistance is either imported, or generated through de novo mutation and poor results if multiresistance arises via horizontal gene transfer (figure 7). Further empirical work will be needed in order to determine which of these channels is most significant.

While the discovery of t_sat, and its use in estimating optimal cycling times is a nice result, there are (inevitably) a number of caveats, conditions and stones still left unturned.

Firstly, it is worth reiterating that we have deliberately based our work here on a classical model [32]. This was done to allow cleaner comparison, and give readers a baseline that we hope they are familiar with. With that said, science moves on. Our modelling tools and understanding of biology have improved since Bonhoeffer’s publication. The significant impact of phenomena such as bystander selection [61] are better understood, and there has been a more recent push toward integrating both within host and between host models [62]. Similarly, in the area of clinical policy, diagnostic-informed strategies such as the ‘search and destroy’ ABR policy implemented in the Netherlands [48,49,60] have been mentioned only briefly in this article. Such diagnostic informed policies can be hugely powerful, and were neglected here simply because it was our intention to model pre-diagnostic ABR policy (in line with the literature we wished to comment upon).

As a more mathematical caveat, many of the asymptotic results here are made on the assumption that both m_A and m_B are rather small; ABR spread is dominated by infection within the hospital (nonsocomal infection). Outside of this parameter range, the results presented here may be less relevant. The second mathematical limitation in the present research is the assumption of continuously varying population; given the relatively small size of a hospital (dozens to hundreds of individuals), this continuity assumption is at best suspicious, especially when many key dynamics of the system occurring when R_A(t), R_B(t) ≪ 1. It would be interesting to explore these results in the stochastic context. It seems likely that some analogue to both ${\overline{X}}_{AB}$ and T_ε can be determined, though it is far from clear that ${\overline{X}}_{AB}$ will be independent of ABR protocol in the stochastic case. Similarly, the addition of a third antibiotic is also predicted to change the behaviour of the system, and the appropriate definition of ${\overline{X}}_{AB}$. Despite these complications, we would still posit that some optimization criteria conceptually equivalent to X_T* are likely to prove useful in all of these cases.

While $\overline{X}$ was found analytically for all management protocols, calculation of mutant arrival rates relied on numeric solutions of ODEs, and no analytic approximation looks forthcoming. This problem is further exacerbated by the fact that it is not even clear which M function is appropriate, and the possibility of horizontal gene transfer between native commensule bacteria carrying resistance genes and invasive pathogenic bacteria [63–65] raises the possibility that none of the M functions explored here give reliable results. Determination of which M function should guide selection of cycle time is an empirical question rather than a mathematical one. We do note, however, that while seldom optimal, T = t_sat is considered ‘good’ for all channels of multiresistance considered here.

Throughout the literature, numerous antibiotic deployment protocols have been proposed, each with the conflicting goals of maximizing patient health and maintaining antibiotic effectiveness. Our focus throughout this article has been to compare these protocols and (more crucially) to compare the various optimization criteria used to rank them. We find criteria that are overly focused on antibiotic stewardship (T_1/2, X_T) tend to recommend patient outcomes which are worse than the long-term outcome of multiresistance, and hence are harmful to patients both in the short and long term. For this reason, we recommend against the use of such optimization criteria. We find that optimization criteria which are more patient-centric (X₃₆₅, X_T*) generally recommend combination therapy as the best method of preventing the creation of multiresistance inside the hospital, but may occasionally recommend cycling if multiresistance is primarily introduced from the community.

Appendix A. Minimizing mortality and hospital stay

Along with the question of ‘which integral of X best represents our ABR goals?’, Uecker & Bonhoeffer also raise the question of whether or not integrals of X are the best starting point for any evaluation criteria, referencing the three competing goals of ‘disease prevalence, mortality rate, length of hospitalization’ ([41], p. 13), after all, the most obvious clinical goal of a hospital is to minimize mortality, and assist in patients' speedy recovery. Maximizing the number of patients in the ward in the uninfected class is not (at face value) the same as minimizing the number of fatalities. In Bonhoeffer’s original investigation [32] infection compartments represented the number of infections in the community—in this context maximizing the uninfected population is a fairly direct ‘maximization of health’. However, later adaptions of the model [33] instead imagine each compartment as populations within a hospital, with immigration and discharge rates back into the community. The exposed class in these models represents patients who have no major infection, but remain in the hospital for other unrelated reasons (for example, post-operative care). In this context, it is not obvious that maximizing X should be our primary goal; instead a hospital director might want to minimize fatalities, or minimize the average number of patients (a rough proxy for the burden on the health system, and the amount of time patients spend in hospital).

In order to study alternative evaluation metrics, we extend the model in two ways; firstly, we separate emigration from the system into two streams: death and emigration. Secondly, we allow the death/emigration rate to vary between infected and uninfected individuals. For the time being, all infections are assumed to have the same death/emigration rates, regardless of the ABR of any given compartment. This leads us to replace the exit rate μ with four parameters: d, d_X, e and e_X; that is to say an infected and uninfected death rate, and an infected and uninfected emigration rate (via hospital discharge).

The total death rate at any given time is given by

$$D=d_{X}X+d(S+R_A+R_B+R_{AB}).$$ A 1

This total population is given by

$$\sigma =X+S+R_A+R_B+R_{AB}.$$ A 2

Noting that the total number of patients within the hospital is conserved, but for immigration, death and discharge, we find

$$\dot{\sigma }=\sum m_i-(d+e)\sigma -(d_X+e_X-d-e)X.$$

Integrating the above over an entire cycle (when cycling drugs) or solving for steady states (for mixing and combination therapy) gives us a relationship between the mean uninfected population and mean total population

$$0=\sum m_i-(d+e)\overline{\sigma }-(d_X+e_X-d-e)\overline{X}$$ A 3

and

$$\overline{\sigma }=\frac{\sum m_i}{(d+e)}-\frac{(d_X+e_X-d-e)}{(d+e)}\overline{X}.$$ A 4

Combining equations (A 1), (A 2) and (A 4)

$$\overline{D}=(d_X-d)\overline{X}+d\overline{\sigma }=\frac{\sum d{m}_i}{(d+e)}-\frac{(d{e}_X-e{d}_X)}{(d+e)}\overline{X}.$$ A 5

For any reasonable illness, where e_X > e, d_X < d, the problem of minimizing death is equivalent to maximizing $\overline{X}$ (the two have a negative linear relationship). The relationship between $\overline{\sigma }$ and $\overline{X}$ will be positive or negative depending on the sign of (d + e − d_X − e_X). When d − d_X < e_X − e, improvements in discharge rate are larger than changes to mortality rate and minimizing hospital load is equivalent to maximizing $\overline{X}$. When d − d_X > e_X − e, the ‘best’ strategy for minimizing hospital load is to maximize the spread of infection; however, in this case, hospital load is reduced purely through patient fatality. In such a case, maximizing $\overline{X}$ would still appear preferable according to any reasonable medical or ethical standard.

It is also worth noting that the various asymptotic results throughout this paper are determined by ${\dot{R}}_A$, ${\dot{R}}_B$ and independent of σ and d_X, e_X, hence they will still apply even when d_X ≠ d, e_X ≠ e. The one exception is equation (B 12), a quadratic in S and σ; given the linear relationship between σ and X, this quadratic can be easily adapted for variable exit rates.

Hence maximizing $\overline{X}$ is an optimal strategy, both for minimizing mortality, and patient recovery time. While not exactly a revelation, it is reassuring to know that these goals are well aligned and in some sense ‘equivalent’ to one another; both downstream goals are not only monotone in $\overline{X}$ but also linear. These results may be more complicated in situations where death or discharge rates vary between different ABR classes, or if we are dealing with multiple different infections of varying severity, but based on what we have seen so far, all reasonable measures of success will generally point in the same direction.

Appendix B. Analytic approximations of $\overline{X}$ for various protocols

In this section, we give derivations for the various $\overline{X}$ approximations of the main text, that is to say equations (3.4), (3.7), (3.8) and (3.11). For each of these approximations, we describe the underlying assumptions, where the approximation is applicable, and how it can fail.

B.1. Combination therapy

In the case of combination therapy, all infected compartments are subject to at least one effective antibiotic. There exists a single stable equilibrium, which can be found numerically. Under the assumption that A and B are effective at keeping infection down (τ + μ) ≫ β_i X for each β_i, and all infected compartments can be well approximated by the balance between immigration and recovery: S ≈ m_S/(τ + μ + γ), R_A ≈ m_A/(τ + μ + γ), R_B ≈ m_B/(τ + μ + γ). The total population of the ward is given by $\sigma =\sum m_i/\gamma$, and hence the healthy population is

$$X\approx \sum m_i/\mu -(m_s+m_A+m_B)/(\tau +\gamma +\mu ).$$

This gives equation (3.4) of the main text.

B.2. Mixing

Antibiotic mixing is best represent by the ‘7-box’ model, as proposed by by Uecker & Bonhoeffer [43] (see §2 for details).

As in the case of combination therapy, the equilibrium state for mixing can be found by solving $\dot{S}={\dot{R}}_A^B=\cdots =0$ numerically. Analytic solutions can be found by first determining what fraction of R_A and R_B are being treated effectively. This can be done by first noting

$${\chi }_B{\dot{R}}_B^A={\chi }_A{\dot{R}}_B^B=0(\mathrm{B}\hspace{0.17em}1)$$ B 1

and

$$\begin{array}{rl}& m_B{\chi }_B{\chi }_A+{\beta }_B(R_B^A+R_B^B)X{\chi }_A{\chi }_B=m_B{\chi }_B{\chi }_A+{\beta }_B(R_B^A+R_B^B)X{\chi }_A{\chi }_B\\ & -\mu R_B^A{\chi }_B-(\gamma +\tau )R_B^A{\chi }_B+q{R}_B^B{\chi }_B-\mu R_B^B{\chi }_A-\gamma R_B^B{\chi }_A-q{R}_B^B{\chi }_A,\end{array}$$ B 2

cancelling common terms (the top row) gives

$$-\mu R_B^A{\chi }_B-(\gamma +\tau )R_B^A{\chi }_B=-\mu R_B^B{\chi }_A-\gamma R_B^B{\chi }_A-q{R}_B^B({\chi }_A+{\chi }_B)$$ B 3

$$\frac{R_B^A}{R_B^B}=\frac{(\mu +\gamma ){\chi }_A+q}{(\mu +\gamma +\tau ){\chi }_B}$$ B 4

$$\mathrm{and}\frac{R_B^A}{R_B^B+R_B^A}=\frac{(\mu +\gamma ){\chi }_A+q}{\tau {\chi }_B+\mu +\gamma +q}.(\mathrm{B}\hspace{0.17em}5)$$ B 5

Let us call this ratio ${\rho }_B=R_B^A/(R_B^B+R_B^A)$, the effective treatment ratio. A similar ratio applies for A, ${\rho }_A=R_A^B/(R_A^A+R_A^B)$. Equipped with an analytic expression for how often R_B is treated effectively, we can now determine for what values of X R_B can be stable

$${\dot{R}}_B^A+{\dot{R}}_B^B=m_B+{\beta }_B(R_B^A+R_B^B)X-(\mu +\gamma )(R_B^A+R_B^B)-\tau R_B^A$$ B 6

$$0=\frac{m_b}{R_B^A+R_B^B}+{\beta }_{B}X-(\mu +\gamma )-\tau {\rho }_B$$ B 7

$$\mathrm{and}X\le \frac{\mu +\gamma +\tau {\rho }_B}{{\beta }_B}.(\mathrm{B}\hspace{0.17em}8)$$ B 8

In the case where $m_b\ll {\beta }_B(R_B^A+R_B^B)$ (resistance importation is rare), the inequality will end up being a reasonably tight estimate of X, which can in turn be used to estimate $S,R_A^B+R_A^A$, and finally (via population conservation) $R_B^B+R_B^A$. A similar X inequality can be found, as dictated by ${\dot{R}}_A^A+{\dot{R}}_A^B$ (equation (3.7a)). Finally, a third ‘successful treatment’ upper bound can be found by assuming all resistant infections are rare ($R_A^A,R_A^B,R_B^A,R_B^B\ll 1$), and X + S make up the vast majority of the population: $X+S\approx \sigma =\sum m_i/\mu$. Here, σ represents the total hospital population at equilibrium. For the particular model considered here, σ is independent of time and AB protocol.

The S dominated equilibrium can be found via the quadratic formula

$$\dot{S}=0\approx m_S+{\beta }_S(\sigma -S)S-(\gamma +\tau +\mu )S$$ B 9

$$S\approx \frac{(\gamma +\tau +\mu -\sigma {\beta }_S)\pm \sqrt{{(\gamma +\tau +\mu -\sigma {\beta }_S)}^2-4{\beta }_S{m}_S}}{2{\beta }_S}$$ B 10

$$\ge \frac{m_S}{\gamma +\tau +\mu -\sigma {\beta }_S}$$ B 11

$$\mathrm{and}X\approx \sigma -S\le \sum m_i\mu -\frac{m_S}{\gamma +\tau +\mu -\sigma {\beta }_S}.$$ B 12

Taken together these three upper bounds (R_A, R_B or S dominated equilibria, equations (3.7a), (3.7b), (3.7c)) constrain the long-term equilibria $\overline{X}$. Diagrams comparing the exact equilibrium (found numerically) to the approximations above for a variety of χ and q values are given in figure 5. As can be seen, all approximations are highly accurate within their domain of applicability. Near the boundary of these regions, where multiple constraints are approximately equal, multiple different strains of infection have non-trivial contributions, and X is correspondingly reduced, leading to a smooth transition.

B.3. Cycling

Let us now turn our attention to cycling protocols. Cycling protocols take one of two major forms: ‘fast’ cycling, in which the system is never allowed to reach equilibrium, and ‘slow’ cycling, in which the system reaches equilibrium with each cycle. Typical trajectories of such cycling protocols are given in figure 4c,d of the main text. Each dynamic regime allows for different simplifying assumptions and gives rise to different approximations of $\overline{X}$ (equations (3.8)).

In the main text, we explore cycling protocols in the context of symmetric parameter values β_A = β_B, m_A = m_B and cycle times T_A = T_B. Here, for the sake of generality, we consider the alternative case, where these parameter values are not assumed to be equal. The symmetric case considered in the main text follows directly as a special case. In what follows, T_A and T_B indicate the amount of time during a cycle spent administering drug A and B, respectively (total cycle time T_A + T_B). This gives

$${\chi }_A(t)\{\begin{array}{cc}1& \text{mod}(t,T_A+T_B)<T_A\\ 0& \text{mod}(t,T_A+T_B)\ge T_A\end{array}$$

with χ_B(t) = 1 − χ_A(t).

Let us first consider the case of ‘fast’ cycling. For fast cycling neither R_A nor R_B approach their equilibrium values and we can integrate over a full A/B drug cycle to find

$${\int }_0^{T_A+T_B}\frac{{\dot{R}}_A}{R_A}\hspace{0.17em}\mathrm{d}t={\int }_0^{T_A+T_B}\frac{m_A}{R_A}\hspace{0.17em}\mathrm{d}t-((T_A+T_B)(\mu +\gamma )+T_B\tau )+{\beta }_A{\int }_0^{T_A+T_B}X\hspace{0.17em}\mathrm{d}t.$$ B 13

By periodicity log (R_A(2T)) = log (R_A(0)), and hence

$$\overline{X}\le {\overline{X}}_{\mathrm{fast}}^A=[\mu +\gamma +\frac{T_B}{T_A+T_B}\tau ]/{\beta }_A.$$ B 14

Similarly, integrating ${\dot{R}}_B$ over a full cycle gives rise to

$$\overline{X}\le {\overline{X}}_{\mathrm{fast}}^B=[\mu +\gamma +\frac{T_A}{T_A+T_B}\tau ]/{\beta }_B.$$ B 15

Much like the mixing case, one of these two bounds will be smaller; the associated strain (either R_A or R_B) will maintain a significant population at all times, while the other strain is pushed to low-population levels maintained only by immigration (figure 9a). Because m_i ≪ R_i at all times for the dominant strain, ${\int }_0^{T_A+T_B}(m_A/R_A)\hspace{0.17em}\mathrm{d}t\approx 0$ and the $\overline{X}\approx {\overline{X}}_{\mathrm{fast}}^{A/B}$. In the symmetric case with β_A = β_B, m_A = m_B, T_A = T_B, we recover equation (3.8a).

Figure 9.

Asymmetric cycling with asymmetric infectivity parameters. (a) when ${\overline{X}}_A>{\overline{X}}_B$, $\overline{X}$ is forced close to ${\overline{X}}_B$ and the X population is not large enough to sustain the R_A population, leading the population to decline with each cycle. (b) When ${\overline{X}}_A\approx {\overline{X}}_B$ both resistant strains coexist; while ${\overline{X}}_A$ is fractionally lower than ${\overline{X}}_B$, decay is slow, and influx via m_A, m_B stabilizes both populations away from zero. Numerical artefacts and the peculiarities of the initial conditions are enough to cause minor drift with each cycle.

Let us now consider the ‘slow’ cycling case. This case is more complicated, because both R_A and R_B dip low enough such that m_A/R_A and m_B/R_B can no longer be ignored but also simpler, because the system returns to equilibrium with each cycle before the new drug regime can begin. In general, X spikes for a brief period following the introduction of each new antibiotic, and then quickly returns to some equilibrium level, $\check{X}$, which may depend on the antibiotic being used (denoted ${\check{X}}^A,{\check{X}}^B$). ${\int }_0^{T_A}X\hspace{0.17em}\mathrm{d}t$ is thus equal to $T_A{\check{X}}^A+C^A$, where C^A represents the area beneath the spike and above the equilibrium when switching from drug B to drug A (figure 10). A similar integral applies when administering drug B. In the symmetric case, we refer to equilibrium values, $\check{X}$, and spike integral, C, without superscripts.

Figure 10.

Schematic depiction of $\int X\hspace{0.17em}\mathrm{d}t$ over one antibiotic cycle. The integral can be broken into two components; one that accounts for the equilibrium X value, and the other that accounts for the excess X levels immediately after switching.

We begin by determining the relevant equilibrium in each part of the cycle. Let $\sigma =\sum m_i/\mu =S+R_A+R_B+X$. The equilibrium level of X while drug A is applied is ${\check{X}}^A=(\mu +\gamma )/{\beta }_A$, and similarly ${\check{X}}^B=(\mu +\gamma )/{\beta }_B$. These two equilibria determine the equilibria of the remaining populations, namely

$${\check{S}}^A=m_s/(\mu +\gamma +\tau -{\beta }_S{\check{X}}^A),{\check{S}}^B=m_s/(\mu +\gamma +\tau -{\beta }_S{\check{X}}^B),$$ B 16

$${\check{R}}_B^A=m_B/(\mu +\gamma +\tau -{\beta }_B{\check{X}}^A),{\check{R}}_A^B=m_A/(\mu +\gamma +\tau -{\beta }_A{\check{X}}^B)$$ B 17

$${\hat{R}}_A^A=\sigma -{\check{S}}^A-{\check{X}}^A-{\check{R}}_B^A,{\hat{R}}_B^B=\sigma -{\check{S}}^B-{\check{X}}^B-{\check{R}}_A^B.$$ B 18

Note here that we use notation in a different way to the 7-box model previously discussed in the main text. While superscripts once again represent the antibiotic currently being applied, it is assumed that a switch in the hospital's drug protocol will shift the entire population (not only freshly incoming patients, as in the 7-box model).

In order to calculate $\overline{X}$ over the time period [0, T_A + T_B], we break the integral into two parts, and make use of the $\dot{R}$ equation for the resistant strain in each part, hence

$${\dot{R}}_A=m_A+{\beta }_{A}X{R}_A-(\mu +\gamma )R_A,$$ B 19

dividing through by R_A and rearranging to make X the subject,

$${\beta }_A{\int }_0^{T_A}X\hspace{0.17em}\mathrm{d}t=(\mu +\gamma )T_A-{\int }_0^{T_A}\frac{{\dot{R}}_A}{R_A}\hspace{0.17em}\mathrm{d}t+{\int }_0^{T_A}\frac{m_A}{R_A}\hspace{0.17em}\mathrm{d}t.$$ B 20

Each of the terms on the right-hand side of the above can be approximated. ${\int }_0^{T_A}(m_A/R_A)\hspace{0.17em}\mathrm{d}t$ corresponds (after some rearrangement) with ${\check{X}}^A$. Simple integration tells us that ${\int }_0^{T_A}\frac{{\dot{R}}_A}{R_A}={[\log (R_A)]}_0^{T_A}\hspace{0.17em}\mathrm{d}t\approx \log ({\hat{R}}_A^A)-\log ({\check{R}}_A^B)$, leaving only ${\int }_0^{T_A}(m_A/R_A)\hspace{0.17em}\mathrm{d}t$.

Directly before drug switching $R_A={\check{R}}_A^B$. Directly after switching drugs, we have ${\dot{R}}_A=\tau R_A$; all remaining terms sum to zero, as otherwise the pre-treatment equilibria would not be an equilibria. Hence, immediately following drug switching $m_A/R_A\approx (m_A/{\check{R}}_A^B)\hspace{0.17em}{\mathrm{e}}^{-\tau t}$. See figure 11 for a comparison of this approximation to numeric solutions of the full ODE system. Straightforward integration gives ${\int }_0^{T_A}(m_A/R_A)\hspace{0.17em}\mathrm{d}t\approx m_A/\check{\tau }{R}_A^B$.

Figure 11.

Trajectories of R_A in a number of different coordinate regimes. Drug A is administered from time 0 to t = 25, while drug B is administered within [−25, 0) and [25, 50). (a) R_A in the original coordinates. (b) m/R_A, the function we must integrate in order to determine $\int X\hspace{0.17em}\mathrm{d}t$. The approximation τe^−τt is given by the black line. The shape of m/R_A in the region t = [T, 2T) follows a clean ‘logistic-like’ curve. (c) A closer examination of (b), comparing m_A/R_A (thick blue line) with the approximation τe^−τt (thin black line). (d) the R_A trajectory in logarithmic coordinates. As can be seen in both (a) and (d), in the time window (25,50], when drug B is administered, the decay trajectory of R_A towards m/τ is rather complicated; by contrast, when drug A is administered m/R_A follows an approximately exponential curve of the form τ e^−τt before converging to ${\hat{R}}_A$ (see the smooth exponential curve in (b) and linear increase in (d)).

Taken together, these elements give

$${\int }_0^{T_A}X\hspace{0.17em}\mathrm{d}t=\frac{\mu +\gamma }{{\beta }_A}{T}_A+\frac{[\log ({\check{R}}_A^B)-\log ({\hat{R}}_A^A)+(m_A/\check{\tau }{R}_A^B)]}{{\beta }_A}$$ B 21

and

$${\int }_0^{T_A}X\hspace{0.17em}\mathrm{d}t={\check{X}}^A{T}_A+C^A.$$ B 22

And similarly, it can be shown

$${\int }_{T_A}^{T_A+T_B}X\hspace{0.17em}\mathrm{d}t=\frac{\mu +\gamma }{{\beta }_B}{T}_B+\frac{[\log ({\check{R}}_B^A)-\log ({\hat{R}}_B^B)+(m_B/\check{\tau }{R}_B^A)]}{{\beta }_B}$$ B 23

and

$${\int }_{T_A}^{T_A+T_B}X\hspace{0.17em}\mathrm{d}t={\check{X}}^B{T}_B+C^B.$$ B 24

In the symmetric case m_A = m_B, β_A = β_B, T_A = T_B, we find $m_A/\check{\tau }{R}_A^B=1$ and ${\int }_{T_A}^{T_A+T_B}X\hspace{0.17em}\mathrm{d}t={\int }_0^{T_A}X\hspace{0.17em}\mathrm{d}t$. Substituting in we recover equation (3.8b).

B.4. Equilibrium post multi-resistance, and construction of X_T*

Suppose we wish to determine ${\overline{X}}_{AB}$, the long-term average number of uninfected individuals, in the presence of multiresistant bacteria. When using cycling therapy, we average ${\overline{X}}_{AB}$ over a complete cycle. If using mixing or combination protocols, ${\overline{X}}_{AB}$ is instead equal to the long-term steady-state value of X(t). We would like to determine this value under the assumption that R_AB > 0. While it is possible to find steady states in the case of mixing and combination therapy, solving equations (2.1) analytically for cycling protocols is not (generically) possible. Fortunately, it also proves unnecessary.

Consider equation (2.1d)

$${\dot{R}}_{AB}=m_{AB}-\mu R_{AB}+{\beta }_{AB}{R}_{AB}X-\gamma R_{AB}.$$ B 25

Assuming that multiresistant bacteria are rare in the community (m_AB ≪ 1) and mutation events producing R_AB strains are rare, it is possible to divide through by R_AB and integrate. This is the approach used by Bonhoeffer et al. [32] in their original paper, although here we will emphasize and make use of the results in a somewhat different manner

$${[\log (R_{AB})]}_{t_s}^{t_f}={\int }_{t_s}^{t_f}\frac{m_{AB}}{R_{AB}}\hspace{0.17em}\mathrm{d}t+{\beta }_{AB}{\int }_{t_s}^{t_f}X\hspace{0.17em}\mathrm{d}t-(\gamma +\mu )(t_f-t_s).$$ B 26

Both m_AB and R_AB are non-negative, hence $\int m_{AB}/R_{AB}\hspace{0.17em}\mathrm{d}t\ge 0$. ${[\log (R_{AB})]}_{t_s}^{t_f}$ is positive when R_AB is increasing, but R_AB is bounded above by the total population size, and hence, R_AB can not increase indefinitely. Hence, in the long run ${[\log (R_{AB})]}_{t_s}^{t_f}\le 0$ regardless of AB protocol.

Taken together, these facts imply

$${\int }_{t_s}^{t_f}\frac{X}{t_f-t_s}\hspace{0.17em}\mathrm{d}t={\overline{X}}_{AB}\le \frac{\gamma +\mu }{{\beta }_{AB}}.$$ B 27

The existence of this equilibrium was implied in [32], though never explicitly stated.

Strategies that would receive higher $\overline{X}$ in the absence of multiresistant infections will instead approach an equilibrium or cycle with mean value close to ${\overline{X}}_{AB}$. Protocols that would otherwise result in $X<{\overline{X}}_{AB}$ (such as withholding antibiotics entirely) will retain their low X value, and will result in the R_AB population decaying at a rate proportional to ${\overline{X}}_{AB}-X$.

With equation (B 26) in mind, it is possible (through some abuse of notation) to ‘integrate’ X from t = 0 to ∞

$${\int }_0^{\mathrm{\infty }}X\hspace{0.17em}\mathrm{d}t={\int }_0^{T_{ϵ}}X-{\overline{X}}_{AB}\hspace{0.17em}\mathrm{d}t+{\int }_{Tϵ}^{\mathrm{\infty }}X-{\overline{X}}_{AB}\hspace{0.17em}\mathrm{d}t+{\int }_0^{\mathrm{\infty }}{\overline{X}}_{AB}\hspace{0.17em}\mathrm{d}t$$ B 28

$$=X_{T\ast }+{[\log (R_{AB})]}_{T_{ϵ}}^{\mathrm{\infty }}+{\int }_{T_{ϵ}}^{\mathrm{\infty }}\frac{m_{AB}}{R_{AB}}\hspace{0.17em}\mathrm{d}t+\mathrm{\infty }.$$ B 29

While infinite, the term ${\int }_0^{\mathrm{\infty }}{\overline{X}}_{AB}\hspace{0.17em}\mathrm{d}t$ is entirely independent of our antibiotic management regime, and hence can be ignored from the perspective of optimization (its derivatives are zero). Assuming a hospital does not pursue an elimination strategy for multiresistance, R_AB will eventually approach $R_{AB}\approx \sigma -{\overline{X}}_{AB}$, hence the log term equation (B29) can also be considered constant (or approximately so). Finally, m_AB/R_AB is approximately zero whenever m_AB ≪ R_AB. Hence, optimization over all time will closely resemble optimization over the far more reasonable timespan $[0,T_{ϵ}]$.

It is worth discussing the similarities between X_T* and the optimization criteria G_1/2, as originally proposed by Bonhoeffer et al. [32]. G_1/2 is defined as ‘the gain in uninfecteds before 50% of infecteds are AB-resistant’, where ‘gain’ is defined as the gain relative to the no antibiotic use equilibrium. Stated in the notation of the current work, this would be written as

$$G_{1/2}={\int }_0^{T_{1/2}}X-{\overline{X}}_{\varnothing }\hspace{0.17em}\mathrm{d}t={\int }_0^{T_{1/2}}X-\frac{\gamma +\mu }{{\beta }_S}\hspace{0.17em}\mathrm{d}t,$$ B 30

where here ${\overline{X}}_{\varnothing }$ represents the ‘no antibiotic treatment’ equilibrium.

This constant term $-{\overline{X}}_{\varnothing }$ is frequently dropped in works following up from Bonhoeffer et al.—an approach which is valid when considering integrals with a fixed endpoint such as X₃₆₅, for which $\int {\overline{X}}_{\varnothing }\hspace{0.17em}\mathrm{d}t$ is itself constant, but significantly alters the optimization criteria for any integral with variable end point, such as G_1/2 and X_T (which are ‘identical’, but for the offset term).

X_T* differs from G_1/2 in two major ways. First, it uses a different ‘offset’ term: this change, while numerically minor, is also fairly critical for the sake of physically meaningful results. Whenever β_AB < β_S there will exist AB management strategies satisfying $(\gamma +\mu )/{\beta }_S<\overline{X}<(\gamma +\mu )/{\beta }_{AB}$ such that T_1/2 = −∞, leading to G_1/2 = ∞. Any protocol designed to achieve this intermediate X will be strictly worse than dealing with multiresistant infections directly, and yet this is precisely the strategy that G_1/2 will optimize towards. Note that no such degenerate results occur for offset terms slightly higher than ${\overline{X}}_{AB}$.

The second difference between G_1/2 and X_T* is the choice of integral endpoint: T_1/2 in the former case and $T_{ϵ}$ in the later. This is done for two reasons: firstly, $T_{ϵ}$ is easier to deal with analytically; it depends only on mutation rate, unlike T_1/2 which depends on both mutation and spread of R_AB. Secondly, $T_{ϵ}$ occurs precisely when R_AB = ε, as opposed to when R_AB = S + R_A + R_B as in the T_1/2 case. This means that the neglected term ${[\log R_{AB}]}_{T_{ϵ}}^{\mathrm{\infty }}$ is constant. By contrast, ${[\log R_{AB}]}_{T_{1/2}}^{\mathrm{\infty }}$ will vary between protocols, leading to a non-constant offset between ${\int }_0^{\mathrm{\infty }}X\hspace{0.17em}\mathrm{d}t$ and ${\int }_0^{T_{1/2}}\overline{X}-{\overline{X}}_{AB}\hspace{0.17em}\mathrm{d}t$. With that said, both these advantages are theoretical, and T_1/2 may well be the more appropriate end point when dealing with clinical trials. In general, we do not anticipate $T_{ϵ}$ or T_1/2 giving radically different clinical recommendations.

Appendix C. Stability of X_T* optima to variation in parameter values

In order for X_T* to provide reliable recommendations, we might require that it not be too sensitive to model specifications (5- versus 7-box model) and parameter values—particularly those parameter values which are hard (or impossible) to measure. Unlike previous optimization criteria, correct evaluation of X_T* requires knowledge of μ, γ and β_AB. While μ and γ can be reasonably calculated in advance, β_AB, the infectiousness of multiresistant bacteria can not be measured in advance. It depends not only on the reproductive cost of resistance, intrinsic to the bacteria itself but may also be influenced by medical interventions, such as quarantine, that may artificially influence infection rates. Figure 12 demonstrates how optimal cycle times vary depending on β_AB; optimal cycle times are relatively stable across a range of β_AB values, but notably have sudden jumps, especially near β_AB → β_A = β_B. Fortunately, these jumps correspond to regions of parameter space where many T values give almost optimal results. While picking truly optimal T becomes more difficult, selecting almost optimal T is relatively easy.

Figure 12.

Optimal cycle times. Coloured lines give optimal cycling protocols for each mutant arrival channel, with T = 0 equivalent to a 50 : 50 mixing protocol. The horizontal black line represents t_sat, while the dashed vertical line represents β_A = β_B; β_AB values above this line represent multiresistant strains which are more infectious than their single resistant counterparts. As can be seen, optimal cycling strategies are broadly stable over a range of β_AB values, with sudden changes in optimal strategy in the vicinity of various ‘tipping points’, for example β_AB ≈ 0.95 when using M_base in the 5-box system.

Another important and less easily determined variable is the ABR importation rates m_A and m_B. Throughout this paper, these rates are assumed to be ‘low’, but how small, and how sensitive optimal switching times are to these parameters is a question of some interest, especially given our previous hypothesis that optimal arrival times scale with t_sat, a function of m_A, m_B.

In order to investigate this, we run numerical simulations and calculate $(\overline{X}-{\overline{X}}_{AB})/\overline{M}$ for a variety of m_A = m_B and T values, and then select the optimal T for each m (figure 13). For each m, we also calculate t_sat. Optimal T for both M_base and M_HGT are roughly parallel to t_sat over several orders of magnitude, with T_HGT ≈ t_sat + 30, and T_base ≈ t_sat − 10. Optimal T for M_select is found between these two values, though where it falls in this range varies depending on m. It seems likely that the offset between t_sat and optimal T will vary depending on other system parameters. We observe remarkable consistency between both 5- and 7-box models; it would appear that antibiotic ‘switching lag’ in the 7-box model has minimal impact on optimal cycling time. This is likely the result of ‘lag’ being significantly smaller than switching time T, and thus not playing into the overall dynamics, so long as T is not too close to 0.

Figure 13.

Optimal cycle times for a given importation rate. Coloured lines give optimal cycling protocols for each mutant arrival channel, with T = 0 equivalent to a 50:50 mixing protocol. The black line represents t_sat. As resistance importation rate increases, optimal cycling time decreases.

Appendix D. Further comparisons of optimization criteria

In the main text (§4), we compared four optimization criteria in the context of both mixing and cycling. In each case, we made simplifying assumptions—namely, in the mixing case, we assumed zero treatment switching, q = 0, and in the cycling case, we assumed symmetric cycling and symmetric parameter values. Here, we (briefly) consider the implications of relaxing these assumptions. In figure 14, we consider mixing, with a drug correction rate q = 1/6. As might be expected, the inclusion of drug switching improves outcomes as measured according to X₃₆₅, X_T*, but reduces T_1/2 and X_T (with the exception of X_T under the influence of M_HGT). Once again, we observe that for X₃₆₅, X_T*, a mixing ratio close to 50:50 is recommended in all cases except M_HGT; in this case, the optimal χ_A ratio remains only slightly offset for X₃₆₅, the maximum value for $X_T\ast$ moves from χ_A ≈ 0.25 to χ_A ≈ 0, that is to say (for the parameter values considered here), horizontal gene transfer is best minimized by having a single ‘front line’ treatment, and only switching the patient to the second treatment if the first does not work. This contrasts with the case of ‘no switching’, where both antibiotics needed to be applied with non-trivial frequency in order to get optimal outcomes.

Figure 14.

Comparison between optimization criteria assuming either q = 0 (thin lines) or q = 1/6 (thick, dashed lines). (a,b) The expected values of X₃₆₅ for high and low mutation/importation rate ν. Each line indicates a different channel of multiresistance arrival. (c,d) Expected values of T_1/2 and X_T. Note that unlike the q = 0 case, the q = 1/6 case does not allow infection time to tend to infinity. Optimal values are still found for the extreme values χ_A = 0, 1 (e) X_T* for a variety of T values. (f) X_T* rescaled such that max (X_T*) = 1. In all cases, drug-switching improves outcomes. With the exception of M_HGT, changing q has no impact on the position of the optimal χ.

In figure 15, we also consider a single case of ‘asymmetric cycling’. In this case, we assume that β_A = β_B, m_A = m_B and $T_A=\frac{3}{2}{T}_B$. In general, it is found the asymmetry reduces X₃₆₅ and X_T*, while increasing T_1/2 and X_T. The one exception being the case of multiresistance arrival via horizontal gene transfer; in this case, fast asymmetric cycling significantly improves X₃₆₅ and X_T* compared to the symmetric case.

Figure 15.

Here, we compare symmetric (thin lines) and asymmetric (thick, dashed) cycle times. In the asymmetric case, drug A is used for $60\text{\%}$ of the cycle time and drug B for $40\text{\%}$. (a,b) The expected values of X₃₆₅ for high and low mutation/importation rate ν. Asymmetry reduces X₃₆₅ in all cases except for M_HGT with short cycle time. If mutation rate is high enough, asymmetry can also provide some improvements for short cycle times for M_selct, though the maximum still remains with long cycle times. (c,d) Expected values of T_1/2 and X_T. In both cases, asymmetry slows development of antibiotic resistance. (e) X_T* for a variety of T values. (f) X_T* rescaled such that the maximum for symmetric cycling equals one. Asymmetric cycling for each M is scaled using the same scaling factor. Asymmetry significantly worsens results in all cases except M_HGT with short cycle time.

One optimization criteria not discussed in detail within the main text is the idea of some ‘exponentially discounted preferences’: $X_{\lambda }={\int }_0^{\mathrm{\infty }}X(t)e^{-\lambda t}\hspace{0.17em}\mathrm{d}t$, (as used in, for example [54]). Exponentially discounting is common in the literature of economics, and has many nice properties, including analytic tractability, and avoiding ‘sharp’ cut offs observed for $X_{365}={\int }_0^{365}X(t)\hspace{0.17em}\mathrm{d}t$, which gives full weight to patients on Day 364, and cares nothing for patients on Day 366.

Preliminary exploration suggests that $X_{\lambda }$ behaves in many ways like X₃₆₅; see figure 16. This makes sense, $X_{\lambda }$ can be approximated by a superposition of X_n, for many different values of n. It seems possible (but not likely) that the two metrics behave differently when using antibiotic cycling strategies. For the time being, such results remain analytically inaccessible and computationally expensive.

Figure 16.

Here, we use the 7-box model to compare ‘1 year’ optimization ($X_{365}={\int }_0^{365}X\hspace{0.17em}\mathrm{d}t$) with ‘exponentially discounted preferences' ($X_{\lambda }={\int }_0^{\mathrm{\infty }}X(t)\hspace{0.17em}{\mathrm{e}}^{-\lambda t}\hspace{0.17em}\mathrm{d}t$) for both high and low mutation rates. We compare under the assumption that drug correction rate q = 0, and with λ = 2/365, such that both X₃₆₅ and $X_{\lambda }$ integrals have the same ‘centre of mass’. We consider both higher and lower mutation rate. Each line indicates a different channel of multiresistance arrival. We observe that $X_{\lambda }$ is qualitatively similar to X₃₆₅, all be it with a angled ‘peak’ in the middle rather than a flat ‘plateau’.

Data accessibility

All scripts used in this study are openly accessible at https://github.com/alastair-JL/ABRcycling.

Authors' contributions

A.D.-J.: conceptualization, formal analysis, investigation, methodology, software, visualization, writing original draft, writing review and editing; A.F.: funding acquisition, supervision, writing review and editing; B.B.: funding acquisition, project administration, supervision, writing review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Competing interests

We declare we have no competing interests.

Funding

A.J-L. acknowledges funding by the School of Medicine and HealthScience, Carl von Ossietzky Universität Oldenburg, https://uol.de/en/medicine. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. B.B. received no specific funding for this work.