Identification of antibiotic collateral sensitivity and resistance interactions in population surveillance data

Abstract

Background

Collateral effects of antibiotic resistance occur when resistance to one antibiotic agent leads to increased resistance or increased sensitivity to a second agent, known respectively as collateral resistance (CR) and collateral sensitivity (CS). Collateral effects are relevant to limit impact of antibiotic resistance in design of antibiotic treatments. However, methods to detect antibiotic collateral effects in clinical population surveillance data of antibiotic resistance are lacking.

Objectives

To develop a methodology to quantify collateral effect directionality and effect size from large-scale antimicrobial resistance population surveillance data.

Methods

We propose a methodology to quantify and test collateral effects in clinical surveillance data based on a conditional t-test. Our methodology was evaluated using MIC data for 419 Escherichia coli strains, containing MIC data for 20 antibiotics, which were obtained from the Pathosystems Resource Integration Center (PATRIC) database.

Results

We demonstrate that the proposed approach identifies several antibiotic combinations that show symmetrical or non-symmetrical CR and CS. For several of these combinations, collateral effects were previously confirmed in experimental studies. We furthermore provide insight into the power of our method for multiple collateral effect sizes and MIC distributions.

Conclusions

Our proposed approach is of relevance as a tool for analysis of large-scale population surveillance studies to provide broad systematic identification of collateral effects related to antibiotic resistance, and is made available to the community as an R package. This method can help mapping CS and CR, which could guide combination therapy and prescribing in the future.

Introduction

The treatment of bacterial infections increasingly relies on antibiotic combination therapy.¹ Although physiological interactions, i.e. synergy and antagonism, between pairs of antibiotics have been explored and exploited for such combination therapies,² evolutionary interactions resulting in collateral effects have only recently started to attract attention.³ Negative evolutionary interactions between antibiotics, known as collateral sensitivity (CS), occur when the emergence of resistance to an antibiotic is accompanied by increased sensitivity to a second antibiotic. On the contrary, positive evolutionary interactions, known as collateral resistance (CR), result in increased resistance to the second antibiotic.⁴

The broad systematic identification of CR can be clinically important to avoid evolutionary unfavourable antibiotic combinations in empirical treatment,⁵ whereas CS can enable the design of antibiotic combination treatment strategies to suppress resistance.^6,7 Although perturbations of gene expression networks that subsequently affect the vulnerability of bacterial cells to chemicals have been proposed as the main mechanism underlying CS, the mechanistic details of CS remain elusive.⁴ For antibiotic pairs that show a CS relationship we may see either a unidirectional or reciprocal relationship, where the latter is most suitable to design such resistance suppressing cycling strategies.^6,8–11 CS has been primarily identified and studied in controlled experimental evolution studies, mostly utilizing laboratory strains.^12,13

In the experimental setting, collateral effects, and CS in particular, are determined by measuring the MIC against multiple antibiotics before and after desensitization, i.e. through development of resistance to a chosen antibiotic by experimental evolution. The fold change in MIC, the ratio of the MIC after and before desensitization, is then used to quantify a collateral response.

The clinical relevance of CS effects remains unclear, due to a lack of studies that characterize collateral effects and CS in particular in clinically isolated bacterial pathogens. Unlike experimentally evolved laboratory strains, clinical bacterial isolates are associated with extensive genetic variability, and a parental wild-type strain is lacking to readily determine collateral effects such as is done experimentally.¹⁴

Increasing availability of large-scale clinical antimicrobial susceptibility surveillance data^15,16 may offer an opportunity to address this knowledge gap. Such datasets include MIC values for commonly used antibiotics in clinically isolated pathogens. Recently, it was shown how dichotomous resistance values, i.e. a classification of sensitivity or resistance, can be used to estimate collateral effects of antibiotics in clinical population data.¹⁷ However, in order to address questions about the therapeutic relevance of CS, it is of specific importance to be able to infer directionality and effect size of collateral effects from available clinical MIC data, which is not possible when only dichotomous MIC values are considered.

Here, we propose a methodology to systematically identify and quantify collateral effects from clinical MIC surveillance data, by comparing two MIC distributions conditional on the resistance to an antibiotic. Specifically, we develop a goodness-of-fit measure for estimating a non-causal collateral effect using the easily interpretable conditional t-test, which allows quantification of collateral effect directionality and effect size (Figure 1). We apply our method to a large public dataset with MIC measurements for multiple antibiotics in clinical Escherichia coli isolates to identify possible collateral effects.

Figure 1.

Collateral effects in the biological process and in population-based inference. Left panel: MICs from the parental strain change due to desensitization to antibiotic B. The MIC for B increases; the effect on the MIC for A indicates whether B has a collateral effect on antibiotic A. Right panel: The parental strain is unknown, so population of strains is divided into a high MIC(B) and low MIC(B) group; instead of MICs for individual strains, the MIC graphs show histograms of strains. Collateral effects are measured by comparing the conditional distribution for high and low MIC for antibiotic B.

Methods

Data pre-processing

Antibiotic MIC data of clinically isolated E. coli strains were obtained from the Pathosystems Resource Integration Center (PATRIC)¹⁵ using the command line interface, resulting in a dataset of 60 antibiotics and 495 strains. Antibiotics with MIC measurements for at least 200 strains were included in the study, resulting in 20 antibiotics (Table S1, available as Supplementary data at JAC-AMR Online). Only strains with data for two or more antibiotics were used for further analysis, resulting in 419 eligible strains (Figure 2a).

Figure 2.

Data pre-processing steps (a) and data analysis strategy (b) schematic overview.

The MICs were measured on a 2-fold concentration scale. The data can be considered discrete, with only 26 unique values and 95% of the measurements falling within 9 of these values. The MICs were log₂ transformed [log₂(MIC)] to put them on a linear scale.

Collateral effect identification

To reflect the biological process of collateral effects, a pair of antibiotics (A and B) was tested by splitting the population of strains in two groups, one with high and one with low MIC for antibiotic B (Figure 1). For computing the collateral effects, only complete pairwise observations were considered (Figure 2b). Next, strains were dichotomized on a dichotomization criterion τ, based on their MIC value for antibiotic B. After dichotomization, the log₂ fold change (FC) was calculated as the difference between the mean log₂(MIC) for A given high MIC for B (MIC_A|B=high) and the mean log₂(MIC) for A given low MIC for B (MIC_A|B=low) (Equation 1).

$${\log }_2\mathrm{ }FC=\mathit{mean}({\log }_2\left({\mathit{MIC}}_{A|B=\mathit{high}}\right))-\mathit{mean}({\log }_2\left({\mathit{MIC}}_{A|B=\mathit{low}}\right)\mathrm{ })$$ (1)

where a log₂ FC > 0 indicates a CR effect (or an MDR phenotype due to the co-presence of distinct antibiotic resistance mechanisms) and a log₂ FC < 0 indicates a CS effect. Collateral effects between two antibiotics were tested in two directions: the collateral effect of B on A and the collateral effect of A on B. To test the fold change, the mean difference between the two groups was compared with an independent sample t-test on the log scale. Let

$${\mathit{\mu }}_{A|B=\mathit{high}}=\mathit{mean}\left({\log }_2\left({\mathit{MIC}}_{A|B=\mathit{high}}\right)\right),$$

then the tested hypotheses can be formulated as

$$H_0:{\mu }_{A|B=\mathit{high}}={\mu }_{A|B=\mathit{low}}$$

$$H_{CS}:{\mu }_{A|B=\mathit{high}}<{\mu }_{A|B=\mathit{low}}$$

$$H_{CR}:{\mu }_{A|B=\mathit{high}}>{\mu }_{A|B=\mathit{low}}$$

The grouping in B depends on the dichotomization criterion τ. We chose the τ in a way to make the two groups (high and low) most equally sized, to maximize the power of detecting collateral effects. In continuous data, this is naturally the median. The dichotomization does not depend on antibiotic A. To study the effect of different values of τ, we evaluated the results for multiple values. The number of options for τ was very limited due to the discrete nature of the MIC observations. Importantly, the MIC for antibiotic A should not be dichotomized, to preserve statistical power and retrieve continuous effect sizes. Between two sets of strains with either a high or low MIC for an antibiotic B, the MIC of antibiotic A is expected to be similar; when this is not the case, this might indicate a collateral effect.

The difference between the mean log₂(MIC) for all the combinations of the 20 antibiotics was tested in two directions, which resulted in 380 statistical tests. A two-sided t-test was used to test both CS and CR. The power, the probability of detecting true effects, was evaluated analytically over different effect sizes of collateral effects in MIC data based on sample sizes, disbalance between the sizes of the high and low MIC(B) groups, and the standard deviation. The values over which the power was calculated were between the ranges found in the extracted data from PATRIC.

The corresponding P values were calculated and adjusted by controlling the false discovery rate (FDR) with the Benjamini–Yekutieli procedure, which corrects for the dependency between the tests.¹⁸ The allowed FDR was set to 0.05. The difference between the means was visualized for the significant results in a heatmap. The distributions of the MICs with the most significant differences in mean for CR and CS were also visualized as histograms.

The data analysis was done in the statistical scripting language R,¹⁹ using the ggplot2²⁰ package for visualization. The code used in this study is available on github (https://github.com/vanhasseltlab/CollateralEffect_MICmethod). The functions for testing collateral effects and producing figures are made available in the R package collatRal (https://github.com/vanhasseltlab/collatRal).

Results

Detection of collateral effects

A total of 419 E. coli strains and 20 antibiotics was included in our analyses after exclusion of antibiotics with low sample size (Figure 2a, Table S1). We identified 14 CS responses and 178 potential CR responses at an FDR of 0.05 with dichotomization at the median (Figure 3). The top five largest and smallest t-statistics, i.e. the most significant collateral responses, are summarized in Table 1, which show a very low P value and FDR-adjusted P value (q value). The largest CS response is ertapenem on cefazolin, with a mean log₂ FC of −1.95 (Figure 4a), which corresponds to a fold change of 0.26. The opposite direction (Figure 4b), the effect of cefazolin on ertapenem, also shows a significant CS response, but with a smaller effect size (−0.86 log₂ FC, q < 0.05). Cefazolin was associated with multiple CS responses with different antibiotics. The most significant CR response is that of meropenem on ertapenem with a mean log₂ FC of 3.66 (Figure 4c), corresponding to a 12.6-fold change.

Figure 3.

Heatmap of collateral responses identified based on clinical surveillance MIC data, for different antibiotic combinations. The effect size (blue/orange colour) log₂ FC is the mean log₂(MIC) shift between the groups low and high MIC for splitting antibiotic B. On the y-axis the tested antibiotics that were split on antibiotic B (x-axis) and the bold lines separate the different antibiotic classes. A darker colour indicates a larger effect, with orange showing CR and blue showing CS. White squares indicate no significant effect at the 0.05 FDR threshold. Reciprocality is denoted by a two-sided arrow.

Table 1.

Top five results of collateral sensitivity and resistance responses in the E. coli data, with lowest and highest value for the t-statistic, respectively

Testing antibiotic (A)	Splitting antibiotic (B)	t-statistic	P value	nB=high	nB=low	Difference in mean (log2 FC)	q value
Collateral sensitivity
CFZ	ETP	−10.04	2.43 × 10−20	69	201	−1.95	2.41 × 10−18
TZP	CFZ	−6.13	4.09 × 10−9	113	110	−1.61	1.25 × 10−7
TZP	CRO	−5.58	5.67 × 10−8	130	156	−1.32	1.46 × 10−6
MEM	CFZ	−5.54	6.68 × 10−8	154	140	−1.23	1.69 × 10−6
CFZ	MEM	−5.13	5.29 × 10−7	64	230	−1.16	1.21 × 10−5
Collateral resistance
ETP	MEM	26.73	5.82 × 10−89	62	316	3.66	1.44 × 10−85
MEM	ETP	23.85	3.00 × 10−77	81	297	3.37	3.71 × 10−74
TOB	GEN	21.99	6.54 × 10−67	117	219	3.00	5.40 × 10−64
CFZ	CRO	21.09	6.61 × 10−60	137	147	2.60	4.09 × 10−57
GEN	TOB	19.74	5.05 × 10−58	146	190	2.82	2.50 × 10−55

The effect size is the difference in mean log₂ FC for the effect of splitting antibiotic B on testing antibiotic A. The number of observations in each group are n_B=high and n_B=low, respectively.

CFZ, cefazolin; ETP, ertapenem; TZP, piperacillin/tazobactam; CRO, ceftriaxone; MEM, meropenem; TOB, tobramycin; GEN, gentamicin.

Figure 4.

Log₂ transformed MIC [log₂(MIC)] distributions of the most significant CS effect (a) and the most significant CR effect (c). The frequency is shown on the y-axis for the different values of the log₂(MIC). The different colours denote high (dark) or low (light) MIC to the splitting antibiotic. The dashed lines show the estimated means for the two groups. (a) log₂(MIC) distribution of cefazolin (CFZ) split by MIC of ertapenem (ETP) and (b) the distribution of ertapenem under influence of cefazolin, which also has a significant CS response, showing a reciprocal effect. (c) Log₂(MIC) distribution of ertapenem under influence of meropenem, which has a significant CR response (3.66 log₂ FC) and (d) the distribution of meropenem under influence of ertapenem, which also has a similarly sized CR response (3.37 log₂ FC). This is a reciprocal collateral response.

We chose the median to define the dichotomization criterion, but due to the discrete nature of the data, many values are equal to the median. Including the strains with a median MIC value in the low or high MIC group was not arbitrary, since it can change the group sizes substantially. The median strains were included in the smallest group to make the sample sizes as equal as possible. For example, in Figure 4a, depicting the effect of ertapenem on cefazolin, the median of ertapenem was −1, and the strains with this median value were included in the B=low group to make the groups most equal in size [hence the high group contained log₂(MIC) ≥0]. This equal splitting was found to improve power (next section).

Power for identification of collateral effects

To understand which collateral effect sizes are detectable using our method, the power to detect collateral effects was calculated for different sample sizes, different group disbalances and different standard deviations, based on the values we found in the PATRIC data. The power to detect different effects greatly depends on the total sample size, with 2000 samples allowing the detection of a log₂ FC of size 0.25 with power over 80%, while for smaller sample sizes, such as 200 total samples, the detectable effect size is 0.75 (Figure 5a). A larger disbalance of the two groups, high MIC and low MIC for antibiotic B, which is often seen in our data, does affect the power, but substantially only if the disbalance is greater than 25/75 (Figure 5b). Finally, the standard deviation of the MIC values has a large effect on how large the power to detect an effect is (Figure 5c), with the effect of the standard deviation on the power being inversely proportional to the effect of the effect size.

Figure 5.

Properties of the collateral effects detection. (a–c) Power analysis for t-test of collateral effects as the mean log₂ FC. The power against effect size. (a) Power for different sample sizes (n), lighter colours show lower n. Power increases greatly with n. (b) Power for different disbalances in the group sample sizes. The most equal groups are divided 50/50, while 10/90 shows a large disbalance in sample sizes, which shows decreased power. (c) The power for different standard deviations of the MIC distributions.

The effect of the dichotomization criterion

For estimating the collateral effects, we dichotomize the sample based on the MIC of one of the two antibiotics, at a chosen criterion (τ). We studied whether this choice influences the results and found that the value of τ does impact our results (Figure 6). The group size equality (transparency of the lines) is a measure for how equal the two groups are, where an equality of 1 indicates a 50/50 split for the groups. The lines themselves show how the test statistics change over different values of τ, showing a dependence of collateral effect size and sometimes even direction (CR or CS) on MIC for some of the antibiotics.

Figure 6.

The effect of changing the dichotomization criterion (x-axis) on the estimated log₂ FC (y-axis). Every panel denotes the antibiotic that is dichotomized (antibiotic B). The colours show the different antibiotics that were tested based on this dichotomization (antibiotic A). The dots denote the value of the log₂ FC found in the main analysis, based on equal split dichotomization. A more transparent line shows a larger inequality in group sizes, where the effect estimation is driven by only a small part of the total sample. Straight horizontal lines denote stable effect size estimation over different dichotomization criteria, while crooked lines show unstable estimation.

The estimated collateral effect is very stable over dichotomization criteria for e.g. amoxicillin/clavulanic acid, piperacillin/tazobactam and tobramycin. Less stable are for example cefazolin and ciprofloxacin. For cefazolin however, the different effect sizes are found where the sample sizes of the group are very different, indicating that the effect is driven by a small part of the data. In case of ciprofloxacin, the change of estimated effect size over the different τ is not driven by a small part of the data.

Discussion

Our analysis demonstrates that empirical determination of collateral effects is possible from MIC population surveillance data, by quantifying shifts in conditional MIC distributions. Importantly, our approach enables detection of collateral responses including directionality and effect size. We demonstrated the utility of our method to a set of E. coli MIC data, identifying CS and potential CR responses from available clinical surveillance data.

Collateral effects are experimentally established to show directionality, that is, the effect size between two antibiotics in terms of their collateral effects is not symmetrical. Thus, a collateral effect of antibiotic A on antibiotic B can differ from the effect of B on antibiotic A. Our method can be employed to detect both one-directional and two-directional (reciprocal) collateral responses. This is in contrast to other statistical methods, such as the odds ratio used in a previous study¹⁷ or a correlation, where both directions yield the same statistic. A correlation will thus either identify two-directional responses or not detect a collateral effect at all, which increases both the number of false discoveries and the number of false rejections, as compared with our method.

The collateral effect metric proposed, using fold change, has a clear interpretation. The effect size is the mean difference in log₂(MIC)s, or log₂ FC, between a group with high MIC for an antibiotic B as compared with the group with a low MIC for the same antibiotic. A fold change value enables interpretation of the clinical relevance of detected MIC changes. This is of relevance, since, especially with large sample sizes, statistical significance does not always imply clinical relevance.²¹ In addition, the fold change measure is comparable to the experimentally determined fold change measures in experimental evolution studies that determine collateral effects.^6–9

The choice of dichotomization criterion is not arbitrary, since it can affect the effect size estimation, such as for ciprofloxacin (Figure 6). The type of the identified collateral effects may even vary depending on the selected dichotomization criterion, which is, for example, the case for ceftriaxone (A) on ciprofloxacin (B). For the choice of dichotomization criterion, three aspects need to be considered. Firstly, the data limit the possible choices of τ. There need to be enough data at both sides of the split to be able to use this test. Secondly, we showed that the power for detecting effects is highest when the two groups are similar in size. To this end, a dichotomization on the equal group sizes yields the highest power and we chose this option for the study. Thirdly, the change of the estimated effect sizes over different dichotomization criteria indicates a dependence between collateral effect and MIC value, which could have an underlying biological cause. The choice of τ could therefore also be decided based on which MIC value you want to know about collateral effects. For instance, in Figure 4a a CS response is shown in strains with a log₂ MIC for cefoxitin larger or equal to 0, indicating that the decrease in sensitivity for cefoxitin is correlated with an increase in sensitivity for cefazolin, above this threshold. Whether the same collateral effect is also detected at other dichotomization criteria can be tested by changing this value.

For our proof-of-concept analysis of our E. coli dataset we identified 192 collateral responses using the equal group splitting for the dichotomization, of which 14 were therapeutically interesting CS responses. Suggestive CR responses were extensively more prevalent than CS responses. This was in line with expectations, given the presence of antibiotics from the same class for which CR is likely to occur. For example, reciprocal CR responses were detected between the aminoglycosides amikacin, gentamicin and tobramycin, and the fluoroquinolones ciprofloxacin and levofloxacin. In accordance with previous studies showing CS between β-lactams,²² we identified reciprocal CS effects between cefazolin and the β-lactams meropenem and ertapenem, as well as between cefazolin and the β-lactam/β-lactamase inhibitor combinations amoxicillin/clavulanic acid and piperacillin/tazobactam. In contrast to the CS repeatedly found in the literature between ciprofloxacin and gentamycin,^7,9–11 our analysis showed a CR response for this antibiotic combination.

The MIC data deposited in PATRIC originate from a variety of studies and MIC determination methods, which leads to large variation and possible error in MIC values. Therefore, the data were used as proof of concept. Large datasets that include MICs determined in a more consistent manner may be able to further improve the performance of our method. In larger datasets, our method could also be used to identify multidrug effects, by splitting the high and low groups conditioned on more than one antibiotic.

This statistical method does not set out to identify causal relationships, so while use of an antibiotic may lead to resistance development, it is not possible to show that the CS or resistance to another antibiotic has been caused by the use of the first antibiotic. Also, due to the observational nature of the data, we are unable to discriminate between CR responses and the occurrence of an MDR phenotype due to the co-presence of distinct resistance mechanisms to each of the individual antibiotic agents.

The occurrence of CS has been previously suggested as a phenomenon that can be utilized to design dosing schedules that prevent emergence of antimicrobial resistance and prolong the efficacy of the existing antimicrobial agents,⁶ but so far it has been mostly studied in reference laboratory strains of a limited number of bacterial species. Although our study suggests the occurrence of CS in clinical populations, it remains unclear to what extent these effects actually occur in clinically occurring pathogens. The developed methodology can be directly applied to clinical datasets of antimicrobial susceptibility that are widely available via national and international surveillance programmes, to both estimate the effect sizes and occurrence of collateral effects and to provide further insight into the clinical relevance of CS effects. These effects can be evaluated for specific antibiotic combinations and pathogen species, which may guide the design of CS-based dosing strategies of high clinical relevance and the selection of empirical treatment.²³ In addition, the quantification of CR in antimicrobial susceptibility surveillance datasets is of interest to identify antibiotic combinations that should be avoided as these could potentially lead to increased risk of treatment failure and the spread of antimicrobial resistance.

We conclude that the proposed methodology is relevant for identification of collateral responses based on clinical surveillance data. We implemented the functions for the method in the R package collatRal to make the method accessible to other researchers. Our method can be applied to larger surveillance datasets that also include MIC data for additional antibiotics and for other clinically relevant bacterial species. Identified collateral effects, and in particular CS, can provide important guidance for combination therapy and in the further design of CS-based dosing strategies that aim to suppress antibiotic resistance.