Deep learning recognises antibiotic modes of action from brightfield images

Daniel Krentzel; Kelvin Kho; Julienne Petit; Nassim Mahtal; Thomas Delerue; Max E Huber; Agnès Zettor; Jeanne Chiaravalli; Spencer L Shorte; Mark Brönstrup; Anne Marie Wehenkel; Ivo G Boneca; Christophe Zimmer

doi:10.1038/s41467-026-76355-0

. 2026 Sep 8;17:10660. doi: 10.1038/s41467-026-76355-0

Deep learning recognises antibiotic modes of action from brightfield images

Daniel Krentzel ^1,^✉,^#, Kelvin Kho ^2,^#, Julienne Petit ^3,^4,^#, Nassim Mahtal ⁵, Thomas Delerue ², Max E Huber ⁶, Agnès Zettor ⁷, Jeanne Chiaravalli ⁷, Spencer L Shorte ⁵, Mark Brönstrup ⁶, Anne Marie Wehenkel ⁴, Ivo G Boneca ^2,^✉, Christophe Zimmer ^1,^8,^9,^✉

PMCID: PMC13655182 PMID: 42860392

Abstract

The antimicrobial resistance crisis urgently calls for antibiotics with novel modes of action (MoAs). While growth inhibition assays can identify antibiotic molecules, they miss promising compounds below inhibitory concentrations and cannot reveal their MoA. Imaging-based profiling of drug-treated bacteria can inform on MoA, but current approaches generally require fluorescent labelling and/or inhibitory concentrations and it remains unclear whether compounds with novel MoAs can be robustly detected. Here, we demonstrate a deep learning approach to recognise antibiotic MoAs from unlabelled images. We train a convolutional neural network to predict treatment conditions from brightfield images of Escherichia coli exposed to antibiotics covering eight MoAs. Our approach can detect drug exposure at subinhibitory concentrations and allows near-perfect MoA recognition, even when trained on only eight images per treatment condition. Previously unseen compounds are assigned to their correct MoA with good accuracy if the MoA is represented in the training data, and our method can robustly detect MoA novelty in five out of six considered MoAs, enabling microscopy-based identification of new antibiotic classes. We also achieve near-perfect MoA recognition in Klebsiella pneumoniae, suggesting applicability to other species. Our approach complements growth inhibition assays and is poised to improve the search for innovative antibiotic compounds.

Subject terms: Machine learning, Image processing, High-throughput screening, Phenotypic screening, Antibiotics

Authors present a deep learning model that recognises antibiotic modes of action (MoA) directly from brightfield images. Their proposed method complements traditional growth inhibition assays in the search for novel antibiotics.

Introduction

Antimicrobial resistance (AMR) caused 1.27 million deaths in 2019¹, with death tolls estimated to rise to as much as 10 million deaths a year by 2050². During the last 40 years, very few antibiotics with novel modes of action (MoAs) have been discovered. New resistance mechanisms of various pathogenic bacteria have thus remained unchecked, thereby rendering the existing arsenal of antibiotics increasingly ineffective³.

Phenotypic drug discovery (PDD) is an attractive approach to discover antibiotics with new MoAs, since no prior target hypothesis is required. A typical read-out in PDD screens is optical density (OD) of compound-treated bacterial cultures, which serves as a proxy for bacterial cell density. While this read-out allows to identify compounds that inhibit bacterial growth, it precludes the identification of antibacterial compounds at sub-inhibitory levels that could be optimised by medicinal chemistry or be effective in combination therapies. Moreover, these single read-out screens cannot provide information on how an antibacterial compound inhibited bacterial growth, i.e. the MoA⁴.

To overcome these limitations, recent antibacterial screening campaigns⁵ have drawn inspiration from eukaryotic PDD approaches using microscopy images as read-outs⁶. Nonejuie et al.⁷ and Zoffmann et al.⁸ acquired fluorescence microscopy images of bacteria and extracted hand-crafted quantitative descriptors (features) from segmented bacterial cells to perform bacterial cytological profiling (BCP). Importantly, compounds with the same MoA induced similar phenotypic changes, thus enabling the computational clustering of compounds by MoA directly from microscopy images. However, a limitation of early BCP is its reliance on cell segmentation, an error-prone step especially at high cell densities, and the potentially insufficient or unspecific information contained in hand-crafted features⁴. Deep learning (DL) approaches, in contrast, hold the potential to extract highly specific phenotypic features from images in an end-to-end manner without segmentation⁴. Prior work on microscopy images of eukaryotic cells labelled with multiple fluorescent dyes (Cell Painting⁹) has shown that features learned by a convolutional neural network (CNN) can outperform handcrafted features in discriminating between different treatment conditions and associating similar treatment conditions, respectively¹⁰. Recently, a CNN trained on over 5000 images has been shown to successfully classify the MoA of antibiotics at 1–5 times the minimum inhibitory concentration (MIC) from multi-channel images of fluorescently stained Mycobacterium tuberculosis bacteria¹¹.

Despite these encouraging efforts, several limitations and questions remain to be addressed before DL analysis of high-throughput microscopy images can be leveraged efficiently in future screens to discover antibiotic compounds with novel MoAs. First, it is unclear if antibiotic compounds can be identified from images at concentrations well below MICs, which would greatly improve the sensitivity of large-scale screens that typically test thousands of compounds at only one or two fixed concentrations. Second, it remains to be determined to what extent unseen compounds can be assigned to their correct MoA and, critically, whether compounds with novel MoAs can be detected, especially at sub-MIC concentrations. Third, fluorescent labelling is expensive, significantly adds complexity to experimental protocols, can introduce batch-to-batch variability, and may interfere with the molecular mechanism of the tested compounds. Whether fluorescent labelling can be dispensed with in favour of brightfield (BF) images in phenotypic screens of bacteria, as recently shown in eukaryotic cells¹², is unclear, since bacterial cells are roughly an order of magnitude smaller, potentially limiting the range of visible morphological phenotypes. Finally, the performance of DL typically comes at a steep cost in training data, as e.g. ~200 fields of view (FoV) per compound were used to train a CNN in ref. ¹¹, imposing limits to scaling up this approach to thousands of compounds and calling for more efficient approaches.

Here, focusing on the model bacterium Escherichia coli, we develop and validate a DL approach to extract bacterial phenotypic features end-to-end directly from BF images, thus dispensing with the need for fluorescent staining. We show that from these BF images alone, our model can reveal alterations induced by antibiotic compounds at concentrations well below the MIC and accurately recognise the MoA of compounds when trained on as few as eight images per well. Moreover, our approach can recognise the MoA of unseen compounds by comparing them to reference antibiotics of the same MoA, and determine if a compound has a novel MoA, again from BF images at sub-MIC concentrations only. We also show the successful application of our method to Klebsiella pneumoniae. Our results lay the foundations for a time- and cost-effective pipeline to identify chemical compounds with potentially novel antibiotic MoAs.

Results

Collecting a high-throughput image dataset of bacterial phenotypes

We collected a high-throughput imaging dataset of E. coli bacteria exposed to 22 widely used reference drugs (Figs. 1a and 2a). We labelled each of these drugs using only one of eight target classes (MoAs) based on available information from the literature (Table 1). Our reference drugs included seven inhibitors of cell wall synthesis targeting penicillin binding proteins (PBPs), which we labelled based on their primary target as inhibitors of PBP1A/B, PBP2 or PBP3; four β-lactamase inhibitors shown to have moderate to weak activity towards these PBPs; four antibiotics targeting ribosomes; three targeting DNA gyrase; two targeting membrane integrity; one targeting RNA polymerase and one targeting DNA synthesis (Supplementary Tables 1 and 2). Relebactam is the only drug for which no targets had been reported in E. coli. As it belongs to the family of diazabicyclooctanes (DBOs) that includes the PBP2 inhibitor avibactam, we putatively labelled relebactam as targeting PBP2. For each of these drugs, we determined the half maximal inhibitory concentration (IC₅₀) ('Methods'), which ranged from 4 ng/mL to 64 μg/mL (Table 1). We distributed these antibiotics onto four 96-well plates with randomised layouts (Supplementary Fig. 1) using an acoustic liquid handler ('Methods') to expose bacteria to one of four subinhibitory concentrations (0.125 × IC₅₀, 0.25 × IC₅₀, 0.5 × IC₅₀ and IC₅₀). Then, we grew bacteria to the exponential phase and seeded them onto the four 96-well plates containing the previously distributed antibiotics ('Methods'). In order to test robustness to biological variability, we performed these experiments as four biological replicates, wherein E. coli were cultured independently and grown with randomised well positions in four 96-well plates (Supplementary Fig. 1). After a 2 h 15 min incubation followed by bacterial membrane staining with FM4-64FX for 15 min, we fixed bacteria with a combination of paraformaldehyde and glutaraldehyde, whereupon we stained DNA with Hoechst 33342. Next, we acquired high-throughput images ('Methods') in three channels (BF, FM4-64FX and Hoechst) using an automated high-content screening system in non-confocal mode with a 63× water-immersion objective (Figs. 1a and 2a).

Fig. 1 — a Bacteria were grown to the exponential phase in culture, antibiotics were distributed onto four 96-well plates at four different concentrations with an acoustic liquid handler, and bacteria were dispatched onto the well plates. Bacteria were then grown with antibiotics for 2 h 15 min and fixed. Plates were imaged on a high-content microscope with a 63× water-immersion objective in non-confocal mode with three channels (BF, FM4-64FX and Hoechst). b The model was trained on high-throughput images from three out of four plates to predict the combination of antibiotic compound and concentration with a cross-entropy loss. Images were first passed through a data augmentation pipeline and then tiled. Each tile was then passed through the same EfficientNet-B0 backbone, whereupon embedding vectors from each image tile were aggregated to obtain a global embedding vector. c Inference/testing was performed on a hold-out test plate with compounds at a given concentration. Confusion matrices compare classifications (predictions) to ground-truth classes. Embedding vectors are visualised in 2D with UMAP dimensionality reduction. Cmpd., Compound; Conc., Concentration; MoA, mode of action.

Fig. 2 — a Representative images of antibiotic-treated *E. coli* (or negative controls, DMSO) from the test plate, with brightfield (BF) shown in grey, the FM4-64FX channel in red and the Hoechst channel in blue. Images were acquired in four biological replicates with 50 FoVs per class (see Supplementary Figs. 2–7 for more examples of BF images). Image contrast was adjusted to improve visualisation. All cropped images from DMSO to clarithromycin are at the same scale. b Confusion matrix of compound predictions per FoV from high-throughput images of bacteria treated at IC₅₀ using a model trained on all imaging channels and all concentrations. Colours encode the MoA as indicated in (c). The confusion matrix corresponds to one hold-out test plate. Supplementary Fig. 9 shows confusion matrices for all four hold-out test plates. c Confusion matrix per FoV for the same test plate obtained by pooling compounds within the same MoA (Table 1), with colours matching those in (b). d Cosine similarities between median feature vectors, with red indicating high similarity and blue indicating low similarity. High similarity tends to be found between antibiotics belonging to the same MoA (e.g. ciprofloxacin, norfloxacin and levofloxacin target the DNA gyrase). Colours encode MoA as in (b, c). e UMAP of image embeddings where smaller dots correspond to individual FoVs and larger dots with black borders to median embeddings over all FoVs in a well. Dots are coloured by antibiotic compounds. f Same UMAP with dots coloured by MoA. Antibiotics belonging to the same MoA tend to co-localise in latent space (e.g. PBP3 inhibitors in green and DNA gyrase inhibitors in red). MoA, mode of action. IC50, half maximal inhibitory concentration. Source data are provided as a Source Data file.

Table 1.

Antibiotics used for E. coli

MoA	Antibiotic	IC₅₀ (μg/mL)	MoA reference
Cell wall (PBP1A/B)	Cefsulodin	23.77	Refs. ^24,42
Cell wall (PBP1A/B)	Penicillin G	25.65	Refs. ^24,43
Cell wall (PBP1A/B)	Sulbactam	15.62	Ref. ²⁵
Cell wall (PBP2)	Avibactam	19.20	Ref. ⁴³
Cell wall (PBP2)	Mecillinam	0.3009	Refs. ^24,42,44
Cell wall (PBP2)	Meropenem	0.1085	Refs. ^43,45
Cell wall (PBP2)	Clavulanate	17.24	Ref. ²⁵
Cell wall (PBP2)	Relebactam	64	N/A for E. coli
Cell wall (PBP3)	Aztreonam	0.01568	Ref. ⁴³
Cell wall (PBP3)	Cefepime	0.007517	Refs. ^43,46
Cell wall (PBP3)	Ceftriaxone	0.004	Ref. ⁴³
Ribosome	Chloramphenicol	3.079	Ref. ⁴⁷
Ribosome	Clarithromycin	22.24	Ref. ⁴⁸
Ribosome	Doxycycline	0.3703	Ref. ⁴⁹
Ribosome	Kanamycin	2.529	Ref. ⁵⁰
DNA gyrase	Ciprofloxacin	0.005723	Ref. ⁵¹
DNA gyrase	Levofloxacin	0.01089	Ref. ⁵²
DNA gyrase	Norfloxacin	0.02538	Ref. ⁵³
RNA polymerase	Rifampicin	3.358	Ref. ⁵⁴
DNA synthesis	Trimethoprim	0.1335	Ref. ⁵⁵
Membrane integrity	Colistin	0.6465	Ref. ⁵⁶
Membrane integrity	Polymyxin B	1.166	Ref. ⁵⁷

Open in a new tab

The 22 antibiotics used to treat E. coli, along with their MoA according to the literature and their potency (IC₅₀). Relebactam has no reported target in E. coli.

Deep learning model and training strategy

We then designed a DL model (Supplementary File 1) to map images of antibiotic-treated bacteria to chemical perturbations ('Methods'). We based our model on the CNN architecture EfficientNet-B0¹³, which was shown to achieve state-of-the-art performance on established image classification benchmarks with much fewer parameters than more widely used architectures like ResNet-50¹⁴ (e.g. 5.3 million parameters vs 26 million parameters and 76.3% vs 76.0% accuracy on ImageNet classification^13,15), thereby keeping memory requirements and computation time moderate.

As input data, we used images of entire FoVs, of 2160 × 2160 pixels (corresponding to an area of 139.32 × 139.32 μm). Images were passed through a data augmentation pipeline and divided into nine tiles, which were processed in parallel using the same convolutional filters. Tile-wise embedding vectors were aggregated before the final classification layer (Fig. 1b and 'Methods'), thus diminishing the influence of tiles without bacteria on model training. More details on image pre-processing, model architecture, training hyperparameters and data augmentation are provided in 'Methods' and Supplementary Tables 3 and 4.

We trained our model to perform image classification, where each distinct antibiotic drug treatment, i.e. each combination of antibiotic and concentration (e.g. ‘aztreonam’ and ‘0.25 × IC₅₀’), was considered as a separate class, resulting in a total of 89 classes (4 concentrations × 22 compounds + 1 negative control treated only with the solvent DMSO; 'Methods'). In order to reduce our model’s sensitivity to possible technical confounders (e.g. plate-specific image features and well position-dependent effects independent of the drug-induced phenotypes) and increase its robustness to biological variability, we used three plates corresponding to three biological replicates to train a single model. Critically, to avoid biases and spurious correlations between training and testing data that could lead to an overestimation of classification performance, we evaluated each model’s classification performance on the biological replicate corresponding to the fourth hold-out test plate (Fig. 1c).

Deep learning accurately recognises the antibiotic mode of action from images

We first asked if and to what extent our model can predict the combination of both drug and concentration from images alone, considering all the available imaging channels (BF, Hoechst, FM4-64FX). This is a very difficult task, since (i) our treatments include multiple compounds with the same targets, expected to induce very similar phenotypes (Table 1) and (ii) changes in drug concentration are not necessarily expected to induce different phenotypes (e.g. if concentrations are either much below or above the IC₅₀). Indeed, although some drug treatments led to easily recognisable phenotypes, such as cell filamentation or shortening, distinguishing between all treatment conditions by visual inspection only remained extremely challenging (Supplementary Figs. 2–7). We assessed classification accuracy (i.e. the percentage of correct classifications) on four rotations of the training and testing plates, with confusion matrices shown in Supplementary Fig. 8. If the model was unable to learn any features specific to the combination of drug and concentration, the expected random classification accuracy would be 1.12% (1/89). Remarkably, the actual classification accuracy ranged from 31.3 to 47.4% at the FoV level, i.e. 28–42 times better than random. Visual inspection of classification errors indicated that the majority of errors were confusions between drugs sharing the same MoA and that misclassifications were more frequent at lower concentrations, where predictions often defaulted to a small number of drugs or the negative control (Supplementary Fig. 8). These results indicate that the model extracted some information specific to drug-concentration combinations from images.

As the effect of antibiotics is likely to be more pronounced at higher concentrations, we next assessed our model (trained on all drug-concentration combinations) only on test data at the highest drug concentrations, i.e. the IC₅₀ and assessed accuracy based only on the predicted antibiotic (irrespective of the predicted concentration; 'Methods'). On these data, our model achieved a very high per-image accuracy of 76.2– 90.2%, compared to 4.3% (1/23) for random classification (Supplementary Fig. 9). Per-well accuracies (obtained using the majority of predictions from all images in the same well) were slightly higher, namely 78.3–91.3%, indicating that at the IC₅₀, the vast majority of the 22 reference drugs used here led to highly specific phenotypes, which our model was able to automatically learn from the training data (Fig. 2b).

Nevertheless, antibiotic classification was not perfect, as the model still tended to confuse certain compounds. Inspection of individual hold-out test replicates revealed that the model tended to systematically confuse a subset of drugs (Fig. 2b and Supplementary Fig. 9). For example, aztreonam was confused with cefepime, cefsulodin or ceftriaxone for three out of four, and relebactam was confused with meropenem in two out of four pairs of model and hold-out test data, and norfloxacin was systematically mistaken for ciprofloxacin in one hold-out test plate. We noted that the confused compounds tended to have similar MoAs (Table 1). Moreover, when computing cosine similarities between median embeddings for all pairs of antibiotics, we could observe clustering by MoA (Fig. 2d), as also illustrated by UMAPs (Fig. 2e, f), suggesting that our model extracted similar feature vectors from images of compounds targeting the same MoA, despite only having been trained on drug labels and not explicitly on MoA labels.

This observation encouraged us to assess our model’s ability to recognise the MoA, as opposed to the identity of the drugs themselves. For this purpose, we first assigned each of the reference drugs to one out of six MoAs or target classes, namely: cell wall, ribosome, DNA gyrase, RNA polymerase, DNA synthesis and membrane integrity (Table 1). Because half of the antibiotics were cell wall inhibitors (11 out of 22), we further subdivided this class by their main target proteins, namely PBP1A/B, PBP2 and PBP3 (Table 1), thereby leading to eight target classes with one to five antibiotics per class. We then assessed the model’s performance at recognising the MoA of each drug by pooling drugs according to their known MoA. We note that this did not involve any retraining on MoAs, as we still used the model trained on combinations of antibiotics and concentrations. Remarkably, our model achieved an accuracy of 84.1–99.3% in classifying drug MoAs at the image-level and of 100% for two out of four replicates at the well-level (on the remaining plates, the classification accuracies were 87.0 and 95.7%), indicating almost perfect recognition of MoA from images (Fig. 2c).

Thus, our data shows that images of bacteria treated with reference antibiotics at 1 × IC₅₀ levels contain highly specific information about drug MoA, allowing accurate MoA recognition from images alone. Moreover, our results indicate that established CNN architectures are well-suited to extract this information in an end-to-end fashion when trained on drug labels alone, without the need for handcrafted features, cell segmentation or prior knowledge of drug MoAs.

Brightfield images alone allow accurate recognition of drug MoA

Fluorescent labelling of various bacterial cell compartments requires the selection of dyes with compatible excitation and emission spectra, as well as the species-specific optimisation of staining protocols. Additionally, fluorescent staining increases cost, acquisition time and data volumes and technical variations in staining may compromise reproducibility. We therefore asked if all three imaging channels, namely BF, FM4-64FX and Hoechst (Fig. 3a), were necessary to accurately distinguish between antibiotic drugs or MoAs. To quantify this, we trained several CNN models as described previously, but with all possible combinations of the three imaging channels as input and assessed classification accuracy on the hold-out test data (at IC₅₀) using the same channels. In some cases, the well-level compound classification accuracy varied significantly between the four hold-out plates, e.g. accuracies ranged from 65.2 to 95.7% for the models trained only on the FM4-64FX channel (Fig. 3b, Supplementary Figs. 10 and 11). We speculate that these differences could be due to variations in staining efficiency between plates. The median well-level compound classification accuracy over the four replicates was highest for the models trained and tested only on the two fluorescent channels (Hoechst and FM4-64FX) and all three channels together (Hoechst, FM4-64FX and BF). To our surprise, however, ignoring one or even two of the channels in training and testing resulted in only small reductions of median accuracy when compared to the differences between plates (Fig. 3b, Supplementary Figs. 10 and 11) and differences in well-level compound classification accuracies between different channel combinations were not significant (Friedman test: p = 0.088), suggesting that the three channels contained largely redundant information. When considering the BF channel alone (Fig. 3d), the drug classification accuracy was 80.4 ± 4.3% (mean ± std.dev.), only 5.4% below the three-channel accuracy, and was more consistent across plates (accuracies ranged from 73.9 to 82.6% vs 78.3 to 91.3%). Strikingly, pooling the predictions from the BF-only model by MoA led to almost perfect well-level MoA classification (95.7 ± 3.6%; Fig. 3c, e) and differences were again not significant (p = 0.053) when compared to other channel combinations. Accordingly, feature vectors obtained from the BF-only model also clustered by MoA (Fig. 3f).

Fig. 3 — a Example images of *E. coli* split by imaging channel. Images were acquired in four biological replicates with 50 FoVs per treatment condition. Image contrast was adjusted to improve visualisation. All images are at the same scale. b Per-well compound classification accuracies for different combinations of imaging channels. Models were trained in a cross-validation experiment on three plates (biological replicate) and tested on a hold-out test plate containing the same imaging channels as the training images. Test images contain bacteria treated at the same relative concentration (IC₅₀). The solid horizontal line indicates the mean, and the dashed line indicates random classification. A Friedman test (n = 4 plates, k = 7 channel combinations) shows no significant difference between the combinations of imaging channels (p = 0.088). c Per-well MoA classification accuracies. The solid horizontal line indicates the mean, and the dashed line indicates random classification. A Friedman test (n = 4 plates, k = 7 channel combinations) shows no significant differences (p = 0.053). d Confusion matrix by compound per FoV as in Fig. 2b but using only BF images for training and testing. e Confusion matrix by MoA per FoV, as in Fig. 2c, but for BF images. f Cosine similarities as in Fig. 2d, but based on the BF images. g, h MoA classification accuracy as a function of the number of training images per condition. The number of images required to reach 95% of the accuracy plateau was estimated by fitting a two-parameter exponential function to averaged MoA classification curves, with an example shown in (g) for predictions obtained from images of bacteria treated at IC₅₀. The solid line is the average, and the shaded area shows the standard deviation across the four replicates. h The number of images per condition required to reach 95% of the accuracy plateau was computed for each concentration on held-out test data, showing that between eight and 15 images are required. i Bars show the number of training images required to reach 95% of the accuracy plateau with (red) or without (blue) pretraining on ImageNet, evaluated for different concentrations by cross-validation on four hold-out test plates. The number of required training images (across all conditions) ranged from 736 (1 × IC₅₀) to 1380 (0.25 × IC₅₀) with pretraining. For **d–i**, all models were trained on BF images only. BF, brightfield; MoA, mode of action; acc., accuracy. Source data are provided as a Source Data file.

To shed light on the image features used by our model, we computed saliency maps using Gradient-weighted Class Activation Mapping (Grad-CAM)¹⁶, which highlights the parts of the image most important for the prediction of each class. Grad-CAM highlighted different features of bacterial cells, rather than the background (Supplementary Fig. 12 and 'Methods'), consistent with the CNN having learned biologically relevant image features as opposed to technical confounders. In composite images of cells treated with antibiotics of distinct MoAs, the class-dependent activations are consistently stronger for the image halves corresponding to the antibiotic class, as expected if the CNN has learned MoA-specific features (Supplementary Fig. 13).

These results show that accurate MoA recognition is possible from BF images alone, thereby removing the need for fluorescent labelling. This could drastically simplify experimental protocols and significantly reduce costs in large microscopy-based phenotypic screens.

CNN learns MoA-specific features from only eight images per condition

A major drawback of using DL algorithms compared to classical analysis methods based on hand-crafted features is the need for large amounts of training data. Assessing the minimum number of images required to train a DL model without losing classification performance can help to optimise and potentially reduce the burden of data collection and increase the rate of assayed compounds. We therefore estimated the minimum number of images required to train our CNN without adversely affecting MoA classification and also assessed the impact of pretraining our model on a large public dataset of natural images (ImageNet¹⁵). For this purpose, we trained our CNN using an EfficientNet-B0 convolutional backbone pretrained on ImageNet (Fig. 1b) with varying numbers of bacterial images per condition ranging from three to 120 and computed mean classification accuracies across four replicates. We observed that classification accuracy stopped increasing with the number of training images and fitted a two-parameter exponential function to determine the accuracy plateau and the number of images required to reach it (within 5%), at four different concentrations (Fig. 3g, h and 'Methods'). We found that between eight (IC₅₀) and 15 images (0.25 × IC₅₀) per condition (736 and 1380 images in total) were sufficient to achieve near-maximum accuracy. This number ranged from seven to nine (at 1 × IC₅₀) when training on the 30% images of highest or lowest cellular density, respectively, roughly consistent with an average 1.33-fold difference in mean cell density (Supplementary Fig. 15). Repeating the analysis after retraining the models with randomly initialised weights (Fig. 3i), we found that pretraining on ImageNet data reduced the number of required images by between 21.1% (for 0.25 × IC₅₀) and 46.7% (for 1 × IC₅₀).

These results show that near-perfect MoA classification performance can be achieved with only eight images per condition for E. coli bacteria exposed to compounds at the IC₅₀ level and highlight the benefit of using pretrained models to reduce the required training data.

Detecting drug phenotypes at subinhibitory concentrations

The above results show that DL can reliably extract MoA-specific features from images of bacteria exposed to drugs at 1 × IC₅₀, which for the reference drugs considered here ranged from 4 ng/mL to 64 μg/mL (Table 1). Visual inspection of drug-treated bacteria shows that the cellular phenotypes tend to be less pronounced at lower concentrations, as expected (Fig. 4a). When screening large libraries of compounds, systematic titration to determine the IC₅₀ for each compound is usually not practical. Instead, chemical screens typically rely on a single concentration, which for most active compounds is generally different from the inhibitory concentration. We therefore asked if our DL method could detect or even classify drug-induced phenotypes at subinhibitory concentrations, and in particular for concentrations at or below the IC₅₀. Since at subinhibitory concentrations, new chemical compounds may induce only subtle effects in phenotypic screening campaigns before optimisation by medicinal chemistry, the ability to detect such weak phenotypes could increase the sensitivity and hit rate for antibiotic drug discovery.

Fig. 4 — a Example images of *E. coli* treated at increasing concentrations (from 0.125 × IC₅₀ on the left to 1 × IC₅₀ on the right) show dose-dependent phenotypic changes. The antibiotic is indicated with colour encoding the MoA (see legend in c). Images were acquired in four biological replicates with 50 FoVs per treatment condition. Image contrast was adjusted to improve visualisation. All images are at the same scale. b Feature vectors of bacteria treated at four different concentrations were jointly projected with UMAP, and dots corresponding to a given concentration were highlighted in each plot. Small dots correspond to individual high-throughput images, and larger dots correspond to median feature vectors. Dots are coloured by antibiotics. c UMAP plots coloured by MoA show that feature vectors tend to move further away from the negative control (DMSO) as concentration increases. d The cosine distance between median feature vectors and DMSO was computed for each antibiotic class at different concentrations. Average distances for each MoA are shown as dashed lines, and shaded areas indicate the standard deviation. A detection threshold (red line) was computed from the intra-class cosine distances of DMSO feature vectors (mean plus three standard deviations). e Drug exposure detection was determined by evaluating cosine distances between median feature vectors and DMSO with respect to the detection threshold. The number of detected drugs at each relative concentration is shown, and bars are coloured by their MoA. The dashed line indicates the total number of antibiotics. f Receiver operating characteristic (ROC) curves for FoV-level drug exposure detection at different concentrations cross-validated on the four hold-out test replicates with the corresponding area-under-the-curve (AUC) values (mean ± std. dev.). Solid lines correspond to mean ROC curves, and shaded areas indicate the standard deviation. Source data are provided as a Source Data file.

We applied our models trained on reference antibiotics at multiple concentrations (as detailed above), on hold-out test data at sub-IC₅₀ levels and observed that, unsurprisingly, our ability to correctly classify bacterial images by compound increased with concentration (Supplementary Fig. 16). We then analysed the embeddings computed by our BF-only model on test data at increasing concentrations, from 0.125 × IC₅₀ to 1 × IC₅₀. For most antibiotics, embeddings were similar to negative controls at the lowest tested concentration (0.125 × IC₅₀) and tended to move away from the control and from one another as concentration increased (Fig. 4b, c). The exceptions were the PBP2 inhibitors avibactam and mecillinam, as well as the DNA synthesis inhibitor trimethoprim, for which embeddings already departed markedly from the controls at the lowest concentration, suggesting that these compounds induce distinctive phenotypic changes at concentrations well below inhibitory values. To quantify this observation, we computed the cosine distances between the median embeddings of drug-treated cell images and the negative control (DMSO) images, at each concentration (Fig. 4d). We also defined a threshold distance as the mean plus three standard deviations of the cosine distance between individual DMSO images. Based on this threshold, we found that even at 0.125 × IC₅₀, compounds from four groups of antibiotics (inhibitors of DNA synthesis, ribosome, PBP1A/B and PBP2) exhibited sufficiently different phenotypes to be detected (Fig. 4d, e). At 0.25 × IC₅₀, all but two groups of antibiotics (RNA polymerase and PBP3 inhibitors) were distinguished from the negative control.

To assess drug exposure detection at different subinhibitory concentrations, we obtained receiver operating characteristic (ROC) curves, which describe the trade-off between the true positive rate (ratio of FoVs with detected drug exposure to actual drug-treated FoVs) and the false positive rate (ratio of FoVs with detected drug exposures to untreated FoVs). These ROC curves were cross-validated on four hold-out plates, and performance was quantified using the area-under-the-curve (AUC), which is 1.0 for perfect detection and 0.5 for random detection. We observed that, in line with Fig. 4e, drug exposure detection improved with increasing concentrations and that even at the lowest concentration (0.125 × IC₅₀), drug exposure was robustly detected with an AUC of 0.81 ± 0.08, while at IC₅₀, detection was perfect (AUC = 1.00) (Fig. 4f).

Overall, our data show that image embeddings learned by our CNN can be used to robustly detect subtle phenotypic changes at subinhibitory concentrations from BF images alone, thereby enabling highly sensitive detection of potentially active compounds in phenotypic screens.

Recognising known modes of action of unseen drugs

We showed above that our model can recognise the identity of a compound or the MoA of several antibiotics after training on images obtained in independent experiments with the same antibiotics. However, in the context of a screening campaign, the goal is to determine the MoA of a novel, i.e. previously unseen drug. Therefore, we next asked if our model could correctly recognise the MoA of an unseen compound, i.e. a compound absent from the training data. To address this question, we performed a leave-one-compound-out (LOCO) analysis for compounds sharing a MoA with at least two other compounds.

We retrained our CNN (using BF images only) after systematically excluding all images for one compound from the training data (Fig. 5a). Then, using a hold-out test plate, we calculated embeddings from images of bacteria exposed to all reference (i.e. training set) compounds as well as to the left-out compound at 1 × IC₅₀. For each FoV, we identified the closest reference compound using a k-NN classifier on embedding vectors (with k = 150 and using cosine distance) (Fig. 5b and 'Methods'). To each left-out compound, we then assigned the MoA of the reference compound that was the closest for most FoVs. We found that at a concentration of IC₅₀, the assignment accuracy of the k-NN classifier across plates ranged from 70 to 80% (Fig. 5c). Moreover, we observed that the k-NN classification on embeddings performed better on average than directly using the MoA predictions of our model (75.0 ± 4.1% vs 63.8 ± 6.3%, Fig. 5c). We then evaluated MoA assignment accuracy for a range of concentrations and found that accuracy tended to improve with higher concentration (Fig. 5d). Interestingly, drugs targeting PBP2 were perfectly classified across the entire range of concentrations, indicating that these compounds induce highly distinctive phenotypes even at low concentrations.

At the highest concentration (1 × IC₅₀), we observed good to excellent MoA prediction accuracy for compounds inhibiting PBP2 (100%), PBP3 (91.7%), DNA gyrase (100%) and ribosome (75%), while compounds targeting PBP1A/B were almost never correctly assigned (1 out of 12 cases; Fig. 5e; see below and Discussion). Overall, the predicted MoA agreed with the ground-truth, i.e. with our literature-based MoA annotations (Table 1), in 56 out of 72 cases (Fig. 5h).

To better understand apparent misclassifications, we investigated MoA assignments in more detail by considering predictions for individual FoVs of bacteria treated with the left-out compounds (Fig. 5f). For instance, in the case of avibactam, out of the 50 FoVs, 45 were assigned to mecillinam and four to meropenem, consistent with the visually similar coccoid morphologies induced by the three compounds (Fig. 5f, h and Supplementary Fig. 3). Since both compounds target PBP2, avibactam was classified as a PBP2 inhibitor, in agreement with its literature-based MoA annotation (Table 1). Similarly, 20 FoVs of bacteria exposed to ciprofloxacin were assigned to norfloxacin and 19 FoVs to levofloxacin, consistent with the visually similar elongated phenotypes induced by all three compounds (Fig. 5f, h and Supplementary Fig. 5). By contrast, the three nominal PBP1A/B inhibitors sulbactam, cefsulodin and penicillin G led to inconsistent MoA predictions (Fig. 5g, h). Penicillin G was associated with the DNA synthesis inhibitor trimethoprim, while cefsulodin was associated with the PBP3 inhibitor ceftriaxone (Fig. 5g). Indeed, visual inspection shows that the morphologies of bacteria exposed to penicillin G resemble those induced by the DNA synthesis inhibitor trimethoprim (long filaments) more so than those induced by cefsulodin, while cefsulodin induced morphologies (shorter filaments) more similar to those induced by the PBP3 inhibitor ceftriaxone than those of penicillin G (Fig. 5g, Supplementary Figs. 2, 4 and 7).

To further investigate these discrepancies in PBP predictions, we performed Bocillin FL competition assays ('Methods') and determined the specificity against different PBPs of a subset of PBP inhibitors studied here, namely clavulanate, cefepime, penicillin G, cefsulodin and sulbactam (Supplementary Fig. 17). Our data confirm that clavulanate, classified by our LOCO analysis as a PBP2 inhibitor (Fig. 5h), is indeed highly specific to PBP2, in agreement with our literature-based annotations (PBP1A/B: 256.9 μg/mL, PBP2: 0.615 μg/mL and PBP3: >1000 μg/mL). Similarly, these data show that cefepime primarily inhibited PBP3, in agreement with our model predictions and annotation (PBP1A/B: 1.724 μg/mL, PBP2: 0.067 μg/mL and PBP3: 0.001 μg/mL). However, the affinity data for two of our three nominal PBP1A/B inhibitors were not consistent with the literature. Indeed, whereas our assay confirmed that cefsulodin primarily targets PBP1A/B (PBP1A/B: 0.748 μg/mL, PBP2: >1000 μg/mL and PBP3: 10.4 μg/mL), in agreement with the literature, neither penicillin G (PBP1A/B: 7.597 μg/mL, PBP2: 64.64 μg/mL and PBP3: 1.977 μg/mL) nor sulbactam (PBP1A/B: 37.4 μg/mL, PBP2: 200.3 μg/mL and PBP3: 102.2 μg/mL) had a specific target, but instead acted as pan-PBP inhibitors. Although these data do not explain the model predictions for cefsulodin, the lack of specificity of penicillin G and sulbactam is a likely cause of their inconsistent assignments by our model (see Discussion).

As pointed out above, we initially labelled relebactam as a PBP2 inhibitor — despite a lack of evidence of its target in E. coli — based only on the drug’s structural similarity with avibactam. Our model indeed classified relebactam as a PBP2 inhibitor from the images alone (Fig. 5h). To verify this prediction experimentally, we measured affinities of relebactam to multiple PBPs using the same competition assay (Fig. 5i, j) ('Methods'). This assay confirmed that relebactam indeed selectively inhibits PBP2 as predicted by our model.

Thus, our CNN can learn to extract embeddings from BF images alone that enable accurate MoA prediction of an unseen drug if the MoA is represented by reference compounds with consistent target specificity.

Detecting drugs with unseen modes of action

While the above results indicate that known MoAs can be assigned to novel compounds, addressing AMR requires the ability to identify compounds with novel MoAs. To address this, we assessed if our approach is capable of detecting compounds with a MoA that — unlike in our previous analysis — was not seen by our model during training. We therefore designed a leave-one-MoA-out (LOMO) experiment where we singled out one MoA and trained our CNN on BF images of drug-treated bacteria after systematically leaving out all compounds from this MoA (Fig. 6a). To serve as negative controls for novel MoA detection, we also randomly removed one compound for each MoA with at least three compounds (Table 1) from the training data. We then computed outlier scores on image embeddings with a local outlier factor (LOF) algorithm¹⁷ (Fig. 6b and 'Methods').

Fig. 6 — a Models were trained on images of drug-treated *E. coli* after removing all images corresponding to antibiotics from one MoA (e.g. PBP2 inhibitors for model 1 and DNA gyrase inhibitors for model M), and models were trained in a cross-validated setting on three out of four plates. b Embeddings from images of bacteria treated with a compound from the left-out MoA were then obtained and outlier scores assigned with a local outlier factor (LOF) algorithm. c UMAPs of embeddings for models trained on compounds from all MoAs except PBP2 (left), DNA gyrase (centre) and ribosome (right). Unsupervised outlier detection was conducted using LOF on embeddings of bacteria exposed to all concentrations (grey dots). Embeddings of compounds from the held-out MoA are shown as stars, negative controls as pentagons and their outlier scores as red and blue circles, respectively. d For each compound, probability scores of MoA novelty were computed by thresholding outlier scores and computing the fraction of detected embeddings. Pale symbols show the novelty detection probability for each plate, and the median is shown in bright colours. e Cross-validated receiver operating characteristic (ROC) curves for FoV-level MoA novelty detection were obtained for each held-out MoA on four hold-out test plates. Solid lines show averages, and shaded areas the standard deviation across the four replicates. Mean area-under-the-curve (AUC) values are shown. In d, e, the colour of each antibiotic encodes its MoA as in Fig. 2c. Source data are provided as a Source Data file.

After model training, we obtained embeddings from images of bacteria on a hold-out test plate treated with all compounds except those of the left-out MoA and the left-out negative control compounds at all four concentrations (grey dots in Fig. 6c). Next, we defined an image as showing a novel MoA if the outlier score of its embedding exceeded a threshold (and as a known MoA otherwise). We then computed novelty probabilities for each left-out compound (at 1 × IC₅₀) by considering the fraction of FoVs that were detected as exhibiting a novel MoA (Fig. 6d). The compounds of the left-out MoA and the negative control compounds allowed us to quantify true and false positives, respectively. We determined ROC curves by cross-validation across four hold-out test plates and computed mean AUC values for each MoA (Fig. 6e).

All compounds targeting PBP2, DNA gyrase, ribosome and membrane integrity exhibited median novelty probabilities above 0.8 and high or very high AUC scores (PBP2: 0.94, DNA gyrase: 0.75, ribosome: 0.92 and membrane integrity: 0.82), indicating that compounds with these MoAs induce phenotypes that can be well differentiated from the MoAs seen during training. In the case of PBP1A/B (AUC = 0.75), penicillin G and sulbactam exhibited high novelty probabilities, but cefsulodin did not (Fig. 6d). For the DNA gyrase inhibitors, median novelty scores across compounds were consistently high (AUC = 0.75), but individual scores by plate exhibited a higher degree of spreading compared to ribosome or PBP2 inhibitors. Conversely, PBP3 inhibitors consistently showed low novelty probabilities (AUC = 0.38), suggesting that these compounds induce phenotypes that do not differ from other MoAs sufficiently to be determined as novel MoAs de novo. Note that the PBP1A/B inhibitor cefsulodin was predominantly classified as a PBP3 inhibitor in our LOCO analysis (Fig. 5h), indicating that cefsulodin induces PBP3-like morphologies. The consistently low novelty scores observed for PBP3 inhibitors could thus also explain the low novelty scores of cefsulodin. However, this was dependent on concentration, since at 4 × IC₅₀ or higher, PBP3 inhibitors led to a clear change in phenotype, enabling good novelty detection (AUC = 0.86) (Supplementary Fig. 18).

Taken together, our results show that compounds from all studied MoAs induce phenotypes that are robustly detected as novel when excluded from the training data at concentrations of IC₅₀, except for PBP3 inhibitors, where concentrations of 4 × IC₅₀ were required. Our approach could thus play a key role in identifying compounds with potentially novel MoAs in phenotypic screens.

MoA recognition from brightfield images of Klebsiella pneumoniae

Finally, we tested the applicability of our approach beyond the model bacterium E. coli. For this purpose, we turned to Klebsiella pneumoniae, another member of the Enterobacteriaceae family. K. pneumoniae has a slightly larger genome than E. coli (~5.5 Mb vs 4.5 Mb and more than 5000 vs ~4400 coding sequences). These differences contribute to different antibiotic susceptibilities, and K. pneumoniae is resistant to four out of the 22 antibiotics used for E. coli. We therefore replaced these by four antibiotics to which K. pneumoniae is sensitive and exposed bacteria grown from a master frozen stock culture to the 22 antibiotics, at concentrations of 0.25 × IC₅₀, 0.5 × IC₅₀, 1 × IC₅₀ and 2 × IC₅₀. We grouped these antibiotics by MoA based on literature annotations as indicated in Table 2. For each drug and each concentration, we acquired 25 BF images of unstained cells on four plates (biological replicates) with randomised layouts using the same automated high-throughput microscopy system as for E. coli. Example images are shown in Fig. 7a and Supplementary Figs. 19–24.

Table 2.

Antibiotics used for K. pneumoniae

MoA	Antibiotic	IC₅₀ (μg/mL)	MoA reference
Cell wall (PBP1A/B)	Cefsulodin	57.98	PBP1, E coli : ref. ⁴²
Cell wall (PBP1A/B)	Sulbactam	19.95	PBP1a, E. coli; ref. ²⁵
Cell wall (PBP2)	Meropenem	0.03311	PBP2, K. pneumoniae; ref. ⁵⁸
Cell wall (PBP2)	Clavulanate	20.94	PBP2, E coli; ref. ⁵⁹
Cell wall (PBP3)	Aztreonam	0.006467	PBP3, K. pneumoniae; ref. ⁵⁸
Cell wall (PBP3)	Cefepime	0.0129	PBP3, K. pneumoniae; ref. ⁵⁸
Cell wall (PBP3)	Cefixime	0.01869	PBP2, Neisseria gonnorhea (homolog of E. coli PBP3); ref. ⁶⁰
Cell wall (PBP3)	Cefoxitin	7.165	PBP1+3, E. coli; ref. ⁶¹
Cell wall (PBP3)	Ceftriaxone	0.009485	PBP3, E coli; ref. ⁴²
Cell wall (PBP3)	Cephalothin	9.647	PBP3, Bacteroides fragilis (homolog of E. coli PBP3); ref. ⁶²
Ribosome	Chloramphenicol	1.978	Ref. ⁴⁷
Ribosome	Clarithromycin	70.61	Ref. ⁴⁸
Ribosome	Azithromycin	5.735	Ref. ⁶³
Ribosome	Doxycycline	2.335	Ref. ⁴⁹
Ribosome	Kanamycin	2.271	Ref. ⁵⁰
DNA gyrase	Ciprofloxacin	0.02229	Ref. ⁵¹
DNA gyrase	Levofloxacin	0.0691	Ref. ⁵²
DNA gyrase	Norfloxacin	0.07824	Ref. ⁵³
RNA polymerase	Rifampicin	16.19	Ref. ⁵⁴
DNA synthesis	Trimethoprim	0.6577	Ref. ⁵⁵
Membrane integrity	Colistin	2.01	Ref. ⁵⁶
Membrane integrity	Polymyxin B	1.638	Ref. ⁵⁷

Open in a new tab

The 22 antibiotics used to treat K. pneumoniae, along with their MoA according to the literature (with the species from which they were inferred) and their potency (IC₅₀).

Fig. 7 — a Brightfield (BF) images of *K. pneumoniae* bacteria treated with antibiotics or negative controls (DMSO). Images were acquired in four biological replicates with 25 FoVs per treatment condition (see Supplementary Figs. 19–24 for more examples). Image contrast was adjusted to improve visualisation. b Confusion matrix for compound prediction per FoV from BF images of *K. pneumoniae* treated with drugs at 1 × IC₅₀ in one hold-out test plate. Confusion matrices for all hold-out test plates are shown in Supplementary Fig. 26. c Confusion matrix by mode of action (MoA) per FoV for the same test plate. Confusion matrices for all hold-out test plates are shown in Supplementary Fig. 27. In b, c, the colour of each antibiotic encodes its MoA as in Fig. 2c. d Well-level classification accuracies of compounds and MoAs across four hold-out test plates (with treatments at 1 × IC₅₀), using models trained on the remaining three plates (biological replicates). The solid horizontal line indicates the mean. Dashed horizontal lines indicate accuracies of random classification. Source data are provided as a Source Data file.

We first attempted to use the model trained on E. coli images directly to analyse the K. pneumoniae images, after application of a sphering transform to align negative control wells ('Methods'). Although this analysis allowed some meaningful clustering of compounds based on feature similarities (e.g. clustering of the ribosome inhibitors azithromycin and clarithromycin or clustering of the PBP2 inhibitors meropenem and clavulanate), it also produced some incorrect or unexpected groupings, e.g. the DNA synthesis inhibitor trimethoprim was clustered with the gyrase inhibitor norfloxacin, while three compounds (cefoxitin, sulbactam and chloramphenicol) were clustered with negative controls (Supplementary Fig. 25).

We therefore retrained our CNN from scratch on K. pneumoniae images to predict treatment conditions (i.e. drug + concentration) from three plates and evaluated classification accuracies on the fourth hold-out test plate, following the same approach as for E. coli. For evaluation, we focused on bacteria treated at 1 × IC₅₀. On the drug classification task (i.e. ignoring concentration), our model achieved per-image accuracies ranging from 79.1 to 86.6%, and per-well accuracies ranging from 82.6 to 86.9% (Fig. 7b, d and Supplementary Fig. 26), a similarly high accuracy as for E. coli. When pooling predicted drugs by MoA (Table 2), we achieved per-image accuracies between 93.7 and 100%, depending on the replicate, and per-well accuracies of 100% for three out of the four replicates (and 95.7% for the remaining replicate; Fig. 7c, d and Supplementary Fig. 27). Thus, our data show that near-perfect MoA-recognition from BF images is possible in another Gram-negative bacterium of clinical interest, suggesting broader applicability across bacterial species.

Discussion

In this study, we introduced a deep learning (DL) approach based on convolutional neural networks (CNNs) that can extract meaningful phenotypic features from microscopy images of bacteria treated with antibiotic compounds and show that these features can be used for multiple downstream tasks with high relevance for phenotypic drug screening. These tasks include the recognition of mode of action (MoA) or even compound identity directly from images (Figs. 2, 3), the detection of weak phenotypes at subinhibitory drug concentrations (Fig. 4), the assignment of drug MoAs for novel compounds (Fig. 5) and the identification of compounds with novel MoAs (Fig. 6). While we focused on E. coli, our analysis of K. pneumoniae data also supports excellent recognition of drug MoA in a second bacterial species of high relevance to AMR (Fig. 7). We propose that our imaging-based approach provides a valuable complement to widely used growth inhibition screens, thanks to its ability to identify compounds at sub-inhibitory concentrations and its ability to either recognise compound MoA by comparison to reference antibiotics or detect novel MoAs (Fig. 8).

Fig. 8 — In chemical screens, compound libraries typically contain inactive compounds (white) and active compounds with strong or subtle effects (red and pink, respectively). Primary high-throughput screens based on microscopy or growth assays usually examine compounds at a single absolute concentration. Imaging-based screens can detect weak phenotypes induced by compounds at sub-inhibitory concentrations (pink), which are missed by growth inhibition screens (see Fig. 4). For the subset of compounds that elicit a phenotype either based on imaging or on growth inhibition, concentration–response curves are measured to determine IC₅₀’s. In a secondary screen, high-throughput imaging of these compounds at IC₅₀ allows to predict if compounds have a novel MoA. If yes, follow-up experiments are in order to elucidate the MoA. If not, MoA can be assigned computationally from the images alone. Green boxes highlight the imaging and analysis methods described in this paper.

Our work builds on prior studies in eukaryotic cells and bacteria that established cytological profiling to analyse drug-induced phenotypes and group drugs by MoA, whether using handcrafted features^7,8, or more recently, DL approaches^4,10,11,18. However, our study represents a significant extension of these early approaches in several ways.

First, we establish that DL can extract both MoA- and compound-specific features from brightfield (BF) images alone, enabling near-perfect MoA recognition without any fluorescent labelling (Figs. 3c, e and 7c). This came to us as a surprise, since most microscopy-based drug screens employ a mixture of fluorescent dyes to characterise subcellular features, as in the popular Cell Painting assay^9,19,20 or bacterial cell profiling assays^5,7,8,11. Our finding that fluorescent labelling of bacterial membranes and DNA is redundant with BF images is consistent with studies showing that DL models can be trained to generate realistic fluorescent labelling or coloration (virtual staining) in silico from label-free images in eukaryotes²¹ and in bacteria, as illustrated by a recent report that Gram-staining can be predicted from darkfield images of E. coli and Listeria innocua²². They are also in line with a recent related study demonstrating MoA recognition in unlabelled E. coli using time-resolved holographic microscopy¹⁸. Our results extend previous work in eukaryotic cells, showing that DL applied to BF images performs on par with classical Cell Profiler features to determine drug MoA¹². More importantly, our findings enable a drastic simplification of the experimental workflow of phenotypic imaging screens by removing the need for staining and protocol optimisation, as well as avoiding the associated costs and potential confounding factors like variations in fluorescent staining. The removal of staining also facilitates phenotypic screening in live cells, potentially circumventing fixation issues and enabling the incorporation of live cell dynamics in phenotyping.

Second, although DL-based image recognition for natural images typically requires thousands of training images per class to reach state-of-the-art classification accuracies, we show that roughly eight FoVs per condition (i.e. compound and concentration) are sufficient to achieve highly accurate MoA recognition, and that acquiring more images does not yield significant improvements (Fig. 3g–i). The main reason for this surprisingly low requirement is probably the large number of cells contained in each FoV, and the relative simplicity of bacterial BF images compared to natural images like those from the ImageNet dataset. Nevertheless, our observation that pretraining on ImageNet significantly reduces the required number of images suggests that many ImageNet features are also relevant for BF E. coli images. Our finding that recognition accuracy saturates with only a handful of FoVs should encourage future phenotypic imaging screens to reduce image acquisition time per well and thereby increase the rate of assayed compounds.

Third, we emphasise that our model was trained solely to predict the combinations of drugs and concentrations and thus requires no explicit prior knowledge of the MoA. Despite this, we show that our model implicitly encodes similarities by MoA in its latent space. This is a welcome feature, as it means that a straightforward supervised learning strategy with well-defined and unambiguous labels (drugs and concentrations) can learn biologically meaningful representations from image data in spite of incorrect MoA annotations. On the contrary, our approach can be used to flag potential MoA annotation errors as showcased in our LOCO analysis for the supposed PBP1A/B inhibitors cefsulodin, sulbactam and penicillin G (see below).

Fourth, we show that DL can robustly detect weak phenotypes induced by drugs at concentrations well below inhibitory levels (Fig. 4). This result implies that large chemical libraries at a fixed concentration can be screened with high sensitivity, i.e. with a high likelihood of picking up active compounds that elicit subtle phenotypes in the images without affecting bacterial growth. In a second step, these hit compounds could then be titrated to determine their IC₅₀ and analysed by imaging for MoA assignment at 1 × IC₅₀ levels, along with active compounds identified by traditional growth inhibition assays (Figs. 5c–h and 8).

Fifth, we have addressed the critical task of determining if a novel compound also has a novel MoA¹⁸. We have shown robust novelty detection for five out of six tested MoAs, with excellent performance (specificity and sensitivity) for PBP2 and ribosome inhibitors, and good to very good performance for inhibitors of PBP1A/B, DNA gyrase and membrane integrity. We acknowledge poor performance for PBP3 inhibitors at concentrations up to 1 × IC₅₀ but note that performance improves at concentrations of 4 × IC₅₀ or higher (Supplementary Fig. 18). Despite imperfections, our method thus opens the door to screening large libraries of compounds using high-throughput microscopy to identify compounds that are likely to act through novel MoAs (Fig. 8).

Finally, we also showed near-perfect MoA recognition in K. pneumoniae, using a model trained only on BF images of this bacterium, indicating that our approach extends to other bacterial species (Fig. 7).

We acknowledge several limitations or open questions. While MoA assignment in the LOCO analysis was good for compounds belonging to four MoA groups, it is notable that almost none of the nominal PBP1A/B inhibitors were correctly assigned to this MoA. Instead, each compound was assigned to a different MoA (Fig. 5h). The phenotypic discordances underlying these predictions and the results of our PBP affinity assays call the literature-based grouping of these PBP1A/B inhibitors into question (Table 1 and Supplementary Table 1). Cefsulodin has been reported to selectively target PBP1A/B (PBP1A: 0.47 μg/mL, PBP1B: 3.7 μg/mL, PBP2: >250 μg/mL and PBP3: >250 μg/mL)^23,24. However, this is not supported by our PBP affinity assay (Supplementary Fig. 17), where we observed inhibition of both PBP1A/B (0.75 μg/mL) and PBP3 (10.4 μg/mL). Sulbactam has been reported to primarily target PBP1A with low affinities for PBP2 and PBP3²⁵, but this is also inconsistent with our affinity assays, where we observe broad inhibition of PBP1A/B, PBP2 and PBP3 (Supplementary Fig. 17). By contrast, the high reported affinity of penicillin G for multiple PBPs (PBP1A: 0.5 μg/mL, PBP1B: 3.0 μg/mL, PBP2: 0.8 μg/mL, PBP3: 0.9 μg/mL and PBP4: 1.0 μg/mL)²⁴ is consistent with our PBP affinity assay showing a pan-PBP inhibitor activity (Supplementary Fig. 17). Taken together, these differences in PBP affinities between putative PBP1A/B inhibitors plausibly explain the observed inconsistencies in bacterial phenotypes. We note that the discrepancies in PBP affinities between our experiments and prior studies could result from variations in bacterial strains due to their individual history of pre-exposure to β-lactams. Cefsulodin is the only drug in our dataset that primarily targets PBP1A/B (0.75 μg/mL), although with high secondary affinity for PBP3 (10.4 μg/mL). Nevertheless, it is classified as a PBP3 inhibitor by our LOCO analysis (Fig. 5g). In this context, we also note that recent work has shown that PBP1A and PBP1B do not regulate the morphology of E. coli bacteria²⁶ and instead contribute to cell-wall integrity, whereas PBP3 directly regulates bacterial cell shape. If inhibition of PBP1A/B does not produce a detectable phenotype, it appears likely that morphological changes induced by cefsulodin will be driven by inhibition of PBP3 alone. This would be consistent with our model’s classification of cefsulodin as a PBP3 inhibitor. In the future, predictions could also benefit from characterising the specificity profile of compounds to non-essential low molecular weight PBPs that we ignored here (e.g. PBP4-7), whose inhibition may affect bacterial phenotypes²⁷. In any case, the discrepancies between the predicted MoAs of compounds that nominally share the same MoA (such as cefsulodin, sulbactam and penicillin G in Fig. 5h) suggest that MoA annotations should be revisited. Corrections of MoA ground-truth assignments flagged by our image-based approach, e.g. by affinity assays, are then likely to yield improved image-based prediction accuracies.

Additionally, our study included four β-lactamase inhibitors (sulbactam, clavulanate, avibactam and relebactam). Although sulbactam has been reported to preferentially bind PBP1A, we show that it is a pan-PBP inhibitor, while we confirm that clavulanate targets PBP2 as reported (Supplementary Fig. 17). Relebactam, however, has not been studied in E. coli, and our LOCO analysis indicates that this drug induces PBP2-like phenotypes similar to avibactam and clavulanate (Fig. 5h). Indeed, our affinity assay confirmed that relebactam selectively targets PBP2 in our E. coli strain (Fig. 5i, j). This is consistent with data on other DBOs like avibactam that target PBP2. Interestingly, prior work shows that in Pseudomonas aeruginosa avibactam preferentially binds PBP1B and PBP4, while relebactam acts as a pan-PBP inhibitor²⁶. However, we note that in the same study, an agnostic target predictor methodology classifies both avibactam and relebactam as PBP2 inhibitors, in agreement with our data.

Intriguingly, while trimethoprim is commonly classified as a DNA synthesis inhibitor and specifically targets folate synthesis, some studies have shown that trimethoprim also directly targets the peptidoglycan pathway^28,29. Indeed, peptidoglycan synthesis relies on nucleotide-activated sugars, such as UDP-N-acetylglucosamine and UDP-N-acetylmuramic acid. Inhibition of nucleotide synthesis by trimethoprim would thus also affect peptidoglycan synthesis and, more generally, cell wall biogenesis, which relies on many nucleotide-activated sugars, including lipopolysaccharides. The assignment of penicillin G to trimethoprim appears consistent with these prior findings.

Taken together, these observations highlight the limitations of grouping compounds into discrete MoA classes and of using reference compounds that inhibit multiple targets. One promising avenue to overcoming this limitation is to predict drug targets by comparing their induced phenotypes to those of large mutant libraries.

The results in our study are limited to E. coli and K. pneumoniae, two Gram-negative bacteria with pronounced morphological changes in response to drug exposure. To what extent our BF-only methodology extends to Gram-positive bacteria and/or other Gram-negative species with more subtle morphological phenotypes remains to be determined. However, we point out that a variation of our deep learning-based analysis method proved effective at analysing images of the Gram-positive bacterium Corynebacterium glutamicum, a proxy for Mycobacterium tuberculosis, despite its significantly smaller size and less pronounced phenotypes, as detailed in ref. ³⁰. Follow-up work could investigate the applicability of our approach to a broader range of bacterial species.

At the technical level, it will be interesting to determine the robustness of our method to changes in experimental conditions, such as microscopy settings. We verified that MoA recognition accuracy is robust to changes in the image bit-depth (16-bit vs 8-bit) and to a roughly 2-fold change in image resolution (500 × 500 vs 256 × 256 pixels) (Supplementary Fig. 14). Additionally, we acquired BF images of unstained E. coli in a different laboratory with a conventional, manually operated imaging system. Compounds were administered at concentrations ranging from 0.25 to 2 × IC₅₀, based on independently determined IC₅₀ values, and many other experimental conditions differed (Supplementary Note 1, Supplementary Tables 5 and 6). After retraining our CNN on this new data set, we achieved a mean MoA recognition accuracy per well on hold-out test plates of 69.6% at 1 × IC₅₀ and 81.9% at 2 × IC₅₀ (Supplementary Fig. 28). Despite the lower accuracies, these data suggest that our approach is relatively robust to changes in experimental conditions and may be applied to characterise compound MoA even without automated microscopy and liquid handling systems. This could considerably extend the reach of our method, including to low-resource settings. Nevertheless, we acknowledge that more analyses beyond these preliminary data will be needed to better define the experimental conditions required for reliable MoA recognition from BF images alone.

Finally, we acknowledge that despite our use of Grad-CAM saliency maps (Supplementary Figs. 12 and 13), our approach provides limited insights into the MoA-specific image features learned by the CNN. Leveraging recent advances in explainable AI to overcome limitations of Grad-CAM or similar methods and extract more detailed information about these features provides another exciting perspective for follow-up studies^31–33. A related limitation is our treatment of concentration labels in the same manner as compound labels. We speculate that better performance on compound classification might be achievable by reframing the prediction of concentration as a regression task or by enforcing the learning of features robust to changes in concentration.

Overall, our work is a step towards deploying DL-based analysis of microscopy images to bacterial phenotypic screens. To facilitate uptake by the community, we make all our analysis methods and pretrained models available, along with the data they were trained on. As sketched in Fig. 8, we propose that our CNN approach and the downstream analysis methods described here complement traditional growth-inhibition assays in a comprehensive phenotypic screening pipeline aiming to discover novel classes of antibiotics to fight AMR.

Methods

Bacterial growth protocol for E. coli

E. coli K-12 MG1655 was grown in 5 mL of cation-adjusted Mueller–Hinton Broth II (MHBII, Fisher Scientific Cat. 11800482) overnight at 37 °C until the stationary phase. The culture was then diluted to 0.05 OD in fresh MHBII and grown at 37 °C for 1 h 45 min until mid-exponential phase between 0.5 and 0.7 OD before further dilution. IC₅₀ values were determined via broth serial dilutions in flat-bottomed 96-well plates (TPP Cat. 92096) with a starting inoculum density of 0.05 OD in a total volume of 100 μL. Micro dilution plates were incubated at 37 °C, 180 rpm for 18 h. The final cell density was measured on a Tecan Spark Multimode Plate Reader and plotted on GraphPad Prism to calculate the IC₅₀ values with the ‘[Inhibitor] vs response – Variable slope (four parameters)’ formula.

For microscopy, antibiotics (see Supplementary Table 2 for further information on the source of the antibiotics) were dispensed into 96-well plates in nanolitre volumes using an Echo 550 liquid handler (Beckman Coulter) before adding 100 µL of bacteria from mid-exponential growth diluted to 0.05 OD. The plates were incubated at 37 °C, 180 rpm for 2 h 15 min. Then, 20 μg/mL of FM4-64FX (Invitrogen Cat. F34653) was added and was further incubated at 37 °C, 180 rpm for another 15 min. The supernatant was then removed after centrifugation at 4000 × g for 5 min. 100 µL of fixative (2.8% paraformaldehyde, 0.04% glutaraldehyde in PBS) was added, and the plates were incubated at room temperature (RT) at 800 rpm for 30 min. Cells were washed twice with 200 µL of PBS by centrifugation and resuspension. Then, cells were suspended in 100 µL of PBS containing 10 µg/mL of Hoechst 33342 (Invitrogen Cat. H1399) and transferred to PhenoPlates (Revvity Cat. 6055300) to determine the OD at 600 nm. Finally, the entire plate was diluted based on the median OD of the negative control wells (DMSO) to a calculated OD of 0.036 in 100 µL. The fixed and diluted cells were stored at 4 °C until imaging.

Bacterial growth protocol for K. pneumoniae

K. pneumoniae KP1 was grown in 5 mL of cation-adjusted Mueller–Hinton Broth II (MHBII, Fisher Scientific Cat. 11800482) overnight at 37 °C until the stationary phase. The culture was then diluted to 0.05 OD in fresh MHBII and grown at 37 °C for 1 h 45 min until mid-exponential phase between 0.5 and 0.7 OD before further dilution. IC50s were determined via broth serial dilutions in 96-well plates with a starting inoculum density of 0.05 OD in a total volume of 100 μL. Micro dilution plates were incubated at 37 °C, 700 rpm for 18 h. The final cell density was measured on a Tecan Spark Multimode Plate Reader and plotted on GraphPad Prism to calculate the IC₅₀ values.

For microscopy, antibiotics were dispensed into 96-well plates in nanoliter volumes using an Echo liquid handler (Beckman Coulter) before adding 100 µL of bacteria from mid-exponential growth diluted to 0.05 OD. The plates were incubated at 37 °C, 700 rpm in an orbi-shaker (Heidolph Titramax 1000) for 2 h 30 min. The OD was measured at 600 nm, and 10 µL of 10× fixative (20% paraformaldehyde, 1% glutaraldehyde in PBS) was added to the cell cultures, followed by a 15 min incubation at RT. Next, 10 µL of fixed cells were transferred to a new plate containing 90 µL of 0.2 M glycine in PBS; the cell density was adjusted in each well with 0.2 M glycine if necessary. Subsequently, 30 µL of the diluted cells were transferred to PhenoPlates (Revvity Cat. 6055300) prefilled with 200 µL of 0.005% PEI in PBS. The plate was centrifuged at 500 × g for 10 min and washed three times with PBS. Finally, the plate was stored overnight at RT prior to imaging. All four biological replicates (plates) were obtained and imaged on the same day.

Penicillin-binding protein affinity assay

E. coli K-12 MG1655 was grown in 5 mL of LB overnight at 37 °C until the stationary phase. The culture was then diluted to 0.05 OD in fresh LB and grown at 37 °C for 1 h 30 min until mid-exponential phase between 0.5 and 0.7 OD. Bacteria were collected by centrifugation at 3220 × g for 5 min and washed with 50 mM Tris, pH 7.8. Then, bacteria were suspended in 50 mM Tris, 200 mM EDTA, pH 7.8 to 20 OD. 25 μL of 10-fold serially diluted antibiotics starting at 2 mg/mL were aliquoted into a V-bottomed plate, and 25 μL of 20 OD bacteria was added for a final starting antibiotic concentration of 1 mg/mL. The plate was incubated at 20 °C for 30 min and then centrifuged at 3220 × g for 5 min to pellet the cells. The supernatant was removed, and cell pellets were suspended in 50 μL of 50 mM Tris, 200 mM EDTA, pH 7.8, containing 25 μM of Bocillin-FL (Invitrogen Cat. B13233). The plate was incubated at 20 °C for 30 min, 1400 rpm. The cells were collected by centrifugation, suspended in 100 μL of 50 mM Tris pH 7.8, and centrifuged again before removing the supernatant. Fifty microliters of Laemmli sample buffer was added, and the plate was incubated at 70 °C for 10 min at 800 rpm before centrifugation at 3220 × g for 10 min to pellet insoluble materials. Twenty-five microliters of each sample were run on 4–15% Tris-Glycine SDS-PAGE (Bio-Rad, Cat. 4568084). The gels were imaged on a Bio-Rad Chemidoc at 488 nm to visualise the Bocillin-FL labelled bands and at stain-free settings to visualise total protein content.

High-throughput imaging

Prior to imaging, plates stored at 4 °C were left at RT for at least 20 min and then centrifuged at 35 × g for 10 min to bring floating bacteria to the well bottom. Images were then acquired using an automated Opera Phenix Plus High-Content Screening System (Revvity) with 63×/NA 1.15 water immersion objective in widefield mode without binning. For E. coli, two imaging sequences were set up: one for the simultaneous acquisition of fluorescent channels by two cameras, and one for the BF acquisition via a dedicated camera. Hoechst 33342 and FM4-64FX were imaged with a 405 and 488 nm excitation laser line and a 435–480 nm and 650–680 nm bandpass filter, respectively. Images were acquired with exposure times of 80 ms for BF, 400 ms for FM4-64FX and 250 ms for Hoechst 33342. In total, 50 FoVs were acquired in non-overlapping, randomly chosen positions within the whole area of each well. These random positions in each well were the same within individual plates but varied between plates. For imaging K. pneumoniae, we used a similar workflow, except that we acquired only BF images in 25 FoVs arranged on a grid at the centre of each well.

Image preprocessing

Images for training were obtained by converting high-throughput images from 16-bit to 8-bit and clipping pixel intensities by percentile between 0.1 and 99.9 to remove spurious saturated pixels. Images from three out of four plates were then randomly split into training (80%) and validation (20%) sets. During training, full-resolution images with 2160 × 2160 pixels were passed through a data augmentation pipeline: images were randomly flipped, rotated and subjected to random changes in brightness, contrast, saturation and hue (for images with more than one channel). To account for potential edge effects due to random rotation, full images were further centre-cropped (1528 × 1528 pixels) (Fig. 1b). Next, from this centre-crop, a random crop of size 1500 × 1500 pixels was obtained, and images were subdivided into nine tiles of size 500 × 500 pixels. Tiles were then resized to 256 × 256 pixels and pixel intensities normalised between −1 and 1. The latter normalisation choice was preferred to the standard normalisation (by subtraction of mean and division by standard deviation) to reduce the effect of changes in cell density. See Supplementary Table 3 for a detailed overview of the data preprocessing pipeline.

Removal of technical variation with sphering

For the analysis of K. pneumoniae data using features learned on E. coli (Supplementary Fig. 25), we used negative control sphering³⁴ to remove unwanted technical variation by assuming that negative controls (DMSO) should lead to consistent feature vectors. Variations that arise between control wells are thus assumed to be technical. The sphering transform seeks to find a new space where negative controls have the same amount of variation in each feature dimension. This is achieved by first calculating a covariance matrix on the feature vectors from the negative controls and obtaining its eigenvalues. The eigenvalues of the covariance matrix are then used to compute a correction matrix that whitens the feature vectors of the negative controls. Here, we used zero component analysis (ZCA) to whiten the data. The correction matrix is then applied to all feature vectors from images of drug-treated bacteria. This reduces large variations that are assumed to be due to technical variations.

Deep learning model training

The DL model was trained for 250 epochs (unless stated otherwise) using an Adam³⁵ optimiser with a learning rate of 0.001. A batch size of 16 images was used. Note that the effective batch size was 144 (16 × 9), since each image was further subdivided into nine tiles. During training, tiles were passed through an EfficientNet-B0 backbone, with weights initialised by pretraining on ImageNet¹ (except where indicated otherwise). Latent feature vectors corresponding to individual tiles were aggregated by averaging to obtain a global feature vector for the full-resolution input image. The model was trained to predict the treatment condition, i.e. the combination of antibiotic and concentration as a multi-class classification task with a cross-entropy loss using L2 regularisation with λ = 0.001 to avoid overfitting. To allow the model to accept images with different numbers of input channels, the first layer of the EfficientNet-B0 architecture was replaced with a convolutional layer (torch.nn.functional.Conv2d) with the appropriate number of input channels. Models were trained with PyTorch 2.3.1³⁶ in Python 3.9 on the HPC cluster at Institut Pasteur Paris using NVIDIA A40 or A100 GPUs with training times of approximately 30–40 h (depending on the GPU). See Supplementary File 1 for the full model architecture and Supplementary Table 4 for an overview of the model hyperparameters.

Computing compound predictions conditioned on the dose

For every FoV, the DL model outputs a logits vector $z \in R^{K}$ , where $K$ corresponds to the total number of compound-dose classes (including controls). Applying the softmax function yields a joint probability $p (k) = e^{z_{k}} / \sum_{k'} e^{z_{k'}}$ over all compound-dose classes where $k$ encodes a unique compound-dose pair. Since the dose is known at inference, we can obtain the conditional probability of compound $c$ at dose $d$ with:

P ((compound = c∣ dose = d) = \frac{p (k_{c, d})}{\sum_{k^{'} \in K_{d}} p (k^{'})}

where $k_{c, d}$ is the class encoding compound $c$ and dose $d$ , and $K_{d}$ is the set of classes at dose $d$ . The predicted compound $\hat{c}$ is then obtained by computing $\hat{c} = {arg \max}_{c} P ((compound = c∣ dose = d)$ and well-level predictions are then obtained by a majority vote over all FoVs in a given well.

Estimating the minimum number of required images

To estimate the minimum number of required training images, we fit an exponential $f (x) = a (1 - e^{- b x})$ (where $x$ is the number of images per condition) to the mean MoA accuracies obtained from hold-out test data with models trained on different numbers of images per well ranging from $x = 3$ to $x = 120$ . From these exponential fits, we obtained the accuracy plateau $a$ and estimated the number of images per well to achieve 95% of this plateau as $x = - \ln (0.05) / b$ . We used the curve fitting function from the scipy³⁷ package in Python 3.9.

Splitting images by bacterial density

The normalised bacterial density $D$ was estimated by thresholding the FM4-64FX channel with Otsu’s method and computing the fraction of foreground pixels as:

D = \frac{#pixel s_{foreground}}{{#pixels}_{foreground} + # {pixels}_{background}} .

Images were then split into low, medium and high-density images based on the normalised bacterial density distribution for each condition. For a given condition, low bacterial density was defined as images below the 30th percentile, medium as between the 30th and 70th percentile and high as above the 70th percentile of the normalised bacterial density distribution.

MoA assignment with k-NN classifier

Embeddings were obtained from a hold-out test plate using a model trained on all except one compound from the other three plates. A k-NN classifier was then applied to predict the MoA of the unseen compound using embeddings from the hold-out test plate corresponding to the highest concentration (IC₅₀) and using MoA labels as targets with the scikit-learn³⁸ package in Python 3.9. The number of neighbours was set to k = 150 and the cosine distance (1 minus the cosine similarity) was used and points were weighted by the inverse of their distance, thereby ensuring that closer neighbours had a greater influence on the MoA assignment than neighbours which were further away. This analysis was repeated for each model with a left-out compound across four hold-out test plates.

MoA novelty detection with the local outlier factor algorithm

Embeddings from images of previously seen conditions were obtained from a hold-out test plate. A LOF classifier was applied to these embeddings using all available concentrations with the scikit-learn package in Python 3.9. The number of neighbours was set to k = 9, and cosine distance (one minus the cosine similarity) was used as the distance metric. To determine if a compound was an outlier, i.e. had a novel MoA, the local outlier factor on embeddings from the highest concentration (IC₅₀) was computed and a threshold applied as described in ref. ¹⁷. For each compound, novel MoA detection probabilities were computed as the fraction of detected outliers considering all FoVs. ROC curves were obtained by computing the local outlier factor for embeddings corresponding to previously unseen compounds and to left-out negative control conditions (one per MoA with at least three compounds) and positive controls (i.e. compounds with the left-out MoA). AUC values were computed with the scikit-learn package. This analysis was repeated for each model with a left-out MoA across four hold-out test plates.

Grad-CAM saliency maps

The Grad-CAM algorithm was implemented as described in ref. ¹⁶. All Grad-CAM maps were obtained from the third convolutional block of the EfficientNet-B0 model as:

L_{Grad - CAM}^{c} = ReLU (\sum_{k} α_{k}^{c} A^{k})

where $A$ are the activations, $c$ is the class for which the Grad-CAM is evaluated, $k$ is the index of the feature map and $ReLU$ is the rectified linear unit. Grad-CAM maps were then rescaled to the size of input images with bicubic interpolation.

Grad-CAM activations $L^{A}$ w.r.t. class $A$ (e.g. avibactam at 1 × IC₅₀) and $L^{B}$ w.r.t. class $B$ (e.g. trimethoprim at 1 × IC₅₀) were computed on composite images composed of a half image from class $A$ and a half image from class $B$ . Grad-CAM values were then summed over the pixels $p$ with x corresponding to the set of pixels belonging either to class x, A or B. Relative activations $R_{A}^{B}$ for class $A$ compared to class $B$ were then computed as:

R_{A}^{B} (x) = \frac{\sum_{p \in x} L_{p}^{A}}{\sum_{p \in x} L_{p}^{A} + \sum_{p \in x} L_{p}^{B}} .

Statistical tests

Rank-based, one-way comparisons of multiple groups on a quantitative variable were performed using the ‘friedman.test’ function (‘stats’ package), stratifying by replicate. Differences were considered significant for p-values below 0.05. In case of statistical significance, a post-hoc analysis was performed using the ‘pairwise.wilcox.test’ function (‘stats’ package, stratifying by replicate, after adjusting for the multiple comparisons by a Benjamini–Hochberg method). These analyses were performed using R (version 4.6.0).

Software packages

Data analysis was carried out in Python 3.9. The following Python packages were used with version numbers in brackets: ‘torch’ (1.12.1), ‘torchvision’ (0.13.1), ‘numpy’ (1.23.1), ‘scipy’ (1.9.1), ‘pandas’ (1.5.0), ‘matplotlib’ (3.6.0), ‘Pillow’ (9.2.0), ‘scikit-image’ (0.19.3), ‘imageio’ (2.22.0), ‘tifffile’ (2022.8.12), ‘imagecodecs’ (2024.1.1), ‘scikit-learn’ (1.1.2), ‘umap-learn’ (0.5.3), ‘xmltodict’ (0.13.0), ‘lxml’ (4.9.1), ‘openpyxl’ (3.1.2), ‘tensorboard’ (2.10.0) and ‘huggingface_hub[torch]’ (1.7.2).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Supplementary information

Supplementary Information^{(47.9MB, pdf)}

Reporting Summary^{(70.4KB, pdf)}

Transparent Peer Review File^{(16.4MB, pdf)}

Supplementary File 1^{(1.2MB, png)}

Source data

Source Data File^{(3.5MB, xlsx)}

Acknowledgements

We thank C. Thépenier, C. Spahn, S. Jang, D. Shum, N. Aulner, T. Walter, M. Mourez, P. Glaser, M. Weigert, D. Garreau and S. Oleshko for discussions or insightful comments on the manuscript and A. Torri for help with coordination. We thank P. Glaser for providing the Klebsiella pneumoniae strain used in this study. We also thank the reviewers for their careful analysis of our paper. We dedicate this work to the memory of our colleague Spencer L. Shorte, who played a key role in the inception of this project.

Author contributions

D.K., K.K. and J.P. developed the experimental protocol. K.K. prepared Escherichia coli plates for imaging and performed PBP affinity assays. K.K. and T.D. adapted the experimental protocol for Klebsiella pneumoniae. T.D. prepared K. pneumoniae plates for imaging. D.K., K.K., J.P. and N.M. optimised imaging parameters. N.M. performed imaging. M.E.H. performed imaging experiments after non-automated plate preparation of E. coli. K.K., M.E.H. and T.D. determined antibiotic IC₅₀ values. A.Z., J.C. and T.D. distributed compounds with a liquid handler. D.K. and C.Z. wrote and revised the manuscript with input from all authors. D.K. developed the computational analysis and made the figures. C.Z., S.L.S. and I.G.B. conceived the project. C.Z., I.G.B., A.M.W. and M.B. supervised the project.

Peer review

Peer review information

Nature Communications thanks the anonymous reviewers for their contribution to the peer review of this work. A peer review file is available.

Funding

D.K. was funded by the Pasteur-Paris University International doctoral program (PPU), the INCEPTION program (Investissement d’Avenir grant ANR-16-CONV-0005) and a Fondation pour la Recherche Médicale (FRM) Fin de Thèse (FDT) grant (FDT202404018132) . K.K. was supported by the INCEPTION program, the Pasteur-Roux-Cantarini Postdoctoral Fellowship (Institut Pasteur) and the Fondation pour la Recherche Médicale (FRM) grant EQU202403018034. J.P. was funded by Fondation pour la Recherche Médicale (grant number EQU202303016284 to Pedro Alzari). Work in the Boneca lab was supported by the Fondation pour la Recherche Médicale (FRM) grant EQU202403018034 and by the Labex 'Integrative Biology of Emerging Infectious Diseases' (ANR-10-LBX-62 IBEID). The UTechS Photonic BioImaging, C2RT, Institut Pasteur, is supported by the French National Research Agency (France BioImaging, ANR-24-INBS-0005 FBI (BIOGEN); Investments for the Future). We acknowledge Institut Pasteur and the Région Île-de-France (DIM1Health program) for the use of the Opera Phenix system. We also acknowledge the INCEPTION program for funding a GPU farm used in this work. C.Z. was supported by the Institut Pasteur, the University of Würzburg and the Bavarian State Ministry for Science and Arts through the Distinguished Professorship Program. M.B. received support from the German Center for Infection Research (DZIF, grant no. TTU 09.732). This work is also supported by the ERC Synergy grant AI4AMR 101167416 funded by the European Union. Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union or European Research Council. Neither the European Union nor the granting authority can be held responsible for them.

Data availability

The imaging data generated in this study have been deposited on Bioimage Archive under accession code S-BIAD1851³⁹. The embeddings and model predictions generated in this study have been deposited on Zenodo⁴⁰. Source data are provided with this paper.

Code availability

The source code for the computational analyses described in this paper is available with MIT license on GitHub (https://github.com/krentzd/ai4ab) and Zenodo⁴¹. Model weights are available on Hugging Face (https://huggingface.co/krentzd/ai4ab).

Competing interests

The authors declare no competing interests.

Footnotes

https://download.pytorch.org/models/efficientnet_b0_rwightman-7f5810bc.pth

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

These authors contributed equally: Daniel Krentzel, Kelvin Kho, Julienne Petit.

These authors jointly supervised this work: Ivo G. Boneca, Christophe Zimmer.

Contributor Information

Daniel Krentzel, Email: daniel.krentzel@pasteur.fr.

Ivo G. Boneca, Email: ivo.gomperts-boneca@pasteur.fr

Christophe Zimmer, Email: christophe.zimmer@uni-wuerzburg.de.

Supplementary information

The online version contains Supplementary material available at https://doi.org/10.1038/s41467-026-76355-0.

References

1.Murray, C. J. et al. Global burden of bacterial antimicrobial resistance in 2019: a systematic analysis. Lancet399, 629–655 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
2.O’Neill, J. Tackling Drug-resistant Infections Globally: Final Report and Recommendations. Review on Antimicrobial Resistance. Wellcome Trust and HM Government. (2016).
3.Brown, D. Antibiotic resistance breakers: can repurposed drugs fill the antibiotic discovery void? Nat. Rev. Drug Discov.14, 821–832 (2015). [DOI] [PubMed] [Google Scholar]
4.Krentzel, D., Shorte, S. L. & Zimmer, C. Deep learning in image-based phenotypic drug discovery. Trends Cell Biol.33, 538–554 (2023). [DOI] [PubMed] [Google Scholar]
5.Zampaloni, C. et al. A novel antibiotic class targeting the lipopolysaccharide transporter. Nature625, 566–571 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Carpenter, A. E. et al. CellProfiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol.7, R100 (2006). [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Nonejuie, P., Burkart, M., Pogliano, K. & Pogliano, J. Bacterial cytological profiling rapidly identifies the cellular pathways targeted by antibacterial molecules. PNAS110, 16169–16174 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Zoffmann, S. et al. Machine learning-powered antibiotics phenotypic drug discovery. Sci. Rep.9, 5013 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Bray, M.-A. et al. Cell Painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes. Nat. Protoc.11, 1757–1774 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Moshkov, N. et al. Learning representations for image-based profiling of perturbations. Nat. Commun.15, 1594 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Quach, D. et al. Deep learning-driven bacterial cytological profiling to determine antimicrobial mechanisms in Mycobacterium tuberculosis. Proc. Natl. Acad. Sci. USA. 122, e2419813122 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Harrison, P. J. et al. Evaluating the utility of brightfield image data for mechanism of action prediction. PLOS Comput. Biol.19, e1011323 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Tan, M. & Le, Q. EfficientNet: rethinking model scaling for convolutional neural networks. In Proc. International Conference on Machine Learning 6105–6114 (PMLR, 2019).
14.He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778. 10.1109/CVPR.2016.90. (2016). [DOI]
15.Deng, J. et al. ImageNet: a large-scale hierarchical image database. In Proc. 2009 IEEE Conference on Computer Vision and Pattern Recognition 248–255. 10.1109/CVPR.2009.5206848. (2009). [DOI]
16.Selvaraju, R. R. et al. Grad-CAM: visual explanations from deep networks via gradient-based localization. 618–626 10.1109/ICCV.2017.74 (2017). [DOI]
17.Breunig, M. M., Kriegel, H.-P., Ng, R. T. & Sander, J. LOF: identifying density-based local outliers. In Proc. 2000 ACM SIGMOD International Conference on Management of Data 93–104. 10.1145/342009.335388. (Association for Computing Machinery, 2000). [DOI]
18.Sedaghat, Z. et al. HoloMoA: a holography and deep learning tool for the identification of antimicrobial mechanisms of action and the detection of novel MoA. Front. Microbiol. 16, 1640252 (2025). [DOI] [PMC free article] [PubMed]
19.Cimini, B. A. et al. Optimizing the Cell Painting assay for image-based profiling. Nat. Protoc.18, 1981–2013 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Willis, C., Nyffeler, J. & Harrill, J. Phenotypic profiling of reference chemicals across biologically diverse cell types using the cell painting assay. SLAS Discov.25, 755–769 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Cross-Zamirski, J. O. et al. Label-free prediction of Cell Painting from brightfield images. Sci. Rep.12, 10001 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Işıl, Ç et al. Virtual Gram staining of label-free bacteria using dark-field microscopy and deep learning. Sci. Adv.11, eads2757 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Fontana, R., Cornaglia, G., Ligozzi, M. & Mazzariol, A. The final goal: penicillin-binding proteins and the target of cephalosporins. Clin. Microbiol. Infect.6, 34–40 (2000). [DOI] [PubMed] [Google Scholar]
24.Curtis, N. A., Orr, D., Ross, G. W. & Boulton, M. G. Affinities of penicillins and cephalosporins for the penicillin-binding proteins of Escherichia coli K-12 and their antibacterial activity. Antimicrob. Agents Chemother.16, 533–539 (1979). [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Moosdeen, F., Williams, J. D. & Yamabe, S. Antibacterial characteristics of YTR 830, a sulfone beta-lactamase inhibitor, compared with those of clavulanic acid and sulbactam. Antimicrob. Agents Chemother.32, 925–927 (1988). [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Montaner, M., Lopez-Argüello, S., Oliver, A. & Moya, B. PBP target profiling by β-lactam and β-lactamase inhibitors in intact Pseudomonas aeruginosa: effects of the intrinsic and acquired resistance determinants on the periplasmic drug availability. Microbiol. Spectr.11, e03038–22 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Meberg, B. M., Paulson, A. L., Priyadarshini, R. & Young, K. D. Endopeptidase penicillin-binding proteins 4 and 7 play auxiliary roles in determining uniform morphology of Escherichia coli. J. Bacteriol.186, 8326–8336 (2004). [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Yaeger, L. N. et al. Perturbation of Pseudomonas aeruginosa peptidoglycan recycling by anti-folates and design of a dual-action inhibitor. Mbio. 16, e02984–24 (2025). [DOI] [PMC free article] [PubMed]
29.Daschner, F. Inhibition of cell wall synthesis by sulfonamides and trimethoprim. Chemotherapy22, 12–18 (1976). [DOI] [PubMed] [Google Scholar]
30.Krentzel, D. et al. Deep learning extracts MoA-specific signatures from high-throughput images of chemically and genetically perturbed Corynebacteria. Preprint at 10.64898/2026.02.23.707449 (2026). [DOI] [PMC free article] [PubMed]
31.van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A. & Viergever, M. A. Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Med. Image Anal.79, 102470 (2022). [DOI] [PubMed] [Google Scholar]
32.Kamakshi, V. & Krishnan, N. C. Explainable image classification: the journey so far and the road ahead. AI4, 620–651 (2023). [Google Scholar]
33.Taimeskhanov, M., Sicre, R. & Garreau, D. CAM-based methods can see through walls. in (eds Bifet, A. et al.) Machine Learning and Knowledge Discovery in Databases 332–348 (Springer Nature, 2024).
34.Arevalo, J. et al. Evaluating batch correction methods for image-based cell profiling. Nat. Commun.15, 6516 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations (2017).
36.Paszke, A. et al. PyTorch: an imperative style, high-performance deep learning library. Advances in neural information processing systems 32, (2019).
37.Virtanen, P. et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods17, 261–272 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
39.Kho, K. et al. Deep learning recognises antibiotic modes of action from brightfield images. Data at 10.6019/S-BIAD1851 (2025). [DOI] [PubMed]
40.Krentzel, D. Source data: deep learning recognises antibiotic mode of action from brightfield images. Data at 10.5281/zenodo.20529250 (2026). [DOI] [PubMed]
41.Krentzel, D. Source code: deep learning recognises antibiotic modes of action from brightfield images. Code at 10.5281/zenodo.22093635 [DOI] [PubMed]
42.Kocaoglu, O. & Carlson, E. E. Profiling of β-lactam selectivity for penicillin-binding proteins in Escherichia coli strain DC2. Antimicrob. Agents Chemother.59, 2785–2790 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
43.Shirley, J. D., Nauta, K. M. & Carlson, E. E. Live-cell profiling of penicillin-binding protein inhibitors in Escherichia coli MG1655. ACS Infect. Dis.8, 1241–1252 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
44.Asli, A., Brouillette, E., Krause, K. M., Nichols, W. W. & Malouin, F. Distinctive binding of avibactam to penicillin-binding proteins of Gram-negative and Gram-positive bacteria. Antimicrob. Agents Chemother.60, 752–756 (2016). [DOI] [PMC free article] [PubMed]
45.Davies, T. A. et al. Binding of ceftobiprole and comparators to the penicillin-binding proteins of Escherichia coli, Pseudomonas aeruginosa, Staphylococcus aureus, and Streptococcus pneumoniae. Antimicrob. Agents Chemother.51, 2621–2624 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]
46.Pucci, M. J., Boice-Sowek, J., Kessler, R. E. & Dougherty, T. J. Comparison of cefepime, cefpirome, and cefaclidine binding affinities for penicillin-binding proteins in Escherichia coli K-12 and Pseudomonas aeruginosa SC8329. Antimicrob. Agents Chemother.35, 2312–2317 (1991). [DOI] [PMC free article] [PubMed] [Google Scholar]
47.Jardetzky, O. & Julian, G. R. Chloramphenicol inhibition of polyuridylic acid binding to E. coli ribosomes. Nature201, 397–398 (1964). [DOI] [PubMed] [Google Scholar]
48.Kakegawa, T. Ogawa, T. & Hirose, S. Mode of inhibition of protein synthesis by TE-031 (A-56268). Chemotherapy. (1988).
49.Chukwudi, C. U. & Good, L. Doxycycline inhibits pre-rRNA processing and mature rRNA formation in E. coli. J. Antibiot.72, 225–236 (2019). [DOI] [PubMed] [Google Scholar]
50.Misumi, M., Nishimura, T., Komai, T. & Tanaka, N. Interaction of kanamycin and related antibiotics with the large subunit of ribosomes and the inhibition of translocation. Biochem. Biophys. Res. Commun.84, 358–365 (1978). [DOI] [PubMed] [Google Scholar]
51.Vance-Bryan, K., Guay, D. R. P. & Rotschafer, J. C. Clinical pharmacokinetics of ciprofloxacin. Clin. Pharmacokinet.19, 434–461 (1990). [DOI] [PubMed] [Google Scholar]
52.Tunitskaya, V. L., Khomutov, A. R., Kochetkov, S. N., Kotovskaya, S. K. & Charushin, V. N. Inhibition of DNA gyrase by levofloxacin and related fluorine-containing heterocyclic compounds. Acta Nat.3, 94–99 (2011). [PMC free article] [PubMed] [Google Scholar]
53.Shen, L. L., Kohlbrenner, W. E., Weigl, D. & Baranowski, J. Mechanism of quinolone inhibition of DNA gyrase: appearance of unique norfloxacin binding sites in enzyme-DNA complexes. J. Biol. Chem.264, 2973–2978 (1989). [PubMed] [Google Scholar]
54.McClure, W. R. & Cech, C. L. On the mechanism of rifampicin inhibition of RNA synthesis. J. Biol. Chem.253, 8949–8956 (1978). [PubMed] [Google Scholar]
55.Baker, D. J. et al. The binding of trimethoprim to bacterial dihydrofolate reductase. FEBS Lett.126, 49–52 (1981). [DOI] [PubMed] [Google Scholar]
56.Sabnis, A. et al. Colistin kills bacteria by targeting lipopolysaccharide in the cytoplasmic membrane. eLife10, e65836 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
57.Dixon, R. A. & Chopra, I. Polymyxin B and polymyxin B nonapeptide alter cytoplasmic membrane permeability in Escherichia coli. J. Antimicrob. Chemother.18, 557–563 (1986). [DOI] [PubMed] [Google Scholar]
58.Sutaria, D. S. et al. First penicillin-binding protein occupancy patterns of β-lactams and β-lactamase inhibitors in Klebsiella pneumoniae. Antimicrob. Agents Chemother.62, 10.1128/aac.00282-18 (2018). [DOI] [PMC free article] [PubMed]
59.Spratt, B. G., Jobanputra, V. & Zimmermann, W. Binding of thienamycin and clavulanic acid to the penicillin-binding proteins of Escherichia coli K-12. Antimicrob. Agents Chemother.12, 406–409 (1977). [DOI] [PMC free article] [PubMed] [Google Scholar]
60.López-Argüello, S. et al. Penicillin-binding protein occupancy dataset for 18 β-lactams and 4 β-lactamase inhibitors in Neisseria gonorrhoeae. Microbiol. Spectr.11, e00692-23 (2023). [DOI] [PMC free article] [PubMed]
61.Labia, R., Baron, P., Masson, J. M., Hill, G. & Cole, M. Affinity of temocillin for Escherichia coli K-12 penicillin-binding proteins. Antimicrob. Agents Chemother.26, 335–338 (1984). [DOI] [PMC free article] [PubMed] [Google Scholar]
62.Yotsuji, A. et al. Mechanism of action of cephalosporins and resistance caused by decreased affinity for penicillin-binding proteins in Bacteroides fragilis. Antimicrob. Agents Chemother.32, 1848–1853 (1988). [DOI] [PMC free article] [PubMed] [Google Scholar]
63.Heidary, M. et al. Mechanism of action, resistance, synergism, and clinical implications of azithromycin. J. Clin. Lab. Anal.36, e24427 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Information^{(47.9MB, pdf)}

Reporting Summary^{(70.4KB, pdf)}

Transparent Peer Review File^{(16.4MB, pdf)}

Supplementary File 1^{(1.2MB, png)}

Source Data File^{(3.5MB, xlsx)}

Data Availability Statement

[CR1] 1.Murray, C. J. et al. Global burden of bacterial antimicrobial resistance in 2019: a systematic analysis. Lancet399, 629–655 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR2] 2.O’Neill, J. Tackling Drug-resistant Infections Globally: Final Report and Recommendations. Review on Antimicrobial Resistance. Wellcome Trust and HM Government. (2016).

[CR3] 3.Brown, D. Antibiotic resistance breakers: can repurposed drugs fill the antibiotic discovery void? Nat. Rev. Drug Discov.14, 821–832 (2015). [DOI] [PubMed] [Google Scholar]

[CR4] 4.Krentzel, D., Shorte, S. L. & Zimmer, C. Deep learning in image-based phenotypic drug discovery. Trends Cell Biol.33, 538–554 (2023). [DOI] [PubMed] [Google Scholar]

[CR5] 5.Zampaloni, C. et al. A novel antibiotic class targeting the lipopolysaccharide transporter. Nature625, 566–571 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR6] 6.Carpenter, A. E. et al. CellProfiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol.7, R100 (2006). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR7] 7.Nonejuie, P., Burkart, M., Pogliano, K. & Pogliano, J. Bacterial cytological profiling rapidly identifies the cellular pathways targeted by antibacterial molecules. PNAS110, 16169–16174 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR8] 8.Zoffmann, S. et al. Machine learning-powered antibiotics phenotypic drug discovery. Sci. Rep.9, 5013 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR9] 9.Bray, M.-A. et al. Cell Painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes. Nat. Protoc.11, 1757–1774 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR10] 10.Moshkov, N. et al. Learning representations for image-based profiling of perturbations. Nat. Commun.15, 1594 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR11] 11.Quach, D. et al. Deep learning-driven bacterial cytological profiling to determine antimicrobial mechanisms in Mycobacterium tuberculosis. Proc. Natl. Acad. Sci. USA. 122, e2419813122 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR12] 12.Harrison, P. J. et al. Evaluating the utility of brightfield image data for mechanism of action prediction. PLOS Comput. Biol.19, e1011323 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR13] 13.Tan, M. & Le, Q. EfficientNet: rethinking model scaling for convolutional neural networks. In Proc. International Conference on Machine Learning 6105–6114 (PMLR, 2019).

[CR14] 14.He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778. 10.1109/CVPR.2016.90. (2016). [DOI]

[CR15] 15.Deng, J. et al. ImageNet: a large-scale hierarchical image database. In Proc. 2009 IEEE Conference on Computer Vision and Pattern Recognition 248–255. 10.1109/CVPR.2009.5206848. (2009). [DOI]

[CR16] 16.Selvaraju, R. R. et al. Grad-CAM: visual explanations from deep networks via gradient-based localization. 618–626 10.1109/ICCV.2017.74 (2017). [DOI]

[CR17] 17.Breunig, M. M., Kriegel, H.-P., Ng, R. T. & Sander, J. LOF: identifying density-based local outliers. In Proc. 2000 ACM SIGMOD International Conference on Management of Data 93–104. 10.1145/342009.335388. (Association for Computing Machinery, 2000). [DOI]

[CR18] 18.Sedaghat, Z. et al. HoloMoA: a holography and deep learning tool for the identification of antimicrobial mechanisms of action and the detection of novel MoA. Front. Microbiol. 16, 1640252 (2025). [DOI] [PMC free article] [PubMed]

[CR19] 19.Cimini, B. A. et al. Optimizing the Cell Painting assay for image-based profiling. Nat. Protoc.18, 1981–2013 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR20] 20.Willis, C., Nyffeler, J. & Harrill, J. Phenotypic profiling of reference chemicals across biologically diverse cell types using the cell painting assay. SLAS Discov.25, 755–769 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR21] 21.Cross-Zamirski, J. O. et al. Label-free prediction of Cell Painting from brightfield images. Sci. Rep.12, 10001 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR22] 22.Işıl, Ç et al. Virtual Gram staining of label-free bacteria using dark-field microscopy and deep learning. Sci. Adv.11, eads2757 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR23] 23.Fontana, R., Cornaglia, G., Ligozzi, M. & Mazzariol, A. The final goal: penicillin-binding proteins and the target of cephalosporins. Clin. Microbiol. Infect.6, 34–40 (2000). [DOI] [PubMed] [Google Scholar]

[CR24] 24.Curtis, N. A., Orr, D., Ross, G. W. & Boulton, M. G. Affinities of penicillins and cephalosporins for the penicillin-binding proteins of Escherichia coli K-12 and their antibacterial activity. Antimicrob. Agents Chemother.16, 533–539 (1979). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR25] 25.Moosdeen, F., Williams, J. D. & Yamabe, S. Antibacterial characteristics of YTR 830, a sulfone beta-lactamase inhibitor, compared with those of clavulanic acid and sulbactam. Antimicrob. Agents Chemother.32, 925–927 (1988). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR26] 26.Montaner, M., Lopez-Argüello, S., Oliver, A. & Moya, B. PBP target profiling by β-lactam and β-lactamase inhibitors in intact Pseudomonas aeruginosa: effects of the intrinsic and acquired resistance determinants on the periplasmic drug availability. Microbiol. Spectr.11, e03038–22 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR27] 27.Meberg, B. M., Paulson, A. L., Priyadarshini, R. & Young, K. D. Endopeptidase penicillin-binding proteins 4 and 7 play auxiliary roles in determining uniform morphology of Escherichia coli. J. Bacteriol.186, 8326–8336 (2004). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR28] 28.Yaeger, L. N. et al. Perturbation of Pseudomonas aeruginosa peptidoglycan recycling by anti-folates and design of a dual-action inhibitor. Mbio. 16, e02984–24 (2025). [DOI] [PMC free article] [PubMed]

[CR29] 29.Daschner, F. Inhibition of cell wall synthesis by sulfonamides and trimethoprim. Chemotherapy22, 12–18 (1976). [DOI] [PubMed] [Google Scholar]

[CR30] 30.Krentzel, D. et al. Deep learning extracts MoA-specific signatures from high-throughput images of chemically and genetically perturbed Corynebacteria. Preprint at 10.64898/2026.02.23.707449 (2026). [DOI] [PMC free article] [PubMed]

[CR31] 31.van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A. & Viergever, M. A. Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Med. Image Anal.79, 102470 (2022). [DOI] [PubMed] [Google Scholar]

[CR32] 32.Kamakshi, V. & Krishnan, N. C. Explainable image classification: the journey so far and the road ahead. AI4, 620–651 (2023). [Google Scholar]

[CR33] 33.Taimeskhanov, M., Sicre, R. & Garreau, D. CAM-based methods can see through walls. in (eds Bifet, A. et al.) Machine Learning and Knowledge Discovery in Databases 332–348 (Springer Nature, 2024).

[CR34] 34.Arevalo, J. et al. Evaluating batch correction methods for image-based cell profiling. Nat. Commun.15, 6516 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR35] 35.Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations (2017).

[CR36] 36.Paszke, A. et al. PyTorch: an imperative style, high-performance deep learning library. Advances in neural information processing systems 32, (2019).

[CR37] 37.Virtanen, P. et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods17, 261–272 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR38] 38.Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).

[CR39] 39.Kho, K. et al. Deep learning recognises antibiotic modes of action from brightfield images. Data at 10.6019/S-BIAD1851 (2025). [DOI] [PubMed]

[CR40] 40.Krentzel, D. Source data: deep learning recognises antibiotic mode of action from brightfield images. Data at 10.5281/zenodo.20529250 (2026). [DOI] [PubMed]

[CR41] 41.Krentzel, D. Source code: deep learning recognises antibiotic modes of action from brightfield images. Code at 10.5281/zenodo.22093635 [DOI] [PubMed]

[CR42] 42.Kocaoglu, O. & Carlson, E. E. Profiling of β-lactam selectivity for penicillin-binding proteins in Escherichia coli strain DC2. Antimicrob. Agents Chemother.59, 2785–2790 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR43] 43.Shirley, J. D., Nauta, K. M. & Carlson, E. E. Live-cell profiling of penicillin-binding protein inhibitors in Escherichia coli MG1655. ACS Infect. Dis.8, 1241–1252 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR44] 44.Asli, A., Brouillette, E., Krause, K. M., Nichols, W. W. & Malouin, F. Distinctive binding of avibactam to penicillin-binding proteins of Gram-negative and Gram-positive bacteria. Antimicrob. Agents Chemother.60, 752–756 (2016). [DOI] [PMC free article] [PubMed]

[CR45] 45.Davies, T. A. et al. Binding of ceftobiprole and comparators to the penicillin-binding proteins of Escherichia coli, Pseudomonas aeruginosa, Staphylococcus aureus, and Streptococcus pneumoniae. Antimicrob. Agents Chemother.51, 2621–2624 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR46] 46.Pucci, M. J., Boice-Sowek, J., Kessler, R. E. & Dougherty, T. J. Comparison of cefepime, cefpirome, and cefaclidine binding affinities for penicillin-binding proteins in Escherichia coli K-12 and Pseudomonas aeruginosa SC8329. Antimicrob. Agents Chemother.35, 2312–2317 (1991). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR47] 47.Jardetzky, O. & Julian, G. R. Chloramphenicol inhibition of polyuridylic acid binding to E. coli ribosomes. Nature201, 397–398 (1964). [DOI] [PubMed] [Google Scholar]

[CR48] 48.Kakegawa, T. Ogawa, T. & Hirose, S. Mode of inhibition of protein synthesis by TE-031 (A-56268). Chemotherapy. (1988).

[CR49] 49.Chukwudi, C. U. & Good, L. Doxycycline inhibits pre-rRNA processing and mature rRNA formation in E. coli. J. Antibiot.72, 225–236 (2019). [DOI] [PubMed] [Google Scholar]

[CR50] 50.Misumi, M., Nishimura, T., Komai, T. & Tanaka, N. Interaction of kanamycin and related antibiotics with the large subunit of ribosomes and the inhibition of translocation. Biochem. Biophys. Res. Commun.84, 358–365 (1978). [DOI] [PubMed] [Google Scholar]

[CR51] 51.Vance-Bryan, K., Guay, D. R. P. & Rotschafer, J. C. Clinical pharmacokinetics of ciprofloxacin. Clin. Pharmacokinet.19, 434–461 (1990). [DOI] [PubMed] [Google Scholar]

[CR52] 52.Tunitskaya, V. L., Khomutov, A. R., Kochetkov, S. N., Kotovskaya, S. K. & Charushin, V. N. Inhibition of DNA gyrase by levofloxacin and related fluorine-containing heterocyclic compounds. Acta Nat.3, 94–99 (2011). [PMC free article] [PubMed] [Google Scholar]

[CR53] 53.Shen, L. L., Kohlbrenner, W. E., Weigl, D. & Baranowski, J. Mechanism of quinolone inhibition of DNA gyrase: appearance of unique norfloxacin binding sites in enzyme-DNA complexes. J. Biol. Chem.264, 2973–2978 (1989). [PubMed] [Google Scholar]

[CR54] 54.McClure, W. R. & Cech, C. L. On the mechanism of rifampicin inhibition of RNA synthesis. J. Biol. Chem.253, 8949–8956 (1978). [PubMed] [Google Scholar]

[CR55] 55.Baker, D. J. et al. The binding of trimethoprim to bacterial dihydrofolate reductase. FEBS Lett.126, 49–52 (1981). [DOI] [PubMed] [Google Scholar]

[CR56] 56.Sabnis, A. et al. Colistin kills bacteria by targeting lipopolysaccharide in the cytoplasmic membrane. eLife10, e65836 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR57] 57.Dixon, R. A. & Chopra, I. Polymyxin B and polymyxin B nonapeptide alter cytoplasmic membrane permeability in Escherichia coli. J. Antimicrob. Chemother.18, 557–563 (1986). [DOI] [PubMed] [Google Scholar]

[CR58] 58.Sutaria, D. S. et al. First penicillin-binding protein occupancy patterns of β-lactams and β-lactamase inhibitors in Klebsiella pneumoniae. Antimicrob. Agents Chemother.62, 10.1128/aac.00282-18 (2018). [DOI] [PMC free article] [PubMed]

[CR59] 59.Spratt, B. G., Jobanputra, V. & Zimmermann, W. Binding of thienamycin and clavulanic acid to the penicillin-binding proteins of Escherichia coli K-12. Antimicrob. Agents Chemother.12, 406–409 (1977). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR60] 60.López-Argüello, S. et al. Penicillin-binding protein occupancy dataset for 18 β-lactams and 4 β-lactamase inhibitors in Neisseria gonorrhoeae. Microbiol. Spectr.11, e00692-23 (2023). [DOI] [PMC free article] [PubMed]

[CR61] 61.Labia, R., Baron, P., Masson, J. M., Hill, G. & Cole, M. Affinity of temocillin for Escherichia coli K-12 penicillin-binding proteins. Antimicrob. Agents Chemother.26, 335–338 (1984). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR62] 62.Yotsuji, A. et al. Mechanism of action of cephalosporins and resistance caused by decreased affinity for penicillin-binding proteins in Bacteroides fragilis. Antimicrob. Agents Chemother.32, 1848–1853 (1988). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR63] 63.Heidary, M. et al. Mechanism of action, resistance, synergism, and clinical implications of azithromycin. J. Clin. Lab. Anal.36, e24427 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Deep learning recognises antibiotic modes of action from brightfield images

Daniel Krentzel

Kelvin Kho

Julienne Petit

Nassim Mahtal

Thomas Delerue

Max E Huber

Agnès Zettor

Jeanne Chiaravalli

Spencer L Shorte

Mark Brönstrup

Anne Marie Wehenkel

Ivo G Boneca

Christophe Zimmer

Abstract

Introduction

Results

Collecting a high-throughput image dataset of bacterial phenotypes

Fig. 1. Data acquisition, model training and inference.

Fig. 2. Deep learning recognises compounds and their MoA from E. coli images.

Table 1.

Deep learning model and training strategy

Deep learning accurately recognises the antibiotic mode of action from images

Brightfield images alone allow accurate recognition of drug MoA

Fig. 3. Classification performance for different numbers of imaging channels and training images.

CNN learns MoA-specific features from only eight images per condition

Detecting drug phenotypes at subinhibitory concentrations

Fig. 4. Detecting drug exposure at subinhibitory concentrations.

Recognising known modes of action of unseen drugs

Fig. 5. Assigning the MoA of unseen compounds.

Detecting drugs with unseen modes of action

Fig. 6. Detecting antibiotics with novel MoAs.

MoA recognition from brightfield images of Klebsiella pneumoniae

Table 2.

Fig. 7. Recognising drug MoA from brightfield images in K. pneumoniae.

Discussion

Fig. 8. Complementarity of the proposed imaging-based pipeline with growth inhibition assays.

Methods

Bacterial growth protocol for E. coli

Bacterial growth protocol for K. pneumoniae

Penicillin-binding protein affinity assay

High-throughput imaging

Image preprocessing

Removal of technical variation with sphering

Deep learning model training

Computing compound predictions conditioned on the dose

Estimating the minimum number of required images

Splitting images by bacterial density

MoA assignment with k-NN classifier

MoA novelty detection with the local outlier factor algorithm

Grad-CAM saliency maps

Statistical tests

Software packages

Reporting summary

Supplementary information

Source data

Acknowledgements

Author contributions

Peer review

Peer review information

Funding

Data availability

Code availability

Competing interests

Footnotes

Contributor Information

Supplementary information

References

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases