SIC50: Determining drug inhibitory concentrations using a vision transformer and an optimized Sobel operator

Summary

As a measure of cytotoxic potency, half-maximal inhibitory concentration (IC50) is the concentration at which a drug exerts half of its maximal inhibitory effect against target cells. It can be determined by various methods that require applying additional reagents or lysing the cells. Here, we describe a label-free Sobel-edge-based method, which we name SIC50, for the evaluation of IC50. SIC50 classifies preprocessed phase-contrast images with a state-of-the-art vision transformer and allows for the continuous assessment of IC50 in a faster and more cost-efficient manner. We have validated this method using four drugs and 1,536-well plates and also built a web application. We anticipate that this method will assist in the high-throughput screening of chemical libraries (e.g., small-molecule drugs, small interfering RNA [siRNA], and microRNA and drug discovery).

Graphical abstract

Highlights

• •SIC50 allows label-free determination of IC50 of anti-cancer drugs
• •It leverages the power of deep learning and enables continuous evaluations
• •SIC50 is faster and more cost effective than conventional methods
• •This method can be adapted for screening drugs that affect cell density and shapes

The bigger picture

One in three people will be diagnosed with a type of cancer in their lifetime.³⁷ In 2019, $141.3 billion USD was spent on oncology medicines globally. This number was projected to reach $394.2 billion USD by 2027.³⁸ The development of cancer therapeutics is expensive, costing approximately $648 million USD per drug in 2017,³⁹ partially due to the necessary screening of tens of thousands of chemicals, which can be prohibitively costly. Thus far, many clinical trials have failed because they were not able to start with the best possible molecule. Therefore, a more affordable method for the efficient evaluation of drug efficacy and the screening of cancer therapeutics is urgently needed to advance the field of cancer treatment.

The cost of developing cancer therapeutics has been increasing in the past decades. To tackle this challenge, we developed a more affordable method for the evaluation of drug potency employing deep learning and validated the method using 4 different drugs. We anticipate that this method will empower the high-throughput screening of chemical libraries that affect cell density and morphologies.

Introduction

Current methods for cell-based screening of anti-cancer compounds require reagents and several steps to evaluate drug potency. For example, developed in 1983, the 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyl-2H-tet-razolium bromide (MTT) assay has been widely adopted to determine the efficacy of drugs (Figure 1A).¹ The positively charged, yellow MTT molecules penetrate viable cells and become reduced to purple formazan crystals by mitochondrial dehydrogenase.^2,3 The crystals are then dissolved in dimethyl sulfoxide (DMSO; 100 μL/well) or sodium dodecyl sulfate (SDS; 10% w/v in 0.01 M hydrochloric acid, 100 μL/well) to become a colored solution with an absorbance maximum near 570 nm, which is quantified by a spectrophotometer. The absorbance is proportional to the number of live cells. The darker the solution, the greater the number of viable cells. Thus, this colorimetric assay measures metabolic activity as an indicator of cell viability.^4,5,6 Cell counting kit-9 (CCK-8) is another commonly used assay. It is based on the reduction of water-soluble tetrazolium 8 (WST-8) to 1-methoxy phenazinium methylsulfate (PMS) by nicotinamide adenine dinucleotide (NAD)- or nicotinamide adenine dinucleotide phosphate (NADP)-dependent dihydrogenases in a cell. Compared with MTT, CCK-8 is more sensitive and does not require dissolving solutions.⁷ In an adenosine triphosphate (ATP) assay, cells are lysed to release the ATP, which activates luciferin and yields a luciferyl-adenylate and pyrophosphate. The luciferyl-adenylate reacts with oxygen to generate carbon dioxide and oxyluciferin in an electronically excited state, which releases bioluminescence (550–570 nm) when returned to the ground state. Proportional to the ATP levels and the number of viable cells, the luminescent signal can be quantified to evaluate the inhibitory concentration 50% (IC50) of a cancer drug, the concentration at which half of the cancer cells are killed. Lastly, the IC50 of a drug can also be determined by counting the nuclei stained with Hoechst dye. However, Hoechst staining inhibits the growth of cells.

Figure 1.

Comparison of SIC50 and the cell-based viability assays commonly used for anti-cancer compound screening

(A) The workflow for an MTT assay. MTT (0.2–0.5 mg/mL) is reduced to its insoluble formazan by nicotinamide adenine dinucleotide phosphate (NADP)-dependent oxidoreductase in metabolically active cells. The absorbance of MTT formazan is measured at 570 nm.

(B) Cell counting kit-8 (CCK-8) assays rely on the reduction of water-soluble tetrazolium 8 (WST-8) to 1-methoxy phenazinium methylsulfate (PMS).

(C) In ATP assays, cells are lysed to release ATP. ATP helps activate luciferin and generate a luciferyl-adenylate and pyrophosphate. Luciferyl-adenylate will be oxidized and yield electronically excited oxyluciferin, which will return to the ground state and release yellow luminescent light.

(D) Hoechst inhibits the growth of cells. Images are obtained from B16-F10 cells. In the control group, the cells were stained with 5 μg/mL Hoechst 33342 for 20 min. In the Hoechst group, the cells were incubated with Hoechst for 24 h. Scale bars are 40 μm. Statistical analyses of the cell numbers from the control group and the Hoechst group suggest that the two groups are significantly different. n = 5, Mann-Whitney test, p = 0.008.

(E) The architecture of the Conv2D used in this study.

(F) The procedures of SIC50. SIC50 determines the IC50 values of drugs by analyzing phase-contrast images modified with a Sobel operator. It avoids operational errors, saves time and reagent cost, will not be interfered by chemicals with absorbances between 450 and 600 nm, and allows for multi-time point assessments of IC50.

(G and H) The overview of a vision transformer. Images are split into patches, position embedded, and fed into a vision transformer encoder. An extra learnable embedding is added for classification.

Deep learning has been used to analyze histological images,^8,9 study the differentiation of induced pluripotent stem cells,^{10,11,12,13,14} and perform binary classifications (live or dead,¹⁵ drug treated or untreated¹⁶) on cancer cells. As a self-attention-based deep neural network, a transformer was developed in 2017¹⁷ for tasks in the field of natural language processing and employed in computer vision applications since 2020¹⁸ thanks to its strong representation capabilities and less need for vision-specific inductive bias.

A vision transformer can be built by splitting images into patches, embedding positions, and adding a learnable “classification token.” A transformer encoder typically consists of multiheaded self-attention and multilayer perceptrons (MLPs) containing two layers with Gaussian error linear unit (GELU) non-linearity. Normalization is applied before every block of the encoder to estimate the normalization statistics from the summed inputs to the neurons within a hidden layer.

In a convolutional neural network (CNN), a non-linear activation function is applied to each layer, followed by a max pooling layer to reduce the dimensionality of images and retain the most prominent features. To prevent the neural networks from overfitting, a dropout layer is added between the hidden layer and the output layer. The neuron in fully connected layers applies a linear transformation to the input vector through a weight matrix. A non-linear transformation is then applied to the product through a non-linear activation function, such as a sigmoid function for the binary classification and a softmax function for the multinomial classification.

Results

Computer-vision-based methods outperform biochemical approaches

In this study, a vision transformer and CNNs (Conv2D) were built to classify preprocessed phase-contrast images and predict the IC50 of drugs (Figure 1). Compared with the widely used MTT assay (Figure 1A), our method has the following advantages: it (1) avoids operational errors and saves time associated with adding the reagents; (2) reduces costs associated with the labeling reagent (i.e., MTT), balancing buffer, filters, and dissolving solutions; (3) does not require incubation time; and (4) can be used to screen a broader range of chemicals because compounds with absorbance from 450 to 600 nm or with antioxidant properties will interfere with the MTT or CCK-8 absorbance measurement,^17,18 and (5) the cells do not need to be in the log phase using our method. In MTT assays, however, cells need to be in the log phase to ensure the linearity between absorbances and cell numbers; (6) our method is not cytotoxic, permitting multi-time point measurements (Figure 1F).

To assess IC50 with deep learning, melanoma cells are treated with different drugs at various concentrations. Data are then collected using a high-throughput automated imaging system (i.e., ImageXpress Pico). Each drug has more than 2,000 images for training and more than 200 images for testing (Tables 1 and S1; Figure S1). For example, the training folder for paclitaxel contains 4,320 images, which are evenly divided between the treated and untreated groups.

Table 1.

Number of images used for training and testing

Drug	Train	Test
Paclitaxel	4,320	480
Cephalotaxine	5,040	560
Fasudil	5,600	600
Irinotecan	4,680	520

The testing folder has 480 images (half from the treated group and half from the untreated group), accounting for 10% of the total paclitaxel dataset. Each image contains various numbers of cells with different densities and morphological features (Figure 2A). The raw image is a 2007 x 2007 16-TIFF file and was converted to an 8-bit PNG and split into 100 patches. The images are augmented by a 40° rotation and a horizontal flip. The pixel values (0–255) of the images are divided by 255 to rescale the data.

Figure 2.

Optimized Sobel operator improves classification accuracy

(A) The B16-F10 cells were treated with 200 μM paclitaxel for 24 h. CNNs cannot accurately classify untreated and treated cells before preprocessing images (accuracy = 50.5%). The RGB values of the background are large and relatively close to that of cells. Scale bars are 15 μm.

(B) Gaussian high-pass (GHP) does not significantly improve the prediction accuracy (51.2%). The difference between the background and the signal is small. Scale bars are 15 μm.

(C) Sobel filter reduces the grayscale values of the background, increases the signal-to-noise ratio, and improves prediction accuracy to 98.2%. Scale bars are 15 μm.

(D) An optimized Sobel (OSobel) operator further improved the prediction accuracy to 99.8%. R, red; G, green; B, blue. Scale bars are 15 μm.

(E) The classification accuracies of CNNs using cells treated with different concentrations of cephalotaxine. The red curve is the same as the one in Figure 3C.

(F) The classification accuracies of CNNs using cells treated with different concentrations of fasudil. The red curve is the same as the one in Figure 3D.

After data preprocessing, binary classification is conducted using untreated cells as the control. We notice that the classification accuracies are low before preprocessing the images (Figure 2A) or modifying the images with a high-pass algorithm (Figure 2B). The accuracy is significantly improved when the images are processed with a Sobel operator (Figure 2C) and an optimized Sobel (OSobel) operator (Figure 2D), likely due to the operator reducing the grayscale values of the background and increasing the signal-to-noise ratio (Figures 2A and 2D). Regardless of the preprocessing methods, the accuracies are equally poor at low concentrations for treated cells (Figures 2E and 2F). However, when the cells are treated with drugs (e.g., cephalotazine and fasudil) at high concentrations, the classification accuracies using images preprocessed with an OSobel algorithm are higher than that of Gaussian high-pass and unprocessed images (Figures 2E and 2F). Thus, OSobel is employed in the experiments thereafter.

Determination of IC50 with deep learning

We observe that the accuracy of the binary classification can be used to predict the IC50 of a drug (Figure 3A). As a reference, the cell viability and IC50 of drugs are first determined using Hoechst staining and are represented as blue dots. Binary classifications are performed using untreated cells and the cells treated with drugs at various concentrations. The classification accuracies at different concentrations are labeled as red squares. The accuracies of binary classification are close to 50% when the cells are treated with drugs at low concentrations, suggesting that the drugs do not result in detectable changes in cellular morphology. When drawing lines connecting every pair of adjacent red squares, the line with the highest slope contains values close to the IC50s of drugs (Figures 3B–3F). Although the average value of these two adjacent concentrations (SIC50) can be different from the IC50s estimated using Hoechst staining by 2-fold (HIC50; Table 2), mapping the IC50 from a large range of concentrations (0.03–200 μM, 6,666-fold) to a 2-fold variation can be helpful and sufficient for many purposes. In addition, 1.5- or 1.25-fold dilutions could be adopted to further improve the prediction accuracy of SIC50. Although the classification accuracies using a vision transformer are close to that of Conv2D (Figures 3B–3E), we expect that the vision transformer would perform better when pretrained at a large scale and transferred to tasks with fewer images.¹⁸

Figure 3.

Determination of IC50s of drugs by SIC50

(A) Binary classifications are performed using the untreated cells as a control.

(B–E) The cell viability (blue curves) and classification accuracies (red curves) in cells treated with drugs at various concentrations respectively, including paclitaxel (B) at 91.4, 30.5, 10.2, 3.4, 1.1, 0.38, 0.13, 0.04, and 0.014 nM, cephalotaxine (C) and fasudil (D) at 200, 66.7, 22.2, 7.4, 2.5, 0.82, 0.27, 0.09, and 0.03 μM, and irinotecan (E) at 138.9, 46.3, 15.4, 5.1, 1.7, 0.57, and 0.19 μM.

Table 2.

IC50 values determined by Hoechst staining and Conv2 and a vision transformer

IC50s	Hoechst (HIC50)	SIC50	Difference
Paclitaxel (nM)	31.50	60.91	1.93
Cephalotaxine (μM)	32.94	44.45	1.35
Fasudil (μM)	60.27	133.35	2.21
Irinotecan (μM)	6.03	10.25	1.70

To compare the performance of the deep-learning models, two experiments are conducted. First, a small number of images (n = 100) are used to train both models. We observe that the vision transformer is faster and more accurate than Conv2D (Table 3). The accuracy of Conv2D is improved and becomes comparable to the accuracy of the vision transformer when a slightly larger dataset (n = 250) is used for training. However, it is still significantly slower than the vision transformer (Table 4). In the task of IC50 prediction, speed (i.e., the time needed to obtain the accuracy curve shown in Figures 3B–3E) is more important than exact accuracy because the goal is to identify the concentrations at which classification accuracy exponentially increases. For this reason, the vision transformer is better than Conv2D in IC50 prediction and is employed in our web application.

Table 3.

Test results using a small dataset

Model	Time (min)	Accuracy	TP	TN	FP	FN	Sensitivity	Precision	Specificity	Neg pred
Conv2D	9:38	0.87	207 (74%)	278 (99%)	2 (1%)	73 (26%)	0.74	0.99	0.99	0.79
Transformer	3:17	0.91	276 (99%)	232 (83%)	48 (17%)	4 (1%)	0.99	0.85	0.83	0.98

TP, true positive; TN, true negative; FP, false positive, FN, false negative, Neg pred, negative predictive.

Table 4.

Test results using a larger dataset

Model	Time (min)	Accuracy	TP	TN	FP	FN	Sensitivity	Precision	Specificity	Neg pred
Conv2D	15:45	0.97	273 (98%)	272 (97%)	8 (3%)	7 (2%)	0.98	0.97	0.97	0.98
Transformer	5:59	0.96	276 (99%)	263 (94%)	17 (6%)	4 (1%)	0.99	1.00	0.94	0.81

Discussion

To develop new therapeutics, companies and research laboratories have been attempting to leverage the power of artificial intelligence for the analysis of different data including genomic information (e.g., Deep Genomics, Verge Genomics, etc.); clinical datasets (Data4Cure, Molecular Health, etc.); small molecular structures (Insilico Medicine, neoX Biotech, etc.); adeno-associated virus sequences (Dyno)^19,20; and cellular images (Recursion Pharma., Noul, etc.).

Deep learning has been employed for cellular analyses including cell segmentation, tracking, and classification²¹; monitoring the morphological dynamics of reprogrammed stem cells¹¹; analyzing histopathology images for cancer diagnoses and prognoses²²; and identifying clinical biomarkers to predict response to chimeric antigen receptor T cell (CAR-T) immunotherapy.²³ Pattarone et al. trained CNN models to classify living and dead breast cancer cells (JIMT-1) treated with doxorubicin at one concentration (0.01 M).¹⁵ ResNET^24,25 can be trained to classify cellular images. It contained 17 convolution layers that employ 3∗3 filters and a stride of 2 and 1 fully connected layer, followed by a softmax layer. The first five layers had 64 kernels. Layers 6–9, layers 10–13, and layers 14–17 have 128, 256, and 512 kernels, respectively. In addition, SqueezeNet²⁶ can be utilized for cell analysis. It begins with a convolution layer, followed by 8 fire models, and ends with a convolution layer and a softmax layer. The fire model consists of a 1∗1 convolution filter named the squeeze layer and a mix of 1∗1 and 3∗3 filters called the expand layer. Support vector machine (SVM) classifiers were used to distinguish drug-treated MCF-7 cells from untreated cells with an accuracy of 92%. Single-cell images were captured by an optofluidic time-stretch microscope after being treated with paclitaxel for 12 and 24 h. These models were used for binary classification of cancer cells (live or dead,¹⁵ drug treated or untreated¹⁶), and the cells were treated with the drug at only one concentration.¹⁵ The classification accuracies did not change significantly when the cells were treated for 48 h, probably due to differences in preprocessing methods.¹⁶ Thus, the accuracies were not used for the assessment of IC50.

The SIC50 models were tested using drugs with different mechanisms of action. For example, paclitaxel stabilizes microtubules, increases microtubule polymerization, decreases microtubule depolymerization, prevents mitosis, and blocks cell-cycle progression.²⁷ Cephalotaxin inhibits the growth of cancer cells by activating the mitochondrial apoptosis pathway.²⁸ Fasudil is a calcium channel blocker and inhibits the Rho-kinase signaling pathway.^29,30 Irinotecan is a prodrug of 7-ethyl-10-hydroxycamptothecin (SN-38), which forms complexes with topoisomerase 1B and DNA and causes DNA misalignment and cell death.³¹ Therefore, we anticipate this method can be used for screening different categories of drugs.

Conclusions

Validated in four drugs, we anticipate that our method will empower drug discovery and research in pharmacology by facilitating the high-throughput screening of chemical libraries using 1,536-well plates and imaging platforms such as the Cytation 5, helping evaluate the potency of other small-molecule drugs, small interfering RNA (siRNA), and microRNA and assessing the cytotoxicity of delivery vehicles for drugs and genes such as lipid nanoparticles,³² polymers, and adeno-associated viruses.^33,34,35,36 In addition, we envision that our method can be modified to facilitate biomedical research related to changes in cellular morphology, e.g., cancer cell metastasis, stem cell differentiation, neural plasticity, and so on.

Experimental procedures

Resource availability

Lead contact

Kit Lam (kslam@ucdavis.edu).

Materials availability

This study did not generate any new reagents or materials.

Methods

Architecture of Conv2D

The Conv2D is constructed using the TensorFlow framework. The network consists of 4 convolutional layers with the rectified linear unit (ReLU). The first two convolutional layers have 64 kernels, while the third and fourth convolutional layers have 128 kernels. At each convolutional layer, the convolution is performed by sliding the filter over the input data to extract the hidden features from the data. Each convolutional layer is followed by a 2∗2 max pooling layer to reduce the feature dimension by keeping only the most relevant features. The output from the last pooling layer is then flattened and presented to a dropout layer with a rate of 0.5 to avoid overfitting. The last 2 layers in the Conv2D are fully connected layers with 512 and 2 neurons, respectively, followed by a to obtain the final classification result. The parameters of the network are trained on over 50 epochs randomly using the training data (90% for training and 10% for testing). The accuracy of the classifier is then computed by evaluating the fitted model using the test data.

Architecture of a vision transformer

Images are resized to 72 × 72 and split into 6 × 6 patches to build a transformer encoder together with position embeddings and a learnable “classification token.” Layer normalization is implemented before every block of the transformer encoder containing multiheaded self-attention and MLPs. Each MLP consists of two layers with GELU non-linearity. We use a learning rate of 0.001 and a weight decay of 0.0001, a batch size of 200, 100 epochs, 4 transformer layers, a dropout rate of 0.1, and MLP head units of [2,048, 1,024].

Cell culture and drug treatment

A vial containing 1 mL cancer cells (B16-F10) is restored in liquid nitrogen and thawed in a water bath at 37°C, followed by mixing with 9 mL of Dulbecco’s modified Eagle medium (DMEM) complete cell culture medium (90% DMEM, 10% FBS, and 1% penicillin/streptomycin) and centrifuged at 800 × g for 4 min. After discarding the supernatant, the cell pellet is resuspended in 10 mL culture medium, seeded in a T75 flask, and placed in an incubator at 37°C and 5% CO₂. Twelve hours later, B16-F10 cells are seeded in plates (96-well plates at a density of 6,000 cells/well, 384 well plates at a density of 1,800 cells/well, and 1,536-well plates at a density of 240 cells/well) in complete growth media and cultured for 12 h in 37°C incubators. The initial cell density can vary among different cell types depending on their proliferation rates. The key is to avoid cell overcrowding 36 h later because cell over-confluency can lead to inaccurate cell count using Hoechst and ImageJ as described below. Before treating the cells with drugs, stock solutions are prepared by dissolving the drugs in DMSO, followed by a serial dilution with the cell culture medium. Different drugs may have different initial concentrations depending on their solubility (e.g., 200 μM cephalotaxine, 138.9 μM for irinotecan, etc.). For the serial dilution, 1 vol of the previous solution is mixed with 2 vol culture medium (i.e., 1:3 dilution). A smaller ratio can be used for better accuracy. The old medium is removed before the drugs are subsequently added at various concentrations and incubated for an additional 24 h in the incubator at 37°C and 5% CO₂. The drugs and materials are procured from different sources: paclitaxel (LC Laboratories, cat. P-9600, CAS no. 33069-62-4); cephalotaxine (Toronto Research Chemicals, cat. C261050, CAS no. 24316-19-6); irinotecan (APExBIO, cat. B2293, CAS no. 136572-09-3); fasudil (Toronto Research Chemicals, cat. R1036); cantharidin (Sigma Aldrich, cat. C7632, CAS no. 56-25-7); B16-F10 cells (ATCC, cat. CRL-6475); DMEM (Thermo Fisher Scientific, cat. 11-995-073); fetal bovine serum (FBS; Thermo Fisher Scientific, cat. 10-437-028); penicillin-streptomycin (PS; Thermo Fisher Scientific, cat. 15070063); T-75 flasks (Corning, product no. 430825); 96-well plates (Corning, product no. 3595, and Thermo Fisher Scientific, cat. no. FB012931); 384-well plates (Corning 3833); and 1,536-well plates (Corning 3712).

Determination of IC50 by Hoechst staining

The cells are incubated with culture media containing 1 μg/mL Hoechst 33342 (Thermo Fisher Scientific, cat. no. H3570), for 10 min. The IC50 is calculated after counting the number of nuclei with CellProfiller and curve fitting with GraphPad Prism 9. Approximately, the IC50 equals the x axis value of the intersecting point of a blue curve in Figure 3 and the horizontal dashed line crossing the y axis at 50%.

Image collection and preprocessing

Images are collected by a high-throughput imaging system (Molecular Devices, ImageXpress Pico) using a 4× objective (29.8% of well) and exported as a 16-bit TIFF. For general information related to Pico, please refer to the “ImageXpress Pico Automated Cell Imaging System User Guide.” The images are converted into 8-bit TIFF, then into 8-bit PNG by ImageJ, and processed using different algorithms (code posted on GitHub). Afterward, each image is split into 100 smaller images using PhotoScape X by opening the file under “viewer,” right-clicking the image, selecting “split,” and specifying the “columns” and “rows.” The step can be completed using online applications or Jupiter Notebook. Next, the data are augmented to generate more images for training.

Published: February 3, 2023

Data and code availability

The codes and images have been deposited at Zenodo (https://doi.org/10.5281/zenodo.7509014) and are publicly available as of the date of publication. Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request. The link to the web application is biochemml.com/image/.