From cell lines to cancer patients: personalized drug synergy prediction

Abstract

Motivation

Combination drug therapies are effective treatments for cancer. However, the genetic heterogeneity of the patients and exponentially large space of drug pairings pose significant challenges for finding the right combination for a specific patient. Current in silico prediction methods can be instrumental in reducing the vast number of candidate drug combinations. However, existing powerful methods are trained with cancer cell line gene expression data, which limits their applicability in clinical settings. While synergy measurements on cell line models are available at large scale, patient-derived samples are too few to train a complex model. On the other hand, patient-specific single-drug response data are relatively more available.

Results

In this work, we propose a deep learning framework, Personalized Deep Synergy Predictor (PDSP), that enables us to use the patient-specific single drug response data for customizing patient drug synergy predictions. PDSP is first trained to learn synergy scores of drug pairs and their single drug responses for a given cell line using drug structures and large scale cell line gene expression data. Then, the model is fine-tuned for patients with their patient gene expression data and associated single drug response measured on the patient ex vivo samples. In this study, we evaluate PDSP on data from three leukemia patients and observe that it improves the prediction accuracy by 27% compared to models trained on cancer cell line data.

Availability and implementation

PDSP is available at https://github.com/hikuru/PDSP.

1 Introduction

Combination therapies, in which multiple drugs are administered concurrently, have emerged as alternatives to single drug therapies for various complex diseases including cancer (Liu et al. 2011, Al-Lazikani et al. 2012, Palmer and Sorger 2017, Lee et al. 2019, Kopetz et al. 2019), diabetes (Home et al. 2009), human immunodeficiency virus (De Clercq 2007), and SARS-CoV-2 (Wang et al. 2022b). These therapies are reported to provide more effective treatment (Csermely et al. 2013, Sun et al. 2015), overcome drug resistance (Hill et al. 2013), and decrease side effects by enabling reduced doses (Al-Lazikani et al. 2012, Zhao et al. 2013). While combination therapies confer benefits to cancer patients compared to single drug ones, personalizing drug combinations for individual patients is challenging. The effectiveness of drug combinations varies across patients due to differences in their genotypic and phenotypic variation (Pang et al. 2014, Day and Siu 2016).

To facilitate drug combination discovery, preclinical studies rely on screening drug combinations in cancer cell lines experimentally. However, even for medium-sized drug panels, exhaustive experimentation with all possible combinations is impossible as the space of possible combinations increases exponentially with the number of drugs (Macarron et al. 2011, Torres et al. 2013, He et al. 2018a). To aid the combination discovery efforts, though most are not personalized, many in silico methods have been proposed (Bulusu et al. 2016, Michael et al. 2019). Various mathematical modeling techniques (Zhao et al. 2011), network-based models (Cheng et al. 2019), and classical machine learning techniques have been applied to predict drug synergy (Bansal et al. 2014, Li et al. 2015, Wildenhain et al. 2015, Joseph et al. 2018). These models have been trained on various types of data, including drug chemical properties, cell line untreated and/or treated gene expression (Yang et al. 2015), drug-target interaction (Bansal et al. 2014), and protein-protein interaction networks (Cheng et al. 2019). Yet, the prediction performance was limited due to limited training data.

Over the last decade, the research community focused on compiling large scale drug synergy datasets and organized several challenges to develop more advanced methods to exploit these resources (Bansal et al. 2014, Yu 2015). For instance, AstraZeneca and Sanger (Yu 2015) released >10 000 experimentally tested drug combinations, where cell viability was measured across various cell lines. Specifically, this DREAM challenge considered 11 759 drug combination screenings that span 85 compounds and 85 cell lines. Similarly, NCI-ALMANAC study (Susan et al. 2017), evaluated the combination effects of 104 FDA-approved cancer drugs on 60 cell lines. This dataset contains nearly 300 000 drug pair-cell line combinations.

With the availability of large-scale training data, we (Kuru et al. 2022) and others (Preuer et al. 2017, Joseph et al. 2018, Liu and Xie 2021, Zhang et al. 2021, Wang et al. 2022a) have developed deep learning based solutions to identify synergistic drug combinations. Preuer et al. presented DeepSynergy, which takes the chemical structures and cancer cell line gene expression data as input to a fully connected neural network to predict the synergy of drug pairings. MatchMaker (Kuru et al. 2022) uses a multi-modal architecture and employs three fully connected neural network modules to learn cell line-specific representations of drugs and predicts drug combination synergy scores. Finally, in a recent approach, DeepDDS (Wang et al. 2022a) uses a graph convolutional neural network to predict combination synergy scores. This approach represents molecules as graphs, where atoms correspond to nodes and bonds correspond to edges.

While the deep learning models showed superior performance in predicting synergistic drug pairs, two critical challenges stand in the way of deploying deep learning models to personalize predictions for cancer patients. Cancer cell line expression data, available for many cancer cell lines, has been used to contextualize the synergy effects in these prior works. However, cell line expressions are only a proxy for the patient expression data; and do not capture the individual patient variation. The models require a large amount of training data to learn patient-specific synergy scores, but measurements on patient-derived samples are not available at large. To tackle patient customized synergy prediction problem, He et al. proposed a computational-experimental setup where they use in silico prioritization and ex vivo testing in patient-derived samples to identify customized synergistic combinations for individual cancer patients. Since the available patient response data is small, they build a random forest model to learn single-drug dose-response curves of patients and use this model to predict drug combinations. Unlike the above-mentioned deep learning models, the model does not benefit from the available large-scale synergy measurements obtained on cell lines. To the best of our knowledge, there is no model that used both the available large-scale cell line data and the limited patient data to achieve patient-level drug synergy prediction. In this work, we address this problem by building upon our earlier model MatchMaker (Kuru et al. 2022), which is trained on and aims at predicting cell-line drug pair synergy. Recognizing that patient drug single drug response data are more widely available than patient drug combination measurements, we integrate prediction components for single drug sensitivity into MatchMaker (Kuru et al. 2022).

Transfer learning has been shown to be useful in several other problems. Dalkıran et al. (2023) employed transfer learning to improve drug sensitivity prediction. It shows the use of deep transfer learning for predicting interactions between drug candidate compounds and understudied target proteins with limited training data. Cai et al. (2020) employed transfer learning strategy for drug-target prediction. Wang et al. (2017) and Li et al. (2017) leverage transfer learning for predicting inter-chain contact predictions of transmembrane protein complexes. In this work, we make use of transfer learning in personalizing drug synergy predictions.

To tackle the patient-specific synergy prediction task, we propose a deep learning-based solution (Fig. 1). In the first stage, we train the model using the large-scale single drug response and synergy scores of drug pairs measured on cell lines (Fig. 1.1). The drug response prediction module enables us later to fine-tune our model with the limited single drug measurements on patient-derived samples. In the second stage, we fine-tune the models using gene expression and drug response data measured on patient-derived ex vivo samples. To customize the models for each patient, we present two novel patient fine-tuning strategies that make use of the patient single drug response data. (i) we use the pre-trained model and then fine-tune the model with drug sensitivity labels of all training patients data (Fig. 1.2.a); (ii) We use patient-specific fine-tuning, in which we fine-tune the pre-trained model for each patient separately (Fig. 1.2.b). We then predict test patients’ synergy data with the corresponding models (Fig. 1.3.a and 1.3.b). We demonstrate that these strategies accurately predict synergy for the patients while maintaining the model’s comparable performance on cell lines. Currently, there is no comparable study that uses deep learning models with measurements on patient data; therefore, this model represents an advance toward personalizing drug synergy predictions.

Figure 1.

The general pipeline for PDSP framework. (1) The architecture and the training procedure of the PDSP model. The chemical features of each drug are concatenated with gene expression features of the cancer cell line and input to drug encoder networks. They learn a representation of each drug separately conditioned on cancer cell line gene expressions. The synergy aggregations network takes both drugs’ representations and applies fully connected layers to predict synergy scores. The drug decoder network takes conditioned drug representations and applies connected layers to predict the drug sensitivities. (2) Fine-tuning the trained PDSP model. The pre-trained PDSP model is used for fine-tuning the drug encoders by using patient single-drug responses and gene expressions for all patients (2.a) and per patient (2.b). (3) Synergy testing with respective fine-tuned models from the previous part.

2 Materials and methods

For a drug pair $<i,j>$ and a cell line k, we aim to predict pair synergy scores $y_{i,j,k}\in \mathbb{R}$ along with each drug’s sensitivities in cell line k. We binarize the individual compound sensitivity scores (see Section 2.1). Let $s_{i,k}\in \{0,1\}$, and $s_{j,k}\in \{0,1\}$ represent drug i and j’s individual sensitivity classes, where 0 indicates resistant and 1 indicates sensitive. We represent each drug with its chemical features $d_i$ and $d_j$, and cancer cell lines with gene expression profiles $c_k$. The model is trained jointly to predict the two drugs’ individual sensitivity classes and pair synergy score by using drug and cell line feature vectors ($d_i$, $d_j$, and $c_k$).

2.1 Datasets

2.1.1 Cancer cell line drug response dataset

We use the Bliss synergy scores measured on cancer cell lines for synergy measurements and half-maximal inhibitory concentration ($I{C}_{50}$) values for sensitivity measurements. The Bliss synergy score is calculated as the difference between the observed and expected effect of the drug combination based on Bliss independence model (Bliss 1939). The Bliss score is positive if the observed effect surpasses the expected effect, indicating synergy. If the observed effect is less than expected, the Bliss score is negative, indicating antagonism. The $I{C}_{50}$ value represents the concentration of a drug needed to inhibit a specific biological or biochemical function or process by 50%, such as growth rate inhibition. Lower $I{C}_{50}$ values suggest the drug is more effective, showing that the cell line is sensitive to the drug. Conversely, higher $I{C}_{50}$ values indicate that the cell line is resistant to the given drug.

We obtain the data from the DrugComb (Zagidullin et al. 2019) database through https://drugcomb.fimm.fi/ (version v1.5, downloaded on February 2022). We binarize $I{C}_{50}$ values as sensitive and resistant. Following Chang et al. (2018), we designate cell lines as sensitive to a drug if the natural logarithm of its $I{C}_{50}$ is < −2 and resistant otherwise. Since DrugComb includes data from multiple sources, it sometimes contains replicates for the same drug pair. In these cases, we average Bliss scores across replicate measurements. Not all cancer cell line gene expression profiles were available at Iorio et al. (2016) [ArrayExpress (Athar et al. 2019) accession number is E-MTAB-3610], so we filtered out cell lines for which gene expression profiles were not available. Our final dataset contains 330 103 samples with 3068 drugs and 81 cancer cell lines.

2.1.2 Patient drug response and synergy dataset

We obtain patient synergy and sensitivity data from He et al. (2018b). This work provides 654 sensitivity measurements on patient-derived samples covering 218 compounds on ex vivo and three patients. He et al. has also made drug combination experiments and reported 20 experimentally verified drug combinations results, of which 10 are synergistic, and the remaining 10 are antagonistic. These 20 combinations include 16 drugs for the three patients. We use the patient drug response data for finetuning the models and use the synergy measurements for evaluating the final model for personalized synergy prediction. In Table 1, we summarize the number of combinations, unique drugs, and cell lines (or patients) after the data processing and filtering steps for both cell line and patient datasets.

Table 1.

DrugComb and patient drug combination dataset details after preprocessing.^a

	Source	Combinations	Drugs	Cell lines/patients
Cell line	DrugComb (Zagidullin et al. 2019)	330 103	3068	81
Patient-derived	He et al. (2018)	20	16	3

^aEach combination consists of two drugs and one cell line.

2.1.3 Model features

Our model relies on two sets of features, one set for representing drugs and the other set for cell line representation. Following the MatchMaker, we represent drugs with their chemical structure features, which we calculate using ChemoPy Python library (Cao et al. 2013) [more information provided in Supplementary Table S4 and in our earlier work (Kuru et al. 2022)]. Cell lines are represented with their untreated gene expression profiles (Iorio et al. 2016), and RMA normalized profiles are obtained from https://www.cancerrxgene.org (downloaded on Dec 2019). Since we apply our model to patient expression profiles from He et al., we use available gene expression profiles from this study. To be able to apply PDSP on patient gene expression profiles, we take the intersection of 17 737 genes in cancer cell lines and 634 genes in patient data. The intersection of these two gene sets contains 542 genes; we use expressions of these genes as gene expression features. The cancer cell line gene expressions and the patient gene expressions are measured under different conditions. We apply quantile normalization (Bolstad et al. 2003). We input two drugs $d_i,d_j\in {\mathbb{R}}^{541}$ and a cell line $c_k\in {\mathbb{R}}^{542}$ to our model and predict $y_{i,j,k}\in \mathbb{R}$, $s_{i,k}\in \mathbb{R}$, and $s_{j,k}\in \mathbb{R}$ simultaneously.

2.2 Training loss

We train the model with a loss function that combines the regression loss $L_{\mathit{reg}}$ and the classification loss $L_{\mathit{cls}}$:

$$\begin{array}{c}L=L_{\mathit{reg}}+\lambda \times L_{\mathit{cls}}\hfill \end{array}$$ (1)

where $\lambda$ is a hyperparameter that adjusts the weight between regression and classification losses. We set $\lambda$ to 10 in our experiments. We adopt mean-squared-error as the regression loss, and weighted binary cross entropy (wBCE) as the classification loss:

$$\begin{array}{c}{L}_{\mathit{reg}}=\frac{1}{N}\sum _{i,j,k\in D}{(y_{i,j,k}-{\widehat{y}}_{i,j,k})}^2\hfill \end{array}$$ (2)

$$\begin{array}{c}\begin{array}{c}{L}_{\mathit{cls}}=\mathit{wBCE}(i,k)+\mathit{wBCE}(j,k)\hfill \\ \mathit{wBC}{E}_{t,k}=-\frac{1}{N}\sum _{t,k\in D}{\lambda }_1{s}_{t,k}\hspace{0.17em}\text{log}({\widehat{s}}_{t,k})+{\lambda }_2(1-s_{t,k})\log (1-{\widehat{s}}_{t,k})\hfill \end{array}\hfill \end{array}$$ (3)

where N is the number of drug combinations in training dataset D. $y_{i,j,k}$ is the ground-truth Bliss score of drug combination <i, j> on cell line k. $s_{t,k}$ is the ground-truth sensitivity class of drug t on cell line k. ${\widehat{y}}_{i,j,k}$ and ${\widehat{s}}_{t,k}$ are the corresponding predictions. ${\lambda }_1$ and ${\lambda }_2$ are the loss weights for sensitive and resistant drugs on cell line k.

2.3 The model architecture

PDSP model builds upon our previous work Matchmaker (Kuru et al. 2022). Figure 1 provides an overview of the PDSP architecture. We first represent cell lines and two drugs as a collection of vectors $d_i$, $d_j$, and $c_k$. With these vector representations, PDSP aggregates both cancer cell line and drugs’ information and predicts drug sensitivities and combination synergy in two stages. In the first stage, PDSP encodes drug and cancer cell line information with Drug Encoder Networks (DENs). DENs take both drug and cancer cell line representations and learn conditional latent representations of the drugs. We force both DENs to share the same architecture and model weights to get order-invariant latent representations. In the second stage, PDSP uses these conditional encodings to predict sensitivity classes and combination synergy score. This second stage contains two Drug Decoder Networks (DDNs) and one Synergy Aggregation Network (SAN). DDN takes encoded conditional latent representation and decodes it to the drug’s sensitivity on a given cell line. Similarly to DENs, DDNs share the same architectures and model weights. While drug encoders learn representations of each drug, they do not provide information about their combinations in a cell line. Therefore, SAN first aggregates drugs’ latent representation by concatenation and provides the combination information. Finally, an aggregated representation is used to predict drug combination synergy.

2.3.1 DEN

The PDSP model consists of two identical drug encoders, $DE{N}_i$ and $DE{N}_j$. These encode the cancer cell line features ($c_k$) and two drugs’ features ($d_i$ and $d_j$) into a conditional latent space. The DEN model takes gene expression data and concatenates it with the features of each drug in the combination. This concatenated information is then passed through a series of fully connected (FC) layers. Each FC layer is followed by a nonlinear activation function (ReLU), except for the last FC layer and dropout layers. The formulation of DEN is as follows:

$$\begin{array}{c}\begin{array}{c}de{n}_1(d,c)=\text{Dropout}(\text{ReLU}(\text{FC}(\text{Concat}(d,c))))\hfill \\ de{n}_2(d,c)=\text{Dropout}(\text{ReLU}(\text{FC}(de{n}_1(d,c))))\hfill \\ \mathit{DEN}(d,c)=\text{Dropout}(\text{Linear}(\text{FC}(de{n}_2(d,c))))\hfill \end{array}\hfill \end{array}$$ (4)

where d and c denotes the drug and cell line features, $de{n}_1$ and $de{n}_2$ are the intermediate layers. The $DE{N}_i$ and $DE{N}_j$ encoders are designed to have the same model parameters and weights, which allows for the same latent representations to be achieved for the two drugs (order-invariance). This shared structure enables the model to learn the relationships between the drugs and cancer cell lines consistently, regardless of the order in which the drugs are presented. The outputs of the encoders, $h_{i,k}$ and $h_{j,k}\in {\mathbb{R}}^m$, are multi-dimensional representations of the drugs in the conditional latent space. We use these drug encodings as inputs to downstream tasks, drug sensitivity, and drug combination prediction. $DE{N}_i$ and $DE{N}_j$ provide drugs encodings $h_{i,k}$ and $h_{j,k}\in {\mathbb{R}}^m$ as follows:

$$\begin{array}{c}\begin{array}{c}{h}_{i,k}=\text{DEN}({\mathrm{d}}_{\mathrm{i}},{\mathrm{c}}_{\mathrm{k}})\hfill \\ h_{j,k}=\text{DEN}({\mathrm{d}}_{\mathrm{j}},{\mathrm{c}}_{\mathrm{k}})\hfill \end{array}\hfill \end{array}$$ (5)

2.3.2 DDN

Based on the latent drug encodings $h_{i,k}$ and $h_{j,k}$, we train two identical drug decoder networks $DD{N}_i$ and $DD{N}_j$ to output drug sensitivities ${\widehat{s}}_{i,k}$ and ${\widehat{s}}_{j,k}$. DDN uses a feed-forward neural network architecture with three FC layers. The first two FC layers apply and ReLU activation and dropout, respectively. Finally, a final FC layer with a linear activation function is used to obtain the drug sensitivity status. The DNN formulation is given below:

$$\begin{array}{c}\begin{array}{c}dd{n}_1(h)=\text{Dropout}(\text{ReLU}(\text{FC}(h)))\hfill \\ dd{n}_2(h)=\text{Dropout}(\text{ReLU}(\text{FC}(dd{n}_1(h))))\hfill \\ \mathit{DDN}(s)=\text{Linear}(\text{FC}(dd{n}_2(h)))\hfill \end{array}\hfill \end{array}$$ (6)

where h corresponds to the conditional drug encodings ($h_{i,k}$ or $h_{j,k}$), $dd{n}_1$ and $dd{n}_2$ are the intermediate layers. Both DDNs use the same model weights, and sensitivity predictions ${\widehat{s}}_{i,k}$ and ${\widehat{s}}_{j,k}$ for drugs i and j on cancer cell line k is calculated as follows:

$$\begin{array}{c}\begin{array}{c}{\widehat{s}}_{i,k}=\text{DDN}({\mathrm{h}}_{\mathrm{i},\mathrm{k}})\hfill \\ {\widehat{s}}_{j,k}=\text{DDN}({\mathrm{h}}_{\mathrm{j},\mathrm{k}})\hfill \end{array}\hfill \end{array}$$ (7)

2.3.3 Synergy aggregation network (SAN)

Similarly to DDN, synergy aggregation network uses feed-forward neural network architecture. SAN takes latent drug representations $h_{i,k}$ and $h_{j,k}$ and predicts drug combination synergy ${\widehat{y}}_{i,j,k}$. Since the inputs of SAN is a drug combination, it first concatenates drug features, then applies similar network architecture to DDN. We apply linear activation at the last layer of SAN since it outputs the Bliss synergy score. The SAN model is given below:

$$\begin{array}{c}\begin{array}{c}sa{n}_1(h_1,h_2)=\text{Dropout}(\text{ReLU}(\text{FC}(\text{Concat}(h_1,h_2))))\hfill \\ sa{n}_2(h_1,h_2)=\text{Dropout}(\text{ReLU}(\text{FC}(sa{n}_1(h_1,h_2))))\hfill \\ \mathit{SAN}(h_1,h_2)=\text{Linear}(\text{FC}(sa{n}_2(h_1,h_2)))\hfill \end{array}\hfill \end{array}$$ (8)

where $h_1$ and $h_2$ denotes conditional drug encodings in the combination, $sa{n}_1$ and $sa{n}_2$ are the intermediate layers. With these formulations, we obtain predicted synergy score as ${\widehat{y}}_{i,j,k}=\mathit{SAN}(h_{i,k},h_{j,k})$.

2.4 Customizing for patients

We initially train PDSP with drug combination data derived from cancer cell line experiments. In this pretrained model, we freeze SAN and fine-tune the model with patient single-drug responses. Fine-tuning provides us to personalize DENs to the patient-derived sample data. We apply two patient fine-tuning strategies and asses their predictive performance.

We denote the initial trained model weights on cancer cell line as ${\Phi }_c$, patients’ gene expression profiles $c_{p_t}\in {\mathbb{R}}^{542}$ ($t\in \{1,2,3\}$), drugs’ chemical features as $d_i\in {\mathbb{R}}^{541}$ and fine-tune ${\Phi }_c$ as follows:

2.4.1 Strategy 1: fine-tune model with all patients

We use ${\Phi }_c$ as the pre-trained model weights for PDSP. We feed patients’ gene expression profiles and drugs’ chemical structure features into the model and run the PDSP training pipeline for single-drug responses. After fine-tuning, we use this new model weights ${\Phi }_{c,\mathit{all}}$ for synergy prediction of all patients. We refer to this model as PDSP^st1. This strategy is useful when we have a set of single-drug responses for multiple patients.

2.4.2 Strategy 2: fine-tune model for each patient

In the second strategy, we fine-tune pre-trained weights ${\Phi }_c$ for each patient separately. This lead to n different models for each of the n patients: ${\Phi }_{c,t}$ ($t\in \{1,\dots n\}$). We evaluate each patient’s drug combination synergy score with the corresponding model’s weights ${\Phi }_{c,t}$. We refer to the models generated using this strategy PDSP^st2.

3 Results

3.1 Experimental setup

We follow the approach outlined by Preuer et al. (2017) for benchmarking model performance and adopt the leave-drug-combination-out strategy, wherein each drug combination is exclusively present in either the training, validation, or test datasets. We divide the whole drug combination data into training, validation, and test splits by considering that a drug pair ¡i, j¿ can only be involved in one of the splits. The ratios of training, validation and test splits are 60%, 20%, and 20%, respectively. We tune the model hyperparameters based on the performance of the validation set. We use these models to report performance on cancer cell line test data. We also use the best PDSP model weights ${\Phi }_c$ as the pre-trained model for patient data fine-tuning as explained in section Customizing for Patients.

3.2 Performance benchmark

3.2.1 Regression performance on cancer cell lines

We compare PDSP’s performance with base model MatchMaker (Kuru et al. 2022) and other state-of-the-art models: DeepSynergy (Preuer et al. 2017), TreeCombo (Joseph et al. 2018) and DeepDDS (Wang et al. 2022a). We also use a standard machine learning algorithm, Random Forest, as the baseline model. We evaluate them based on MSE, Pearson, and Spearman rank correlation coefficients between the actual Bliss synergy scores and the predicted scores on the unseen cell line test set. For each model, we tune its hyperparameters (see Supplementary Tables S5–S8) based on the performance of the validation set. Table 2 shows the model performances for these performance metrics. Among these methods, PDSP and MatchMaker achieve the top performances. Both models show comparable performances across all metrics. While MatchMaker shows slightly better performance on MSE (41.60 versus 41.95), PDSP is better on the Spearman correlation coefficient ($64\%$ versus $65\%$). They achieve the same performance in the Pearson correlation coefficient: $91\%$. These results show that adding drug response prediction nodes to enable patient personalization did not sacrifice on the synergy prediction performance; PDSP achieves on par synergy prediction performance on cell lines with MatchMaker (i.e. no patient fine-tuning). When we compare PDSP’s performance with the remaining models, PDSP achieves $\sim 20\%$ MSE and $2\%$ to $7\%$ correlation improvements over the next best model, DeepSynergy. DeepDDS, a model that uses graph neural networks to encode drug chemical structure, performs worse than the baseline method, Random Forest.

Table 2.

Comparison of synergy predictions of the models based on MSE, Pearson and Spearman correlation coefficients.

	MSE	Pearson	Spearman
PDSP	41.95	0.91	0.65
MatchMaker	41.60	0.91	0.64
DeepSynergy	50.65	0.89	0.59
TreeCombo	61.89	0.87	0.54
DeepDDS	177.33	0.54	0.26
Random Forest	165.72	0.60	0.31

We further assess the cell line specific performances of the models by measuring the regression performance metrics for each cell line separately. We provide the list of 10 cell lines for which our method performed the best. We also provide the number of instances that cell line is observed in training data in Supplementary Tables S1–S3. We observe that the number of cases does not have a substantial effect on the performance.

3.2.2 Classification performance on cancer cell lines

We also evaluate the classification performance of the synergy model. We binarize the actual Bliss scores: synergistic pairs have positive Bliss scores, and antagonistic pairs have negative Bliss scores. We perform binary classification to evaluate the predictive performance of the models and present the results in Table 3 using various metrics such as balanced accuracy (mean of recall values calculated for each class, BACC), partial AUC (false positive rate up to 0.1; pAUC) and Cohen’s Kappa score (Cohen 1960). Note that we do not perform any training for this task and use the discretized predictions and ground truth to measure classification performance.

Table 3.

Classification performance comparison of the models based on different classification metrics.^a

	AUC	pAUC	AUPR	F1	Precision	Recall	Kappa	BACC
PDSP	0.79	0.64	0.81	0.72	0.70	0.75	0.40	0.70
MatchMaker	0.79	0.64	0.80	0.73	0.69	0.76	0.40	0.70
DeepSynergy	0.76	0.62	0.78	0.70	0.68	0.73	0.35	0.68
DeepDDS	0.61	0.53	0.63	0.62	0.59	0.65	0.16	0.58
TreeCombo	0.73	0.60	0.75	0.68	0.67	0.79	0.32	0.66
Random Forest	0.63	0.55	0.66	0.54	0.63	0.48	0.17	0.59

^aExperiments with a Bliss score >0 were classified as positive (synergistic) samples and those below 0 are classified as negative (antagonistic) samples.

PDSP and MatchMaker outperform other methods in classification performance. They exhibit similar performances, with only slight differences in different metrics. PDSP achieves slightly higher scores in AUPR (0.81 versus 0.80) and Precision (0.70 versus 0.69), while MatchMaker archives slightly higher F1 (0.73 versus 0.72) and Recall (0.76 versus 0.75) scores.

3.2.3 Patient personalized classification performance

While we compare the synergy prediction performance of PDSP with others on cancer cell lines and show above that it is on par with the state-of-the-art methods, its ultimate goal is to perform personalized predictions for cancer patients. Cancer cell line measurements provide large scale training data, but they fail to capture patient variation as cancer is a complex and heterogeneous disease. We have very limited patient-level ground truth data to train a complex machine learning model. For this reason, we employ transfer learning approach in the design of PDSP and use the single drug responses of patients to fine-tune the model as explained in Section Customizing for Patients. We compare the classification performances of PDSP (nonpersonalized), MatchMaker, DeepSynergy, TreeCombo, DeepDDS and Random Forest models with two personalized PDSP^st1 and PDSP^st2 models. Figure 2 shows the number of True Positives (TPs), True Negatives (TNs), False Positives (FPs) and False Negatives (FNs).

Figure 2.

The classification results of patient drug combination synergy data. All models are trained with DrugComb data with cell line gene expressions. PDSP^st1 and PDSP^st2 correspond to personalized models that are fine-tuned based on Strategy 1 and Strategy 2 respectively. In the figure, accurate predictions are denoted by blue rectangles, while false predictions are indicated by red rectangles. Synergistic predictions are represented by a plus (+) sign, and antagonistic predictions are signified by a minus (−) sign.

Figure 2 shows nonpersonalized models are not good at detecting antagonistic drug combinations. For example, Random Forest method predicts all drug pair-patient trios as synergistic. Similarly, MatchMaker and TreeCombo predict 17 and 18 (out of 20) drug pairs as synergistic, respectively. They mainly act as a one-class-predictor, which leads to a large number of FPs and low number of TNs. These results lead to low precision values ($47\%$ and $56\%$, respectively) for these models. Compared to models that are not fine-tuned, PDSP^st1 and PDSP^st2 achieve better precision ($63\%$ and $83\%$) and separation between synergistic and antagonistic drug combinations. Specifically, the patient-personalized PDSP^st1 model decreases the number of FPs to 3 and increases the number of TNs to 7 compared to PDSP. This shows that fine-tuning with patient gene expression improves precision ($63\%$ versus $43\%$) and accuracy ($60\%$ versus $40\%$). PDSP^st2 model performs even better with 9 TNs at the cost of 1 FP. This indicates that it is nearly perfect at detecting antagonistic drug combinations while preserving the performance at synergistic drug combinations. Thus, using per-patient fine-tuning (Strategy 2) in particular, yields the best-personalized drug synergy prediction performance.

While the ideal scenario would be a model performing exceptionally well in detecting both synergistic and antagonistic drug combinations, the observed performance of the fine-tuned models indicates a higher accuracy in detecting antagonistic combinations compared to synergistic ones. However, we should note that antagonistic drug combinations may lead to adverse side effects or ineffective treatment outcomes, which can be detrimental to a patient’s well-being. Accurate identification of antagonistic drug combinations allows clinicians to avoid potentially dangerous combinations and make informed decisions in designing cancer treatment regimens (Hackman et al. 2020). In contrast, while the detection of synergistic combinations is also valuable, the lower performance in this aspect may be mitigated by further research and adjustments to the model.

4 Discussion

This study is the first attempt to perform in-silico personalized drug synergy prediction for cancer patients that uses large-scale gene expression data. With limited patient-level data, we opted for a transfer learning based approach to first train a complex model using large scale cell-line-based drug-pair perturbation datasets and used patients’ drug response profiles to fine-tune and personalize the model. We show favorable results compared to the state-of-the-art models that do not allow incorporating personal features.

The results indicate that fine-tuning a cell line pre-trained model with a small number of single-drug responses enables PDSP to identify synergistic and antagonistic drug pairs for specific patients better than other models. While the nonpersonalized PDSP model’s accuracy is 40% when tested for patients, fine-tuned PDSP^st1 and PDSP^st2 models’ accuracies are 60% and 70%, respectively. We observe that models, that are not fine-tuned, tend to make more synergistic predictions, and they cannot detect antagonistic drug pairs. This problem is alleviated by fine-tuning with our personalized models. We argue that in cancer therapy, treatment safety, and efficacy are critical, and a focus on the accurate detection of antagonistic drug interactions is important to prioritize patient well-being and treatment success. It is critical to improve its ability to detect synergistic interactions as well, but in the context of cancer therapy, focusing on identifying antagonistic combinations is as important as detecting synergistic pairs.

This study uses the drug’s chemical structure and cell line specific gene expression profiles as the feature set. As indicated in Kuru et al. (2022), these features are selected as they are widely available for most drugs and provide a large training set to train a complex model like ours. If available on a large scale, our model can always be supplemented with other sources of information such as drug-disease or drug-target interactions and network-based methods (Cheng et al. 2019). Similarly, it is straightforward to adapt the model to use other synergy scores such as Loewe, ZIP and HSA.

We hypothesize that personal single-drug responses can be indirectly used to fine-tune the PDSP model toward better personal drug synergy prediction. We modify the MatchMaker architecture to use this information. We show that, indeed it benefits the drug synergy prediction task in patients. However, as personal drug synergy labels become widely available, we foresee that transfer learning using actual synergy labels will result in even better performance. This is because models will be able to incorporate directly relevant labels, and the model architecture will be less complex with fewer parameters to optimize.

Supplementary data

Supplementary data are available at Bioinformatics online.

Conflict of interest

None declared.

Funding

This work was supported in part by TUBA—GEBIP 2017 award to AEC. O.T. and A.E.C. acknowledge support from Bilim Akademisi BAGEP awards.