Machine Learning for Pharmacogenomics and Personalized Medicine: A Ranking Model for Drug Sensitivity Prediction

Abstract

It is infeasible to test many different chemotherapy drugs on actual patients in large clinical trials, which motivates computational methods with the ability to learn and exploit associations between drug effectiveness and patient characteristics. This work proposes a machine learning approach to infer robust predictors of drug responses from patient genomic information. Rather than predicting the exact drug response on a given cell line, we introduce an elastic-net regression methodology to compare a drug-cell line pair against an alternative pair. Using predicted pairwise comparisons we rank the effectiveness of different drugs on the same cell line. A total of 173 cell lines and 100 drug responses were used in various settings for training and testing the proposed models. By comparing our approach against twelve baseline methods, we demonstrate that it outperforms the state-of-the-art methods in the literature. In contrast to most other methods, the algorithm is able to maintain its high performance even when we use a large number of drugs and few cell lines.

1. Introduction

Performing drug screening and selecting appropriate personalized treatment based on individual genomic or proteomic profiles is one of the paramount goals of precision medicine.

Since the effectiveness of each drug can be varied among patients, finding the right drug for each cancer patient is quite a challenging task [1]. One possible option is to assess the efficacy of drugs using large clinical trials. This path makes it possible to capture most of the pertinent biological features of a patient [2]. However, this approach is impractical for several reasons: (i) large clinical trials are time-consuming and expensive; (ii) a clinical trial typically evaluates one (or very few drugs) at a time; and (iii) even a large clinical trial may not include enough patients to cover all cancer genomic variations.

Cell lines contain a large number of the molecular features of tumors; as a result, they are expected to reflect the properties (mutation status, gene expression, drug sensitivity, and so on) of the original cancer type from which they were cultured [3]. Thus, they provide practical preclinical models in order to analyze strategies for predictive marker development [4]. Consequently, an alternative to clinical trials is drug response prediction using large-scale screenings of cancer cell lines against libraries of pharmacological compounds [5]. This can lead to genomic predictors of drug responses from large panels of cancer cell lines [3], [6].

Hansch et al. [7] were among the first who revealed the existence of mathematical relations between the biological activity of a chemical compound and its physicochemical properties. Pan-cancer repositories (e.g., the molecular analysis for a therapy choice trial at the National Cancer Institute, the Cancer Genome Atlas) provide the foundation for joint analysis of cancer cell lines and their drug responses. Furthermore, over the past few years, two important studies, namely the cancer cell line encyclopedia [3], and the genomics of drug sensitivity projects [8] revealed the applicability of machine learning algorithms in predicting drug response based on panels of cancer cell lines.

Different methodologies have been utilized in the literature for drug sensitivity prediction based on genomic data. Barretina et al. [3] employed elastic net regression [9] to predict drug sensitivity for the Cancer Cell Line Encyclopedia (CCLE). Geeleher et al. [10] used pre-treatment baseline gene expression data to predict the chemotherapeutic response in patients. They applied a ridge regression model to predict drug response for breast cancer cell lines using baseline gene expression data. This method was compared against several methods, such as nearest shrunken centroids, principal component regression, Least Absolute Shrinkage and Selection Operator (LASSO) regression [11], elastic net regression, and random forests [12]. They observed that ridge regression yielded the best performance. However, other studies comparing multiple algorithms on a drug sensitivity database have observed that random forests perform better than ridge regression or other regularized linear regression approaches [13]. Undoubtedly, the random forest algorithm has been one of the top-performing algorithms in drug sensitivity prediction, and this has been proven in multiple other drug sensitivity studies [14], [15], [16], [17]. Nevertheless, in all of these studies, the authors did not consider the effect of molecular feature data, the type of drug used, the sensitivity of the response (discrete or continuous), and methods for summarizing compound sensitivity values.

In a comprehensive comparative study in drug sensitivity prediction, Jang et al. [6] considered these modeling factors to compare various machine learning algorithms, including principal component regression, partial least square regression, least squares support vector machine regression with linear kernels, random forests, LASSO, ridge regression, and elastic net regression. They considered more than 110,000 various models based on a multifactor experimental design. Their analysis suggested that elastic net or ridge regression will most likely yield the most accurate predictors. However, some studies argue that the existing regression-based methods for drug selection may sacrifice performance on very few but sensitive drugs to achieve better performance on the majority of insensitive drugs [18]. Therefore, when using the regression-based models to predict how a cell line responds to sensitive drugs, the prediction could lead to incorrect drug selection or prioritization [18].

Motivated by the high performance of regression-based methods in drug sensitivity prediction, we propose a novel elastic-net-based regression that utilizes gene expression features and drug sensitivity data to build a predictor. Most of the conventional regression methods learn a vector of coefficients for each cell line or drug and use these vectors to predict the sensitivity values. We believe this is the main reason that may lead to sacrificing performance on very few but sensitive drugs to achieve better performance on the majority of insensitive drugs. Moreover, because the number of cell lines is significantly less than the number of features, these approaches usually lead to “overfitting.” In addition, they fail to capture the drug-cell line relation across various cell lines. As a result, they are unable to effectively prioritize sensitive drugs over insensitive drugs.

Our method uses regularized regression to model the difference in sensitivity of a given drug acting on different cell lines or different drugs acting on the same cell line. Specifically, we do not predict the exact drug response for each drug, but rather develop a model that estimates the difference in sensitivity obtained by comparing two different drug-cell line pairs. This score function can compare drug sensitivities for a given cell line and then use these pairwise comparisons to rank the drugs. Using this model, we can rank the sensitivity of different drugs acting on a novel cell line. The use of regularization as part of the regression is motivated by the earlier work we reviewed and recent results on the connection between regularization and robustness to the potential presence of outliers [19], [20].

The remainder of this paper is organized as follows. Section 2 outlines our method, presents the data we used, and discusses model training and performance evaluation. Section 3 presents our main results and how our method compares to alternatives. Section 4 discusses the results and draws some conclusions.

Notational conventions:

We use boldfaced lowercase letters to denote vectors, ordinary lowercase letters to denote scalars, boldfaced uppercase letters to denote matrices, and calligraphic capital letters to denote sets. All vectors are column vectors. For space saving reasons, we write $\mathbf{x}=\left(x_1,\dots ,x_n\right)$ to denote the column vector $\mathbf{x}\in {ℝ}^n$. For any matrix A, we let a_ij denote its (i, j) element, a_i the ith row and, with some abuse of our conventions, A_j the jth column. I denotes the identity matrix. We use prime to denote the transpose of a vector, and ${\Vert \mathbf{x}\Vert }_p={\left(\sum _{i=1}^n{\left|x_i\right|}^p\right)}^{1/p}$ for the ${\ell }_p$ norm, where p ≥ 1. For any matrix $\mathbf{A}\in {ℝ}^{n\times m}$ will denote the ${\ell }_p$ norm of a vectorized form of the matrix, i.e., ${\Vert \mathbf{A}\Vert }_p={\left(\sum _{i=1}^n\sum _{j=1}^m{\left|a_{ij}\right|}^p\right)}^{1/p}$. We also use e_i to denote the ith unit vector, a vector of all zeros except the ith element which is set to 1.

2. Materials and Methods

2.1. Model description

We are given a matrix $\mathbf{S}\in {ℝ}^{N_C\times N_G}$, where each row corresponds to one of N_C cell lines (patients). Each cell line is characterized by a gene expression vector containing N_G features. We are also given a matrix $\mathbf{R}\in {[0,1]}^{N_C\times N_D}$ of drug responses to N_D different drugs; each row corresponds to a cell line and each column lists the response of a specific drug to all cell lines. Drug responses are in [0, 1] and a higher response implies higher sensitivity of the cell line to the drug (i.e., a more effective drug). Our ultimate goal is to rank the N_D drugs based on their response, so that the most effective ones are ranked on top. Importantly, we care about the relative ordering of the most effective drugs; predicting the exact value of the drug response is of no consequence.

In our specific application, we do not have any information about the drugs (e.g., their mechanism of action, physical and chemical properties, metabolism, therapeutics, and toxicity). We just know the name of each drug, which makes the prediction even more difficult. Therefore, we use so-called “one-hot” encoding to represent drugs; i.e., for each of the N_D drugs we create an indicator variable taking values in {0,1}. Application of the ith drug will simply be represented by the ith unit vector. If we had access to the drug-specific features, we can easily substitute the one-hot encoding vector with such features.

Define now the (N_D + N_G)-dimensional vector ${\mathbf{p}}_{ij}=\left({\mathbf{e}}_i,{\mathbf{s}}_j\right)$, $i=1,\dots ,N_D$, $j=1,\dots ,N_C$, formed as the concatenation of the one-hot vector representing drug i and the j row vector of S containing the gene expression for cell line j. Per our notational conventions, r_ij denotes the (i, j) element of the response matrix R, that is, the response of drug j in cell line i. Denote by N = N_CN_D the number of all possible combinations of a drug with a cell line. Define the matrix $\mathbf{X}\in {ℝ}^{N\times \left(N_D+N_G\right)}$ with rows p₁₁, p₂₁,...,p_N corresponding to drug-cell line pairs and let $\mathbf{y}=\left(r_{11},r_{21},\dots ,r_N\right)$ be the vector of corresponding responses. For any $i=1,\dots ,N$, (x_i, y_i) will consist of the ith row of X and the ith element of y.

We will view $\left\{\left({\mathbf{x}}_i,y_i\right);i=1,\dots ,N\right\}$ as a training set and we are interested in training a model that predicts the difference in sensitivity between two different drug-cell line combinations x_i, x_j. We start by defining appropriate labels. Define:

$$a_{ij}={\xi }_{ij}\left(y_i-y_j\right)\times 100,i,j=1,\dots ,N,$$

where

$${\xi }_{ij}=\{\begin{array}{ll}{\varphi }_1,& \text{when}i\text{and}j\text{refer to same drug in different cell lines,}\\ {\varphi }_2,& \text{when}i\text{an}j\text{refer to different drugs}\text{in the same cell line,}\\ 0,& \text{otherwise,}\end{array}$$ (1)

and ${\varphi }_1$, ${\varphi }_2$ are positive constants. Let also

$${\psi }_{ij}=\{\begin{array}{ll}1,& \text{when}i\text{and}j\text{refer to the same drug in}\text{different cell lines or different drugs in the same cell line,}\\ 0,& \text{otherwise}.\end{array}$$

The training problem amounts to finding a matrix W that solves the following optimization problem

$$\underset{\mathbf{W}}{\min }\frac{1}{Z}\sum _{\begin{array}{c}i,j=1\\ i\ne j\end{array}}^N{\left[{\psi }_{ij}\left({\mathbf{x}}_i^{\prime }\mathbf{W}{\mathbf{x}}_j-a_{ij}\right)\right]}^2+{\lambda }_1{\Vert \mathbf{W}\Vert }_2^2+{\lambda }_2{\Vert \mathbf{W}\Vert }_1,$$ (2)

where $Z=N\left(N_D+N_C-2\right)$ and ${\lambda }_1,{\lambda }_2\ge 0$ are regularization parameters.

Having solved the regression problem in (2), we can rank the drugs on how they affect a new cell line using the following procedure. Let s be the gene expression vector of the new cell line. For any $i=1,\dots ,N_D$, we compare the drug-cell line pairs g_i = (e_i, s) against themselves and other pairs x_n, $n=1,\dots ,N$, already seen in the training set. Specifically, the score T_i of drug $i=1,\dots ,N_D$, on the cell line s is obtained by:

$$T_i=\frac{1}{\left(N_D+N_C-1\right)}\left(\sum _{\begin{array}{c}j=1\\ j\ne i\end{array}}^{N_D}{\mathbf{g}}_i^{\prime }\mathbf{W}{\mathbf{g}}_j+\sum _{n=1}^N{\psi }_{in}{\mathbf{g}}_i^{\prime }\mathbf{W}{\mathbf{x}}_n\right).$$ (3)

Ranking these scores T_i provides a ranking of the effectiveness of the drugs on the new cell line s.

We present an example to illustrate the scoring process. Suppose that we have 3 drugs D₁, D₂, and D₃ and 4 cell lines C₁, C₂, C₃, and C₄. The structure of the training matrix X is shown in Figure 1. The arrows in this figure depict the comparisons our model performs during training for D₁ in C₁. To rank the drugs in a new cell line C_Test, use (3) to compute T_i, i = 1,2,3. Figure 2 demonstrates the comparisons (3) performs to evaluate the score for D₁.

Fig. 1:

The structure of X and the comparisons our model will carry out for D₁ in C₁.

Fig. 2:

The process of calculating the score for D₁ in a new cell line.

2.2. Data sets

We designed, evaluated, and trained our method using the cell line data and drug sensitivity data from the Cancer Cell Line Encyclopedia (CCLE) [21] and the Cancer Therapeutics Response Portal (CTRP v2) [22]. We used the “Act Area” (the area above the fitted dose-response curve) to quantify drug sensitivity. With this metric, lower response indicates higher drug sensitivity. It worth mentioning that approximately 20% of the drug responses are missing; this portion of the data were excluded from our analysis.

2.3. Gene selection scheme

Each gene in our data set has approximately 20,000 features. A snapshot of all the transcriptional activity in a cell line can be obtained by gene expression microarrays. Please refer to Appendix A and [23] for more information. Since the number of features is significantly higher than the number of training samples, using all these features may result in overfitting. Instead, we selected an appropriate subset of features to use for all our experiments. To that end, and consistent with the discussion in Section 1, we used LASSO regression to select a subset of features.

Specifically, consider the matrix X and the response vector y defined in Section 2.1. For the rows (e_i, s_j), $i=1,\dots ,N_D$, $j=1,\dots ,N_C$, of X define a common coefficient vector $\mathit{\beta }=\left({\mathit{\beta }}_1,{\mathit{\beta }}_2\right)$, ${\mathit{\beta }}_1\in {ℝ}^{N_D}$, ${\mathit{\beta }}_2\in {ℝ}^{N_C}$. To develop a regression-based model for predicting y, we solve the following loss minimization problem:

$$\underset{\mathit{\beta }}{\min }{\Vert \mathbf{y}-\mathbf{X}\mathit{\beta }\Vert }^2+\lambda {\Vert {\mathit{\beta }}_2\Vert }_1,$$ (4)

where λ > 0 is a scalar that modulates the strength of the regularizer. Gene features were eliminated using a recursive feature elimination procedure. In particular, we solve (4) using cross-validation to select λ. We eliminate 5% of the gene features whose corresponding coefficient in β₂ is among the 5% smaller absolute values. We reformulate (4) using the remaining gene features and repeat this process while the validation loss keeps decreasing. Further details of the gene selection scheme and the list of the selected genes can be found in Appendix A.

2.4. Converting the model to elastic net regression

The following Lemma is almost immediate; hence, we skip the proof. It establishes that Problem (2) can be reformulated as a standard elastic net regression.

Lemma 2.1

Formulation (2) can be converted to the following elastic net regression problem:

$$\underset{\mathbf{w}}{\min }\frac{1}{Z}\sum _{\begin{array}{c}i,j=1\\ i\ne j\end{array}}^N{\left[{\psi }_{ij}\left({\mathbf{v}}_{ij}^{\prime }\mathbf{w}-a_{ij}\right)\right]}^2+{\lambda }_1{\Vert \mathbf{w}\Vert }_2^2+{\lambda }_2{\Vert \mathbf{w}\Vert }_1,$$ (5)

where v_ij is the vectorization of ${\mathbf{x}}_i{\mathbf{x}}_j^{\prime }$, and w is the vectorization of W.

2.5. Iterative thresholding for complexity reduction

Regularizing with the ${\ell }_1$-norm is known to induce sparsity in the coefficient vector w. To reduce the complexity of our model, we can set to zero some elements of w that appear less important in making a prediction; this has also the effect of reducing running time and providing a higher level of interpretability [24] to the model. To that end, we used an iterative procedure similar to the one mentioned in Section 2.3.

More specifically, in each iteration, we use cross-validation to select λ₁, λ₂ in (5) that minimize the loss on the validation set. Then, we select 10% of the elements of w with the smallest absolute value and set them to zero. We continue iterating in this fashion until we do not see any considerable decrease in the (validation) performance of our model. The end result will be a parameter vector w with much less non-zero elements than the total number of (N_D + N_C)² elements. In this way, we not only prevent over fitting, but also decrease the complexity of the model.

2.6. Closed form solution

After fixing a number of elements of w to zero using the approach described in Section 2.5, we remove the ℓ₁-norm penalty from the problem formulation (5). The resulting problem is equivalent to ridge regression. Let $\mathbf{V}\in {ℝ}^{N(N-1)\times {\left(N_D+N_G\right)}^2}$ be the matrix with rows $\left\{{\psi }_{ij}{\mathbf{v}}_{ij};i,j=1,\dots ,N,i\ne j\right\}$. Define also the vector $\mathit{\alpha }=\left({\psi }_{ij}{a}_{ij};i,j=1,\dots ,N,i\ne j\right)\in {ℝ}^{{\left(N_D+N_G\right)}^2}$. Then, Problem (5), with λ₂ = 0, is written as:

$$\underset{\mathbf{w}}{\min }\frac{1}{Z}{\Vert \mathbf{V}\mathbf{w}-\mathit{\alpha }\Vert }_2^2+{\lambda }_1{\Vert \mathbf{w}\Vert }_2^2.$$ (6)

The following lemma provides a closed-form solution to Problem (6). A proof is provided in Appendix B.

Lemma 2.2

The closed form solution for Problem (6) can be written as follows:

$$\mathbf{w}={\left({\mathbf{V}}^{\prime }\mathbf{V}+{\lambda }_{1}Z\mathbf{I}\right)}^{-1}{\mathbf{V}}^{\prime }\mathit{\alpha }.$$ (7)

Notice that the matrix V^′V is positive semi-definite and, for large enough λ₁, V^′V + λ₁ZI is positive definite and thus invertible. When w is dense, the asymptotic time complexity of the closed form solution would be $O\left({\left(N_D+N_G\right)}^6+N(N-1){\left(N_D+N_G\right)}^4\right)$. However, as we discussed in Section 2.5, we can greatly decrease the number of non-zero elements of w and instead solve for the remaining non-zero elements of w, where the matrix to be inverted has dimension n×n with n being the number of non-zero elements of w. A schematic of the proposed model is summarized in Figure 3.

Fig. 3:

Schematic overview of the proposed method. In the first step (Data preparation), the model creates the pairwise drug-cell line comparison matrix using one-hot vectors of drugs and the gene expressions for training cell lines. Then, the proposed model converts the problem to an elastic net regression by changing and expanding the pairwise drug-cell line comparison matrix. Finally, iterative thresholding reduces the complexity and running time of the model. In the training step, w will be learned using Equation (7). Finally, in the last step, the scores of drugs for a new cell line are calculated using Equation (3). Eventually, the ranking of the drugs in that cell line is obtained as a permutation π which sorts the drugs in the decreasing order of their score.

2.7. Experimental Setting

The data were standardized so that variables lie between zero and one. The dataset was further split into a training and a test set. Details of the pre-processing steps can be found in Appendix C.

We applied iterative thresholding on the training set and found the non-zero elements of w. Performance was evaluated on the test set. To calculate some of the metrics we classified each drug into two classes – sensitive/insensitive – for each cell line. Specifically, a fixed percentile of all drug response values in the training set for each cell line was used as a threshold to determine drug sensitivity in that cell line.

2.8. Performance metrics

To evaluate the accuracy of our approach, we used two ranking metrics, namely AH@k, and CI^s. The first metric is the average-hit at k, which is the average number of sensitive drugs that are ranked among the top k of a ranking list [25], [26]. This metric is important because in this specific application we care more about sensitive drugs. It evaluates the ability of a model to place sensitive drugs on top of insensitive drugs. However, a high value for AH@k does not necessarily guarantee that the ordering among the topranked drugs is correct. For that purpose, we consider the concordance index CI. The concordance index measures the ratio of correctly ordered drug pairs among all possible pairs [25]. In other words, it measures whether the ordering structure of a ranking list is close to its ground truth or not. Since we care more about the sensitive drugs, we use CI to evaluate the ranking structures among only sensitive drugs and denote it by CI^s. If we use $d_m\succ d_n$ to represent that drug d_m is more sensitive, and thus ranked higher than drug d_n, CI^s can be defined as follows:

$$C{I}^s=\frac{1}{\left|d_m\stackrel{\top }{\succ }{d}_n\right|}\sum _{d_m\succ d_n}\mathbb{I}\left(d_m\stackrel{p}{\succ }{d}_n\right),$$

where $d_m\stackrel{\top }{\succ }{d}_n$ represents that drug d_m is ranked higher than drug d_n based on the ground truth ranking, $d_m\stackrel{p}{\succ }{d}_n$ represents that drug d_m is ranked higher than drug d_n based on the predicted ranking list of sensitive grants, $\mathbb{I}$ is the indicator function, and $|\cdot |$ denotes the cardinality of a set [18].

2.9. Baseline methods

We compared the proposed method against twelve state-of-the-art approaches for drug sensitivity prediction including Elastic Net (EN) [9], Kernel Ridge Regression (KRR) [27], Bayesian Multitask Multiple Kernel Learning (BMTMKL) [28], Partial Least Square regression (PLS) [29], Principal Component Regression (PCR) [24], Sparse Principal Component Regression (SPCR) [30], Gaussian Process Regression (GPR) [31], LASSO [11], Robust Support Vector Regression (RSVM) [32], LambdaMART [33], Neural Network Regression (NNR) [34] and Deep neural network (DNN) [35]. For more information about these algorithms, please refer to the above-mentioned references. Furthermore, the details of LambdaMART and NNR can be found in the supplementary materials. In general, the typical practice for regression models is to design supervised predictive models for each drug based on the gene expression profiles of cell lines. Then, the ranking of the drugs in a specific cell line is obtained as a permutation, sorting the drugs in the decreasing order of predicted drug response values.

2.10. Hyper-parameter optimization

There are three hyper-parameters in our model, namely ${\varphi }_1$, ${\varphi }_2$ (cf. (1)) and λ₁ (cf. (2)). We tuned these hyper-parameters with a grid search on the training set, using cross-validation. To that end, we tried different values to find the best values of these hyper-parameters. Similarly, we conducted a grid search for each of the parameters of the baseline methods. The details of the parameter-tuning procedure can be found in Appendix D.

3. Results

We consider three case studies for ranking cancer drugs acting on cancer cell lines: (i) lung cancer cell lines, (ii) blood cancer cell lines, and (iii) myeloma and lymphoma cell lines. In each case, we compare the performance of the proposed method on ranking drugs with baseline methods introduced in Sec. 2.9. The source code is provided on GitHub. ¹

In most of the prior work in the literature, all the cell lines and drugs in CCLE and CTRP v2 (see Sec. 2.2) were used for ranking. In this work, we elected to develop separate drug lists for each of the cancer cases studies outlined above. One reason is that cancer care remains specialized to the type of tissue where the primary cancer developed. The drawback is that the ranking problem is more challenging, since, in each case, we work with fewer cell lines. As a result, part of the goal is to assess whether the proposed model is able to overcome this challenge. From a histological angle perspective, there are six broad categories of cancers based on tissue type: carcinoma, sarcoma, myeloma, leukemia, lymphoma, and mixed types [36]. Moreover, depending on the primary site of origin, cancers may be of specific types such as lung cancer, liver cancer, breast cancer, etc.

In Case Study 1, we used the cell lines which were derived from the lung cancer tumors (carcinoma group). After removing missing data, we have 51 cell lines and 100 drugs. According to our gene selection results, 88 genes are selected as a minimal subset of all genes. The details of the gene selection scheme and the list of the selected genes can be found in Appendix A. We present two different experiments, one involving the ranking of 50 drugs, and the other involving 100 drugs. In all experiments, we set the sensitivity threshold so that the number of sensitive drugs in that experiment is exactly 10 drugs. Therefore, even when we have 100 drugs, we would have 10 sensitive drugs and we will measure the ability of the models to rank these sensitive drugs at the very top of the ranking list. It worth mentioning that in the second experiment, we added 50 new drugs to the existing drugs in the first experiment. However, the sensitive drugs in the first experiment are not exactly the same as the sensitive drugs in the second experiment because in each experiment, the threshold is defined based on a different set of drugs.

In Table 1, we compare the proposed method with the baseline methods for 50 and 100 drugs. In this and the tables corresponding to the other two case studies, the last two columns report the average AH@5 and AH@10 for testing data that correspond to the best parameter combinations. Bold and underlined numbers indicate the best performance among all methods for each metric. Bold numbers demonstrate the second-best performance among all methods for each metric. In all tables, Ω denotes the kernel parameter for RSVM, NPComp is the number of principle components, PLScomp is the number of PLS components, Pcomp is the number of principle components, Card is the desired number of non-sparse components (cardinality) of output for each principal component, τ is the learning rate, D_max is the maximum depth of a tree, Sub refers to the subsample ratio of the training instances, and n_T is the number of trees in the model. Additionally, Drop refers to hidden dropout ratio, and ${\ell }_2$ is the parameter of the ${\ell }_2$-norm penalty for the DNN method.

TABLE 1:

Performance Comparison of Ranking Methods for 50 and 100 Drugs (Lung Cancer Cell Lines).

	Methods	Parameters	AH@5	AH@10
50 Drugs	RSVM	Ω = 0.1    γ = 100    Kernel = polynomial	3.7	6.4
	LASSO	λ = 10	3.6	6
	KRR	λ = 105    σ2 = 10    Kernel = RBF	3.6	6
	EN	α = 0.1    λ = 100	3.6	6
	BMTMKL	αb = 10−10    βb = 1010	3.3	6.1
	PLS	PLScomp = 28	3.4	5.2
	PCR	Pcomp = 32	3.4	5.4
	SPCR	Pcomp = 29    Card = 25	3.4	6.4
	GPR	Kernel = Matern 5/2	3.7	6.1
	NNR	Epochs = 50    Batch size = 5	3.6	6.4
	LambdaMART	τ = 0.01  nT = 100  Dmax = 5  Sub = 0.7	3.8	6.4
	DNN	Epochs = 100  Batch size = 5  Drop = 0.3  ${\mathcal{l}}_2$ = 0.001	3.6	6.4
	Proposed Model	${\varphi }_1$ = 2.5  ${\varphi }_2$ = 1  NonZ = 5010  λ1 = 1	3.8	6.6
100 Drugs	RSVM	Ω = 0.1    γ = 1    Kernel = polynomial	3.4	5.4
	LASSO	λ = 100	3.4	5
	KRR	λ = 10    σ2 = 10  Kernel = RBF	3.4	5.2
	EN	α = 0.1    λ = 10	3.4	5.4
	BMTMKL	αb = 10−10    βb = 1010	3.2	5.2
	PLS	PLScomp = 26	3	4.5
	PCR	Pcomp = 22	3.2	4.9
	SPCR	Pcomp = 28    Card = 75	3.5	4.9
	GPR	Kernel = exponential	3.5	5
	NNR	Epochs = 100    Batch size = 5	3.4	5.1
	LambdaMART	τ = 0.01  nT = 1000  Dmax = 5  Sub = 1	3.2	5.3
	DNN	Epochs = 100  Batchsize = 5  Drop = 0.1  ${\mathcal{l}}_2$ = 0.01	3.2	4.9
	Proposed Model	${\varphi }_1$ = 4  ${\varphi }_2$ = 1  NonZ = 2694  λ1 = 1	3.8	5.5

In general, the predictive performance obtained by our model for both scenarios is significantly higher than the baseline methods. As mentioned earlier, only the top few most sensitive drugs will be of great interest in practice. Clearly, the proposed method performs considerably well in this aspect. Evidently, the difference between the best and the second-best AH@5 for 100 drugs, is greater than what we find for 50 drugs. In our experiment on lung cancer cell lines, RSVM shows a reasonably good overall performance and it is the second-best methods. Moreover, LambdaMART, DNN, and NNR which perform well in terms of AH@5 and AH@10 for 50 drugs, performed poorly for 100 drugs. As can be seen, due to the limited number of samples available for training and testing, the performance of the baseline methods decreases significantly when we increase the number of drugs. Nevertheless, the proposed method is able to maintain its high performance. The data augmentation step (i.e., comparing each drug with other drugs in the same cell line and comparing that drug with itself in other cell lines) and predicting scores instead of the exact drug responses are two main reasons behind this superior performance.

In Case Study 2, we used the cell lines which were derived from the blood cancer tumors (leukemia group). After removing missing data, we retain 63 cell lines and 100 drugs. Based on our gene selection results, 134 genes are selected as a minimal subset of all genes. The details of the gene selection scheme and the list of the selected genes can be found in Appendix A. The performance comparison of ranking methods for both 50 and 100 drugs is presented in Table 2. The experimental setting for these experiments is exactly like Case Study 1.

TABLE 2:

Performance Comparison of Ranking Methods for 50 and 100 Drugs (Blood Cancer Cell Lines).

	Methods	Parameters	AH@5	AH@10
50 Drugs	RSVM	Ω = 0.1    γ = 0.1    Kernel = polynomial	3.42	5.50
	LASSO	λ = 1000	3.42	5.50
	KRR	λ = 1000    σ2 = 10    Kernel = RBF	3.42	5.50
	EN	α = 0.3    λ = 10	3.42	5.50
	BMTMKL	αb = 10−10    βb = 1010	3.17	5.42
	PLS	PLScomp = 30	3.00	5.00
	PCR	Pcomp = 26	3.25	5.33
	SPCR	Pcomp = 25    Card = 50	3.42	5.50
	GPR	Kernel = Matern 3/2	3.42	5.50
	NNR	Epochs = 100    Batch size = 5	3.42	5.50
	LambdaMART	τ = 0.1  nT = 1000  Dmax = 5  Sub = 0.7	3.33	5.58
	DNN	Epochs = 100  Batchsize = 5  Drop = 0.3  ${\mathcal{l}}_2$ = 0.001	3.33	5.42
	Proposed Model	${\varphi }_1$ = 5  ${\varphi }_2$ = 1  NonZ = 2886  λ1 = 105	3.42	5.92
100 Drugs	RSVM	Ω = 2    γ = 0.1    Kernel = polynomial	3.42	5.08
	LASSO	λ = 100	3.42	5.08
	KRR	λ = 100    σ2 = 100    Kernel = RBF	3.42	5.08
	EN	α = 0.8    λ = 10	3.42	5.08
	BMTMKL	αb = 10−10    βb = 1010	2.83	5.08
	PLS	PLScomp = 29	2.92	4.83
	PCR	Pcomp = 27	3.17	5.08
	SPCR	Pcomp = 25    Card = 25	3.42	5.08
	GPR	Kernel = squared exponential	3.42	5.08
	NNR	Epochs = 50    Batch size = 5	3.42	5.08
	LambdaMART	τ = 0.01  nT = 100  Dmax = 5  Sub = 0.7	3.42	5.17
	DNN	Epochs = 50  Batchsize = 5  Drop = 0.3  ${\mathcal{l}}_2$ = 0.01	3.42	5.08
	Proposed Model	${\varphi }_1$ = 2.5  ${\varphi }_2$ = 1  NonZ = 2664  λ1 = 10	3.58	5.17

For 50 drugs, the proposed method consistently outperforms all baseline methods across all performance metrics except AH@5 where it is tied with several methods (RSVM, LASSO, KRR, EN, SPCR, GPR, and NNR). However, these methods do not have a comparable performance in other metrics. As we can see, AH@10 is the highest for the proposed method and the difference between the best and the second-best AH@10 is considerably high. For 100 drugs, the performance of the proposed method is even better. Most of baseline methods demonstrate equal performance, to which our model compares favorably. In these experiments, LambdaMART, NNR, RSVM, and EN show a reasonably good overall performance and are the second-best methods.

In Case Study 3, we used the cell lines which were derived from myeloma and lymphoma groups. Lymphoma and myeloma are cancers of the immune system. After removing missing data, we have 59 cell lines and 100 drugs. According to our gene selection results, 115 genes are selected as a minimal subset of all genes. The details of gene selection scheme and the list of the selected genes can be found in Appendix A. Table 3 shows the performance comparison for 50, and 100 drugs.

TABLE 3:

Performance Comparison of Ranking Methods for 50 and 100 Drugs (Myeloma and Lymphoma Cell Lines).

	Methods	Parameters	AH@5	AH@10
50 Drugs	RSVM	Ω = 1    γ = 1    Kernel = polynomial	4.00	6.82
	LASSO	λ = 10	4.00	6.91
	KRR	λ = 1    σ2 = 1    Kernel = RBF	4.00	6.91
	EN	α = 0.9    λ = 10	4.00	6.91
	BMTMKL	αb = 10−10    βb = 1010	4.00	6.54
	PLS	PLScomp = 24	4.09	6.91
	PCR	Pcomp = 26	4.09	6.82
	SPCR	Pcomp = 24    Card = 75	4.00	6.82
	GPR	Kernel = Matern 3/2	4.00	6.91
	NNR	Epochs = 100    Batch size = 25	4.00	6.82
	LambdaMART	τ = 0.01  nT = 100  Dmax = 5  Sub = 1	4.00	7.00
	DNN	Epochs = 50  Batchsize = 5  Drop = 0.3  ${\mathcal{l}}_2$ = 0.01	4.00	6.54
	Proposed Model	${\varphi }_1$ = 0.1  ${\varphi }_2$ = 1  NonZ = 3980  λ1 = 0.1	4.09	7.00
100 Drugs	RSVM	Ω = 1    γ = 0.1    Kernel = polynomial	3.82	6.64
	LASSO	λ = 10	4.00	6.64
	KRR	λ = 10    σ2 = 10    Kernel= RBF	4.09	6.64
	EN	α = 0.1    λ = 10	4.09	6.73
	BMTMKL	αb = 10−10    βb = 1010	3.82	6.64
	PLS	PLScomp = 28	4.00	6.45
	PCR	Pcomp = 32	3.82	6.45
	SPCR	Pcomp = 28    Card = 100	3.82	6.64
	GPR	Kernel = exponential	4.00	6.64
	NNR	Epochs = 100    Batch size = 5	3.82	6.64
	LambdaMART	τ = 0.1  nT = 10  Dmax = 10  Sub = 0.7	3.64	6.36
	DNN	Epochs = 100  Batchsize = 5  Drop = 0.3  ${\mathcal{l}}_2$ = 0.01	3.82	6.45
	Proposed Model	${\varphi }_1$ = 0.3  ${\varphi }_2$ = 1  NonZ = 4180  λ1 = 50	4.18	6.82

For 50 drugs, the proposed method outperforms all baseline methods across all performance metrics except AH@5 where it is tied with two methods (PLS and PCR). However, these methods do not have a comparable AH@10. When we increase the number of drugs to 100, the predictive performance obtained by our method is found to be significantly higher than the baseline methods. In our experiment for myeloma and lymphoma cell lines, EN shows a reasonably good overall performance and it is the second-best method. Furthermore, LambdaMART and PLS that perform really well in terms of AH@10 for 50 drugs, demonstrated a poor performance for 100 drugs.

Even though in personalized medicine and drug selection, we just care about the top few most sensitive drugs, knowing the right order of those sensitive drugs will be of great interest. In this regard, CI^s measures whether the ordering structure of a ranking list for sensitive drugs is close to its ground truth or not. As demonstrated in Figure 4, the proposed method has a higher ability to put the sensitive drugs in the right order compared to the baseline methods. LambdaMART is the only method that has a comparable or better CI^s. However, when we consider the main evaluation metrics (AH@5 and AH@10), it does not have a comparable performance.

Fig. 4:

CI^s comparison of ranking methods for 50 and 100 drugs.

In a nutshell, these results suggest that predicting drug sensitivities with the proposed method leads to superior predictive performance than the baseline methods across all performance metrics. Moreover, the proposed method is not only able to push the most sensitive drugs to the top of the ranking list, but it can put them in the right order.

4. Discussion and Conclusion

Since the effectiveness of medicines and therapies varies among patients, a conventional clinical practice is to treat cancer patients with a variety of therapeutic options. However, using molecular data, we can use the biological differences among patients’ cancers to choose precise and individualized therapeutic options. To that end, we proposed a novel elastic-net-based regression that utilizes gene expression features and drug sensitivity data to build a predictor. In general, this model provides the following considerable advantages:

• Instead of predicting the exact drug response for each drug, we calculate a score for each drug and then use these scores to rank the drugs for a specific cancer cell line. This approach is able to solve the problem of fitting on the insensitive drugs that is usually seen in regression-based models.

• We learn a coefficient matrix for all the cell lines and drugs, and then define a score function using that matrix to prioritize drugs within each cell line. In this way, we are able to capture drug-cell line relations between various cell lines.

• Since we usually do not have access to enough data, we need to achieve high performance with a limited amount of data. The specific structure of this model (comparing each drug with other drugs in the same cell line and comparing that drug with itself in other cell lines) substantially improves the accuracy of the drug prediction model under limited data.

• The proposed model is able to maintain its high performance even when we use a large number of drugs and a few cell lines. As we saw in Section 3, when we increase the number of drugs, we increase the complexity of the problem and predicting the true ranking becomes more difficult. We observed that when increasing the number of drugs, the ability of other methods to predict the right ranking decreases substantially. However, the proposed method maintains its high performance even when we run it for 100 drugs.

• A great number of methods in the literature try to learn either a vector or a matrix for each drug or cell line. Such methods need to keep a huge amount of information to find the ranking for new samples. In contrast, the proposed method maintains one sparse matrix.

• Learning one sparse matrix for all cell lines and drugs reduces the possibility of over fitting in the proposed model.

Although our model is quite promising, it also suffers from some limitations that can be addressed in future work. First, in the proposed model, we only used gene expression data. However, one way to increase the prediction accuracy of this model is to incorporate various genomic information, such as epigenomic characterizations and protein level information. Furthermore, an interesting future direction is to incorporate the toxicity of drugs in our predictions. Due to the biological differences among patients, the side effects of medicines and therapies may vary. However, when we predict the most effective drugs, we do not consider the side effects and toxicity of each drug. Therefore, by considering this factor, we can further optimize our predictions. Finally, graph neural networks have been shown to yield state-of-the-art results in some applications compared to other deep learning-based approaches [37]. It has been demonstrated that representing compound structures as molecular graphs can improve the performance of drug response prediction [37]. Thus, in future work, and assuming we have drug composition information, we can represent drug molecules as graphs instead of using one-hot encoding vectors.

Biographies

Shahabeddin Sotudian received the B.S. and M.S. degrees both in industrial and systems engineering from Tehran Polytechnic, in 2015 and 2017, respectively. He is currently a Ph.D. candidate in the Department of Electrical and Computer Engineering, Division of Systems Engineering, Boston University, Boston, MA, USA. His research interests lie in optimization, applied probability and statistics, and machine learning, with applications in healthcare and biology.

Ioannis Ch. Paschalidis received the M.S. and Ph.D. degrees both in electrical engineering and computer science from the Massachusetts Institute of Technology(MIT), Cambridge, MA, USA, in 1993 and 1996, respectively. In September 1996 he joined Boston University where he has been ever since. He is a Professor at Boston University with appointments in the Department of Electrical and Computer Engineering, the Division of Systems Engineering, and the Department of Biomedical Engineering. He is the Director of the Center for Information and Systems Engineering (CISE). He has held visiting appointments with MIT and Columbia University, New York, NY, USA. His current research interests lie in the fields of systems and control, networking, applied probability, optimization, operations research, computational biology, and medical informatics. Dr. Paschalidis is a recipient of the NSF CAREER award (2000), several best paper and best algorithmic performance awards, and a 2014 IBM/IEEE Smarter Planet Challenge Award. He was an invited participant at the 2002 Frontiers of Engineering Symposium, organized by the U.S. National Academy of Engineering and the 2014 U.S. National Academies Keck Futures Initiative (NAFKI) Conference. He is the inaugural Editor-inChief of the IEEE Transactions on Control of Network Systems.