Machine learning–based integration of network features and chemical structure of compounds for SARS‐CoV‐2 drug effect analysis

Abstract

High drug development costs and the limited number of new annual drug approvals increase the need for innovative approaches for drug effect prediction. Severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2), the cause of coronavirus disease 2019 (COVID‐19), led to a global pandemic with high morbidity and mortality. Although effective preventive measures exist, there are few effective treatments for hospitalized patients with SARS‐CoV‐2 infection. Drug repurposing and drug effect prediction are promising strategies that could shorten development time and reduce costs compared with de novo drug discovery. In this work, we present a machine learning framework to integrate a variety of target network features and physicochemical properties of compounds, and analyze their influence on the therapeutic effects for SARS‐CoV‐2 infection and on host cell cytotoxic effects. Random forest models trained on compounds with known experimental effects on SARS‐CoV‐2 infection and subsequent feature importance analysis based on Shapley values provided insights into the determinants of drug efficacy and cytotoxicity, which can be incorporated into novel drug discovery approaches. Given the complexity of molecular mechanisms of drug action and limited sample sizes, our models achieve a reasonable mean area under the receiver operating characteristic curve (ROC‐AUC) of 0.73 on an unseen validation set. To our knowledge, this is the first work to incorporate a combination of network and physicochemical features of compounds into a machine learning model to predict drug effects on SARS‐CoV‐2 infection. Our systems pharmacology–based machine learning framework can be used to classify other existing drugs for SARS‐CoV‐2 infection and can easily be adapted to drug effect prediction for future viral outbreaks.

STUDY HIGHLIGHTS.

• WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?

Drug discovery and development is a complex process that aims to bring a new drug from bench to bedside, which is costly, time‐consuming, and high risk. Of drug candidates in clinical trials, 90% fail overall, and 40%–50% of clinical failures are attributed to a lack of clinical efficacy.

• WHAT QUESTION DID THIS STUDY ADDRESS?

This study is the first work to incorporate both network and physicochemical features of compounds into a machine learning model to predict drug effects on severe acute respiratory syndrome coronavirus 2 infection.

• WHAT DOES THIS STUDY ADD TO OUR KNOWLEDGE?

Random forest models followed by feature importance analysis based on Shapley values provided insights into the determinants of drug efficacy and cytotoxicity.

• HOW MIGHT THIS CHANGE DRUG DISCOVERY, DEVELOPMENT, AND/OR THERAPEUTICS?

This systems pharmacology–based machine learning framework can be adapted to drug effect classifications for other diseases. It has the potential of saving a significant amount of time and cost by narrowing the drug list for potential clinical trials.

INTRODUCTION

Drug discovery and development is a complex process that aims to bring a new drug against validated biological targets associated with a particular disease from bench to bedside, which is costly, time‐consuming, and high risk. ¹ It takes, on average, 10–15 years and $1.0–2.6 billion ² , ³ for a drug to complete all three phases of clinical trials before being approved for use in patients. Moreover, 90% of drug candidates in clinical trials fail overall, and 40%–50% of clinical failures are attributed to a lack of clinical efficacy. ⁴ , ⁵ For these reasons, there are only a limited number of new drug approvals annually (5‐year average for Food and Drug Administration–approved therapeutics is 51 drugs/year). ⁶

To accelerate drug discovery, a method that accurately predicts drug effects early in the drug discovery process is sorely needed. Prediction of drug effects is a challenging problem owing to the mechanistic complexity of drug action, but it has the potential of saving a significant amount of time and cost by narrowing the list of potential clinical trials. Many computational methods for drug efficacy prediction and repositioning have been proposed, including machine learning–based approaches. ⁷ , ⁸ , ⁹ Although most of the extant machine learning methods are based solely on the input of drugs' chemical structure (or structure–activity profile) or induced transcriptional profiles, ¹⁰ , ¹¹ , ¹² drug effects also depend on the mode of action and the microenvironment of molecular interactions within which drug targets operate biochemically. Indeed, network‐based approaches have been used to explore the relationship between drugs and diseases as well as to identify drug repositioning opportunities through molecular interaction network‐based insights into drug target actions. ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷

Severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) causes coronavirus disease 2019 (COVID‐19) and led to a global pandemic with high morbidity and mortality. Despite effective preventive measures, there are currently no highly effective treatments for severely infected patients. Nevertheless, many insightful data related to SARS‐CoV‐2 infection have been generated, including interactions between SARS‐COV‐2 proteins and human target proteins, ¹⁸ and SARS‐COV‐2‐induced (differential) gene and protein expression. ¹⁹ , ²⁰ Considering these omics data, drug repurposing and drug effect prediction based on knowledge of available drugs are promising strategies that could shorten the time and reduce the cost of drug discovery compared with de novo drug discovery. ²¹ Morselli Gysi et al. proposed a network medicine‐based framework for identifying potential drug‐repurposing opportunities for COVID‐19. ²² Patten et al. experimentally validated a 6710 compound library and identified potent SARS‐CoV‐2 infection inhibitors. ²³ Yet, neither of these studies explored the potential predictive advantages of integrating targets' protein–protein interaction network features with physicochemical properties of compounds in a comprehensive computational strategy.

To explore to what extent the network features of drug targets in the human interactome and the physicochemical properties of compounds can contribute to drug effects, in this work, we developed a machine learning framework to integrate physicochemical properties and target interactions of compounds and analyze their influence on drug effects (therapeutic effects and cytotoxic effects) for SARS‐CoV‐2 infection. In particular, we employed a set of compounds with known therapeutic effects on SARS‐CoV‐2 infection as the training data. We mapped SARS‐CoV‐2 targets and drug targets into the human interactome and explored a variety of network parameters of drug targets and the physicochemical properties of drugs. The random forest model trained on the benchmark data set and subsequent feature importance analysis provided insightful interpretations of the properties of effective drugs as well as drugs with cytotoxic effects, which can be included in novel drug repurposing strategies. The results confirm the role of the human protein–protein interactome in drug effects. To the best of our knowledge, this is the first work to incorporate network features with physicochemical features into a machine learning model to analyze and predict drug effects for SARS‐CoV‐2 infection. The proposed drug effect prediction pipeline could be effective in future viral outbreaks, where no vaccines will be available rapidly.

METHODS

Human interactome and covidome

The interactome used in this work consolidates physical protein–protein interactions directly from the most high‐profile, high‐throughput studies into one network compiled by Wang and Loscalzo, ¹⁵ including binary protein–protein interactions, protein complexes, kinase–substrate interactions, and signaling pathways. High‐quality protein–protein interactions are compiled from several high‐throughput yeast‐two‐hybrid studies, mass spectrometry, and the curated literature. In addition, experimental signaling interactions; kinase–substrate interactions; and high‐quality, literature‐based signaling interactions were also incorporated. The consolidated interactome so defined contains 16,454 proteins and 233,957 interactions. In addition to the human interactome, some of our features use the information on the 332 human host proteins physically targeted by SARS‐CoV‐2, henceforth referred to as the covidome. This information was determined experimentally in 2020 by Gordon et al. using affinity‐purification mass spectrometry. ¹⁸

Compound data

We further collected information about potentially repurposable candidate drugs from the Broad Institutes' Drug Repurposing Hub (DRH) library. We used the drug information (Version 3/24/2020) for compounds currently in clinical trials and their corresponding protein targets. In addition, we collected sample annotation (Version 3/24/2020), including an identifier for compound‐name mapping and further compound‐level information. We used the experimental results of Patten et al., ²³ which tested the SARS‐CoV‐2 viral load of four doses of each compound of the DRH library using immunofluorescence‐based screening. Thereafter, each compound was classified into strong, weak, very weak, inactive, or cytotoxic using a log‐logistic regression for viral load and nuclei count, surrogates for viricidal activity (decreased viral load compared with control) and host cell cytotoxicity (decreased nuclei count compared with control), respectively.

Feature extraction and preprocessing

The network properties of compound targets were calculated using a python library graph‐tool, ²⁴ which provides a graphical representation of the human interactome as a protein–protein interaction network with proteins as nodes and their interactions as edges. In addition to various network centrality measures for drug targets and their first neighbors, we also calculated the shortest path length from a drug target to the covidome (two different proximities) for each drug and then took the minimum, mean, median, and maximum of the shortest path lengths over all the targets of the drug as network features of this drug. If the targets of a drug have no paths to the covidome, we used a large constant (i.e., the longest shortest path length between any drug target and any covidome node) to impute these features. In total, 67 network features of drug targets were considered (Table 1).

TABLE 1.

The 67 network features of drug targets extracted from the human interactome.

Topological measures	Groups	Aggregations over multiple drug targets	Descriptions
Pagerank (2 × 4 features)	Targets, first neighbors	Mean, median, maximum, minimum	Measures the importance of a node based on the number of incoming edges and their corresponding nodes' importance
Betweenness centrality (2 × 4 features)	Targets, first neighbors	Mean, median, maximum, minimum	Measures the influence of nodes on the information flow through a graph by scoring the number of shortest paths passing through them
Closeness centrality (2 × 4 features)	Targets, first neighbors	Mean, median, maximum, minimum	Measures how efficiently a node can spread information in the graph by calculating the inverse distance to all other nodes
Eigenvector centrality (2 × 4 features)	Targets, first neighbors	Mean, median, maximum, minimum	Measures the transitive influence of nodes
Katz centrality (2 × 4 features)	Targets, first neighbors	Mean, median, maximum, minimum	Measures the influence of nodes using the total number of walks between the node of interest and any other node
Local clustering coefficient (2 × 4 features)	Targets, first neighbors	Mean, median, maximum, mininum	Describes the likelihood of a node's neighbor being connected
Degree centrality (2 × 4 features)	Targets, first neighbors	Mean, median, maximum, minimum	Measures the number of connections a node has in the graph
Average shortest distance to covidome (four features)	Targets	Mean, median, maximum, minimum	Among all the shortest distances of a target to nodes in the covidome, the average is taken
Minimum shortest distance to covidome (four features)	Targets	Mean, median, maximum, minimum	Among all the shortest distances of a target to nodes in the covidome, the minimum one is taken
Number of drug targets (one feature)	Targets	—	Total number of drug targets in the interactome
Number of first neighbors (one feature)	Targets	—	Total number of first neighbors in the interactome
Drug targets in the covidome (one feature)	Targets	—	Total number of drug targets in the covidome

Based on the Simplified Molecular Input Line Entry System (SMILES) codes of the compounds, the physicochemical properties of each compound were extracted from PubChem using the python package pubchempy. ²⁵ We removed the perfectly correlated features “monoisotopic mass” and “exact mass” and kept only “molecular weight.” After extracting the features, we removed compounds with no targets. We also dropped compounds with very weak and low effects, such that only strong+ weak (= effective), no‐effect, and cytotoxic compounds were left in the data set. Furthermore, we retained compounds with missing structural annotations, as these will be imputed in the machine learning workflow. In total, 26 physicochemical properties were included in the analysis (Table 2, Table S1).

TABLE 2.

Physicochemical properties of each compound.

Physicochemical properties	Descriptions
Molecular weight	Sum of all atomic weights in a compound, measured in g/mol
XLogP 26	Computationally predicted octanol–water partition coefficient, measuring the hydrophobicity of a compound
TPSA 27	Estimate of the TPSA in A
Complexity 28 , 29	Estimate of structure complexity
Charge	Net charge of the compound
Various 2D counts	Several counts of structural properties: H‐bond donors, H‐bond acceptors, rotatable bonds, heavy atoms, stereo atoms, stereo bonds, and covalent units
Volume 3D	Volume of the compound
Steric quadrupole 3D	Steric quadrupole property in 3D
Various 3D counts	Several features of 3D: acceptor features, donor features, anion features, cation features, ring features, hydrophobic features, effective rotors, and conformers
Conformer model RMSD 3D	The RMSD of the conformer model

Abbreviations: 2D, two‐dimensional; 3D, three‐dimensional; RMSD, root mean squared distance; TPSA, topological polar surface area.

Evaluation workflow

Our machine learning workflow is implemented using the scikit‐learn Python package ³⁰ (Figure S1). We split our data into an 80% training set and a 20% unseen validation set (label stratified). We used the training set for hyperparameter optimization (randomized search with 100 iterations), cross‐validation (CV), and model training. The pipeline consists of dropping low‐variance variables, standardization of the data (scaling the data to zero mean and unit variance), and K‐nearest neighbors imputation of missing values. Owing to the high imbalance in our class labels, we additionally used a random undersampling of the majority class in the pipeline. This means that in each fold of the CV, the majority class was undersampled in the training fold, but never in the test folds and unseen validation set. At the end of this pipeline, a balanced random forest classifier is trained, taking into consideration the imbalance in each bootstrap sample. The sets of hyperparameters from which 100 combinations were randomly chosen in the randomized search can be found in Appendix S1.

We used the area under the receiver operating characteristic curve (ROC‐AUC) to evaluate our model and assess potential label imbalance. ROC‐AUC scores were generated through scikit‐learn's roc_auc_score function using probability estimates and a 0.5 cutoff. We used a stratified 5 × 5–fold nested CV on the training data to evaluate the model performance and prevent undetected overfitting. The stratification accounts for the class imbalances, whereas the repetitions and nestedness avoid common validation pitfalls. ³¹ The unseen validation set is only used in the final validation of the best‐performing models. Data and code are available in our GitHub repository: https://github.com/julianspaeth/sars‐cov‐2‐classification.

Feature importance analysis

Shapley values borrow insights from cooperative game theory and measure the average expected marginal contribution of one feature to a model's predictions at the instance level after all possible combinations have been considered. ³² Therefore, the Shapley value shows how much a feature of each sample contributes to the model prediction of this sample. To calculate the feature importance of our model, we used the TreeExplainer model of the python package SHAP. SHAP is based on a game‐theoretic approach using Shapley values that have various advantages over previous feature importance methods, such as directly measuring local feature effects and combining local explanations with global insights. ³³ , ³⁴ In this study, the Shapley value analysis was conducted using the final cross‐validated models trained with the optimal hyperparameters on the unseen validation set, and feature importance is calculated using the mean absolute Shapley values over all samples for the feature. The colors of feature values are based on a 0–1 minimum–maximum scaling of each feature. The Shapley value of a feature group for a sample was obtained by summing the Shapley values of the individual features in the group for the sample. Feature groups were then ranked by the feature group importance, defined as the mean absolute Shapley values of the feature group over all the samples.

RESULTS

In this work, we implemented a machine learning framework to integrate network features and physicochemical properties of compounds to predict drug effects on SARS‐CoV‐2 infection. Our framework, graphically illustrated in Figure 1, consists of extracting physicochemical properties of compounds and topological network features of compound targets in the human protein–protein interactome. The resulting combined feature set was used to train random forest models to classify effective versus no‐effect, cytotoxic versus no‐effect, and cytotoxic versus effective compounds (where effect refers to viricidal efficacy, and cytotoxic refers to host cell cytotoxicity, both in high‐throughput assays). The reason we chose three binary models instead of one multiclass model was that binary models are easier to understand, evaluate, and interpret. Another advantage is that the models can be used separately, depending on the user's purpose. After the models were trained, feature importance was calculated to identify potential hallmarks of compounds that lead to a specific drug effect.

FIGURE 1.

Overview of our machine learning framework for compound effect prediction and analysis. Using experimentally validated compounds, their physicochemical properties, and their target interactions in the human interactome, we trained random forest models to predict effective, cytotoxic, and no‐effect compounds and performed a subsequent feature importance analysis using Shapley values. SARS‐CoV‐2, severe acute respiratory syndrome coronavirus 2.

Targets and physicochemical properties of compounds

We used graph‐tool ²⁴ to produce a graphical representation of the human interactome as a protein–protein interaction network with proteins as nodes and their interaction as edges. In addition, we annotated the 332 human SARS‐CoV‐2 target proteins, ¹⁸ henceforth denoted the covidome. Of the 332 covidome proteins, 326 were present in our consolidated interactome. We merged the sample and drug information files from DRH with the corresponding screening outcomes and dropped “very weak” and “low” effect compounds, as their z‐scores were below 2.5 compared with controls. ²³ Accordingly, only strong (116), weak (140), cytotoxic (78), and no‐effect (3526) compounds remained. To simplify the model complexity, we merged strong‐effect and weak‐effect labels together to new “effective” labels (256). Now we have three mutually exclusive drug groups: effective, no‐effect, and cytotoxic.

For each node in the interactome, we calculated various network features, including seven centrality measures and two different proximities (shortest distances) of nodes to the covidome, as shown in Table 1. We next calculated the mean, median, minimum, and maximum of each network centrality measure over all annotated targets of a compound as well as for their first neighbors. In addition, we calculated the mean, median, minimum, and maximum of two different shortest distances over all annotated targets of a compound. Furthermore, we added the total number of targets, neighbors, and targets in the covidome as features, resulting in 67 target‐related network features classified into 12 categories (Table 1).

We analyzed the occurrence of targets between the different classes: effective, cytotoxic, and no‐effect. As shown in the Venn diagram in Figure 2, many targets appear in more than one class. Specifically, 58 targets appear at least once in effective (red circle), cytotoxic (blue circle), and no‐effect (green) compounds. A total of 66 targets appear exclusively in effective compounds, 24 in cytotoxic, and 1395 in the no‐effect compound classes. Notably, 209 targets appear in both the effective drug class and the no‐effect drug class, indicating that some drugs from different effect categories partially share targets. We also found a number of drugs from different categories that have identical target sets, which demonstrates the necessity of incorporating chemical structures into drug effect prediction.

FIGURE 2.

Venn diagram showing the target overlap of the compound classes. The circles represent the number of no‐effect targets (green), effective targets (red), cytotoxic targets, and their corresponding intersections. A total of 66 targets appear exclusively in effective drug, 24 in cytotoxic drug, and 1395 in no‐effect drug classes.

We next used the python package, pubchempy, to extract 26 physicochemical properties of each compound from PubChem, ²⁵ classified into 10 groups. The physicochemical properties we extracted can be found in Table 2 and Table S1, and the drug × property matrix is provided in Table S3. Some physicochemical properties correlate with other features (Figure S2). However, we only removed features with perfect correlations and retained the rest for the models to determine if they are important. Combining all features resulted in 93 physicochemical and network features that are used in the following analyses.

Physicochemical and target interactome properties significantly differ across drug classes

Before incorporating our features into the machine learning framework, we first conducted an exploratory analysis of these features to examine whether or not they differ across the three compound classes (effective, no‐effect, and cytotoxic). We performed a Bonferroni‐corrected independent t‐test on all features across different effect classes to avoid false positives. The corrected p‐values are visualized in the heatmap of Figure 3, where red illustrates a highly significant p‐value (p < 0.001), orange a significant p‐value (p < 0.05), and blue a nonsignificant p‐value for each class pair (row) and feature (columns). As an overview, we created a heatmap separately for each feature subgroup: physicochemical properties, network targets, network first neighbors, and network proximity.

FIGURE 3.

Heatmap showing the Bonferroni‐corrected p‐values of independent t‐tests for each feature. Blue indicates nonsignificant p‐values, orange indicates significant p‐values (p < 0.05), and red indicates highly significant p‐values (p < 0.01). E‐N: effective vs. no‐effect; C‐N: cytotoxic vs. no‐effect; E‐C: effective vs. cytotoxic.

As visible in the heatmap, significant differences in physicochemical properties (upper) almost exclusively appear in a comparison of effective versus no‐effect classes. Only one significant difference was found (three‐dimental [3D] volume) between the cytotoxic and no‐effect classes. Between effective and cytotoxic classes, no significant differences were found. Structure properties seem to play a major role in predicting the efficacy of a compound. In particular, various 3D features, size (steric quadrupole), molecular weight, and volume 3D are key predictive features.

The network properties of the drug targets (e.g., centrality measures, degree) showed the most significant differences among groups. Of the features, 62% show significant differences between effective and no‐effect compounds with a high tendency to centrality measurements and the pagerank of the targets. This implies that the network properties of the drug targets and their importance in the human interactome play a major role in the efficacy of a drug. An even higher number of features showed significant differences between cytotoxic and no‐effect targets (79%). In contrast to effective compounds, the targets of cytotoxic compounds show a significantly different degree compared with no‐effect compounds, with these targets having a higher number of connections to other nodes (higher degree). Between cytotoxic and effective compounds, only the degree and closeness centrality significantly differ.

The network properties of the drug target's first neighbors showed fewer significant differences across compound categories than those of the actual targets, suggesting that the direct microenvironment of the targets matters more for drug action than does a more extensive, meso‐scale network structure. Network proximity parameters examine the proximity of drug targets to the covidome at the human protein–protein interactome. A few differences in the network proximity features were identified in cytotoxic versus effective and effective versus no‐effect comparisons. No significant differences in network proximity features were observed in the effective versus cytotoxic comparison, highlighting the necessity of incorporating physicochemical properties of compounds for drug effect prediction.

Random forest models achieve reasonable prediction performance

We then integrated the physicochemical properties of drugs and network features of drug targets into a random forest–based machine learning model. Specifically, we created three models: no‐effect versus effective, no‐effect versus cytotoxic, and effective versus cytotoxic. After performing a cross‐validated hyperparameter search, the model with the best hyperparameters for each binary classification was evaluated using CV (stratified 5 × 5 folds). For choosing the best hyperparameters, we considered both mean ROC‐AUC and a possibly high minimum ROC‐AUC to increase the robustness of the model. Specifically, we first chose a set of top 10 models with the highest mean of 25 ROC‐AUCs. Next, to ensure that the worst performance of a model is still reasonable, we chose the model that showed the highest minimum ROC‐AUC of the 10 models as our best model. The results of the hyperparameter optimization can be found in Appendix S1 and Table S2. Figure 4 shows a grouped boxplot for the ROC‐AUC scores of the three best‐performing models trained on real drug effect labels (in blue) and the three best‐performing models trained on randomly shuffled drug effect labels with real features (in orange, 100 permutations), which summarize the predictive power of the trained models for the test folds in the 5 × 5–fold nested CV. The ROC‐AUC scores imply that all the three models are equally well predictive of drug effects.

FIGURE 4.

Evaluation of our random forest models. For all class pairs, the real models (blue) performed significantly better (Welch's t‐test) than the models trained on randomly shuffled drug effect labels with real features (orange). The 25 dots in blue represent each single result of the fivefold cross‐validation repeated five times. Each dot in orange represents the average area under the receiver operating characteristic curve (ROC‐AUC)  of cross‐validations over 100 permutations of drug effect labels. The asterisks indicate the level of significance. **** indicates a p‐value ≤1e‐4, *** ≤ 1e‐3, and so on.

In addition, all three models trained on real drug effect labels performed significantly better than the models trained on randomized labels. In all of the 100 permutations (i.e., 100%), significant performance differences were observed according to Welch's t‐test (Table S4). Notably, although few network features and physicochemical properties are significantly different in cytotoxic versus effective classes when considered individually, the machine learning model shows that integrating them together can distinguish cytotoxic compounds from effective compounds. For all models, cytotoxic versus no‐effect, cytotoxic versus effective, and no‐effect versus effective, the mean ROC‐AUCs were greater than 0.7, significantly increasing the correct classification in contrast to a random guess. In the final evaluation of the models on the 20% unseen validation data set, the models performed within the range of the boxplot whiskers. With a ROC‐AUC of 0.68, the no‐effect versus effective model performed slightly worse than our CV result but is still in a reasonable range. The no‐effect versus cytotoxic model even outperformed the CV result with a ROC‐AUC of 0.83, and the cytotoxic versus effective model performed comparably to the CV mean with a ROC‐AUC of 0.69. Together, our models achieve a mean ROC‐AUC of 0.73 on the unseen validation data. Given the limited size of training samples and the high overlap of targets between effective compounds and no‐effect compounds, our models achieve a very reasonable performance.

Feature importance analysis identifies predictive properties of drug effects

We performed a feature importance analysis based on final cross‐validated models trained with the optimal hyperparameters for the unseen validation set. The feature importance was obtained using the TreeExplainer of the python SHAP package to identify a feature's Shapley values driving decisions in our trained models (Figure 5). The absolute Shapley value documents the importance of a sample's feature in separating positive and negative classes. A positive Shapley value indicates that the higher the value of a sample's feature, the more it affects the model predicting the positive class for that sample. In contrast, a negative Shapley value suggests the higher the value of a sample's feature, the more it affects negative class predictions. Each sample–feature set combination (Figure 5a,b) and each sample–feature combination (Figure 5c) generate one Shapley value visualized as a dot in the beeswarm plot. Initially, we assigned the features to smaller subgroups. All features were grouped into physicochemical properties, network features of targets, first neighbors, and network proximity features by summing the individual Shapley values of the feature of each group. This classification can lead to more importance in feature groups with a higher number of features; however, it reveals the overall importance of this group to our models. Furthermore, we grouped the network features into different aggregation types, that is, minimum, mean, median, and maximum, and evaluated the feature groups in the same way.

FIGURE 5.

Feature importance of broader feature groups (a, b) and individual features (c). The beeswarm plots show the importance of different features (c), feature groups (a, b), and their corresponding feature value (color) on the output of the model. Each sample–feature set combination (a, b) and each sample–feature combination (c) generate one Shapley value that is visualized as a dot in the beeswarm plot.

From Figure 5a, we can see that physicochemical properties (“structure”) had the highest influence on distinguishing effective compounds from no‐effect compounds, followed by the network properties of the first neighbors, proximity, and targets. Compared with the physicochemical properties, the network properties have much less influence on the prediction of effective compounds. This finding indicates that network properties alone cannot predict drug effects well, consistent with earlier findings where three different methods were assembled together for optimal prediction. ²² In contrast, the network properties of the targets had the greatest impact, followed by the properties of the first neighbors, on distinguishing cytotoxic compounds from no‐effect compounds. Physicochemical properties had the greatest impact again on distinguishing effective compounds from cytotoxic compounds; however, the network properties retained importance in this comparison, as well. The proximity of compound targets to the covidome was generally the weakest feature group over all three models.

From Figure 5b, we can see that median or mean values of network properties influence distinguishing effective drugs from no‐effect or cytotoxic drugs. Interestingly, we also see that the maximum value of network properties of all targets had the highest importance in distinguishing no‐effect from cytotoxic compounds, but the lowest importance in our other models. This result could indicate that cytotoxicity is driven by extreme network parameters. Overall, all aggregation types added some predictive contributions to the models, showing that it makes sense to incorporate different aggregation types into the feature set.

Although the grouped features provided interesting insights into important properties for our models, a single feature Shapley value analysis allowed us to identify individual features that were important for each model, as depicted in Figure 5c. For the no‐effect versus effective model, three of the five most important features were physicochemical. High feature cation counts, y‐steric quadrupole, and the log of partition coefficient of an organic compound from its topological structure (XLogP) were important for the prediction of effective compounds. The most important network feature was a high median degree of the first neighbors and a low minimum mean shortest path length to the covidome. Our feature importance analysis suggests that effective compounds are rather large and of higher molecular weight, are more lipophilic (XLogP), and have targets close to or inside the covidome.

In contrast to effective compounds, network properties did play a more important role in distinguishing cytotoxic compounds from no‐effect compounds. Seven of the top 10 important features were the network properties of compound targets and first neighbors. The number of neighbors was the most important feature, followed by many properties that define highly important nodes in the interactome (target Katz centrality, neighbor local clustering coefficient, target degree, target pagerank), leading to the hypothesis that cytotoxicity might be caused by compounds whose targets are well connected and have high importance. Moreover, a low minimum mean shortest path length to the covidome was favorable for cytotoxicity. The only physicochemical features with reasonable importance were associated with the size and weight (Y‐steric quadrupole, molecular weight, volume, heavy atom count, and complexity) of the compound.

Similar to the feature importance of the no‐effect versus cytotoxic model, the cytotoxic versus effective model's highest important features are mostly based on network topology (seven of the top 10 features). The targets of effective compounds seem to have fewer neighbors and, therefore, a lower degree than the targets of cytotoxic compounds. This finding indicates that the targets of cytotoxic compounds, in contrast to the targets of effective compounds, are well‐connected nodes in the human interactome that might lead to off‐target effects. An interesting observation is that the neighbors of effective targets have higher degrees than those of cytotoxic targets. This finding indicates that cytotoxic compounds might target hub nodes directly, whereas effective compounds target the neighborhood of hubs. Effective compounds have a higher XLogP and higher y‐steric quadrupole.

DISCUSSION

In this work, we present a random forest–based machine learning framework to analyze and predict drug effects on SARS‐CoV‐2 infection by integrating the physicochemical properties of compounds and the network features of compound targets. Based on the models, we identified important predictive properties for effective, cytotoxic, and no‐effect compounds. Considering the difficulty of the classification task, our models achieved reasonable accuracy and performed significantly better than random models, indicating that they could detect signals for all classes.

There are several reasons responsible for not achieving even higher accuracies. First, the compound data might include inaccuracies. In particular, the compound target sets are not necessarily complete (i.e., actual targets are missing), or not all assigned targets are always activated (i.e., expressed in a given cell type). An important characteristic we ignored is drug target affinities. The Broad Institute's DRH library does not provide target affinity information that might be retrieved from some other databases, for example, BindingDB. ³⁵ The second reason is the small number of effective and cytotoxic compounds. Both classes were highly underrepresented, forcing us to use an undersampling strategy that drastically reduced the data set size. Moreover, a large portion of identical target sets were observed across different drug categories, which further complicates drug effect prediction.

Despite the difficulties in performance, our models could still detect patterns that enabled them to distinguish different compound classes significantly better than a random model. Our subsequent feature importance analysis reveals some insights about the properties of effective, cytotoxic, and no‐effect compounds. Based on the top five features of our models, effective compounds consist of a higher number of positive ions (high cation count), a large physical extent of the compound in width (high y‐steric quadrupole), and more rings than no‐effect compounds. In addition, they are more likely to be nonpolar soluble compounds than no‐effect compounds. Cytotoxic compounds were not distinguished from no‐effect compounds by physicochemical features but by the network features of their targets. Cytotoxic compounds have targets with more neighbors, with greater control over the network (high betweenness centrality), with greater importance (pagerank) and higher degrees, and with neighbors having a high influence in the interactome (Katz centrality) than no‐effect compounds. Compared with effective compounds, cytotoxicity is associated with targets having higher maximum target degrees, a higher number of neighbors, and a lower mean neighbor degree in the interactome. It seems that the targets of cytotoxic compounds have features indicating control in the network and are, for example, more likely to be hubs; in addition, however, a target's neighbors are of high importance, which could cause a chain of effects and, therefore, amplify off‐target effects. These valuable insights into the properties of effective and cytotoxic compounds can be incorporated into novel drug discovery approaches.

Physiochemical properties influence both pharmacokinetic and pharmacodynamic processes of a drug in a complex way. As mentioned previously, drug target affinities were not included in the models because of the lack of complete, reliable information. Drug target affinities, influenced by factors such as concentration, molecular shape, charge distribution, and noncovalent interactions between the drug and the target can play a significant role in determining a drug's cytotoxic and antiviral effects. If a drug has higher affinity for target proteins involved in cell division (cytotoxic) or viral replication (antiviral), it could lead to more effective inhibition of these processes, resulting in stronger therapeutic effects. Therefore, incorporating reliable drug target affinities into the models would be potentially useful.

Importantly, this study introduces a new framework for analyzing and predicting drug efficacy and cytotoxicity, which can be easily adapted for general drug effect prediction for other diseases. By integrating human interactome data and physicochemical properties of compounds, the models can decipher the underlying determinants of drug effects when the sample size is adequate, along with more precise interactomes and target annotations in the future. We envision our approach will be a useful prediction tool for drug efficacy, possibly reducing the time and expense of high‐throughput, wet‐laboratory experiments, and improving the outcome of subsequent clinical trials.

AUTHOR CONTRIBUTIONS

J.S. and R.‐S.W. wrote the manuscript. J.S., R.‐S.W., and J.L. designed the research. J.S. performed the research. J.S., R.‐S.W., M.H., J.B., and J.L. analyzed the data.

FUNDING INFORMATION

This work was supported by a fellowship within the International Research Stays for Computer Scientists (IFI) programme of the German Academic Exchange Service (to J.S.); Brigham Research Institute Microgrant Award (to R.S.W.); and by National Institutes of Health Grants HL155107, HL155096, HG007691, and HL119145 and American Heart Association (AHA) grants D700382 and 957729 (to J.L.). This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement nos. 777111 and 826,078. This publication reflects only the authors' view, and the European Commission is not responsible for any use that may be made of the information it contains (to J.B. and J.L.).

CONFLICT OF INTEREST STATEMENT

J.L. is scientific cofounder of Scipher Medicine, Inc., which uses network medicine analyses to identify disease biomarkers and potential therapies. All other authors declared no competing interests for this work.

Supporting information

Figure S1

Figure S2

Table S1

Table S2:

Table S3:

Table S4:

Späth J, Wang R‐S, Humphrey M, Baumbach J, Loscalzo J. Machine learning–based integration of network features and chemical structure of compounds for SARS‐CoV‐2 drug effect analysis. CPT Pharmacometrics Syst Pharmacol. 2024;13:257‐269. doi: 10.1002/psp4.13076