VCTatDot and VCTatMLP: novel deep learning models with triadic attention embeddings for synergistic drug combination prediction

Abstract

Computational drug repurposing is vital in drug discovery research because it significantly reduces both the cost and time involved in the drug development process. Additionally, combination therapy—using more than one drug for treatment—can enhance efficacy and minimize the side effects associated with individual drugs. However, there is currently limited research focused on computational approaches to combination therapy for viral diseases. This paper proposes AI-based models to predict novel drug combinations that can synergistically treat viral diseases. To achieve this, we have compiled a comprehensive dataset containing information on viruses, drug compounds, and their approved interactions. We introduce two attention-based models and compare their performance with traditional machine learning and deep learning models in predicting synergistic drug pairs for treating viral diseases. Among all the methods tested, the random forest algorithm and one of the attention-based models utilizing a customized dot product as a predictor showed the highest performance. Notably, two predicted combinations—acyclovir + ribavirin and acyclovir + Pranobex Inosine—have been experimentally validated to produce a synergistic antiviral effect against the herpes simplex virus type 1, as reported in existing literature.

Introduction

Viruses are among the oldest biological entities on Earth, having existed for billions of years¹. They play essential roles in ecosystems, such as controlling the populations of organisms like bacteria and archaea², returning nutrients to ecosystems by breaking down plankton³, and driving evolutionary processes⁴. However, viruses are often associated with diseases and have caused significant morbidity and mortality in recent decades. Within just two decades, three deadly single-stranded viruses—SARS-CoV, MERS-CoV, and SARS-CoV-2—have caused severe acute respiratory infections, leading to a substantial number of deaths regionally and globally⁵. Monkeypox, a double-stranded virus from the smallpox family, can cause symptoms such as rash, fever, and swollen lymph nodes^6,7. Additionally, HMPV (human metapneumovirus), a negative-sense single-stranded RNA virus from the Pneumoviridae family, causes respiratory tract infections in children, adults, and the elderly⁸.

All of these viruses, among many others, emerge unexpectedly and often cause deadly diseases without any prescribed treatments readily available. Therefore, discovering effective drugs to stop or hinder these viral diseases is of utmost importance. On the other hand, drug discovery is a time-consuming process and an extremely expensive industry⁹. Relying solely on proposing new drugs in emergencies can endanger the lives of thousands or even millions of people. Thus, drug repurposing can be an effective solution in such scenarios. However, this approach also requires significant effort¹⁰.

Using computational methods for screening and proposing approved drugs for repurposing is an efficient and effective strategy that significantly reduces costs and time¹¹. Drug repurposing, especially with the advent of deep learning and transformer-based methods, has had a high impact on drug research and discovery. Most studies focus on monotherapy, where drug target predictions typically suggest single drugs for treating diseases. These targets can include proteins, viruses, and other biomolecules.

However, in recent years, there has been growing interest in predicting combination therapies for curing diseases¹². The goal is to increase treatment effectiveness while simultaneously reducing side effects compared to higher doses of a single drug. Cancer treatment and oncology, in particular, have seen extensive investigations into synergistic combination therapies using machine learning and deep learning methods in recent years^13–16.

In contrast, combination therapies for treating viral diseases remain rare^17,18. This is due to challenges such as drug resistance, the difficulty of treating such viruses, and the lack of knowledge during emergency situations¹⁹. As a result, proposing synergistic antiviral combination therapies suffers from a lack of sufficient research. This work focuses on attention-based methods for predicting drug pairs to treat viral diseases.

The proposed approach for viral combination therapy in this work utilizes an attention mechanism for generating a significant embedding for the prediction of the interaction more effectively and efficiently. We prepared the attention-based models in two ways, i.e., prediction using an MLP and prediction using a customized inner product. The former is called the Virus Combination Therapy using attention and MLP, or VCTatMLP, and the latter is called Virus Combination Therapy using attention and Dot product, or VCTatDot. Figures 2 and 3 show two approaches, respectively.This paper makes four key contributions, detailed as follows.

• Preparing and customizing the dataset for predicting new drug combinations for therapy.

• Proposing two attention-based deep learning models for effective drug prediction.

• Conducting a comprehensive analysis of the performance of the proposed models.

• Predicting several combination therapies for viral disease treatment, particularly focusing on two confirmed combinations for treating HSV-1.

We organized the paper as follows. “Related work” section reviews previous studies in this field. “Basic definitions and methods” section outlines key concepts used in our approach. “Dataset” section describes the dataset employed in this study. “Proposed methods” section presents two approaches for virus combination therapy. “Results” section reports the evaluation metrics and experimental findings. Finally, “Conclusion” section summarizes the work and its contributions.

Fig. 2.

VCTatMLP architecture, the encoder section consists of attention mechanism blocks, and the prediction section consists of an MLP block applied on the concatenation of the embedding vectors.

Fig. 3.

VCTatDot architecture, the encoder section consists of attention mechanism blocks and the prediction section consists of applying a sigmoid function on the tri-dot product of the embedding vectors.

Related work

El-Hafeez et al.²⁰ developed a framework for classifying and recommending synergistic drug combinations. They utilized the O’Neil drug interaction dataset²¹, which contains 22737 drug combinations tested against 39 cancer cell lines. To classify the data into three categories—synergistic, additive, or antagonistic—they applied various classification methods, including naïve Bayes, random forest, k-Nearest Neighbors, and Logistic Regression. For the regression task, they employed linear regression, ridge regression, and random forest regression to predict combination sensitivity scores (CSS). Additionally, they enhanced model performance through data preprocessing by identifying outliers using the Interquartile Range (IQR) method and addressing data imbalance through random undersampling.

Jiang et al.²² developed a graph convolutional network (GCN) for predicting synergistic drug combinations. Their approach utilized multiple biological interaction networks as input data. Specifically, they took advantage of three types of networks: drug-drug synergy (DDS), drug-target interaction (DTI), and protein-protein interaction (PPI) to construct their heterogeneous multi-type graph using graph embedding techniques. They formulated the problem as a link prediction task and trained their model using the O’Neil dataset. Their model was compared against traditional machine learning models, including random forest, support vector machine, elastic net, and deep neural networks, demonstrating superior performance. Furthermore, the GCN model successfully predicted synergistic drug combinations for various cancer cell lines, such as breast, colon, lung, and skin cancer, with some predictions later validated in clinical trials.

Yuan et al.²³ proposed a machine learning framework to model cellular responses to perturbations, including drug combinations. Unlike traditional black-box machine learning models, CellBox combines explicit mathematical models of molecular interactions with machine-learning-based parameter inference. Using ordinary differential equations (ODEs), CellBox learns molecular interaction networks from perturbation data and can predict responses to unseen drug combinations. The model was trained using reverse-phase protein array (RPPA) data, which includes measurements of protein and phosphoprotein levels taken before and after perturbation. In addition, the model was tested on a dataset of melanoma cell lines treated with various drugs. The study demonstrated that CellBox could accurately predict dynamic cellular responses to novel drug combinations, achieving an average Pearson correlation of 0.93 in cross-validation experiments. Furthermore, the inferred interaction networks showed significant overlap with known biological pathways, indicating that CellBox not only predicts drug synergy but also provides valuable mechanistic insights.

Güvenç et al.²⁴ provided a comprehensive review of machine learning approaches for drug combination therapies, organizing them into three main categories: drug combination sensitivity prediction, drug synergy prediction, and drug synergy classification. They highlighted that integrating diverse data sources enhances drug synergy prediction. However, challenges such as data standardization and missing values persist in this field of research.

The drug combination sensitivity prediction category focuses on determining how specific drug combinations impact cell viability or growth. Various machine learning (ML) models have been applied, including deep learning (DNNs), matrix factorization methods, and regression models. For instance, the BestComboScore²⁵ model utilizes DNNs to integrate multiple types of molecular and drug features. In contrast, the ComboFM²⁶ model employs factorization machines to learn higher-order drug interactions.

The drug synergy prediction category quantifies the interaction strength between pairs of drugs. Güvenç et al. emphasized that the most effective models combine multi-source data, such as gene expression profiles, chemical structures, and protein-protein interactions. They discussed the DrugComboRanker²⁷ model, which uses Bayesian non-negative matrix factorization to cluster functionally similar drug pairs and predict their likelihood of synergy. Other studies have employed matrix completion techniques, like DECREASE²⁸, which reconstructs dose-response matrices to estimate synergy scores.

The last category is drug synergy classification, which involves classifying drug combinations as synergistic, additive, or antagonistic interactions. Classification models, including random forest, SVMs, and deep learning architectures, have been widely adopted. SyDRa²⁹, a random forest-based approach, integrates drug–chemical structure, drug–target network and pharmacogenomics data to identify effective drug combinations.

Several recent works have also focused on creating datasets and knowledge bases to support AI models for combination therapy for antibiotic synergy prediction. For example, the study by Lv et al.³⁰ introduced ACDB, a manually curated Antibiotic Combination Database, comprising 6175 combinations from over 6000 studies. It includes various molecular features and chemogenomic data, facilitating machine learning-based antibiotic synergy prediction. In a related effort, the authors developed AADB, which collects 1238 antibiotic-adjuvant pairs, annotated with detailed metadata and labels for synergistic or antagonistic behavior³¹. These resources represent valuable efforts toward organizing large-scale, diverse synergy data. Moreover, another contribution from these researchers proposed a graph-learning framework that integrates synergy labels and drug feature embeddings to predict novel antibiotic combinations, highlighting the utility of graph-based representations in synergy discovery³². In further work, they reviewed computational models, databases, and tools for antibiotic combinations, emphasizing the role of high-throughput screening and multi-omics data in advancing synergy prediction³³. The authors categorized computational approaches into drug-response prediction, mechanistic models, machine learning, and network-based models, noting that integrating multi-source data (e.g., chemical structures, targets, phenotypes) improves accuracy. This review also highlighted challenges such as experimental noise in synergy scores and the lack of standardized evaluation metrics. In another study, a similarity matrix fusion (SMF) method was proposed to integrate structural, pharmacological, phenotypic, and therapeutic drug features for clustering drug-drug interaction (DDI) networks. This approach confirmed that fused similarity matrices enhance the interpretability of drug groups and group-group interactions, enabling the prediction of new drug combinations³⁴. Expanding on this, a novel drug similarity measure was introduced based solely on DDI network topology, eliminating reliance on external drug features³⁵. By projecting signed networks into consistent and inconsistent neighbor subsets, this method achieved superior clustering performance and semi-supervised prediction of unknown DDIs. This work underscored the potential of network-derived similarity for functional annotation of compounds with unknown mechanisms of action. Most recently, Lv et al.³⁶ integrated multi-species DDI networks to identify robust antibiotic subgroups. While the above proposals and studies focused on antibiotic synergy prediction, our proposed models concentrate on virus synergistic drug combinations.

Virus combination therapy receives limited attention due to the narrow-spectrum effectiveness of antiviral treatments and the persistent challenge of drug resistance¹⁹. Computational and AI methods have been explored for COVID-19 combination therapy suggestions^17,37. Myhre³⁸ compiled a list of virus therapy drug combinations reported in various experiments (e.g., in vitro, in vivo, and clinical trials). Using Myhre’s list, previous research established an initial dataset for virus synergistic combination therapy³⁹ and employed machine and deep learning models to predict novel combination therapy candidates. New transformer and attention-based methods have been largely overlooked in predicting virus combination therapy. This work proposes attention-based methods to predict novel synergistic drug combinations for virus treatment.

Basic definitions and methods

This paper proposes two attention-based methods for predicting synergistic effects of pair drugs for viral disease treatments and compares them with a machine learning algorithm consisting of a support vector machine, and random forest and a convolutional deep model (DRaW). We use several concepts for machine and deep learning, therefore, it is necessary to introduce the fundamental concepts. This section introduces and reviews the important concepts for the proceeding of the paper.

Support vector machine

Support Vector Machine (SVM)⁴⁰ is a supervised learning algorithm that finds an optimal hyperplane to separate data into two classes by maximizing the margin between them. For cases where data is not linearly separable, kernel functions such as linear, polynomial, and radial basis function (RBF) are used to map data into a higher-dimensional space where separation becomes possible. The regularization parameter C controls the trade-off between maximizing the margin and minimizing classification error: a larger C aims to reduce misclassification, while a smaller C allows for a wider margin with more tolerance for errors⁴¹.

Random forest

Random Forest⁴² is an ensemble method that builds multiple decision trees using randomly selected feature subsets and aggregates their predictions via majority voting. We used Gini and Log Loss⁴³ as criteria to evaluate feature importance. Feature selection strategies such as and were employed to ensure diversity. Each input vector consists of features from two drugs and one virus: + + .

Multilayer perceptron

MLPs are feedforward neural networks inspired by biological neurons^44,45. They consist of multiple layers of perceptrons, where each applies an activation function and learns weights through backpropagation by minimizing prediction errors⁴⁶. MLPs are suitable for capturing nonlinear relationships in the input data.

Convolutional neural network

Convolutional Neural Networks (CNNs) are deep learning architectures initially developed for image analysis, but they have since been adapted for various other domains⁴⁷. These networks utilize convolutional layers to extract features, pooling layers to reduce dimensionality, and fully connected layers for decision-making⁴⁸. In our work, we adopt a customized CNN architecture proposed for drug repurposing^49,50, which has been specifically adapted for predicting antiviral combinations³⁹. Therefore, it is a proper candidate for comparing the performance of the proposed methods.

Attention mechanism

The attention mechanism is a computational process that identifies and focuses on the important parts of the input data for higher-performance learning. Its main idea is based on assigning different weights to different parts of the input vector regarding the importance of each part automatically⁵¹. This leads to recognizing the dependencies of the far parts of the data. The attention mechanism is based on three main concepts query, key, and value, which we describe shortly. The query is a vector that the model looks for to find some information based on that. The key is a vector representing the information available in each element of the input. This vector is compared with the query vector. In other words, the similarity among keys and queries is calculated according to the following inner product of key and query.

1

where and are vectors and Eq. (1) represents the similarity score computed based on inner product.

In the next step, the computed scores of all keys are converted to the interval [0, 1] using softmax. These outputs are known as attention weights, shown in the following equation.

2

where represents the similarity score between query vector and the key vector . The final step of the attention mechanism involves summing the weighted value vectors as follows.

3

where each input value vector is multiplied by its corresponding attention weight and summed to obtain the overall weighted values. The whole procedure is compacted in the following equation.

4

These attention weights are learned during the training phase. This paradigm, due to giving higher weights and attention to the more important parts of the inputs for outcome prediction has improved the performance of deep learning enormously.

Self- vs. cross-attention

It is worth noting that the attention mechanism has two categories of self-attention and cross-attention⁵². The former considers all three vectors of query, key, and value the same; that is, each input vector is fed to the model as key, query, and value. In other words, this type of mechanism concentrates on the impact or the importance of the different parts of the same vector. The latter is another attention mechanism in which the query, key, and value vectors are not the same. For example, it accepts a sentence as a key and a value, and its corresponding translation to another language as the query. In this work, we consider one drug as the query and another drug as the key and value, and vice versa.

Tri-dot product

The inner product or dot product on a vector space is a function as,

5

Which has the following properties⁵³.

1. . iff .

2. :

   • .
   • .

3. : .

4. : .

In this paper, the vectors receive values from a subspace of . Therefore, the inner product of two vectors and is,

6

where, and are the k-th values of and , respectively. The inner product has many applications in different areas of research and studies. For example, they are used for similarity comparison in quantum mechanics and computing⁵⁴. Furthermore, it can be used in decoding or prediction of values in machine and deep learning approaches⁵⁵. This paper utilizes an inner product for the prediction of synergistic combination therapy for viruses. However, it uses a special type of that which is the generalization of three vectors instead of two vectors. The inner product of three vectors is called generalized dot product or tri-dot product⁵⁶, which is defined as follows.

7

where, , , and are the k-th values of , , and , respectively. Tri-dot product has the last three properties of the dot product, but it does not follow the first properties and can accept negative values for some of the vectors.

As mentioned above, due to generating a significant embedding vector by attention approaches, the decoding (prediction) step can be a tri-product. The results show its high performance, which we mention in “Results” section.

Dataset

This section provides information on the dataset utilized in this paper. Any deep learning model needs a suitable dataset to learn from and infer new predictions⁵⁷. The starting point of this work is a dataset that contains information on the combination of viruses and drugs used for combination therapy. Myhre collected a list of combination therapies used for emerging infectious diseases from the literature. He showed that developing new tools for combating ineffective viruses plays an important role in disease treatment. He correctly mentioned that the use of more than one drug can lead to higher performance in the treatment of diseases rather than a single drug, so it is necessary to introduce combination therapy as an effective approach to handle viral diseases. In his list, the synergism effect of pair drugs on viruses has been collected from approved clinical trials, in vitro, and in vivo. However, the list does not follow a proper standard for drugs’ names and viruses. Therefore, we updated the list with the standard names of drugs and viruses regarding DrugBank and NCBI. Then, we generated a dataset from the information in the revised list of viral combination therapies. The drug SMILES⁵⁸ was collected from the DrugBank⁵⁹, and the nucleotide sequences of viruses were collected from NCBI⁶⁰. Using Tanimoto score and sequence alignment, we computed the similarity among drugs and viruses, respectively. Then, for each drug, we produced a feature vector using its similarity values with other drugs. The same happens with viruses. Therefore, the data set contains the drug characteristic vectors , in which the i-th row () represents the characteristic vector of the i-th drug. Furthermore, a virus feature vector has been generated using similar values between viruses called , where the j-th row shows the virus feature vector j-th virus. This dataset is called CombTVir³⁹. The drug-virus interactions are considered as labels in this dataset. The dataset contains 44 viruses, 211 drugs, and 372 approved drug-virus interactions. Therefore, it has a high percentage of sparsity.

Furthermore, we present the frequency of each virus in the dataset, as illustrated in Fig. 1. This visualization highlights that while some viruses, such as HSV-1, are quite common, others, like EBV and VZV, occur only once or infrequently. Importantly, our framework exhibits predictive capabilities, even for these underrepresented viruses. As shown in the “Results” section, Table 8 reveals that the model successfully predicts four combinations involving VZV (which appears only four times in the dataset) and one combination involving EBV (which appears just once). This demonstrates that the model does not merely memorize the more frequent viruses but is also capable of generalizing to rarer cases.

Fig. 1.

The frequency of each virus present in positive samples.

Table 8.

Predicted drug combinations that are either individually approved or shown to have synergistic effects in the literature.

Antiviral 1	Antiviral 2	Virus
Acyclovir65	Ribavirin66	HSV-163
Acyclovir65	Inosine (Inosine Pranobex)67	HSV-164
Acyclovir65	Cidofovir68	HSV-1
Acyclovir65	Zidovudine69	HSV-1
Acyclovir65	Didanosine69	HSV-1
Acyclovir65	Genistein70	HSV-1
Acyclovir65	Idoxuridine71	HSV-1
Acyclovir65	Resveratrol72	HSV-1
Acyclovir65	Stavudine69	HSV-1
Acyclovir65	Adefovir73	HSV-1
Acyclovir65	Trifluridine74	HSV-1
Acyclovir65	Trifluridine74	HSV-2
Acyclovir75	Cidofovir68	HSV-2
Acyclovir65	Inosine67	HSV-2
Acyclovir76	Foscarnet77	VZV
Acyclovir76	Brincidofovir78	VZV
Acyclovir76	Brivudine79	VZV
Acyclovir76	Brincidofovir78	VZV
Alisporivir80	Ribavirin81	HCV
Alisporivir80	Mycophenolic Acid82	HCV
Alisporivir80	Taribavirin83	HCV
Acyclovir76	Maribavir84	CMV
Acyclovir76	Brincidofovir85	CMV
Acyclovir86	Foscarnet77	EBV
Amantadine87	Caffeine88	FLUAV

Proposed methods

The proposed methods in this paper, VCTatMLP and VCTatDot, are based on the attention mechanism, illustrated in Figs. 2 and 3. Both methods consist of two steps, that is, encoding and decoding. The former generates a significant embedding vector of each input entity. Encoding is an attention-based approach to generating embeddings, and its general procedure has been defined in “Attention mechanism” section. The encoding is identical for both proposals. The decoding predicts the synergistic effect of drugs on a given virus. We propose two methods for the prediction of combination therapy for viruses. The two proposed methods differ in the decoding step.

VCTatMLP is the first model, which consists of two sections: encoder (embedding generation for viruses and drugs) and decoder (prediction of the drug-drug-virus interaction), shown in Fig. 2. It aims to predict the synergism effect of combination drug i and drug j on virus k, where and and . It utlizes three attention blocks for the encoding section. For drug embeddings, the first two attention blocks are utilized for their embedding generation. The first attention block uses drug i as the key and value, drug j as the query, and its output is the embedding for drug i. The next attention block for the drugs is drug j as the key and value and drug i as the query. The two mentioned attention blocks belong to the cross-attention category. Therefore, the cross-relation among two drugs leads to a higher significant embedding for each drug. Lastly, for viruses, it uses the third attention block, a self-attention block, in which its q, k, and v vectors are set to virus input vectors. Therefore, three embeddings are generated for drug i, drug j, and virus k, called , , and , respectively. Notably, the encoding section for both proposed methods is identical; therefore, VCTatDot follows the same line of the embeddings for drugs and viruses.

The rationale behind the encoder is that drug synergy depends not only on the individual effects of each drug but also on how their features interact. To capture this, cross-attention is used between drugs, allowing one drug’s representation to be shaped by the other. This mimics how complementary or competing mechanisms of action may influence therapeutic effects. For viruses, self-attention emphasizes the internal relations among their features, ensuring subtle differences between viral strains are reflected in their embeddings.

The significance of the encoder section is that we can use a lighter version of methods for decoding or prediction. VCTatMLP utilizes an MLP for the goal of prediction. Figure 4 shows the MLP architecture used for the prediction. It contains one batch normalization layer as the first layer, five fully connected layers, each followed by a batch normalization layer, and a final fully connected layer for outputting the prediction value. The activation function of the inner FC layers is ReLU, and the last activation layer is sigmoid.

Fig. 4.

Architecture of MLP section of VCTatMLP.

VCTatDot utilizes a customized version of the inner product for synergism prediction, tri-dot product, introduced in “Tri-dot product” section-Eq. (7). By having embeddings , , and , the following equation computes the synergism prediction.

8

Here, F is the embedding dimension, and , , and represent the f-th elements of embeddings , , and , respectively. As shown in Eq. (8), VCTatDot calculates the element-wise product of these three embeddings and sums the results. Briefly, both methods employ the proposed attention mechanism to generate significant representations of two drugs and the virus. VCTatMLP and VCTatDot utilize MLP and tri-dot product, respectively, for downstream tasks, both using binary cross-entropy (9) as the loss function.

9

.

The MLP decoder is designed to capture highly nonlinear interactions between drugs and viruses, offering flexibility in modeling complex relationships. In contrast, the alternative decoding method (tri-dot product) focuses on efficiency.

In summary, the overall procedure of our proposed framework can be described in the following steps:

• Preparation of feature vectors for drugs and viruses as model inputs.

• Encoding through three attention blocks: two cross-attention blocks for drug pairs and one self-attention block for viruses, generating embeddings , , and , respectively.

• Decoding phase for synergism prediction, where VCTatMLP employs a multi-layer perceptron (MLP) and VCTatDot uses a tri-dot product formulation.

• Optimization of model parameters using the binary cross-entropy loss function.

Results

The proposed methods were implemented in Python using scikit-learn for machine learning and PyTorch for deep learning models. Experiments were conducted on a system running Ubuntu 22.04 LTS with an Intel Xeon E5 v4 (4 cores), 16 GB RAM, and an RTX 3060 12G GPU. Table 1 details the specific libraries and versions used.

Table 1.

Library/Framework used for developing the proposed methods.

Library/Framework	Version
numpy	1.24.3
matplotlib	3.4.2
pandas	1.5.2
rdkit	2024.3.3
sklearn	1.2.1
torch	2.0.1 + cu118

We evaluate algorithm performance using MCC, AUC-RoC, and AUPR, calculated from the true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) of the confusion matrix, using the following equations.

10

11

12

13

14

Due to the highly imbalanced nature of our dataset, accuracy is not a reliable metric for evaluating model performance. Accuracy may present an overly optimistic view by being biased toward the majority (negative) class. Therefore, we focus on more informative metrics such as MCC, AUC-ROC, and AUPR, which are better suited for imbalanced classification tasks.

For imbalanced datasets, the Matthew Correlation Coefficient (MCC) is a robust metric that considers all values in the confusion matrix. MCC, unlike the F1-score⁶¹, achieves high values only when all four confusion matrix parameters are high. We also prioritize the Area Under the Precision-Recall Curve (AUPR) due to the dataset imbalance. AUPR, which analyzes the precision-recall trade-off across different thresholds, provides a more informative evaluation than AUC-ROC⁶².

As described in “Basic definitions and methods” section, we outlined the various configurations used for Support Vector Machine (SVM) and Random Forest (RF). The best-performing SVM utilized a polynomial kernel with a C value of 10, while RF employed log loss as the criterion and set the maximum number of features to . The learning rate was .

Performance was evaluated using four positive to negative (P2N) sampling ratios: 1:3, 1:5, 1:10, and 1:100. For example, a P2N ratio of 1:10, indicates that ten randomly selected negative samples were included for each positive sample.We implemented 10-fold stratified cross-validation, ensuring an equal proportion of positive and negative samples in the training, validation, and testing sets. Table 2 displays the frequency of each sample type in the different scenarios mentioned.

Table 2.

Distribution of positive and negative samples across training, validation, and test sets under different positive to negative (P2N) ratios.

P2N ratio	Set	Positives	Negatives	Total
1:3	Training	298	894	1192
	Validation	37	112	149
	Test	37	112	149
1:5	Training	298	1,490	1788
	Validation	37	186	223
	Test	37	186	223
1:10	Training	298	2,970	3268
	Validation	37	372	409
	Test	37	372	409
1:100	Training	298	29,700	29,998
	Validation	37	3,720	3757
	Test	37	3,720	3757

Each setup uses 80% for training, 10% for validation, and 10% for testing.

Tables 3 through 6 present the results for the 1:3, 1:5, 1:10, and 1:100 P2N scenarios, respectively, reporting MCC, AUC-ROC, and AUPR. Specifically, Table 3 shows results for the 1:3 P2N scenario. DRaW¹⁸ (a CNN model) exhibited the lowest performance in this scenario. While other methods achieved near-perfect AUC-ROC and AUPR, Random Forest yielded the best MCC. VCTatMLP and VCTatDot also demonstrated strong performance.In addition, RF achieved the best balance of Recall (0.97) and Specificity (0.99), while SVM, VCTatDot, and VCTatMLP also maintained high values. DRaW showed lower recall, reflecting weaker sensitivity.

Table 3.

Performance comparison for positive to negative sample ratio 1:3.

Method	Specificity	Recall	MCC	AUC-RoC	AUPR
SVM	0.99 ± 0.0	0.96 ± 0.03	0.96 ± 0.0	0.99 ± 0.0	0.99 ± 0.0
RF	0.99 ± 0.0	0.97 ± 0.03	0.98 ± 0.02	0.99 ± 0.0	0.99 ± 0.0
DRaW18	0.98 ± 0.02	0.81 ± 0.18	0.83 ± 0.16	0.97 ± 0.02	0.95 ± 0.03
VCTatDot	0.98 ± 0.01	0.96 ± 0.03	0.95 ± 0.02	0.99 ± 0.0	0.99 ± 0.0
VCTatMLP	0.99 ± 0.01	0.96 ± 0.04	0.96 ± 0.02	0.99 ± 0.0	0.99 ± 0.0

Highest values are in [bold].

Table 6.

Performance comparison for positive to negative sample ratio 1:100.

Method	Specificity	Recall	MCC	AUC-RoC	AUPR
SVM	0.99 ± 0.0	0.87 ± 0.05	0.85 ± 0.03	0.99 ± 0.0	0.92 ± 0.03
RF	0.99 ± 0.0	0.86 ± 0.04	0.92 ± 0.02	0.96 ± 0.02	0.92 ± 0.03
DRaW18	0.99 ± 0.0	0.78 ± 0.05	0.87 ± 0.02	0.98 ± 0.01	0.86 ± 0.0
VCTatDot	0.99 ± 0.0	0.84 ± 0.07	0.88 ± 0.04	0.99 ± 0.01	0.92 ± 0.04
VCTatMLP	0.99 ± 0.0	0.83 ± 0.02	0.85 ± 0.12	0.93 ± 0.03	0.86 ± 0.06

Highest values are in [bold].

Table 4 presents the results for the P2N 1:5. A similar pattern is observed for performance; DRaW shows the lowest performance, while the attention-based methods demonstrate high performance. However, the random forest surpasses all methods in terms of MCC.Recall and Specificity analysis confirms that the attention-based methods maintain a strong balance, while DRaW suffers from weaker sensitivity despite high specificity.

Table 4.

Performance comparison for positive to negative sample ratio 1:5.

Method	Specificity	Recall	MCC	AUC-RoC	AUPR
SVM	0.99 ± 0.0	0.96 ± 0.03	0.96 ± 0.03	0.99 ± 0.0	0.99 ± 0.0
RF	0.99 ± 0.0	0.95 ± 0.04	0.97 ± 0.02	0.99 ± 0.0	0.99 ± 0.0
DRaW18	0.99 ± 0.0	0.79 ± 0.21	0.84 ± 0.21	0.96 ± 0.06	0.94 ± 0.06
VCTatDot	0.99 ± 0.0	0.97 ± 0.01	0.96 ± 0.02	0.99 ± 0.0	0.99 ± 0.0
VCTatMLP	0.99 ± 0.0	0.97 ± 0.02	0.96 ± 0.02	0.99 ± 0.0	0.99 ± 0.0

Highest values are in [bold].

Table 5 still indicates that DRaW has the poorest performance.Its recall remains low despite keeping high specificity. In contrast, attention-based methods (VCTatDot and VCTatMLP) maintain a stronger balance between recall and specificity, ensuring reliable detection while minimizing extreme false positives. RF and SVM also retain high levels on both measures, confirming their robustness. Random forest surpasses other methods in terms of MCC and AUPR. VCTatDot and VCTatMLP demonstrate high performance as well.

Table 5.

Performance comparison for positive to negative sample ratio 1:10.

Method	Specificity	Recall	MCC	AUC-RoC	AUPR
SVM	0.99 ± 0.0	0.96 ± 0.03	0.96 ± 0.02	0.99 ± 0.0	0.98 ± 0.01
RF	0.99 ± 0.0	0.95 ± 0.03	0.97 ± 0.02	0.99 ± 0.0	0.99 ± 0.0
DRaW18	0.99 ± 0.0	0.80 ± 0.24	0.84 ± 0.25	0.97 ± 0.02	0.93 ± 0.04
VCTatDot	0.99 ± 0.0	0.90 ± 0.1	0.91 ± 0.02	0.99 ± 0.0	0.98 ± 0.0
VCTatMLP	0.99 ± 0.0	0.96 ± 0.02	0.95 ± 0.03	0.99 ± 0.0	0.97 ± 0.01

Highest values are in [bold].

Table 6 shows that with the most challenging 1:100 positive-to-negative sampling ratio, VCTatDot outperforms other methods regarding three metrics: AUC-ROC, AUPR, and MCC.Although specificity remains consistently high across all models, recall becomes a clear differentiator. SVM achieves the highest recall, while the random forest also maintains a strong recall. VCTatDot balances high specificity with competitive recall, making it overall the most consistent performer under extreme imbalance. For the first time, VCTatDot outperforms random forest regarding AUC-ROC. SVM has a similar performance to VCTatDot. Furthermore, VCTatDot outperforms VCTatMLP. Random forest performs better compared to other approaches regarding MCC.

To further assess the generalization ability of our approach, we introduce a virus-split scenario in which 10% of the viruses are randomly excluded from training and reserved for testing. This setup reflects the real-world challenge of predicting drug combinations for previously unseen viruses. The results presented in Table 7 demonstrate that both VCTatDot and VCTatMLP maintain strong predictive power across various positive to negative ratios. Even in more imbalanced scenarios, the models achieved performance levels comparable to those of the standard K-fold setting. These findings underscore the robustness of our approach and confirm its effectiveness in generalizing to unseen viruses while managing class imbalance.

Table 7.

Performance of VCTatDot and VCTatMLP under the virus-split scenario, where 10% of viruses are excluded from training and used only for testing.

Ratio	Method	Specificity	Recall	MCC	AUC-RoC	AUPR
1:100	VCTatDot	0.99 ± 0.0	0.78 ± 0.02	0.88 ± 0.01	0.97 ± 0.01	0.90 ± 0.02
	VCTatMLP	0.99 ± 0.0	0.80 ± 0.05	0.88 ± 0.0	0.98 ± 0.0	0.92 ± 0.02
1:10	VCTatDot	0.99 ± 0.0	0.87 ± 0.01	0.92 ± 0.01	0.99 ± 0.0	0.98 ± 0.0
	VCTatMLP	0.99 ± 0.0	0.87 ± 0.05	0.92 ± 0.02	0.99 ± 0.0	0.98 ± 0.0

The first column indicates the positive to negative (P2N) sample ratios, while the second column is labeled as the name of the model.

Figure 5 illustrates the loss convergence of VCTatMLP and VCTatDot, confirming the superior performance of VCTatDOT. The loss of VCTatMLP loss converges to 0.70, while VCTatDot’s loss converges to below 0.01, signifying an enhanced prediction of original labels due to the lower loss value. Furthermore, this improved convergence is observed in VCTatDot, which is a lighter and faster model than VCTatMLP.

Fig. 5.

Convergence of loss functions for a positive to negative sampling ratio of 1:100.

Table 8 lists predicted synergistic drug pairs for viral diseases, focusing on combinations where each drug is independently prescribed for the corresponding virus and. For example, Acyclovir and Cidofovir are both prescribed for HSV-1. Critically, this work identified two synergistic combinations for HSV-1: acyclovir and ribavirin⁶³ and acyclovir and inosine⁶⁴. Our methods correctly predicted both of these combinations. In addition, the complete list of predicted combination therapy for viruses is reported in Table 9.

Table 9.

All predicted drug–drug–virus combinations with frequencies

Antiviral1	Antiviral2	Virus	Frequency
acyclovir	brivudine	CMV	22
acyclovir	brincidofovir	CMV	17
acyclovir	cidofovir	HSV-2	17
acyclovir	cidofovir	HSV-1	11
acyclovir	zidovudine	HSV-1	7
acyclovir	foscarnet	VZV	6
acyclovir	telbivudine	CMV	6
acyclovir	telbivudine	HSV-2	6
acyclovir	zidovudine	HSV-2	6
acyclovir	didanosine	CMV	5
acyclovir	idoxuridine	CMV	5
acyclovir	telbivudine	HSV-1	5
adefovir	zidovudine	HBV	5
acyclovir	adefovir	CMV	4
acyclovir	baloxavir	CMV	4
acyclovir	brivudine	HIV-1	4
acyclovir	quercetin	CMV	4
acyclovir	trifluridine	HSV-1	4
adefovir	brivudine	HBV	4
acyclovir	azacitidine	HSV-1	3
acyclovir	azacitidine	HSV-2	3
acyclovir	brincidofovir	VZV	3
acyclovir	brivudine	VZV	3
acyclovir	cenicriviroc	CMV	3
acyclovir	gemcitabine	CMV	3
acyclovir	idoxuridine	HSV-1	3
acyclovir	idoxuridine	HSV-2	3
acyclovir	maribavir	CMV	3
alisporivir	ribovirin	MERS-CoV	3
amantadine	ribovirin	FLUAV	3
acyclovir	adefovir	HSV-1	2
acyclovir	adefovir	HSV-2	2
acyclovir	baloxavir	HBV	2
acyclovir	baloxavir	SARS-CoV	2
acyclovir	docosanol	EBV	2
acyclovir	doxycycline	CMV	2
acyclovir	letermovir	HSV-1	2
acyclovir	ribavirin	HSV-2	2
acyclovir	stavudine	CMV	2
alisporivir	ribovirin	HCV	2
alisporivir	zidovudine	HBV	2
artemether	brincidofovir	CMV	2
2’-fluoro-5-methylarauracil (fmau)	adefovir	HBV	1
5-trifluorothymidine(tft)	adefovir	HBV	1
acyclovir	atazanavir	CMV	1
acyclovir	azacitidine	CMV	1
acyclovir	azithromycin	CMV	1
acyclovir	baloxavir	B19V	1
acyclovir	baloxavir	CVB3	1
acyclovir	baloxavir	DENV	1
acyclovir	baloxavir	EBOV	1
acyclovir	baloxavir	EBV	1
acyclovir	baloxavir	EV71	1
acyclovir	baloxavir	FLUAV	1
acyclovir	baloxavir	FLUBV	1
acyclovir	baloxavir	HEV	1
acyclovir	baloxavir	HIV-1	1
acyclovir	baloxavir	HMPV	1
acyclovir	baloxavir	HPV	1
acyclovir	baloxavir	HSV-1	1
acyclovir	baloxavir	HSV-2	1
acyclovir	baloxavir	JCV	1
acyclovir	baloxavir	JUNV	1
acyclovir	baloxavir	KSHV	1
acyclovir	baloxavir	LASV	1
acyclovir	baloxavir	MERS-CoV	1
acyclovir	baloxavir	RVFV	1
acyclovir	baloxavir	SFTSV	1
acyclovir	baloxavir	VACV	1
acyclovir	baloxavir	VZV	1
acyclovir	baloxavir	YFV	1
acyclovir	baloxavir	hPIV-3	1
acyclovir	bepridil	CMV	1
acyclovir	boceprevir	HSV-1	1
acyclovir	brivudine	EBV	1
acyclovir	cenicriviroc	EBV	1
acyclovir	cenicriviroc	HIV-1	1
acyclovir	clofarabine	HSV-2	1
acyclovir	daclatasvir	CMV	1
acyclovir	didanosine	HSV-1	1
acyclovir	didanosine	HSV-2	1
acyclovir	didanosine	VZV	1
acyclovir	digoxin	CMV	1
acyclovir	diltiazem	CMV	1
acyclovir	diltiazem	HSV-2	1
acyclovir	docosanol	HIV-1	1
acyclovir	docosanol	ZIKV	1
acyclovir	eucalyptol	CMV	1
acyclovir	foscarnet	EBOV	1
acyclovir	foscarnet	EBV	1
acyclovir	genistein	CMV	1
acyclovir	genistein	HSV-1	1
acyclovir	genistein	HSV-2	1
acyclovir	glycyrrhizin	HSV-2	1
acyclovir	hydroxyurea	HSV-2	1
acyclovir	inosine (inosine pranobex)	CMV	1
acyclovir	inosine (inosine pranobex)	HSV-1	1
acyclovir	inosine (inosine pranobex)	HSV-2	1
acyclovir	maribavir	HSV-1	1
acyclovir	oseltamivir	CMV	1
acyclovir	remdesivir	CMV	1
acyclovir	resveratrol	HSV-1	1
acyclovir	ribavirin	CMV	1
acyclovir	ribovirin	HSV-1	1
acyclovir	ribovirin	ZIKV	1
acyclovir	sirolimus	HSV-1	1
acyclovir	stavudine	HSV-1	1
acyclovir	tenofovir	HSV-1	1
acyclovir	tipranavir	CMV	1
acyclovir	trifluridine	HSV-2	1
acyclovir	zidovudine	HBV	1
adefovir	disoproxil fumarate	HBV	1
adefovir	edoxudine	HBV	1
adefovir	famciclovir	HBV	1
adefovir	hbpg	HBV	1
adefovir	mmudr	HBV	1
adefovir	stavudine	HBV	1
adefovir	vesatolimod	HBV	1
adefovir	vidarabine	HBV	1
alisporivir	mycophenolic acid	CMV	1
alisporivir	mycophenolic acid	HCV	1
alisporivir	tacrolimus	NoV	1
alisporivir	taribavirin	HCV	1
amantadine	caffeine	FLUAV	1
artemether	artesunate	CMV	1
artemether	brincidofovir	FLUAV	1
artemether	brincidofovir	HBV	1
artemether	brincidofovir	HCV	1
artemether	brincidofovir	HSV-1	1
artemether	brivudine	CMV	1
artemether	brivudine	HBV	1
artemether	brivudine	HCV	1
artemether	brivudine	HSV-1	1
artemether	brivudine	HSV-2	1

Conclusion

Synergistic combination therapy is valuable in drug repurposing for enhanced efficacy and reduced side effects. While computational drug discovery is rich with research in this area, studies on combination therapy for viral diseases remain scarce. This work introduces two attention-based approaches for synergistic antiviral combinations: one using an MLP, and the other a customized, tri-dot product. The tri-dot product, leveraging significant embeddings from attention-based generation, outperforms the MLP method, suggesting that substantial embeddings paired with a lighter prediction model are more effective. Also, our methods predict two drug combinations for the HSV-1 virus: acyclovir combined with ribavirin and acyclovir with inosine, both of which are approved by the literature. Furthermore, the proposed approach maintains and even improves its performance with increasing dataset size compQ1ared to other methods. These machine learning approaches also achieve high performance with reduced computational resources. Future work will focus on enhancing these attention-based approaches on larger datasets to further advance combination therapy for viral diseases. Additionally,the integration of explainable artificial intelligence (XAI) methods represents an important direction for advancing the interpretability of predictive models in synergistic antiviral drug discovery. In future work, we plan to incorporate drug- and virus-specific features directly into our framework and apply XAI techniques to clarify the contribution of these features to model predictions.

Author contributions

M.H. and S.M. conceptualized the study. M.H. provided supervision, managed the project, and led the writing of the original draft and subsequent revisions. S.M. contributed to method development, data curation, validation, and visualization, as well as the review and refinement of the manuscript.

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Data availibility

The datasets and code generated in this study are available in the GitHub repository: https://github.com/BioinformaticsIASBS/VirCombAtt.

Declarations

Competing interests

The authors declare that they have no competing interests.