AMIUgraph: analysis and modeling of interactions for utility-driven benchmarking of graph-based models in healthcare

Abstract

Background

Graph-based machine learning approaches, including Knowledge Graph Embedding (KGE) methods and Graph Neural Networks (GNNs), have emerged as powerful tools for modeling complex biomedical data. However, a systematic and clinically grounded comparison of these approaches across heterogeneous healthcare graphs, accounting for both predictive performance and real-world deployment constraints, is still lacking.

Methods

We introduce AMIUGraph, a comprehensive benchmarking framework for healthcare link prediction that integrates real-world clinical data with external biomedical knowledge bases. AMIUGraph evaluates eight state-of-the-art models, of which four are knowledge graph embedding (KGE) methods DistMult, CP, ComplEx, and ConvE and four are graph neural network (GNN) architectures GCN, GraphSAGE, GAT, and GIN. The models are evaluated across three heterogeneous bipartite graphs representing Patients–Diseases, Diseases–Drugs, and Drugs–Targets interactions. Models are assessed under both transductive and inductive learning settings using accuracy, AUC, precision, recall, F1-score, and training time as evaluation metrics.

Results

Experimental results show that model performance is strongly influenced by graph structure and sparsity. GNNs consistently achieve superior predictive performance on sparse interaction graphs, particularly for Diseases–Drugs and Drugs–Targets prediction tasks. In contrast, KGE models demonstrate competitive accuracy with substantially lower computational costs in inductive clinical scenarios involving unseen patients. These trends are especially relevant in clinically realistic settings characterized by multimorbidity, such as gastrointestinal and liver diseases, where frequent patient updates and complex therapeutic interactions are common.

Conclusion

AMIUGraph provides a clinically grounded and utility-driven benchmarking framework that jointly evaluates KGE and GNN models across multiple healthcare graph types and learning settings. The findings offer practical guidance for selecting graph-based models in medical decision-support systems, including applications in gastrointestinal healthcare, while promoting transparency and reproducibility through the public release of all datasets, protocols, and code.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12911-026-03717-5.

Introduction

With the advancement in technology, the volume of data in the biomedical field keeps growing at an exponential rate. When represented as a knowledge graph, such data can effectively model the complex structure of entities and different relations within biological systems. By leveraging powerful representation learning models in Deep Learning (DL), such as Knowledge Graph Embedding (KGE) models [1–4] and graph neural networks (GNNs) [5–8], it is possible to uncover hidden interaction patterns underlying such data, enabling a more granular, patient-centered analysis of interactions. In this respect, the literature generally recognizes three types of interactions: Drugs—Targets, Drugs—Diseases, and Patients—Diseases.

Such a representation is particularly well suited for complex chronic conditions like liver diseases, where patients often present with multiple metabolic comorbidities, long and evolving disease trajectories, and highly heterogeneous therapeutic strategies. In this context, the coexistence of hepatic and extra-hepatic conditions, the progressive nature of liver damage, and the frequent need for gastrointestinal polypharmacological treatments generate a dense web of interdependencies that cannot be adequately captured by isolated or pairwise analyses. Graph-based representations allow these multifactorial relationships to be modeled explicitly, providing a more realistic depiction of clinical complexity and enabling the investigation of multimorbidity patterns, treatment interactions, and disease progression within an integrated analytical framework.

An essential component of genomic drug discovery is the research of drug and protein (target) interactions [9, 10]. The current state-of-the-art involves integrative methods that account for protein sequences, drug structures, and known interaction networks. Computational methods for this task are often categorized into recommender systems, Machine Learning (ML)-based, DL-based, and network-based methods [11].

Identifying new indications for existing drugs (drug repositioning) is a promising approach for treating complex diseases, but traditional screening is slow and expensive [12]. Computational methods have been then developed to support traditional methods using matrix factorization [13–16] and machine learning [17, 18]. However, many existing methods do not fully leverage the role of proteins as intermediaries, and their performance can be limited by data sparsity. The application of GNN-based methodologies has recently shown promise in improving these investigations [19].

The rising incidence of chronic diseases has spurred interest in predictive models that use electronic health records to identify at-risk patients early [20].

This is especially relevant for chronic liver diseases such as dismetabolic associated liver disease or cirrhosis where early identification of high risk patients is crucial to prevent disease progression, reduce the burden of irreversible liver damage, and limit the onset of severe systemic complications. Timely risk stratification in these patients can support targeted clinical monitoring, personalized therapeutic interventions, and more effective allocation of healthcare resources, ultimately improving long-term outcomes and patient survival.

While numerous studies apply ML or DL for specific disease prediction, there is a lack of decision-support tools based on graph methods (Graph Neural Networks (GNNs) or KGEs) that can be applied to predict a variety of diseases within a unified framework.

ML – and DL–based streams are individually valuable, but they share a fundamental limitation: they often analyze specific biomedical interactions in isolation, without jointly considering patient-level clinical profiles, disease associations, therapeutic links, and molecular targets within a unified benchmarking setting. This gap is also reflected in recent benchmarking efforts. For instance [21], conducted an extensive evaluation of KGE models, but explicitly identified the application of GNNs as a promising yet unexplored direction for future work.

Our study is designed to bridge all such outlined significant gaps. First, we conduct one of the first comprehensive benchmarking studies comparing KGE and GNN models for link prediction across three heterogeneous biomedical graphs with distinct structural characteristics: Patients-Diseases, Diseases-Drugs, and Drugs-Targets interactions constructed from real clinical data and external publicly-available datasets. Second, we perform a large-scale comparative evaluation of 8 state-of-the-art models accounting for KGE models—DistMult [1], CP [2], ComplEx [3], and ConvE [4]—and GNNs approaches, Graph Convolutional Networks (GCN) [5], GraphSAGE [6], Graph Attention Networks (GAT) [7], Graph Isomorphism Network (GIN) [8]—on complex, multi-relational biomedical data, directly addressing the gap in systematic model comparison identified in prior literature. Then, we provide an in-depth utility-driven analysis of model performance in both transductive settings, where models predict missing links among nodes already observed during training, and inductive settings, where models are evaluated on previously unseen nodes or interactions. This analysis combines predictive accuracy with computational efficiency metrics to rigorously assess model generalization on unseen patient data, which is critical for clinical deployment. Finally, we release a fully reproducible, open-source experimental pipeline with standardized protocols, hyperparameter configurations, and baseline results to ensure transparency and enable fair comparison in future healthcare graph learning research.

To guide our analysis and structure this investigation, our study seeks to answer the following research questions (RQs):

• RQ1 (Overall Link Prediction Performance): Which model family, GNN or KGE, achieves the best overall performance for link prediction in a transductive setting?

• RQ2 (Hyperparameter Sensitivity): How sensitive are the models to changes in their key hyperparameters, and what impact does this have on their predictive accuracy?

• RQ3 (Training Statistics): What do the training dynamics, such as convergence speed and resource consumption, reveal about the practical usability of different models?

• RQ4 (Inductive Generalization): How well do the models generalize in an inductive setting, where they must predict links for entities not seen during training?

By addressing these questions, this paper aims to identify the most effective graph-based techniques for modeling complex biomedical interactions, thereby enhancing predictive power and advancing their potential for clinical relevance in healthcare. We dub our benchmarking analysis AMIUGraph, short for Analysis and Modeling of Interactions for Utility-driven Benchmarking of Graph-Based Models in Healthcare. We acknowledge that several biomedical knowledge graphs, such as BioKG [22, 23], Hetionet [24], and the Clinical Knowledge Graph (CKG) [25], already integrate multiple biomedical entity types, including drugs, diseases, genes, proteins, pathways, and biological processes. Therefore, the novelty of AMIUGraph does not lie in the general idea of connecting heterogeneous biomedical entities. Rather, it lies in the construction of a clinically grounded benchmarking framework that combines patient-level cohort data with external disease–drug and drug–target knowledge sources, and in the systematic comparison of KGE and GNN model families across Patients–Diseases, Diseases–Drugs, and Drugs–Targets link prediction tasks under both transductive and inductive evaluation settings.

The remainder of this paper is structured as follows. Section 2 reviews related work. Section 3 details our experimental setup, datasets, and models. Section 4 presents and discusses the results, and Sect. 6 concludes the paper.

Related work

In this section, we review representative graph learning-based approaches applied to biomedical and clinical data, with a particular focus on Knowledge Graph Embedding (KGE) models and Graph Neural Networks (GNNs). Table 1 summarizes the comparison between AMIUGraph and these prior studies. The works reported in the table were selected to provide a representative, rather than exhaustive, view of graph-based biomedical link-prediction research and benchmarking. We included: (i) established surveys and benchmarks on biomedical graph embedding; (ii) widely used biomedical knowledge-graph resources or graph-based benchmarks; (iii) representative KGE, GNN, or graph-based methods addressing at least one of the relation families considered in AMIUGraph, namely patient–disease (PD), disease–drug (DD), and drug–target (DT), or closely related biomedical link-prediction tasks; and (iv) recent works that introduce inductive evaluation, clinical/utility-driven grounding, or large-scale biomedical graph reasoning. The goal of this section is therefore not to enumerate all existing biomedical graph-learning papers, but to highlight the dimensions most relevant to AMIUGraph: patient-level clinical grounding, coverage of multiple biomedical relation families, comparison between KGE and GNN model families, inductive evaluation, and deployment-oriented evidence.

Table 1.

Representative graph-based methods, biomedical resources, and survey/benchmark papers related to AMIUGraph. The works were selected according to the criteria described in Sect. 2: established survey/benchmark relevance, coverage of AMIUGraph-related relation families (PD, DD, DT) or closely related biomedical link-prediction tasks, use or systematic evaluation of graph embedding/KGE or GNN methods, inductive evaluation, and clinical/utility grounding

Papers	Venue	Graph models		Datasets			Ind.	Clin./Util.
		GE/KGE	GNN	PD	DD	DT
[26]	Bioinformatics	×	×	×	×	\checkmark	×	×
[17]	PLOS One	×	×	×	×	\checkmark	×	×
[27]	WWW	×	×	×	\checkmark	×	×	\checkmark
[28]	Bioinformatics	×	×	×	×	\checkmark	×	×
[29]	Bioinformatics	×	\checkmark	×	\checkmark	×	×	×
[23]	CIKM	×	×	×	×	×	×	×
[24]	eLife	×	×	×	\checkmark	\checkmark	×	×
[25]	bioRxiv	×	×	×	\checkmark	\checkmark	×	×
[30]	Bioinformatics	\checkmark	\checkmark	×	\checkmark	×	×	×
[31]	Inf. Fusion	\checkmark	\checkmark	×	\checkmark	\checkmark	×	×
[32]	AMIA J.S.T.S.P.	\checkmark	×	×	\checkmark	×	×	×
[33]	J. Biomed. Inf.	×	×	×	×	×	×	\checkmark
[34]	Nature Comm.	×	×	×	\checkmark	×	×	×
[21]	Bioinf. Adv.	\checkmark	×	×	\checkmark	\checkmark	×	\checkmark
[35]	AI in Med.	×	\checkmark	×	\checkmark	×	\checkmark	×
[36]	Nature Med.	×	\checkmark	×	\checkmark	×	\checkmark	×
AMIUGraph	BMC Med. Inf. & Decis. Mak.	\checkmark	\checkmark	\checkmark	\checkmark	\checkmark	\checkmark	\checkmark

Note. For survey, benchmark, or resource papers, checkmarks in the model-family columns indicate the methodological scope or systematic evaluation covered by the work rather than a single proposed architecture. GE denotes graph embedding

Surveys and biomedical graph embedding benchmarks

Established surveys and benchmarks provide an important reference point for positioning AMIUGraph. [30] present BioNEV, a systematic benchmark of biomedical graph embedding methods covering several biomedical link-prediction and node-classification tasks, including drug–disease association, drug–drug interaction, protein–protein interaction, medical-term semantic type classification, and protein function prediction. Their work provides useful methodological guidance for selecting graph embedding methods and tuning their hyperparameters. More recently [31], provide a broad survey of graph embedding techniques for biomedical data, organizing the literature around homogeneous, heterogeneous, and dynamic biomedical graphs, and summarizing applications, datasets, open-source implementations, and open challenges.

AMIUGraph is complementary to these efforts. While these works provide broad methodological taxonomies and benchmarking evidence on general biomedical networks, AMIUGraph focuses on a clinically grounded, patient-level healthcare benchmark. In particular, AMIUGraph jointly evaluates patient–disease, disease–drug, and drug–target relations; compares KGE and GNN families under a unified protocol; considers both transductive and inductive settings; and reports clinical/utility grounding together with convergence and computational cost.

Drug–target interaction prediction and target discovery

A first line of research focuses on the prediction of drug–target interactions and the identification of potential therapeutic targets. [26] introduce bipartite local models to predict drug–target interactions from chemical structures and genomic sequences by learning separate local classifiers on a bipartite graph. Their method reports Area Under the Receiver Operating Characteristic Curve (AUC) values above 97% and Area Under the Precision–Recall Curve (AUPR) values up to 84% on enzymes, ion channels, G protein-coupled receptors (GPCRs), and nuclear receptors. [17] propose Heterogeneous Graph-Based Inference (HGBI), a heterogeneous drug–target graph inference approach that integrates drug–drug and target–target similarities with known drug–target interactions to recover known associations and infer novel ones through guilt-by-association and information flow on the network. [37] introduce a topology-driven tool for Plasmodium falciparum metabolic networks, identifying essential enzyme targets through simulated reaction knock-outs and alternative pathway searches. [28] propose DDR, a method for Drug–Target Interaction (DTI) prediction that combines multiple drug–drug and target–target similarity measures through nonlinear similarity fusion and trains a random forest classifier on graph-based features.

Drug–drug interaction and polypharmacy modeling

A second group of studies investigates graph-based methods for drug–drug interaction prediction and polypharmacy modeling. [27] develop an end-to-end pipeline that constructs a knowledge graph from web data, computes large-scale similarity features, and predicts drug–drug interactions using logistic regression. Their system also provides Application Programming Interfaces (APIs) and a web-based user interface to support explanations of predicted interactions. [29] introduce Decagon, a multirelational Graph Convolutional Network (GCN) that models polypharmacy side effects by message passing over heterogeneous drug–protein–disease graphs, establishing GNNs as strong baselines for multirelational biomedical link prediction. [32] show that tensor factorization and KGE formulations can achieve competitive AUPR values and favorable computational complexity for polypharmacy side-effect prediction, suggesting that embedding-based models remain robust and efficient on biomedical knowledge graphs.

Biomedical knowledge graphs, drug repurposing, and benchmarking

More recent studies exploit biomedical knowledge graphs for drug repurposing, knowledge graph completion, and benchmarking. [23] release BioKG, a broad biomedical knowledge graph designed for reproducible relational learning across diverse entity and relation types, enabling scalable link prediction benchmarks and downstream transfer tasks. [33] apply knowledge graph completion to COVID-19 drug repurposing by integrating literature, ontologies, and molecular data to rapidly generate treatment hypotheses under time and data constraints. [34] extend guilt-by-association to multiple layers of large biomedical graphs for drug repurposing, showing that structured propagation across heterogeneous networks can improve indication discovery and interpretability at scale. [21] revisit KGE performance on BioKG using best-practice training protocols, including one-vs-all cross-entropy, tuned regularization, and negative sampling, reporting substantial gains in link prediction and downstream transfer tasks. [35] propose GCNGAT, a model that combines graph convolution and graph attention for drug–disease association prediction, aiming to better handle heterogeneity and sparsity in biomedical graphs. [36] introduce TxGNN, a graph foundation model for clinician-centered zero-shot drug repurposing that provides interpretable multi-hop rationales across thousands of diseases.

Patient-level graph reasoning and clinical decision support

Graph-based methods have also been explored in patient-level clinical decision-support scenarios. [38] present a graphical method that jointly models physician–patient features and labels using a factor graph and sum-product message passing. Their approach improves targeting accuracy on claims and prescription data while providing visual interpretability of the relationships between clinical variables, physicians, and patients. However, this line of work does not provide a unified benchmark across patient–disease, disease–drug, and drug–target interactions, nor does it compare KGE and GNN families under both transductive and inductive settings.

Positioning of AMIUGraph

Unlike broad survey and benchmark efforts such as BioNEV [30] and the biomedical graph embedding survey by [31], AMIUGraph is not intended as a general methodological taxonomy of biomedical graph embedding methods. Instead, it provides a task-oriented and clinically grounded benchmark for healthcare link prediction. Similarly, unlike broad biomedical knowledge resources such as BioKG [23], Hetionet [24], and the Clinical Knowledge Graph (CKG) [25], AMIUGraph is designed around patient-level clinical grounding and benchmark-oriented model comparison.

The final benchmark includes real patient-level clinical data and external biomedical interaction resources, and evaluates models under both transductive and inductive settings. In addition to predictive performance, AMIUGraph also quantifies convergence behavior and computational cost, which are essential dimensions for real-world clinical deployment. Earlier studies typically cover only part of this setting: some focus on a single interaction type, such as drug–target or drug–disease prediction; others evaluate only one model family, such as KGEs or GNNs; and most do not jointly consider inductive generalization, patient-level clinical data, and computational efficiency. AMIUGraph integrates these dimensions within a unified and reproducible benchmark, providing practical guidance for the selection of graph-based models in healthcare applications.

Methodology

This section details our experimental study designed to evaluate link prediction performance on a novel, heterogeneous biomedical graph. We refer to this benchmarking analysis as AMIUGraph, short for Analysis and Modeling of Interactions for Utility-driven Benchmarking of Graph-Based Models in Healthcare.

We begin by describing the source datasets and the pre-processing pipeline used to construct our final graphs. We then present the final statistics of the generated graphs: Patients—Diseases, Diseases—Drugs, and Drugs—Targets. Subsequently, we provide formal definitions for the transductive and inductive learning settings, which form the basis of our evaluation. Finally, we outline the model architectures and the hyperparameters considered to ensure the reproducibility of our results.

Data sources and pre-processing

Our final graph is constructed by integrating data from three distinct sources, each corresponding to a specific type of biomedical interaction.

Source 1: patient and disease data

The first data source consists of patient-level clinical and disease information obtained from two population-based studies conducted by the Laboratory of Epidemiology and Biostatistics at the National Institute of Gastroenterology “Saverio de Bellis” Research Hospital, Castellana Grotte, Italy: the MICOL Study [39] and the NUTRIHEP Study [40, 41]. The dataset includes 2,487 individuals and contains demographic, anthropometric, clinical, and biochemical variables. Specifically, the available patient-level variables include sex, age, body mass index, height, weight, systolic and diastolic blood pressure, thyroid-stimulating hormone (TSH), free triiodothyronine (FT3), and free thyroxine (FT4). In addition to these patient characteristics, the dataset contains 14 binary disease-related variables, indicating the presence or absence of the following conditions: diabetes mellitus, ulcer, gallstones, intestinal polyps, cancer, hyperuricemia, dyslipidemias, hypertension, heart attack, stroke, hyperthyroidism, liver disease, hepatitis C virus infection (HCV), and fatty liver. Table 2 summarizes the baseline characteristics of the study population and reports the prevalence of each disease variable. The most frequent disease-related conditions in the sample were fatty liver, observed in 1,051 participants (42.3%), hypertension, observed in 1,049 participants (42.2%), and dyslipidemias, observed in 576 participants (23.2%). Less frequent conditions included stroke, observed in 22 participants (0.9%), liver disease, observed in 39 participants (1.6%), and heart attack, observed in 67 participants (2.7%). All procedures were conducted in accordance with the ethical standards of the institutional research committee of the National Institute of Gastroenterology, IRCCS “S. de Bellis” Research Hospital, and with the principles of the 1964 Declaration of Helsinki and its subsequent amendments. Written informed consent was obtained from all participants prior to enrollment.

Table 2.

Descriptive statistics of the combined MICOL and NUTRIHEP study population

Variable	Unit / Categories	Value
Sample Size	N	2487
Gender	Female / Male	1256 (50.5%) / 1231 (49.5%)
Age	years	60.47 ± 14.27
Body Mass Index		28.08 ± 5.19
Height	cm	161.81 ± 9.93
Weight	kg	73.51 ± 14.67
Systolic BP	mmHg	124.81 ± 16.10
Diastolic BP	mmHg	78.21 ± 8.01
TSH	μIU/mL	1.82 ± 4.08
FT3	pg/mL	3.26 ± 0.48
FT4	ng/dL	0.88 ± 0.31
Diabetes Mellitus	No / Yes	2246 (90.3%) / 241 (9.7%)
Ulcer	No / Yes	2340 (94.1%) / 147 (5.9%)
Gallstones	No / Yes	2314 (93.0%) / 173 (7.0%)
Intestinal Polyps	No / Yes	2404 (96.7%) / 83 (3.3%)
Cancer	No / Yes	2321 (93.3%) / 166 (6.7%)
Hyperuricemia	No / Yes	2394 (96.3%) / 93 (3.7%)
Dyslipidemias	No / Yes	1911 (76.8%) / 576 (23.2%)
Hypertension	No / Yes	1438 (57.8%) / 1049 (42.2%)
Heart Attack	No / Yes	2420 (97.3%) / 67 (2.7%)
Stroke	No / Yes	2465 (99.1%) / 22 (0.9%)
Hyperthyroidism	No / Yes	2303 (92.6%) / 184 (7.4%)
Liver Disease	No / Yes	2448 (98.4%) / 39 (1.6%)
Hepatitis C Virus (HCV)	No / Yes	2402 (96.6%) / 85 (3.4%)
Fatty Liver	No / Yes	1436 (57.7%) / 1051 (42.3%)

Continuous variables are expressed as mean ± std

Categorical variables as frequency (percentage)

Source 2: disease and drug data

To model disease-drug interactions, we leveraged the dataset provided by Luo et al. [42]. This resource contains multiple types of biomedical relations, but for the scope of this work, we exclusively extracted the disease-drug associations, which originally included 5,603 diseases, 708 drugs, and 199,214 known interactions.

Source 3: drug and target data

Drug-target interactions were sourced from the popular DrugBank database (version 5.0) [43]. While DrugBank is a comprehensive resource, we focused solely on drug-target links, which in the original version comprised approximately 13,000 drugs, 4,563 unique protein targets, and 14,150 recorded interactions.

Graph construction and final datasets

A multi-step pre-processing pipeline was implemented to harmonize and integrate data from these sources.

First, we selected 14 distinct diseases from the patient cohort data, consistently with the clinical variables reported in Table 2: Diabetes Mellitus, Cancer, Hypertension, Dyslipidemias, Fatty Liver, Hepatitis C Virus (HCV), Liver Disease, Stroke, Ulcer, Heart Attack, Hyperuricemia, Gallstones, Hyperthyroidism, and Intestinal Polyps. After removing entries with missing values to ensure data completeness, we obtained a cleaned cohort of 2,487 patients.

To construct the Patients—Diseases bipartite graph, each positive patient-disease association was treated as an individual edge. Since patients can be diagnosed with multiple diseases, a single patient may contribute multiple edges to the graph. This resulted in a binary clinical profile involving 14 disease nodes and 3,976 patient-disease associations, representing all positive disease occurrences observed in the cohort.

For consistency with the Diseases—Drugs dataset, we performed a normalization step to map the patient diseases to the standardized taxonomy provided by the drug interaction database. However, only 10 out of the 14 selected diseases had corresponding matches in the Diseases—Drugs dataset. Specifically, the matched diseases were: Diabetes Mellitus, Dyslipidemias, Fatty Liver, Heart Attack, Hepatitis C Virus (HCV), Hypertension, Hyperuricemia, Liver Disease, Stroke, and Ulcer. The remaining four diseases, namely Cancer, Gallstones, Hyperthyroidism, and Intestinal Polyps, lacked drug association data in the external knowledge base and were therefore excluded from the disease-drug interaction graph.

Accordingly, the final disease subset used for the integrated graph-based analysis consisted of 10 clinically grounded and externally mappable diseases. This subset was used to construct the harmonized Patients—Diseases and Diseases—Drugs components, ensuring consistency across the patient cohort and the external drug interaction knowledge base.

The patient data was structured into two components: (i) a feature matrix containing patient attributes, including demographic, anthropometric, and biochemical variables, and (ii) a binary clinical profile indicating the presence or absence of the selected diseases for each patient.

This process yielded three final bipartite graphs for our experiments. Their key statistics are summarized in Table 3.

Table 3.

Statistics of the final datasets after pre-processing

Dataset	Node types	# Nodes	# Edges	Sparsity
Patients—Diseases	Patients	2,042	4,127	0.8653
	Diseases	14
Diseases—Drugs	Diseases	10	2,630	0.5798
	Drugs	569
Drugs—Targets	Drugs	569	2,575	0.9941
	Targets	770

The current benchmark intentionally adopts a task-specific bipartite formulation rather than a single ontology-enriched multi-relational knowledge graph. Biomedical ontologies and terminologies, such as SNOMED CT1, ICD-102, or related disease hierarchies, could provide additional semantic relations, including parent–child disease links, synonym mappings, and cross-terminology correspondences. However, the objective of AMIUGraph is to evaluate KGE and GNN models under controlled and comparable link prediction tasks with explicit target relations: Patients–Diseases, Diseases–Drugs, and Drugs–Targets. A flat bipartite formulation preserves the interpretability of each edge type and avoids introducing auxiliary ontology-derived relations whose density, semantics, and coverage may differ substantially across data sources. Moreover, because patient-level disease labels are derived from cohort variables, whereas disease–drug and drug–target links are obtained from external biomedical resources, ontology enrichment would require an additional harmonization layer and could introduce mapping uncertainty. We therefore leave the integration of ontology-based and terminology-based relations to future extensions of AMIUGraph, where their contribution can be evaluated through a dedicated multi-relational protocol.

Experimental settings

We evaluate all models under two distinct learning settings: transductive and inductive, to assess both their ability to model existing graph structures and their capacity to generalize to unseen data.

Transductive link prediction

In the transductive setting, the model has access to the entire set of nodes in the graph during training, validation, and testing. The goal is to predict missing links (edges) between these known nodes. For example, when training on the Patients—Diseases graph, the model learns embeddings for all 1,814 patients and all 14 diseases. The task is then to predict whether a specific patient is associated with a specific disease, given that the link was held out from the training set. This setting primarily tests a model’s ability to learn and reconstruct the patterns within a fixed graph structure.

Inductive link prediction

In the inductive setting, the model is evaluated on its ability to generalize to entirely new nodes that were not seen during training. This scenario is more challenging and better reflects real-world clinical applications, where a model must make predictions for a new patient entering the system.

To create this setting, we split the nodes into training, validation, and test sets. For instance, in the Patients—Diseases graph:

• A subset of patients is used for training the model.

• The model’s ability to generalize is then tested by asking it to predict disease associations for a completely separate set of patients.

In this setting, models cannot rely on learned node-specific embeddings for the new patients. Instead, representations for unseen nodes must be generated from available input features. We therefore define the inductive evaluation in AMIUGraph as a feature-based generalization task: the model is trained on a subset of patients and tested on patients that were not observed during training, using their clinical feature vectors to predict patient–disease associations. This protocol is applied only to the Patients–Diseases graph because it is the only component of the current benchmark that contains real node features, namely demographic, anthropometric, clinical, and biochemical attributes associated with patient nodes (e.g., those reported in Table 2). By contrast, the Diseases–Drugs and Drugs–Targets graphs contain interaction links extracted from external biomedical resources, but they do not include feature vectors for diseases, drugs, or target proteins in the current benchmark. Applying the same inductive protocol to these graphs would require introducing additional external descriptors, such as ontology-based disease representations, chemical or molecular drug descriptors, or sequence-based target features, thereby defining a different experimental setting with additional preprocessing and harmonization choices.

This protocol tests the model’s capacity to map node features into a predictive representation space, which is a critical requirement for deployment scenarios involving newly observed patients. GNN models can exploit these input features through their message-passing layers, whereas KGE models require an explicit feature-to-embedding mechanism for unseen patients. Specifically, after normalizing the patients’ input features, we train a 1-layer MLP to project each patient feature vector into the same latent space as the disease embeddings learned during training. The resulting patient embedding is then used by the KGE scoring function to predict patient–disease associations.

Tested models and reproducibility

A link predictor is a differentiable model in which entities and relation types are mapped to a continuous embedding space. The likelihood of a link between two entities is modeled as a function of their corresponding embeddings. Formally, link predictors are defined by a parametric scoring function, where θ denotes the model parameters. Given a triple of subject (s), predicate (p), and object (o), the function computes a score that reflects how likely it is that s and o are connected by relation p.

Knowledge graph embedding models (KGEs)

Knowledge Graph Embedding (KGE) models aim to learn low-dimensional vector representations of entities and relations in a knowledge graph, such that the semantic and structural information of the graph is preserved in the embedding space. By optimizing the scoring function , KGEs capture the likelihood of observed triples and enable the prediction of missing links. Popular KGE approaches differ primarily in how they parameterize .

DistMult. In DistMult [1], the scoring function is defined as:

1

where denotes the tri-linear dot product across the k-dimensional embedding space. DistMult models the interaction between the subject, predicate, and object embeddings via element-wise multiplication, which makes it computationally efficient and easy to optimize. However, due to its symmetric nature in the scoring function, DistMult is limited in its ability to model asymmetric relations.

CP. The Canonical Tensor Decomposition model (CP) [2] extends DistMult by assigning each entity x two distinct embeddings: and , used depending on whether the entity acts as subject or object, respectively. The corresponding scoring function is defined as:

2

Unlike DistMult, which uses a single shared embedding per entity, CP’s use of separate embeddings for subjects and objects allows it to better capture asymmetric relationships by modeling different roles explicitly.

ComplEx. ComplEx [3] is a further generalization of DistMult in which the embeddings of s, p, and o are represented as complex-valued vectors, i.e., . The scoring function is defined as:

3

where denotes the real part of , and is the complex conjugate of . The use of complex embeddings allows ComplEx to naturally model asymmetric relations by capturing phase differences between entities and relations.

ConvE. ConvE [4] is a knowledge graph embedding model where the interactions between entities and relations are modeled via a 2-dimensional convolutional layer. The scoring function is defined as:

4

where is a relation embedding and are the entity embeddings. The vectors and are reshaped into two-dimensional matrices and , which are then concatenated and processed by a 2D convolutional layer with filters ω. The resulting feature map tensor is flattened, projected into a k-dimensional space via a weight matrix W, and finally matched with the object embedding through an inner product.

Graph neural networks (GNNs)

Graph Neural Network (GNN) models leverage message-passing mechanisms to aggregate and transform information from a node’s local neighborhood, enabling the learning of context-aware node embeddings. Unlike KGEs that treat entities and relations as static embeddings, GNNs dynamically update node representations by integrating both node features and graph structure.

GCN. The Graph Convolutional Network (GCN) [5] is a foundational GNN model. Each layer is defined as:

5

where is the symmetrically normalized adjacency matrix with added self-loops, is a learnable weight matrix, and is an activation function.

GraphSAGE. GraphSAGE [6] introduces a general inductive approach by learning aggregator functions (Eq. 6) that sample and aggregate features from a node’s local neighborhood. For a node v , its representation is updated by aggregating its neighbors’ representations and combining the result with its own previous-layer representation (Eq. 7):

6

7

This design allows GraphSAGE to efficiently generate embeddings for previously unseen nodes.

GAT. The Graph Attention Network (GAT) [7] incorporates attention mechanisms to weight the contribution of neighboring nodes. Its message passing scheme is:

8

where Λ is a matrix of attention weights learned during training, computed based on the node and its neighbors’ features.

GIN. The Graph Isomorphism Network (GIN) [8] is a powerful GNN architecture shown to be as expressive as the Weisfeiler-Lehman graph isomorphism test. The GIN layer is formulated as:

9

where is a multi-layer perceptron and ε is a tunable parameter.

Reproducibility

To ensure reproducibility, we provide complete implementation details including preprocessing scripts, model configurations, and training protocols. All experiments are conducted using PyTorch Geometric3, with fixed random seeds and standardized data splits. The optimal hyperparameter configurations for each model were selected via grid search on the validation sets. The complete parameter spaces explored for KGE and GNN models are reported in Appendix A. The best configurations per model and dataset correspond to the settings that achieved the highest validation performance. We make our codebase and processed datasets publicly available4, enabling other researchers to replicate our experiments and build upon our work.

Results

This section reports the empirical results obtained on the three bipartite graphs and the two evaluation settings. All metrics are computed on the test sets using the best checkpoints selected on validation.

Transductive link prediction performance

Table 4 reports the test performance across all three bipartite graphs in the transductive setting. Overall, results highlight a strong dependence on graph structure and sparsity, highlighting that GNN architectures and ConvE generally outperform simpler bilinear KGE models across tasks.

Table 4.

Transductive link prediction performance of KGE and GNN models on the three bipartite graphs. The × link denotes no link, whereas the \checkmark link denotes an observed link. Boldface indicates best values within each dataset-family block

Datasets	Families	Models	× link			\checkmark link			Acc	AUC
			Prec	Rec	F1	Prec	Rec	F1
Patients—Diseases	KGE	DistMult	0.65	0.75	0.69	0.70	0.59	0.64	0.67	0.72
		CP	0.64	0.76	0.70	0.71	0.58	0.63	0.67	0.71
		ComplEx	0.64	0.77	0.70	0.71	0.57	0.63	0.67	0.69
		ConvE	0.75	0.81	0.78	0.79	0.73	0.76	0.77	0.84
	GNN	GCN	0.74	0.77	0.75	0.76	0.73	0.74	0.75	0.80
		SAGE	0.75	0.81	0.78	0.79	0.72	0.76	0.77	0.81
		GAT	0.74	0.77	0.75	0.76	0.74	0.75	0.75	0.77
		GIN	0.77	0.71	0.74	0.73	0.78	0.76	0.75	0.81
Diseases—Drugs	KGE	DistMult	0.59	0.56	0.57	0.58	0.61	0.60	0.59	0.59
		CP	0.60	0.54	0.57	0.58	0.65	0.61	0.59	0.59
		ComplEx	0.64	0.57	0.60	0.61	0.68	0.64	0.62	0.63
		ConvE	0.82	0.53	0.65	0.65	0.89	0.75	0.71	0.76
	GNN	GCN	0.79	0.70	0.74	0.73	0.81	0.77	0.75	0.83
		SAGE	0.81	0.61	0.70	0.69	0.85	0.76	0.73	0.80
		GAT	0.78	0.72	0.75	0.74	0.79	0.76	0.76	0.83
		GIN	0.74	0.79	0.77	0.78	0.72	0.75	0.76	0.83
Drugs—Targets	KGE	DistMult	0.82	0.51	0.63	0.64	0.89	0.75	0.70	0.87
		CP	0.81	0.51	0.62	0.64	0.88	0.74	0.69	0.85
		ComplEx	0.73	0.49	0.59	0.62	0.82	0.71	0.66	0.80
		ConvE	0.74	0.61	0.67	0.67	0.79	0.72	0.70	0.79
	GNN	GCN	0.76	0.83	0.79	0.81	0.75	0.78	0.79	0.87
		SAGE	0.72	0.82	0.77	0.79	0.67	0.73	0.75	0.82
		GAT	0.80	0.83	0.81	0.82	0.80	0.81	0.81	0.87
		GIN	0.81	0.89	0.85	0.88	0.79	0.83	0.84	0.91

On the Patients–Diseases graph, ConvE achieves the best overall performance, reaching an Accuracy of 0.77 and an AUC of 0.84. GraphSAGE and GIN provide competitive alternatives, both achieving AUC values up to 0.81 and comparable F1 scores. In contrast, simpler bilinear KGE models (DistMult, CP, and ComplEx) obtain a lower performance, with Accuracy values around 0.67 and AUC in the range 0.69–0.72.

On the Diseases–Drugs graph, message-passing models clearly dominate. GNN architectures reach AUC values up to 0.83 and Accuracy up to 0.76, outperforming the best KGE configuration (ConvE, AUC 0.76).

On the Drugs–Targets graph, GIN achieves the highest overall performance (Accuracy 0.84, AUC 0.91), followed closely by GAT and GCN. While KGE models, such as DistMult, remain competitive in terms of AUC (0.87), GNNs provide a more balanced performance across precision, recall, and F1.

The corresponding confusion matrices are reported in Appendix B, where they provide a graphical summary of the classification outcomes for the evaluated configurations. In the main text, the discussion focuses on the aggregate performance metrics reported in Table 4 and on the main trends observed across graph types and model families.

Inductive generalization on patients–diseases

Table 5 reports test performance when evaluating on unseen patients in the inductive setting. Compared to the transductive scenario, results are more tightly clustered: most models achieve Accuracy values in the range 0.78–0.79, indicating comparable generalization ability across architectures.

Table 5.

Inductive link prediction performance of KGE and GNN models on the patients–Diseases graph. The × link denotes no link, whereas the \checkmark link denotes an observed link. Boldface indicates best values within each dataset-family block

Datasets	Families	Models	× link			\checkmark link			Acc	AUC
			Prec	Rec	F1	Prec	Rec	F1
Patients—Diseases	KGE	DistMult	0.75	0.83	0.79	0.81	0.73	0.77	0.78	0.85
		CP	0.75	0.84	0.79	0.82	0.72	0.77	0.78	0.85
		ComplEx	0.77	0.85	0.80	0.83	0.74	0.78	0.79	0.85
		ConvE	0.78	0.81	0.79	0.80	0.77	0.78	0.79	0.85
	GNN	GCN	0.76	0.81	0.79	0.80	0.74	0.77	0.78	0.82
		SAGE	0.76	0.84	0.80	0.83	0.74	0.78	0.79	0.79
		GAT	0.76	0.81	0.79	0.80	0.74	0.77	0.78	0.78
		GIN	0.77	0.84	0.80	0.82	0.75	0.79	0.79	0.83

KGE models with feature-based initialization achieve the highest AUC (0.85), while GNN models remain competitive, with GIN reaching the best AUC among GNNs (0.83). Although absolute performance differences are modest, these results suggest that both model families are capable of effective inductive inference when patient features are available.

The confusion matrices for the inductive experiments are reported in Appendix B. These results complement the aggregate metrics in Table 5 by illustrating the distribution of true positives, true negatives, false positives, and false negatives across the evaluated models. Overall, the matrices show moderate and relatively balanced classification errors, with a slight tendency toward false-positive predictions in some configurations.

Convergence and computational efficiency

Table 6 summarizes the total training time for all models across the three bipartite graphs and evaluation settings. Since AMIUGraph is designed as an exploratory and benchmarking-oriented study rather than a deployment-latency evaluation, we focus on training time as an indicator of the computational cost required for model fitting, convergence, and repeated model selection under a standardized experimental protocol. While inference time is an important metric for fully deployed clinical decision-support systems, its evaluation would require a separate deployment-oriented protocol, including fixed serving conditions, batch sizes, hardware constraints, and target clinical use cases. We therefore leave inference-time evaluation to future deployment-oriented extensions of the benchmark. In the transductive setting, GNN models (trained with the Adam optimizer) consistently converge faster than KGE models trained with AdaGrad, completing training in 118–137 seconds versus 144–207 seconds for KGEs. GAT is consistently the fastest model across datasets (118–126s), while ComplEx is the slowest on the Drugs–Targets graph (207s). ConvE, trained with Adam despite belonging to the KGE family, exhibits intermediate training times (151–179s), reflecting the additional cost of its convolutional scoring function.

Table 6.

Total training time for all models across the three bipartite graphs and evaluation settings. GNN models use ADAM optimizer, KGE models use AdaGrad optimizer

Dataset	Family	Model	Transductive	Inductive
			time (s)	time (s)
Patients—Diseases	KGE	DistMult	184.43	19.92
		ComplEx	144.10	24.50
		CP	145.18	14.46
		ConvE	151.00	17.71
	GNN	GCN	133.65	134.08
		SAGE	136.97	96.86
		GAT	118.57	197.18
		GIN	127.75	186.01
Diseases—Drugs	KGE	DistMult	167.89	—
		ComplEx	173.17	—
		CP	173.00	—
		ConvE	178.38	—
	GNN	GCN	128.72	—
		SAGE	136.44	—
		GAT	125.92	—
		GIN	132.76	—
Drugs—Targets	KGE	DistMult	167.06	—
		ComplEx	206.95	—
		CP	192.76	—
		ConvE	174.76	—
	GNN	GCN	126.68	—
		SAGE	134.72	—
		GAT	123.07	—
		GIN	132.30	—

In the inductive setting on Patients–Diseases, the computational trend reverses sharply. Feature-initialized KGE models complete training in only 14–25 seconds, whereas GNN models require substantially longer runtimes, ranging from 96.9s for GraphSAGE up to 197.2s for GAT. This difference reflects the cost of inductive message passing and neighborhood aggregation in GNNs, compared to the lightweight feature-to-embedding mapping used by KGE models in the inductive pipeline.

Overall, these results suggest distinct operational niches that must be considered when selecting models for real-world clinical deployment: GNNs generally offer superior efficiency and performance in the transductive setting, whereas KGEs become the clear choice for inductive scenarios due to their drastic reduction in computational cost with comparable predictive ability.

Comprehensive training and validation curves (loss and accuracy metrics) for all evaluated models and datasets are provided in Appendix C. The optimal checkpoints are identified at the validation loss minima across all runs.

Gastrointestinal disease as a benchmarking use case

Gastrointestinal Disease (GD) represent a clinically relevant target domain for graph-based machine learning models due to their high prevalence, complex etiopathogenesis, and frequent coexistence with metabolic and systemic comorbidities. These conditions commonly occur in multimorbid settings involving diabetes, dyslipidemia, and cardiovascular disorders, giving rise to heterogeneous and densely connected clinical data structures. This complexity makes gastrointestinal healthcare environments a suitable testbed for evaluating the ability of graph neural networks and knowledge graph embedding models to capture meaningful interaction patterns. In AMIUGraph, gastrointestinal and liver diseases are explicitly represented within the patient-disease, disease-drug, and drug-target interaction graphs, reflecting the real-world clinical complexity of multiple interacting conditions and drug exposures. This project enables systematic evaluation of model performance in clinically relevant scenarios, while emphasizing utility-based criteria beyond predictive accuracy, such as generalizability, robustness to sparsity, and computational feasibility.

Discussion

This section analyzes our experimental findings with respect to the research questions guiding the study, highlighting model strengths, limitations, and practical implications for biomedical deployment.

RQ1: overall link prediction performance

The experimental results highlight that link prediction performance is strongly dependent on graph topology and sparsity, confirming that no single architecture is universally optimal.

This behavior can be interpreted in light of the different inductive biases of the two model families. KGE models learn compact latent representations and relation-specific scoring functions, which can be advantageous when graph patterns are relatively regular or when computational efficiency is important. Conversely, GNNs rely on message passing and neighborhood aggregation, allowing them to exploit local and multi-hop structural information. This explains why GNNs are particularly effective on sparse biomedical interaction graphs, whereas more expressive KGE models such as ConvE remain competitive when richer scoring functions are required.

Patients–Diseases: Heterogeneity favors deep scoring and aggregation. On the Patients–Diseases graph, the superior performance of ConvE and inductive-friendly GNNs (GraphSAGE, GIN) contrasts with the lower results of bilinear KGEs. This suggests that patient–disease associations are not merely structural but reflect heterogeneous clinical patterns: models capable of modeling richer feature interactions or leveraging neighborhood propagation are better equipped to capture these complex signals compared to simple factorization methods.

Interaction Graphs: Sparsity favors Message Passing. On the sparser Diseases–Drugs and Drugs–Targets graphs, the trend shifts noticeably. Here, GNN architectures consistently outperform KGE baselines. This dominance indicates that when direct associations are sparse, the ability of GNNs to aggregate information from multi-hop neighborhoods provides crucial context that factorization models fail to capture effectively. In particular, GIN’s strong performance on targets highlights the effectiveness of isomorphism-based aggregation in biological networks.

From a clinical translation perspective, these findings suggest that model selection should be guided by the structure and intended use of the target graph. On dense clinical association graphs such as Patients–Diseases, ConvE and inductive-friendly GNNs such as GraphSAGE and GIN appear particularly suitable because they can capture heterogeneous patient–disease patterns and richer feature interactions. Conversely, on sparse biomedical interaction graphs such as Diseases–Drugs and Drugs–Targets, GNNs provide a stronger option because neighborhood aggregation helps recover informative relational context when direct links are limited.

RQ2: hyperparameter sensitivity

Hyperparameter tuning remains important for both families, but with different practical profiles. KGE models show stronger sensitivity to embedding size and learning rate, while GNNs are generally more stable but depend on architectural choices such as depth and attention heads. These trends support the view that KGEs can be very effective when carefully tuned, whereas GNNs provide a more robust default choice on sparse biomedical graphs.

RQ3: training statistics

Training-time results (Table 6) confirm two operationally relevant patterns. First, in the transductive setting, GNN models (ADAM) train faster than KGE models (AdaGrad), with runtimes of 118–137s versus 144–207s. Second, in the inductive Patients–Diseases setting, the situation reverses sharply: KGE models become extremely fast (∼ 14–25s), whereas GNNs require 97–197s. This highlights that, under feature-based inductive inference, KGEs can be preferable when computational budget and rapid updates are priorities. This accuracy–efficiency trade-off is particularly relevant for clinical scenarios requiring frequent model updates, such as the integration of newly observed patients or periodically refreshed clinical records. In these settings, lightweight KGE-based pipelines may offer practical advantages because they reduce the computational burden associated with repeated training and model selection.

RQ4: inductive generalization

The inductive evaluation is clinically relevant because real deployments must generalize to new patients. Our updated inductive results show that performance is competitive but not saturated. Most models achieve Accuracy in a narrow band (∼ 0.78–0.79), indicating that generalization is feasible but still affected by non-negligible error rates.

KGE models: strong AUC with conservative link prediction. In the inductive setting, feature-initialized KGEs obtain the highest AUC (0.85). However, confusion matrices reveal a moderate number of errors and a mild tendency toward false negatives (missed links). For instance, the best KGE settings yield FP counts in the range 20–25 and FN counts in the range 30–36 (e.g., ComplEx: TP = 96, TN = 109, FP = 20, FN = 33; ConvE: TP = 99, TN = 104, FP = 25, FN = 30). This suggests that models may be slightly conservative in predicting links under a fixed decision threshold, a behavior that could be addressed in practice through threshold calibration depending on whether recall or precision is clinically prioritized.

GNN models: competitive accuracy but higher compute cost. GNNs remain close in Accuracy and F1 to KGEs in the inductive regime, with GIN providing the strongest inductive GNN performance (AUC 0.83). However, the computational cost is substantially higher than for KGEs (Table 6), which may be a limiting factor in workflows requiring frequent retraining or rapid model updates.

Failure cases and less effective scenarios. The inductive setting also highlights the main conditions under which the framework may be less effective. Prediction errors are more likely for new patients whose clinical profiles are poorly represented in the training cohort, such as patients with rare disease combinations, atypical multimorbidity patterns, or borderline biochemical and anthropometric values. In these cases, both KGE-based and GNN-based models may assign uncertain scores to candidate patient–disease links because the learned representations are derived from limited examples of similar clinical profiles. This limitation is particularly relevant in realistic clinical scenarios, where patient populations are heterogeneous and disease prevalence is often imbalanced. Rare diseases or under-represented comorbidity patterns may lead to missed links, while highly prevalent or strongly connected conditions may increase the risk of over-predicting associations. Consequently, the framework may be less reliable when applied to external cohorts with different demographic, clinical, or epidemiological characteristics from those observed during training. These findings suggest that threshold calibration, subgroup-level error analysis, and external validation are necessary before using AMIUGraph in clinical decision-support workflows.

Implications for deployment. Given the relatively small performance gaps in inductive accuracy, model selection should be guided by operational constraints: KGEs offer the best efficiency and top AUC, while GNNs may be preferable when richer structural inductive biases or more complex feature integration are required. Therefore, in inductive clinical scenarios involving unseen patients, KGEs provide an attractive accuracy–efficiency trade-off, while GNNs remain preferable when the clinical task requires richer structural reasoning over stable and densely connected biomedical graphs.

Framework improvement directions. Beyond the comparison of individual models, the results suggest several directions for improving AMIUGraph as a benchmarking framework. In terms of graph construction, future versions should incorporate richer clinical attributes, additional biomedical entity types, temporal information, and more systematic strategies for representing missing, uncertain, or weakly supported links. This would allow the benchmark to better capture the longitudinal and heterogeneous nature of healthcare data.

In terms of model selection, the framework could be extended with hybrid architectures that combine the global relational scoring capabilities of KGE models with the local neighborhood aggregation mechanisms of GNNs. Calibration-aware and uncertainty-aware models could also be incorporated to support more reliable interpretation of predicted links, especially in clinical screening scenarios. Robustness should be assessed through subgroup-level error analysis, sensitivity analyses, temporal validation, and evaluation under distributional shifts. Finally, scalability should be investigated on larger graphs and multi-institutional datasets, including standardized inference-time measurements under realistic serving conditions.

Limitations and future directions

The results of AMIUGraph provide useful evidence on the comparative behavior of KGE and GNN models across heterogeneous biomedical graphs. At the same time, their interpretation should take into account methodological and practical aspects that may affect the translation of the framework to broader clinical or experimental settings.

First, AMIUGraph relies on the quality, completeness, and harmonization of the input graphs. Missing links, noisy associations, or inconsistent mappings between clinical disease labels and external biomedical knowledge bases may affect both model training and evaluation. This aspect is particularly relevant in healthcare, where disease definitions, coding practices, and drug annotations may vary across institutions and data sources.

Although our inductive evaluation simulates the arrival of new patients, performance remains below saturation and confusion matrices indicate non-trivial error rates. Prediction errors may be more likely for patients with rare disease combinations, atypical multimorbidity patterns, or clinical profiles that are under-represented in the training cohort. In realistic clinical settings, such errors may have different practical implications: false negatives may delay the identification of relevant patient–disease or treatment-related associations, whereas false positives may increase the burden of downstream clinical validation.

Another aspect to consider is that the current framework formulates link prediction as a binary classification task over sampled candidate pairs. While this design enables standardized benchmarking across models, it does not fully capture uncertainty, clinical severity, temporal dynamics, or causal relationships among biomedical entities. Therefore, predicted links should be interpreted as candidate associations requiring expert review and external validation, rather than as direct clinical evidence.

From a clinical applicability perspective, AMIUGraph should currently be interpreted as a benchmarking framework rather than as a clinical decision-support tool ready for routine use. Its translation into real-world clinical practice may be limited by the use of a single-center cohort, the absence of external and prospective validation, and the lack of longitudinal patient trajectories capturing disease progression, treatment changes, and follow-up information. Moreover, the current framework relies primarily on structured clinical variables and does not yet integrate potentially relevant multimodal data in clinical practice, such as medical imaging, genomic data, or unstructured clinical notes. These limitations may affect the generalizability, reliability, and practical utility of the framework across institutions and patient populations. Future studies should therefore prioritize multicenter and prospective validation, longitudinal and multimodal data integration, subgroup-level error analysis, calibration and uncertainty assessment, and evaluation within real-world clinical workflows. Moreover, we acknowledge that additional ranking-based evaluation metrics, such as MRR and Hits@K, which are commonly used in standard link prediction benchmarks, could be considered in future extensions of this work. In the current version, however, we restrict our evaluation to a binary classification formulation of the clinical prediction task (e.g., whether a patient will develop a given disease). We adopt this setting because we consider it more aligned with a realistic clinical scenario, where the goal is to support decision-making through a yes/no risk prediction rather than to rank all possible disease candidates. In this context, we argue that ranking-based metrics may be less directly informative, as they assume a complete ranking over all possible entities, whereas clinical decision-making typically focuses on the identification of high-risk conditions above a decision threshold.

Another limitation of the present study is that computational efficiency is assessed in terms of training and convergence time, whereas inference latency is not directly measured. This choice is consistent with the exploratory and benchmarking-oriented scope of AMIUGraph, which focuses on model comparison, repeated training, and model-selection costs rather than real-time deployment. Future work will extend the benchmark with inference-time measurements under standardized serving conditions to better assess online clinical decision-support scenarios.

Conclusion

In this study, we proposed AMIUGraph, a comprehensive framework integrating patients, diseases, drugs, and protein targets within a unified graph for biomedical link prediction. By benchmarking both KGE and GNN models across transductive and inductive settings, we showed that performance depends strongly on graph characteristics and deployment constraints rather than on model family alone.

In the transductive setting, ConvE achieves the strongest results on the Patients–Diseases graph, while GraphSAGE and GIN provide competitive alternatives. On the larger and sparser interaction graphs (Diseases–Drugs and Drugs–Targets), GNN architectures consistently outperform KGEs, confirming the value of neighborhood aggregation in capturing sparse structural signals.

In the inductive Patients–Diseases evaluation, performance is tightly clustered across all models. Feature-initialized KGEs achieve slightly superior ranking metrics, though confusion matrices highlight that inductive errors remain non-negligible for all architectures. This suggests that while generalization is feasible, threshold calibration and cost-sensitive evaluation are crucial for clinical usage.

Finally, our training-time analysis reveals distinct advantages for each family depending on the scenario. GNNs converge faster in the transductive setting, whereas feature-initialized KGEs are substantially more efficient in the inductive regime, requiring a fraction of the time needed by GNNs.

From a clinical perspective for gastrointestinal diseases, this trade-off is particularly relevant for chronic liver diseases, such as metabolic dysfunction–associated steatotic liver disease, where patient–disease associations must be frequently updated over long and heterogeneous disease trajectories, reflecting changes in clinical status, comorbidities, and treatment response over time, while drug–target knowledge remains comparatively stable. In such settings, models capable of efficiently incorporating newly observed patient information without requiring costly retraining are especially valuable for longitudinal monitoring and real-world clinical deployment.

This makes KGEs particularly attractive for scenarios requiring rapid updates and continual integration of new patient data, while GNNs remain the robust choice for static, structure-heavy tasks.

Future work will focus on hybrid architectures combining global relational scoring with local message passing, calibration strategies to control false negatives in screening tasks, and evaluation on larger and more heterogeneous clinical cohorts.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

AI

Artificial intelligence

ML

Machine learning

DL

Deep learning

KGE

Knowledge graph embedding

GNN

Graph neural network

GNNs

Graph neural networks

KGEs

Knowledge graph embeddings

DTI

Drug–Target interactions

DDI

Drug–Drug interaction network

DT

Drugs–Targets

DD

Diseases–Drugs

PD

Patients–Diseases

GCN

Graph convolutional networks

GAT

Graph attention networks

GIN

Graph isomorphism network

HGBI

Heterogeneous graph-based inference

RF

Random forest

LRW

Local random walk

ACT

Average commute time

AUC

Area under the receiver operating characteristic curve

AUPR

Area under the precision–recall curve

STARD

Standards for reporting diagnostic accuracy studies

STROBE

Strengthening the reporting of observational studies in epidemiology

GPCR

G-protein-coupled receptor

GD

Gastrointestinal disease

NAFLD

Non-Alcoholic fatty liver disease

AFLD

Alcoholic fatty liver disease

HCV

Hepatitis C virus

BMI

Body mass index

BP

Blood pressure

TSH

Thyroid-Stimulating hormone

FT3

Free triiodothyronine

FT4

Free thyroxine

Author contributions

PS (Paolo Sorino), ADB (Alessandro De Bellis), and DM (Daniele Malitesta) contributed equally to conceptualization, formal analysis, investigation, writing of the original draft, and writing, review, software and editing. DL (Domenico Lofù) contributed to investigation, data visualization, writing of the original draft, and writing, review and editing. AF (Antonio Ferrara) contributed to data curation and writing, review and editing. CB (Caterina Bonfiglio) contributed to data curation, clinical data interpretation, and writing, review and editing. RD (Rossella Donghia) contributed to data curation, clinical data interpretation, and writing, review and editing. GG (Gianluigi Giannelli) contributed to clinical supervision, supervision, and writing, review and editing. FN (Fedelucio Narducci), TDN (Tommaso Di Noia), and EDS (Eugenio Di Sciascio) contributed to supervision and writing, review and editing. All authors read and approved the final manuscript.

Funding

This study was supported by the Italian Ministry of Health through Ricerca Corrente 2025–2026.

Data availability

The individual-level clinical data used in this study are not publicly available due to ethical and privacy restrictions. De-identified data may be made available from the corresponding author upon reasonable request, subject to approval by the data owner and the relevant institutional and ethical regulations. The disease–drug and drug–target interaction datasets were obtained from publicly available third-party resources cited in the manuscript. The source code and experimental pipeline are publicly available at: https://github.com/sisinflab/AMIUGraph.

Declarations

Ethics approval and consent to participate

The study was approved by the Ethics Committee of the National Institute of Gastroenterology “Saverio de Bellis” Research Hospital (DDG-CE-589/2004, 18 November 2004) in accordance with the Declaration of Helsinki. Written informed consent to participate in the study was obtained from all participants prior to enrollment.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.