Multimodal AI in precision medicine: linking omics, imaging and clinical decisions

Abstract

Precision medicine is shifting from a single-biomarker paradigm toward multimodal integration of molecular, imaging, and clinical data, with artificial intelligence (AI) serving as a key enabling technology. Multimodal imaging, computational pathology, spatial omics and liquid biopsy have all become more adept at capturing disease and biological variation, and foundation models, especially vision-language models, are now creating a common language between histology, molecular and biomedical text. AI-powered pipelines are also improving data quality upstream, which ultimately means that multimodal inference can be applied to tasks of real clinical importance: biomarker screening, modelling treatment outcomes, even autonomous coordination among diagnostic systems, rather than just retrospective prediction. But none of this means more modalities necessarily translate into more value. Additional data may introduce redundancy, technical artefacts, institutional bias, and missingness. The future of multimodal AI should therefore emphasize sufficiency rather than maximalism, with systems designed to accommodate incomplete or asynchronous inputs, quantify uncertainty, and demonstrate added value over established clinical standards. Ultimately, clinical utility will depend on external and prospective validation showing improvements in calibration, efficiency, net clinical benefit, and patient outcomes. The promise of AI-enabled multimodal precision medicine lies not in combining every available data stream, but in determining which molecular, morphological, and clinical information is necessary, complementary, and actionable for an individual patient at a specific decision point.

Quantitative computational models of tissue and molecular states are increasingly complementing conventional single diagnostic markers in precision medicine [1,2]. BioCompNet, for example, extracts automated body-composition features from magnetic resonance imaging for cardiometabolic stratification [3]. This shift toward reusable representations is also evident in disease-specific and cross-task imaging models: RETFound supports label-efficient transfer across retinal disease tasks, UMedPT learns transferable features across heterogeneous biomedical imaging tasks, and MedFMC provides a real-world benchmark for adapting foundation models to medical image classification [4-6]. In computational pathology, systematic evaluation of foundation-model feature extractors using whole-slide multiple-instance learning has demonstrated that pretrained representations can transfer across grading and biomarker-prediction tasks [7], while CHIEF extends this approach toward pan-cancer detection and prognosis-oriented representation learning [8]. Doppler ultrasonography combined with magnetic resonance imaging has also been used to predict blood flow under simulated microgravity [9], whereas RadFM extends generalist multimodal modeling across web-scale two- and three-dimensional radiological data [10]. Similarly, Prov-GigaPath scales self-supervised learning to gigapixel whole-slide pathology while incorporating both local and global tissue context [11], and FedM2CT addresses cross-institutional variability through federated computed-tomography reconstruction without pooling patient data [12]. Collectively, these advances are moving medical imaging beyond passive visual documentation toward scalable computational phenotyping.

Molecular profiling augments this imaging dimension by providing mechanistic insight into disease and its diversity. With the support of machine learning, single-cell and spatial technologies are increasingly able to map molecular drivers within local tissue environments; in lung adenocarcinoma, for example, such approaches have revealed MAPK-driven transcriptional programs within specific immune niches [13]. The same convergence now extends to spatial modeling: STAIG integrates expression, spatial position and image information [14], while GHIST and iSCALE use histology or tissue-scale structure to infer or extend molecular states beyond the native assay resolution [15,16]. Recent approaches have further bridged morphology and molecular state. OmiCLIP links histopathology with spatial transcriptomics to enable cell-type decomposition and gene-expression prediction [17], while SEQUOIA and mSTAR provide complementary routes from histology to transcriptomic or multimodal slide representations [18,19]. A third, less invasive layer is added by liquid biopsy: plasma and tissue DNA methylation signatures can detect early-stage non-small-cell lung cancer [20], and combining liquid-biopsy signals with radiology has improved non-invasive discrimination of gallbladder cancer from benign disease [21]. Vision-language models extend this integration further. MCPL aligns medical text and image representations through collaborative multimodal prompting across downstream medical tasks [22], while VLM-CPL, generative and foundation-model approaches in digital pathology, and PathChat illustrate complementary strategies for annotation-efficient classification, language-guided representation learning, and interactive multimodal reasoning [23-25]. Similar principles apply beyond oncology; in breast capsular contracture, transcriptomic and machine-learning analyses have identified metabolic and immune mechanisms associated with disease progression [26]. More broadly, pan-cancer histology-genomic fusion, multi-omic prediction of breast-cancer therapy response and systematic multimodal fusion studies show how complementary modalities can sharpen patient stratification and biomarker discovery [27-29].

The integration of molecular and morphological information is being further accelerated by foundation models and artificial intelligence (AI)-enabled analytical pipelines. One example of AI’s upstream contribution is mtFFPECleaner, which filters spurious mitochondrial DNA variants caused by formalin-fixation artefacts in sequencing data, highlighting that reliable multimodal inference depends on analytical accuracy before integration [30]. At the molecular-integration level, Flexynesis provides a reusable framework for bulk multi-omics fusion, feature prioritization, and clinical-outcome modeling [31]. MUSK advances shared representation learning by combining large-scale pathology images and biomedical text within a unified vision-language model for cancer detection, biomarker prediction, and outcome modeling [32]. A systematic assessment of multimodal generative AI in clear-cell renal cell carcinoma similarly showed that general-purpose models can capture diagnostically relevant pathological features, although their performance varies substantially [33]. Recent work on AI and digital tools in cancer pathology extends this transition from image classification toward language-aware, generative, and workflow-oriented systems [34]. Another important interface between molecular measurement and patient care is clinical laboratory medicine, where AI is being investigated for quality control, interpretation, risk stratification, and clinical decision-making [35]. HONeYBEE demonstrates how foundation-model-derived embeddings can organize clinical, radiological, pathological, and molecular information at scale, including when modality availability is heterogeneous [36]. Together, these developments indicate that multimodal AI is moving beyond simple feature concatenation toward shared representations that preserve and connect molecular measurements, morphology, imaging, and language.

The central question, however, is whether multimodal integration can improve clinical decision-making rather than simply increase retrospective discrimination on benchmark datasets. Explainable AI applied to more than 15,000 patients with multiple solid cancers has integrated clinical variables, image-derived body-composition features, and tumor mutational profiles to identify patient-specific predictors of treatment outcome [37]. Across cancer types, multimodal systems have integrated radiology, pathology, and genomics to predict response to PD-(L)1 blockade [38], predicted prognosis in adult and pediatric brain tumors [39], inferred transcriptomic information from histology for treatment-response modeling [40], used self-supervised histopathology to predict molecular classes in endometrial cancer [41], and derived pathomics signatures associated with prognosis and chemotherapy benefit in stage III colon cancer [42]. Histology-derived AI biomarkers have also approached the performance of transcriptomic predictors of anti-angiogenic treatment response in renal cancer [43]. More recent models incorporate clinical-information prompting [44], prior knowledge-guided multimodal learning [45], explicit robustness to missing modalities [46], and deep fusion of patho-radiomic and clinical data [47]. EAGLE moves closer to clinical implementation by predicting EGFR mutation status from routine lung adenocarcinoma histology; external validation and a prospective silent trial suggest that AI-assisted screening may reduce the need for rapid molecular testing while maintaining high screening accuracy [48]. An even broader transition is emerging from predictive models toward systems capable of orchestrating multiple specialized tools. An autonomous precision-oncology agent combining a large language model with multimodal diagnostic and analytical tools has already been tested in clinically representative decision-making scenarios [49]. These studies redefine the goal itself: the clinically meaningful endpoint is not maximal predictive accuracy, but the ability to judge when further testing is required, synthesize complementary evidence and support an actionable decision for an individual patient.

Multimodality, however, does not necessarily imply clinical value. Additional data can just as easily add redundancy, technical artefacts, institutional signatures and population-specific biases, and complete molecular, imaging and clinical measurements are seldom all available in routine care. The future of multimodal systems should therefore focus on sufficiency rather than maximalism: models that can cope with missing and asynchronous modalities, preserve modality-specific provenance, quantify uncertainty and show incremental benefit over robust clinical baselines rather than merely increasing complexity. Recent frameworks that explicitly evaluate heterogeneous or missing modalities illustrate one route towards this goal [36,46], but robustness must ultimately be demonstrated prospectively.

Validation is as important as design: external validation across institutions, devices and populations should precede claims of generalizability, and prospective evaluation should include calibration, net clinical benefit, workflow efficiency and patient outcomes rather than isolated gains in area under the curve. Reporting and evaluation frameworks such as DECIDE-AI [50], TRIPOD+AI [51], STARD-AI [52] and FUTURE-AI [53] provide increasingly concrete standards for this transition. Methodological work has also converged on practical design principles: cross-modal interactions should be modeled explicitly rather than reduced to simple feature concatenation [54-56], while multi-omics integration should preserve biologically meaningful structures across data types. Translation also depends on demonstrating that added modalities improve biomarker discovery, diagnosis or prognosis in clinically relevant cohorts rather than only on retrospective benchmarks. The actual potential of AI-powered precision medicine is therefore not the combination of every data stream that happens to exist. It lies in determining which combination of molecular, morphological and clinical evidence is necessary, complementary and actionable at a specific decision point for an individual patient.

Disclosure of conflict of interest

None.