Harnessing AI in Multi-Modal Omics Data Integration: Paving the Path for the Next Frontier in Precision Medicine

Abstract

The integration of multi-omics data with detailed phenotypic insights from electronic health records (EHRs) marks a paradigm shift in biomedical research, offering unparalleled holistic views into health and disease pathways. This review delineates the current landscape of multi-modal omics data integration, emphasizing its transformative potential in generating a comprehensive understanding of complex biological systems. We explore robust methodologies for data integration, ranging from concatenation-based to transformation-based and network-based strategies, designed to harness the intricate nuances of diverse data types. Our discussion extends from incorporating large-scale population biobanks to dissecting high-dimensional omics layers at the single-cell level. The review underscores the emerging role of large language models in artificial intelligence, anticipating their influence as a near-future pivot in data integration approaches. Highlighting both achievements and hurdles, we advocate for a concerted effort towards sophisticated integration models, fortifying the foundation for groundbreaking discoveries in precision medicine.

1. Introduction

The landscape of biological research has undergone a remarkable transformation, courtesy of breakthroughs in high-throughput techniques that have ushered in the era of ‘omics’ science. Spanning across genomics, epigenomics, transcriptomics, proteomics, and metabolomics, the realm of omics offers a kaleidoscopic view of the constituent elements that make up human biology. These layers, each a unique facet of the broader biological puzzle, have long been the subject of unimodal analyses, yielding critical insights into discrete regulatory mechanisms within our biological systems.

However, the advent of sophisticated artificial intelligence (AI), particularly through deep learning and machine learning algorithms, marks a seminal point in our scientific journey. Traditional machine learning frameworks, such as support vector machines and random forests, laid the groundwork for integrating and interpreting the tapestry of multi-omics data. The rise of advanced paradigms—convolutional neural networks, graph neural networks, and recurrent neural networks—has exponentially amplified our capabilities, enabling the decoding of complex patterns and nonlinear relationships sprawling across multi-omics data.

This revolution in data analytics stands at the cusp of a new frontier: the comprehensive and holistic exploration of biological processes. By harnessing the singular attributes and synergistic interactions across various omics strata, machine learning serves as the linchpin in our pursuit of a deeper understanding of life’s fundamental mechanisms, thereby paving the way for innovations in precision medicine.

Significant strides have been made, transitioning from mono-omics to sophisticated multi-omics analyses, spurred by the maturation of data integration techniques and an ever-expanding repository of biological intelligence. Concurrently, the emergence of diverse phenotypic data from electronic health records (EHRs) has added nuanced dimensions that are intimately tied to patient health narratives—ranging from clinical diagnoses and therapeutic histories to vital parameters and advanced imaging findings.

The convergence of multi-layered omics data with rich phenotypic breadcrumbs from EHRs signifies an exciting convergence in biomedical research. Particularly, the systematic amalgamation of large-scale, multi-omics data with nuanced, patient-centric information from EHRs promises to unlock previously cryptic corners of human biology. By shedding light on the subtle dance between molecular constituents and clinical phenomenology, this integrative approach heralds a transformative era in precision medicine.

In this review, we embark on a journey through the dynamic evolution, inherent challenges, innovative methodologies, and untapped potential residing within the realm of multi-modal omics data integration. Standing on the brink of this scientific odyssey, we are witnesses to a nascent revolution in integrative biomedical analysis, one that holds the promise to redefine our understanding of health and the therapeutic strategies of tomorrow.

2. Opportunities in integrative analysis of multi-modal omics data

With the precipitous decline in the costs associated with high-throughput sequencing and other massively paralleled biomolecular technologies, a novel window of opportunity has opened up, offering enhanced accessibility to these advanced tools (1). This progression facilitates their seamless integration into both clinical research and practical applications. In this review, we distinguish between traditional and recent multi-omics analysis, emphasizing the emerging opportunities in multi-modal omics data integration. Specifically, we highlight how the expanded scale and improved resolution of samples now available are forging new frontiers for multi-omics data acquisition.

Conventional multi-omics approaches largely center on the analysis of bulk samples. Although these methodologies have been instrumental in providing insights into specific tissues or cell populations and in furthering our understanding of dominant pathways or disease pathogenesis, they inherently average data across numerous cells. Consequently, while notable achievements have been made in describing a broad overview of biological processes, integrative analysis with these traditional techniques often falls short when it comes to deciphering the intricate cellular heterogeneity within samples. Moreover, these approaches may not capture subtle yet critical differences at the population level, such as variation/diversity in biological pathways across races or genders. Instead, the remaining gaps in our comprehension of human biology could potentially be bridged with the integrated analysis of emerging multi-omics datasets—specifically, multi-modal omics data sourced from large-scale biobanks and high-resolution single-cell multi-omics data.

2.1. Multi-modal omics data integration analysis at the population level

Large-scale, national, or community-based biobanks have started to incorporate heterogeneous phenotypic data derived from EHRs alongside conventional multi-omics data (2) (Table 1). Biobank-scaled multi-omics data integration is transformative in the sense that it offers a shift from random bulk samples to organized population-level datasets. This shift allows for better representation of individual characteristics, which represent a broader spectrum of the population, incorporating variation due to factors like age, ethnicity, lifestyle, and more. This improved representation lends itself to a more holistic and real-world applicable understanding of human biology, particularly unveiling new insights into disease risk and manifestation. Notably, there is a significant advantage in integrating omics data with the observable traits or characteristics of individuals, in that doing so enables determination the etiology or biomarkers for specific diseases (3).

Table 1.

Multi-Omics Projects in the National and Community-Based Cohorts for Biomedical Research: List of omics-specific projects and consortiums were listed in Supplementary Table 1.

Biobank	Project Characteristics				Omics Data					References
	Major Ethnicity (Region)	Donor status	Cohort size (participants)	Year of launch	Genomics	Epigenetics	Transcriptomics	Proteomics	Metabolomics
TOPMed Program	Mixed (USA)	Various	~205,000	2014	✔	✔	✔	✔	✔	(13, 14)
IPOP	Mixed (USA)	Pre-diabetic	100	2012	✔	✔	✔	✔	✔	(15, 16)
UK Biobank	European (UK)	Various	~500,000	2007	✔			✔	✔	(17–20)
INTERVAL trial	European (UK)	Healthy	~50,000	2012	✔		✔	✔	✔	(21–25)
BIOS Consortium	European (Netherlands)	Various	3,841	-	✔	✔	✔		✔	(26–28)
Lifelines Biobank	European (Netherlands)	Healthy	167,000	2006	✔	✔	✔		✔	(29–31)
Estonian Biobank	European (Estonia)	Healthy	> 200,000	2002	✔				✔	(32, 33)
China Kadoorie Biobank	East Asian (China)	Various	> 512,000	2004	✔			✔	✔	-
GNHS	East Asian (China)	Various	> 5,000	2008	✔			✔	✔	(34)
Taiwan Biobank	East Asian (Taiwan)	Healthy	> 150,000	2012	✔	✔			✔	(35)
BioBank of Japan	East Asian (Japan)	Various	260,000	2003	✔			✔	✔	(36)
jMorp	East Asian (Japan)	Healthy	> 5,000	2015	✔	✔	✔	✔	✔	(37, 38)
KoGES	East Asian (Korea)	Various	> 245,000	2001	✔				✔	(39, 40)
KoCAS	East Asian (Korea)	Healthy	> 1,500	2005	✔				✔	(41, 42)
CHAIN Network	African and South Asian	Children with acute illness	3,101	2016	✔			✔	✔	(43, 44)
SPHS	South Asian (Singapore)	Various	364	2003	✔	✔	✔			(45)

Abbreviations: USA, United States of America; UK, United Kingdom; TOPMed, Trans-Omics for Precision Medicine; IPOP, Integrated Personal ‘Omics Project; BIOS, Biobank-based integrative omics study; jMorp, Japanese Multi-Omics Reference Panel; GNHS, Guangzhou Nutrition and Health Study; KoGES, Korean Genome and Epidemiology Study; KoCAS, Korean Children-Adolescent Cohort Study; CHAIN, The Childhood Acute Illness and Nutrition; SPHS, Singapore Population Health Study.

2.1.1. Exploring disease etiology with multi-omics at the population level.

Integration of multi-omics and phenotypic data from biobanks enables the development of molecular-level disease understanding at the population level. Biomarkers have unique advantages in providing early assessment of patient health, prognostic efficacy analysis, and accurate disease staging and typing. Biomarkers discovered through the incorporation of EHR and omics data can provide guidance and evidence for disease diagnosis and prognosis in clinical use.

2.1.2. Longitudinal multi-modal omics integration.

Integrated analysis of phenotypic and multi-omics data facilitates longitudinal integration of multi-modal omics data, an emerging approach that combines data collected over extended periods from the same samples (4). This longitudinal perspective can reveal how biological systems evolve over time, highlighting trends, patterns, and associations that might not be apparent from cross-sectional studies. Applications of longitudinal multi-omics integration include studying disease progression, identifying biomarkers for early diagnosis, understanding the effects of therapies, and uncovering previously hidden relationships between different biological layers (5). A longitudinal approach can thus offer a holistic view of the underlying mechanisms driving health, disease, and responses to treatments.

2.1.3. Incorporating heterogeneous clinical data with multi-omics.

The integration of omics data with diverse clinical information, including medical images, has become increasingly feasible. Imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) offer detailed insights into the structure and function of tissues and organs within the body. To effectively analyze the nuanced interplay between omics data and medical images, it is imperative that both kinds of data be sourced from the same sample. However, traditional multi-omics data analyses often don’t support such cross-examinations unless structured within meticulously designed cohorts. This challenge comes from the need for highly structured and meticulously designed cohorts to guarantee matching samples for comparison. Biobanks present a solution as they collect both imaging phenotypes and omics data from the same individuals, thereby enabling more straightforward combined analysis.

In addition to integrative and longitudinal analyses with heterogeneous phenotypic data, a significant advantage of multi-omics approaches is the ability to extract traditional single omics data within a population-level level. This feature is particularly beneficial for integrative genetics research, allowing for the use of diverse methodologies. Multi-omics approaches lies in their ability to synthesize and interpret these data types collectively, enhancing our understanding of complex biological systems and phenotypes:

• Epigenome-wide association studies (EWAS) focus on the integration of epigenomic data, such as DNA methylation and histone modifications, with clinical phenotypes. This approach is particularly useful for investigating the impact of epigenetic modifications on disease susceptibility and progression (6). EWAS can uncover epigenetic markers associated with diseases, shedding light on how environmental factors interact with an individual’s genetic makeup to influence gene expression patterns.

• Transcription-wide association studies (TWAS) involve the integration of transcriptomics and genome-wide association study (GWAS) data. By leveraging gene expression information from tissues or cell types relevant to a specific trait, derived from sources like GTEx, PsychENCODE, and CommonMind, TWAS can identify genes whose expression levels are associated with a trait of interest (7–9).

• Proteome-wide association studies (PWAS) have emerged to explore the connections between protein profiles and health conditions. PWAS identifies associations between changes in protein expression levels and specific health outcomes, which offers insights into complex disorders and provides potential biomarkers and treatment targets for precision medicine (10, 11).

• Metabolome-wide association studies (MtWAS) explore the connection between metabolites—small molecules involved in biochemical processes—and traits or diseases. Integrating metabolomic data with genetic information can reveal metabolic pathways and networks that are linked to specific physiological conditions (12).

2.2. High-resolution multi-omics data integration analysis from single cells

While biobanks offer scale and better population representation, single-cell omics dives deeper into specific biological and cellular mechanisms, enhancing the resolution to the level of individual cells. That is, instead of understanding a cell population as a homogenous entity and aggregating data from millions of cells, single-cell analyses recognize and chart the heterogeneity within, providing a detailed, cell-specific landscape that reveals cellular intricacies often overshadowed in bulk analyses. By comprehensively profiling biological molecules—such as DNA (genomics), RNA (transcriptomics), proteins (proteomics), and metabolites (metabolomics)—from individual cells, single-cell multi-omics data elucidate the unique molecular signature of each cell (46). Understanding the heterogeneity within cell populations is crucial for understanding complex biological processes, disease mechanisms, and developmental pathways; most of all, it is essential for understanding the comprehensive interplay of molecular processes in health and disease. In this review, we introduce single-cell omics technology by highlighting its utilization of various omics and discuss opportunities for integration analysis accordingly.

2.2.1. Integration of genomic and transcriptomic data

Genome and Transcriptome Sequencing (G&T-seq) simultaneously measures genomic DNA and mRNA from single cells, elucidating the genetic makeup, variations, and gene expression profile (47). gDNA-mRNA sequencing (DR-seq) likewise investigates the relationship between genome and transcriptome in individual cells, enabling an understanding of cellular behavior and functional characteristics (48). Both of these methods unveil genetic and transcriptomic heterogeneity within tissues, improving precision in the identification of cellular origins of diseases and potential therapeutic targets. Detailed descriptions of each single-cell omics method are provided in the Supplementary Text 1.

2.2.2. Integration of transcriptomic and proteomic data

Cellular Indexing of Transcriptomes and Epitopes by sequencing (CITE-seq) combines the specificity of antibody detection with next-generation sequencing through tagging antibodies with unique oligonucleotide barcodes to label proteins in cells (49); mRNA from the same cells is sequenced in parallel. This method provides a comprehensive view of cell functions, accounting for differences between protein and mRNA levels. RNA expression and protein sequencing (REAP-seq) is an integrated single-cell sequencing method likewise designed to concurrently profile the transcriptomes and proteomes of individual cells (50), which provides invaluable insights into cellular function and regulation. Both of these methods combine RNA sequencing with protein-level insights in the same cell.

2.2.3. Integration of epigenetic and transcriptomic data

Targeted Epitranscriptomic Amplification sequencing (TEA-seq) focuses on RNA modifications at the single-cell level, elucidating how these modifications interact with other regulatory layers (51). It is tailored to profile targeted modifications of interest rather than to conduct a global survey. Accelerated Single-stranded Adenine Profiling sequencing (ASAP-seq) quantifies nascent RNA molecules and so measures the rate of transcription in single cells (52), offering insights into transcriptional dynamics and gene expression regulation. Together, these methods provide granular insights into RNA modifications and transcription rates at the individual cell level, a resolution not attainable with traditional methods.

2.2.4. Integration of multi-modal single-cell omics with phenotypic data

Integrating single-cell multi-omics with phenotypic data involves associating the detailed molecular profiles of individual cells with their observable characteristics or functions. This comprehensive approach allows for a more holistic understanding of how molecular changes influence cell behavior and the overall organismal phenotype; furthermore, by correlating molecular profiles with phenotypic responses, researchers can better identify therapeutic targets and predict drug responses (53).

3. Machine learning for multi-modal omics data integration analysis

Many researchers have examined the challenges that arise during data analysis of either single omics or multi-omics data and have proposed solutions tailored to each scenario (54, 55). In this review, we briefly introduce representative machine learning approaches for data integration. Subsequently, we provide a more detailed discussion on integrating imaging phenotypes with multi-omics and the analysis of longitudinal multi-omics data.

3.1. Integration strategies for utilizing machine learning with multi-modal data

Analyzing multiple omics data presents significant challenges due to the inherent data heterogeneity that arises from the diverse techniques and platforms used for generating different omics data types. These datasets can typically be visualized as structured matrices, with rows representing samples and columns representing the biological features specific to each omics category. Tasks may involve classification at the sample level, such as predicting cell types or stratifying patients, or identifying biomarkers at the feature level. In either case, machine learning methodologies provide robust solutions for the integration and interpretation of such intricate data.

A major hurdle with multi-omics data is its high dimensionality, a common characteristic even for data of a single omics type. This problem is further complicated when integrating data from multiple omics sources, as overlapping samples may be lacking. To combat the issue of high dimensionality, researchers often turn to methods for feature selection, such as filter-based, wrapper-based, or embedded methods. Feature extraction techniques like principal component analysis, canonical correlation analysis, non-negative matrix factorization, and auto-encoder also come in handy.

Another layer of complexity arises from issues related to data sparsity and the task of interpreting the results. To weave together and make sense of multi-omics data, researchers employ a spectrum of methodologies ranging from traditional machine learning to state-of-the-art deep learning, all tailored to data characteristics and the objective of the analysis. Here, we discuss three representative integration strategies: concatenation-based integration, transformation-based integration, and network-based integration (Fig. 1).

Figure 1.

Overview of multi-modal omics data integration analysis

Concatenation-based integration stands out as one of the most straightforward strategies for amalgamating multi-omics data. It involves directly combining (or joining) features from different omics datasets through aligning them side-by-side (56–58). To illustrate, if one possesses two matrices of data, one from genomics and another from transcriptomics, pertaining to the same set of samples, the concatenation approach would position the columns of one matrix adjacent to those of the other, culminating in a broader matrix. With such a concatenated feature matrix, we can use any kind of supervised learning for analysis; moreover, this approach is straightforward to implement and understand because the matrix preserves the original features derived from each omics method. Nonetheless, the concatenated matrix often demands supplementary preprocessing steps, such as normalization or batch effect mitigation, as data from different omics sources might operate on different scales or distributions.

Transformation-based integration is a technique used to blend multi-omics data by shifting the data from various omics sources into a shared feature space or framework. By mapping data to these unified spaces, it becomes feasible to identify relationships across different omics layers; hence, this method excels at revealing hidden relationships and patterns within and between omics sets (59). However, in contrast to the direct concatenation-based method, transformation-based integration can obfuscate the understanding of the original data, making it difficult to derive direct biological or clinical insights. Several machine learning methods are frequently employed for transformation-based integration. For example, canonical correlation analysis discerns linear combinations of features from two omics datasets that correlate maximally, making it suitable for two-omics data integration. Multiple kernel learning (MKL) combines several kernels (each corresponding to a different omics data type) to optimize a single learning task. Each kernel gauges similarity within its respective omics type, and MKL fuses them into a consolidated kernel space. Non-negative matrix factorization breaks down data into products of non-negative matrices, and when applied to multi-omics data, reveals shared patterns or meta-genes. Finally, deep learning techniques such as an auto-encoder are trained to jointly represent multi-omics data, with the bottleneck layer encapsulating a compressed, integrated representation of the input datasets.

Network-based integration is used to integrate and analyze multi-omics data in the context of biological networks (60–62); it leverages the structured relationships between biological entities, such as genes, proteins, or metabolites, to provide context and improve the interpretation of multi-omics data. This method essentially visualizes and analyzes data in the form of networks, where nodes represent biological entities and edges represent relationships or interactions between them. Once the multi-omics data are converted to an adjacency or similarity matrix, kernel-based algorithms or graph neural networks can be applied. Thus, network-based integration transforms multi-faceted omics data into a structured, interconnected framework that facilitates a holistic view of biological systems. The interconnected nature of networks naturally complements the intricate relationships within and between omics datasets, offering an intuitive platform for multi-omics data exploration and hypothesis generation.

3.2. Integrative analysis through combining multi-omics and imaging phenotypes

Biobanks that link patient EHRs with omics data have paved the way for developing an unprecedented degree of insight into disease risk and manifestation. As imaging modalities such as CT, MRI, and PET provide detailed insights into the structure and function of tissues and organs, the recent inclusion of medical imaging in EHRs has opened up the potential for finer phenotyping and developing an enhanced understanding of how various structures correlate with disease. Radiomics transforms these routine medical images into quantitative data that can be mined, allowing image features to be analyzed alongside other biological information. However, image features encompass comprehensive characteristics, including pathological conditions, and differ in modality from other data sources; as such, integrated analysis of imaging with omics data demands significant computational effort. Machine learning is essential for the various stages of this integrated analysis, including data preprocessing, feature selection and extraction, designing integration approaches, model selection, and model evaluation. Notably, deep learning-based algorithms are well-suited for handling the complexity of multi-modal data and also for identifying intricate patterns and correlations (63). By building descriptive and predictive models, deep learning methods enable researchers to extract valuable insights from the integrated data, contributing to a more comprehensive understanding of complex diseases and enhancing healthcare decision-making.

Radiogenomics, which specifically integrates medical imaging with genetic data, aims to identify disease mechanisms, predict prognosis, and assess treatment responses through investigating the relationships between imaging features and genetic factors; in particular, it can elucidate genomic risk factors associated with a phenotype of interest and the genetic architecture shared between phenotypes and morphologies. This approach has been applied to phenotypes derived from imaging of specific tissues or organs, including structural and functional brain imaging (64–66), cardiac imaging (67), retinal imaging (68, 69), and lung imaging (70). Beyond genomics, further incorporating diverse multi-modal omics data into such analyses can provide supplementary insights into the pathobiological pathways and disease-associated risk factors. Such integrated studies have been systematically conducted across a broad spectrum of diseases such as autism spectrum disorder, oral diseases, and lung cancer (70, 71). In addition, a metagenomic study of the gut microbiome has revealed associations between protective bacteria, cognitive traits, and brain regional volumes derived from imaging (72). The integration of microbiome, metabolomics, cytokine measurements, cognitive assessments, and brain imaging data in that study provides further evidence of the greater insights that can be obtained by utilizing more data modalities.

3.3. Machine learning for improved phenotyping

Machine learning plays a critical role in enhancing phenotyping accuracy, enabling quantitative measurements to more precisely represent disease-related features. For instance, machine learning algorithms have been employed to predict vertical cup-to-disc ratio, a critical parameter for glaucoma diagnosis. By overcoming the need for costly and time-intensive expert labeling, machine learning facilitates the identification of significant loci; thus, future work in machine-learning-based phenotyping will facilitate omics integration studies (73).

3.4. Longitudinal multimodal omics data integration

Longitudinal multi-omics integration combines data obtained over time from the same samples, shedding light on biological evolution and patterns not seen in cross-sectional studies. It is therefore beneficial in studying disease progression, pinpointing early diagnostic biomarkers, evaluating therapy effects, and understanding interconnected biological mechanisms influencing health and treatment responses. However, longitudinal multi-omics integration also presents numerous challenges that demand sophisticated computational and analytical strategies. Specific to longitudinal data, challenges like temporal misalignment, missing data, and irregular sampling intervals require robust methods for data preprocessing and normalization. Interdependency between multiple time points from the same sample, unbalanced datasets, and unexpected outlier events add to the data complexity. Finally, the time element further amplifies the high-dimensional nature of multi-omics data, an issue compounded by the typically limited number of samples and high sample variability.

Methods developed for longitudinal multi-omics data integration typically have the following components: 1) preprocessing, 2) modeling, and 3) analysis (e.g. predicting a phenotype, identifying omics biomarkers for the progression of a phenotype, extracting clusters of omics features that have similar patterns, etc.). Preprocessing tasks may involve excluding subjects with limited time points or low variation, imputation of missing data, normalization, and feature selection to identify the most informative elements for subsequent analysis. Preprocessing and modeling may be performed sequentially or simultaneously depending on the framework. For example, Bodein et al. developed timeOmics, a longitudinal multi-omics integration framework that performs each component sequentially (74). Preprocessing in this framework includes a fold-change-based filter to focus on time-sensitive molecules; after preprocessing, a linear mixed model spline framework is used to model the expression of each biological feature while considering inter-individual variability. It then uses six unsupervised integration methods to cluster multi-omics expression profiles and identify key molecular features per cluster. Metwally et al. follow a similar structure with their proposed method, OmicsLonDA (75), in which they first input preprocessed data, then apply a Gaussian smoothing spline regression model in a semi-parametric approach to discern significant time intervals of omics features between study groups.

In addition to statistical methods, recurrent neural networks (RNNs) are a prevalent deep learning method for integrating longitudinal multi-omics or multi-modal data. Their effectiveness is clear from their success in other areas such as natural language processing, time series prediction, and speech recognition, just to name a few (76, 77). The advantages of using RNN-based models are three-fold. First, they process sequences by iterating through individual elements while retaining a memory state that captures the context of prior elements. Second, the same parameters are used across different time steps, promoting compactness and generalization. Third, RNNs are naturally equipped to handle sequences of varying lengths, making them flexible and able to address a variety of tasks (78). However, while vanilla RNNs introduced the concept of processing sequences, they struggle with long sequences due to challenges like vanishing and exploding gradients (79). Modern adaptations, such as long short-term memory networks (LSTM) and gated recurrent units (GRUs), were designed to overcome these issues and are now more prevalent in practice.

When applied to longitudinal multi-omics integration, some frameworks using these models combine the preprocessing, modeling, and analysis stages, while others approach them as separate, consecutive steps. Lee et al. used the latter approach by training a unique GRU for each data modality to transform longitudinal data into fixed-size vectors (80). The respective vectors for each modality were then concatenated and merged into an integrated vector, which was used to predict progression from mild cognitive impairment to Alzheimer’s disease (AD) using an l1-regularized logistic regression. The former approach of combining steps is possible because RNNs are able to impute missing values or interpolate between observed time points, creating continuous sequences for further analysis. Nguyen et al. adapted minimalRNN to simultaneously impute missing data and predict AD progression (81), and found it to perform better than typical interpolation methods such as forward and linear filling and other baseline methods. Jung et al. developed a deepRNN framework which combines imputation, encoding, and analysis to predict clinical status trajectories (82).

Ultimately, longitudinal multi-omics integration has the potential to uncover previously hidden relationships between different biological layers and glean insights that remain concealed in cross-sectional studies. However, the path to such insights is strewn with challenges—from the intricacies of temporal data to the high dimensionality of multi-omics data. The advent of deep learning, especially RNNs and their advanced derivatives, offers compelling strategies for managing the complexities inherent to this data. As this field continues to evolve, it will be crucial to refine existing methodologies and develop new tools, ensuring realization of the full potential of longitudinal multi-omics integration for health and disease.

4. Downstream multi-modal omics data integration analyses

In this section, we review several studies for downstream applications that leverage the vast resources of large biobanks to uncover valuable insights into complex traits and diseases. These illustrate the scale and potential of multi-modal omics data integration.

4.1. Biobank-scaled multi-modal omics data integration

Large-scale biobanks have paved the way for extensive multi-modal omics data integration, which in turn enables a deeper understanding of genetic, epigenetic, and metabolic contributions to human health. We explore the key applications of GWAS, EWAS, TWAS, and MtWAS, which incorporate phenotypic data with genetic, epigenetic, transcriptomic, and metabolomics data respectively.

TWAS can identify genes whose expression levels are associated with a particular trait. Integrating TWAS results into polygenic risk scores (PRS) can enhance the predictive power of PRS models, more accurately capturing the functional consequences of genetic variants and hence improving disease risk and other phenotypic outcome predictions. Recent studies on complex neuropsychiatric traits (83), chronic obstructive pulmonary disease (COPD) (84), and other phenotypes have demonstrated improved prediction performance and enhanced cross-ancestry PRS portability (84, 85). EWAS aids in discovering epigenetic markers related to a disease, providing insights into the interaction between environmental factors and individual genetics that influence gene expression. It also highlights the genes, pathways, and molecular mechanisms associated with common and complex traits like attention-deficit hyperactivity disorder (ADHD) (86), speech sound disorders (87), and coronary artery disease (CAD) (88). In addition, EWAS has been instrumental in identifying specific genetic factors that contribute significantly to diseases known for their high polygenic risk burdens, such as ADHD (89) and autism (90). Integrating metabolomic data with genetic information can further unveil metabolic pathways and networks that are tied to specific physiological states; for instance, Kojouri et al. examined the causal associations between physical activity, body mass index, and metabolites (12). More broadly, MtWAS can identify metabolites linked to disease risk, giving a more comprehensive view of disease etiology and potential biomarkers, and has been employed for conditions like depression, chronic kidney disease (CKD), and neovascular age-related macular degeneration (91, 92). Several studies have further inferred the causality of metabolites in human diseases using medallion randomization (93, 94).

Multi-omics projects and consortia focused on specific diseases are also noteworthy (Supplementary Table 2). Exploration of disease etiology at the molecular level increasingly requires multi-omics and computational algorithms. For clinical diseases, disease biomarkers play key guiding roles in diagnosis and prognosis, as they have unique advantages in evaluating early, low-level damage and so provide for early warning, prognostic efficacy analysis, and accurate staging and typing. Although many cancer-associated biomarkers have been identified through single omics, multiple omics approaches may provide enhanced benefits by uncovering biomarkers that are shared across different cancer types. Multi-omics also has many applications in the study of neurodegenerative biomarkers and therapeutic targets. In addition, recent years have seen increased acknowledgment of the human microbiome’s crucial role in health and disease, leading to many microbiome-wide association studies (MWAS) exploring the intricate relationship between the human microbiome and health outcomes; these have revealed potential biomarkers, therapeutic targets, and new pathways (95, 96). Similarly to MtWAS, causal links have been found between microbiome data and diseases like heart failure (97) and COVID-19 severity (98) through Mendelian randomization. These comprehensive repositories shaped by multi-omics-driven data will play a pivotal role in unraveling the complex nuances of various diseases, thereby advancing biomedical research toward the realm of precision health.

4.2. Multi-omics with imaging phenotypes

Medical imaging has vastly improved our understanding of complex diseases, a fact that underscores the importance of transpathology, the combination of molecular imaging and pathology data, in investigating disease mechanisms (99). In transpathology, specific attributes of diseases, such as phenotypic traits, structural abnormalities, and progression patterns, are directly compared to imaging, providing a more comprehensive view of disease-related biological processes. An advantage of this approach is that it provides deeper insights into disease etiology by identifying relationships between molecular imaging and pathology data; disadvantages are that the methods can be computationally intensive and may require prohibitively large datasets to achieve accurate results. Transpathology approaches have been particularly successful in elucidating biological features of neurological diseases, with hundreds of studies connecting brain network activity from imaging studies with the likelihood of developing mental illness at the individual level (100). Recent work has also shown the benefit of integrating imaging data with expression data to learn more about disease. For example, tri-modal transcriptomics, genomics, and imaging data were combined in a federated model to understand the relationships between omics levels and how these biologically important factors relate to AD (101). Similarly, Bao et al. recently integrated genomic data, multiple types of expression data, imaging data from 145 brain regions, and AD status using a co-localization framework to distinguish causal AD pathways (102). Imaging analyses have also been implemented for spatial transcriptomics, enabling a more comprehensive view of molecular processes (103).

4.3. Precision medicine approaches with multi-omics data integration

Precision medicine represents the pinnacle of personalized medicine, in which medical decisions, treatments, and interventions are tailored to each individual based on their unique characteristics (104). Central to this vision is the integration of multi-omics data: joining genomics, transcriptomics, proteomics, metabolomics, and other high-throughput datasets to obtain a holistic view of an individual’s biological makeup. Integrating these vast datasets allows clinicians and researchers to identify complex patterns, making it possible to predict susceptibility to specific diseases, understand disease progression, and create personalized treatment strategies. In essence, multi-omics integration is a key element that will transform the ambitious vision of precision medicine into a tangible reality, ushering in a new era of personalized and predictable healthcare.

With the incorporation of multi-omics data and EHR-derived phenotypic data from large-scale biobanks, numerous studies have been undertaken to stratify or predict patient risk (24). Thompson et al. explored the potential of epigenetic information to enhance phenotype inference in combined biobank-EHR systems by developing a methylation risk scoring model that integrated DNA methylation data and 607 EHR-derived phenotypes from the UCLA health biobank (105) and demonstrated its potential to significantly improve clinical phenotype inferences compared to traditional polygenic risk scores. Talmor-Barkan et al. integrated extensive clinical and multi-omics profiling of 199 patients with acute coronary syndrome (ACS) to understand the multifaceted nature of CAD (106). By integrating serum metabolomics, gut microbiome data, and data from two major Israeli hospitals, the study identified distinct serum and gut microbial signatures in ACS patients compared to controls: the former lacked a previously unidentified bacterial species from the Clostridiaceae family, the absence of which was connected to various circulating metabolites known to increase CAD risk. This finding emphasizes the personalized nature of metabolic deviations in ACS patients and their ties to important clinical factors and cardiovascular outcomes; it also underscores the potential early involvement of metabolic disturbances linked to the microbiome and diet in dysmetabolic phases that predate clinically apparent CAD. Furthermore, using a metabolomics-based model, the authors observed that ACS patients’ predicted body mass index exceeded actual measurements, and these predictions correlated with diabetes mellitus and CAD severity. All told, this study highlights the potential of the serum metabolome in comprehending the diverse risk factors associated with CAD. In a different study, Parisot et al. utilized graph convolutional networks (GCNs) to introduce a comprehensive framework that integrates both imaging and non-imaging data for brain analysis in large populations (107). In their approach, the population is represented as a sparse graph where nodes correspond to imaging-based feature vectors and edge weights to phenotypic data, including genetic information. Such network-based integration captures the interplay of individual features and their interactions holistically. They tested this method on two broad datasets: ABIDE for predicting autism spectrum disorder and ADNI for predicting conversion to AD. The results showed a remarkable performance improvement over existing techniques, achieving a classification accuracy of 70.4% for ABIDE and 80.0% for ADNI. This highlights the potential of neuroimaging and phenotypic data integration using GCN to improve disease prediction. Mathew et al. undertook a comprehensive immune profiling of hospitalized COVID-19 patients using high-dimensional flow cytometry (108). Their analysis at the single-cell omics level revealed three distinct immunotypes, each associated with differing degrees of disease severity and clinical outcomes, and the longitudinal analysis highlighted stability and fluctuations in patient responses. These findings not only offer a comprehensive map of immune cell responses in COVID-19 but also highlight potential avenues for therapeutic interventions.

5. Future directions of multi-modal omics data analysis

In the previous section, we discussed how the scale of data integration and the utilization of machine learning have revolutionized the landscape of multi-modal omics data analysis, with a focus on advancing precision medicine. However, the rapid and large-scale development of state-of-the-art large language models (LLMs), including GPT-4 (109), PaLM2 (110), and LLaMA2 (111), has introduced artificial intelligence methods with unprecedented and remarkable performance capabilities. These developments anticipate a new paradigm shift in multimodal omics data analysis. Accordingly, this section explores the trends, opportunities, and challenges that will emerge as the field embraces the potential of LLMs and other cutting-edge technologies to unlock the complexity of multimodal omics data for the benefit of precision medicine and biomedical research.

Multimodal LLMs are designed to accommodate inputs of multiple modalities, extending beyond text-based data (Table 2). Notably, ChatGPT has profoundly influenced a number of fields, reshaping our interaction with technology. In addition to the increasing public interest in ChatGPT, as LLMs demonstrate remarkable achievements in the biomedical domain, numerous experimental methodologies applied to multimodal data are emerging, promising to open new avenues of discovery and innovation. In particular, building on established medical LLMs (112–114), several medical multimodal LLMs have been proposed (115, 116). These models are poised to revolutionize medical research and healthcare by enabling interactive data-driven communication, information extraction, and insights into model results. One key differentiator between LLMs and traditional machine learning is their ability to engage in data-driven communication, extract information, and provide insights into the underlying reasons and basis for the model’s results. In essence, as new multimodal omics data is generated and the analyzed knowledge is fed into LLMs, they can infer new insights based on previously accumulated data, further advancing our understanding of complex diseases and enabling more informed healthcare decision-making.

Table 2.

General-purpose and Biomedical-specified large language model (LLM)

Models		Modalities	Foundation models	Model size	LLM Finetuning
General purpose multi-modal LLM	Flamingo	Language, Vision	Chinchilla, NFNets	3B, 9B, 80B	No finetuning
	LLaVA	Language, Vision	Vicuna/LLaMA2 (111, 121) CLIP ViT (122)	7B, 13B	LLM finetuning
	BLIP2	Language, Vision	OPT/Flan-T5 (123, 124) CLIP ViT+Q-former (122)	3B, 7B, 12B	No finetuning
	MiniGPT4	Language, Vision	Vicuna (121) CLIP ViT+Q-former (122)	7B, 13B	No finetuning
	InstructBLIP	Language, Vision	BLIP2	7B, 13B	No finetuning
	KOSMOS-1	Language, Vision	Magnato (125) CLIP ViT (122)	1.6B	LLM finetuning
	PaLM E	Language, Vision	PaLM (126) ViT (127)	12B, 66B, 84B, 562B	LLM finetuning
	mPLUG-OWL	Language, Vision	LLaMA (128) CLIP ViT (122)	7.2B	LLM finetuning
	mPLUG- Doc OWL	Language, Vision, Chart Document	mPLUG-OWL	Unknown	LLM finetuning
	GPT4	Language, Vision, Chart, Document		Unknown	Unknown
Medical multi-modal LLM	LLaVA-Med	Language, Medical Vision: [X-ray, MRI, Pathology, Gross pathology, CT]	LLaVA	7B, 13B	No
	MedVInT	Language, Medical Vision: [X-ray, MRI, Pathology, Gross pathology, CT]	PMC-CLIP (129) PMC-LLaMA (130)	7B	Yes
	Med-Flamingo	Language, Medical Vision: [X-ray, MRI, Pathology, Gross pathology, CT]	OpenFlamingo (131)	9B	Yes
	Visual Med-Alpaca	Language, Medical Vision: [X-ray, MRI, Pathology, Gross pathology, CT]		7B	No
	RadFM	Language, Medical Vision: [X-ray, MRI, Pathology, Gross pathology, CT]	PMC-LLaMA (130)	14B	Yes
	PathAsst	Language, Pathology	PLIP (132) Vicuna (121)	13B	No
	PaLM-Med M	Language, Medical Vision: [X-ray, MRI, Pathology, Gross pathology, CT]	PaLM-Med (114)	12B,84B,562B	No

We briefly describe general-purpose LLMs and biomedical-specific LLMs in Table 2. For example, LLaVA can pinpoint anomalies in unconventional images (Supplementary Figure 1). LLaVA-Med, when presented with a radiology image, does not just decode its features but synthesizes medical knowledge to diagnose (Supplementary Figure 2). As multi-modal LLMs prove capable of such multimodal reasoning, we foresee vast potential in tackling complex challenges like multi-omics analysis. A conceptual framework named Generalist Medical AI (GMAI) (117) has been proposed with the aim of flexibly processing diverse medical modalities from imaging to genomics. Subsequently, Acosta et al. underscored the potential of multimodal AI in harnessing a myriad of biomedical datasets, from expansive biobanks to cost-effective sequencing (118). We likewise anticipate that multimodal medical AI will steer the next era of multi-omics analysis (Figure 2). A recent endeavor tasked ChatGPT with single-cell type prediction (119), which yielded accurate outcomes due to GPT-4’s extensive training on scientific literature enabling it to recognize marker genes and their cellular associations. This capability might be harnessed for more intricate tasks like multi-omics analysis.

Figure 2.

Potential ability of Large Language Models with Multi-modal omics data

Recent endeavors have demonstrated the efficacy of ChatGPT in tasks like single-cell type prediction, with impressive prediction performance outcomes (119). The impressive accuracy come from GPT-4’s comprehensive training on scientific literature, equipping it to identify marker genes and their corresponding cellular associations. There is potential to leverage this capability for multi-omics analysis interpretation. While there is a growing interest in employing foundation models for radiomics (120), and several multi-modal LLMs are able to stride in this integration (Table 2), they are currently lacking in handling molecular-level omics data. For instance, Med GPT-M have been conducted to analyze genomics, yet they still represent a proof-of-concept study with preliminary results. With technological advancements in the foundation model, advancements in understanding of molecular omics will enable researchers to become more adept at interpreting and integrating vast and diverse multi-omics data, thereby overcoming the current limitations of multi-modal LLMs. We describe a list of models trained on datasets like transcriptomes, DNA sequences, and protein sequences that may potentially underpin future multi-modal LLM development (Supplementary Table 3). While forging a multi-modal omics-centric LLM is a formidable challenge, we believe such harnessing of diverse modalities will redefine future research paradigms.

6. Conclusion

As we navigate the intricate landscape of biological systems, the integrative analysis of multimodal omics data emerges as a cornerstone in decoding the complexities inherent in health and disease. This approach, characterized by the confluence of genomics, transcriptomics, proteomics, metabolomics, and nuanced clinical data, including imaging phenotypes, represents a quantum leap in our collective scientific methodology.

This review encapsulated the multifaceted dimensions of this integrative process, addressing inherent challenges and spotlighting strategic methodologies that transcend traditional data analysis. From connection-based to transformation-based, and network-centric integrations, the scientific community now wields the power to unify the once-disparate data realms, offering fresh perspectives on the molecular underpinnings of pathological states.

The implications of this synthesis are profound, extending beyond theoretical research into the practical sphere of precision medicine. By harnessing the wealth of information within large-scale biobanks and electronic health records, we enhance our capacity for patient risk stratification, disease prognostication, and the customization of therapeutic interventions. Such endeavors are transforming the healthcare paradigm, empowering clinicians to curate treatment modalities reflective of individual biological signatures.

Peering into the future, we recognize the impending influence of advanced computational models, especially large-scale language models capable of multimodal reasoning. These innovative systems promise to redefine problem-solving in biomedical contexts, despite existing obstacles like resource constraints and data accessibility. Nonetheless, through economical and collaborative ventures, the pursuit of comprehensive, tailored solutions for multi-omics analyses is well underway.

In essence, the journey toward a thorough understanding of biological entities, via multimodal omics data integration, is leading biomedical research into uncharted territories. The evolution of machine learning strategies and the advent of sophisticated multimodal platforms are not just enhancing our present capabilities but are also charting the course for future breakthroughs. It is through these prisms that we foresee a revolution in personalized healthcare delivery, anchored by precision medicine’s tenets.