Impact of matched samples on domain harmonization in cross-cohort NMR metabolomics

Abstract

Introduction

Machine learning models trained on metabolomics data frequently show reduced performance when applied to independent cohorts because of domain shifts introduced by differences in analytical platforms, sample handling, storage conditions, and metabolite quantification procedures. Robust approaches for harmonizing metabolomics datasets across cohorts are therefore needed to improve model transferability and reproducibility.

Objectives

To evaluate the impact of matched samples on domain harmonization and cross-cohort prediction performance in NMR metabolomics data, and to compare multiple harmonization strategies for improving model generalization across independently acquired datasets.

Methods

A machine-learning framework was developed to classify ICD-10 code I50.9 (unspecified heart failure) using plasma NMR metabolomics data from the Estonian Biobank. Models were trained on a source dataset comprising 12,000 samples measured at Nightingale Health and evaluated on an independently measured destination dataset of 295 samples acquired on a different NMR platform. Harmonization approaches included within-domain normalization, Optimal Transport, Bridge mapping, and Empirical Bayes correction. The effect of matched samples was assessed using up to 108 duplicated biospecimens measured in both datasets. Additional validation was performed using independently generated human gestational diabetes and mouse diabetes metabolomics datasets from Metabolomics Workbench.

Results

Domain harmonization generally improved cross-cohort prediction compared with unaligned data, particularly when matched samples were available. Predictive performance increased as the number of matched samples used for calibration increased, although gains varied across methods. Nevertheless, simple within-domain normalization consistently provided strong performance and often matched or exceeded more complex alignment approaches. External validation using fully unmatched human and mouse metabolomics datasets demonstrated that harmonization strategies improved classification performance relative to raw data, supporting the broader applicability of the framework.

Conclusion

Matched samples substantially enhance domain harmonization and improve the transferability of metabolomics prediction models across analytical platforms and cohorts. However, robust normalization remains a strong baseline, and the benefits of more sophisticated alignment methods depend on the availability and representativeness of matched reference samples. These findings provide practical guidance for developing reproducible cross-cohort metabolomics machine-learning models while preserving biologically relevant signals.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1007/s11306-026-02543-0.

Introduction

Cardiovascular diseases (CVDs) are a significant disease burden that affects 523 million people worldwide (Vos et al., 2020). The onset and progression of a CVD can be attributed to an individual’s genetic background (Kavousi et al., 2023; Robinson-Cohen et al., 2023), but the five most significant modifiable risk factors are body mass index, systolic blood pressure, non-HDL cholesterol, current smoking status, and diabetes (Consortium, 2023). The diagnosis of various CVDs is associated with multiple clinical symptoms, mainly dyspnea and chest pain. For measurable values, high-sensitivity cardiac troponin values can be indicative of myocardial injury or acute coronary syndrome (Golino et al., 2022). Various ’omics’ analyses have paved the way for improved disease-related biomarker discovery and a better understanding of disease pathology (Reitz et al., 2023), but CVD research remains relevant with access to data from various biobanks and the application of novel laboratory and statistical methods (Liu et al., 2025).

Data from metabolomics experiments have been used in multiple studies involving cardiovascular diseases. A lipidomic-focused cohort study on incident primary non-fatal, fatal myocardial infarction and stroke found associations with several monoacylglycerols, diacylglycerols, cholesteryl esters, free fatty acids and sphingolipids (Eichelmann et al., 2022). Another clinical trial found higher levels of ceramide ratio C24:1/C24:0, acylcarnitine ratio C4/C18:2 as predictors of coronary artery disease (Karagiannidis et al., 2022). Due to the substantial number of different cardiovascular diseases and their associated factors, research interest remains high in determining the most significant predictors and biomarkers using optimal statistical methods.

In addition to established statistical analysis methods, the application of machine learning (ML) models is becoming more widespread in metabolomics research. A study focusing on sphingolipids applied random forest (RF) and least absolute shrinkage and selection operator (LASSO) regression approaches to generate CAD risk scores (Poss et al., 2020). The study illustrated the usefulness of ML methods in combination with conventional clinical CVD biomarkers, including serum LDL cholesterol and triglycerides. Another study used plasma samples analyzed with liquid chromatography tandem mass spectrometry (LC-MS) to assess the risk of CVD. Elastic net regression revealed changes associated with CVD in amino acids, energy intermediates, lipids, and xenobiotic compounds (Rios et al., 2023). In general, the use of ML methods in CVD research is still novel but is more widely accepted as an addition to established statistical analysis methods.

The application of machine learning models involves the training of models on a subset of the whole data, for instance 4/5 and validation using various metrics on the remaining 1/5 of data. The data is split and used for training and validation in multiple different batches, i.e. cross-validated, and the results are concatenated to assess the model’s accuracy. The best practice to obtain a robust estimate of performance is to test the models on data from another cohort or independently measured samples. The lack of test datasets has been cited as a limitation in many research papers (Poss et al., 2020; Karagiannidis et al., 2022). The measurement of samples for a test dataset introduces variability in the data, where the machine operator, the metabolite quantification protocol, the storage and handling conditions all contribute to the variability in results (Stavarache et al., 2022). In the UK Biobank, the inconsistency of results from different NMR platforms and analysis times, highlighted the need for a software that attempts to harmonize measurement values and make data batches more comparable (Ritchie et al., 2021). Due to the differences in datasets from different laboratories, the challenge of generating an independent test dataset remains a major limitation in many research projects. Having a reliable method for data transformation and harmonization of multiple independent datasets would improve the quality of future research where comparisons are needed.

In this research article we address the above mentioned issue, namely the harmonization and calibration of data coming from different experimental devices, setups, sample storage conditions and collection time points. More specifically, we deal with the real-world scenario in which a machine learning model has been trained and validated on an existing dataset and subsequently tested on a new dataset which has been obtained using measurements from a different instrument, distinct metabolite quantification protocol and a different set of samples. In our case the samples for the training dataset (the training cohort) were collected between the years 2002–2011 and measured at Nightingale Health OY, Finland. The test dataset was obtained instead by measuring again a subset of the previous cohort samples together with new samples (collected between 2018-2019). All these new measurements were performed at the Istituto Italiano di Tecnologia, Genoa, Italy with a different experimental apparatus.

Within this setting we designed a software tool that alternatively applies and evaluates diverse data adaptation approaches on such data. It supports several techniques: Optimal Transport theory (OT) (Perrot et al., 2016), Bridge mapping (BM) (either nearest-neighbors pseudo-pairs combined with linear regression (Cover & Hart, 1967) or Correlation Alignment for Deep Domain Adaptation (Sun et al., 2016)) and Empirical Bayes (EB) (Johnson et al., 2006). The techniques described above rely on matched sample pairs measured across the two experiments conducted in Finland and Italy. Here we systematically evaluate the performance of such approaches in relation to the number of matched sample pairs. Finally, to further compare methods and validate the software, we test the same techniques on additional datasets from freely available repositories.

Materials and methods

Study population

The Estonian Biobank (EstBB) constitutes a unique volunteer-based cohort, reflecting the demographic composition of the adult population in Estonia. The recruitment of participants was carried out on a nationwide scale through collaborative efforts involving general practitioners and medical personnel. This involved the collection of blood samples and medical histories from willing participants, as described in the seminal work by Leitsalu et al. (2015); Milani et al. (2025). The purpose of this recruitment initiative was to establish a foundation for subsequent omics studies, connecting data on genetic, environmental, and lifestyle factors in the context of health and disease.

In this study, EstBB data on 12,000 individuals had been measured using Nuclear Magnetic Resonance (NMR) spectroscopy at Nightingale Health OY, Finland. The dataset encompassed 4,641 samples from male participants (mean age 46.1 years) and 7,359 samples from female participants (mean age 46.5 years).

NMR spectroscopy for metabolite measurements

The sample measurements for the first dataset (), which were used for the training and validation of ML models, are from non-fasting plasma samples obtained from 12,000 volunteers from the Estonian Biobank. The samples were stored in liquid nitrogen and underwent proton NMR measurements on a 500 MHz spectrometer at Nightingale Health OY, Finland. Metabolite quantification was obtained using proprietary software (Julkunen et al., 2023). The dataset contained quantified concentrations of 168 different measurements in absolute levels, along with 81 diverse metabolite ratios, resulting in 249 biomarkers. The measurements span a wide array of biological components, encompassing lipoproteins, lipids, fatty acids, amino acids, ketone bodies, apolipoproteins, as well as various compounds related to glycolysis, fluid balance, and inflammation. Detailed information on experimental procedures, sample preparation, and the application of proton NMR spectroscopy for measurements of the first dataset () has been previously documented and published (Soininen et al., 2009; Huhtala et al., 2021; Würtz et al., 2017).

The second dataset (), used for evaluation of the machine learning models, comprised 295 non-fasting plasma samples from the Estonian Biobank. NMR measurements were performed on an independent 600 MHz spectrometer at the Istituto Italiano di Tecnologia (Genoa, Italy). Metabolite concentrations were quantified using the PULCON (Pulse Length-Based Concentration Determination) approach. A detailed description of the NMR acquisition parameters, sample preparation, spectral processing, and metabolite quantification workflow for dataset is provided in Supplementary Information Section titled "NMR methods".

Medical history analysis for target disease selection

In the context of our study focusing on cardiovascular diseases, various criteria were applied for the selection of codes from the International Classification of Diseases, Tenth Revision (ICD-10) (Organization, 2004). These criteria were for diagnoses made within 60 days after blood sample donation, to find biomarker data relevant to the onset of CVDs. In addition, individuals with a CVD diagnosis 10 years before their recruitment were excluded from the analysis. This constraint aimed to mitigate the potential impact of long-term pre-existing cardiovascular conditions and the effect of medications on the interpretation of the results. Furthermore, a limit for a minimum of 30 documented cases was set to enhance statistical robustness and ensure an adequate sample size for subsequent analyzes. No constraints were imposed on the presence of concomitant diseases, nor were limitations based on the age or sex of the participants. This was made to preserve the inclusivity of the study population, acknowledging the complex interplay of multiple factors in the onset and progression of cardiovascular diseases. Thus, the absence of such restrictions serves to enhance the generalized approach and robustness of our findings.

Data preparation and acquisition

The initial NMR dataset () contained 249 biomarkers generated using the primary analytical platform. For cross-platform comparison with , this set was restricted to metabolites that could be confidently identified and quantified in both analytical platforms. The inclusion of biomarkers was based on successful spectral assignment and quantification using Chenomx NMR Suite under the standard analysis workflow, without the application of additional targeted deconvolution procedures, custom metabolite fitting algorithms, or metabolite-specific quantification approaches. Metabolites that could not be unequivocally identified on both platforms were excluded from further analyzes. This selection process resulted in a final set of 11 metabolites that were available for direct cross-platform harmonization and agreement evaluation. For dataset , samples were prepared using automated pipetting and metabolite concentrations were generated using platform-specific proprietary quantification software. For dataset , samples were prepared manually and metabolite concentrations were quantified using Chenomx NMR Suite according to the procedure described in the Supplementary Information Section titled "Metabolite Quantification". The selection of metabolites was performed solely to establish a common set of compounds measurable across both platforms and was not based on downstream agreement metrics, correlation coefficients, or harmonization performance.

Evaluation of data harmonization methods

Unsupervised visualization of between-dataset structure

Principal component analysis (PCA) was used for the unsupervised visualization of the structure between datasets and was performed on the combined dataset () after applying each transformation method. Specifically, PCA projections were computed using data transformed with k = 0 (no matched samples) or k = 108 (all matched samples), while keeping dataset sizes fixed. The only difference between projections is whether matched-sample information was used during transformation.

Harmonization assessment of matched samples

To evaluate the performance of different data harmonization approaches between the 108 matched samples acquired by two NMR platforms, we utilized several complementary statistical metrics. Linear association was assessed using Pearson correlation coefficients, while monotonic association was evaluated using Spearman rank correlations. Mean differences between paired measurements were tested with paired t-tests, and distribution-free comparisons were performed using paired Wilcoxon signed-rank tests. The general distributional similarity between platforms was further examined using the Kolmogorov-Smirnov test. Agreement beyond simple correlation was quantified using the intraclass correlation coefficient (ICC), calculated using a two-way agreement model, and Lin’s concordance correlation coefficient (CCC), which simultaneously measures precision and accuracy. Additional measures included root mean square error (RMSE), mean absolute error (MAE), paired Cohen’s d effect size, and Bland-Altman statistics including mean bias and limits of agreement. ICC is widely recommended for assessing the reliability and interchangeability of quantitative measurements, while CCC provides a more stringent assessment by incorporating both correlation and systematic bias (Koo & Li, 2016; Lawrence & Lin, 1989). Bland-Altman analysis was used to quantify systematic differences and estimate the range within which most inter-platform measurement differences fell (Bland & Altman, 1986).

Machine learning for the prediction of cases and controls

The dataset was divided into training and validation partitions, with a 4-1 ratio and a five-fold cross-validation. The algorithms used in the training of ML models included Random Forest (RF) (Ho, 1995), Decision Trees (DT) (Wu et al., 2008), Support Vector Machine (SVM) (Cortes & Vapnik, 1995), Logistic Regression(LR) (Cox, 1958), Gaussian Naive Bayes (GNB) (Pedregosa et al., 2011), Stochastic Gradient Descent (SGD) (Robbins & Monro, 1951) and k-nearest neighbors (KNN) (Cover & Hart, 1967). Hyperparameter tuning was used for all models that supported it.

The results of the machine learning models were assessed using balanced accuracy (BA), which is the arithmetic mean of sensitivity and specificity, calculated using the formula: balanced accuracy = (sensitivity + specificity)/2, where the results can range from 0 (worst) to 1 (best). We used this metric as it is highly reliable and avoids the well known issues associated with accuracy for datasets with unbalanced classes. Additionally, the Matthews Correlation Coefficient (MCC score) was used, which is a metric for evaluating binary classification models. The values range from −1 to +1, where +1 is considered a perfect prediction, 0 is a random guess, and −1 is a complete inverse prediction. MCC is a valuable metric for imbalanced datasets as it takes all four confusion matrix values (TP, TN, FP, FN) into consideration for a balanced view of the model’s performance. A detailed description of the ML workflow can be found in the Supplementary Information Section titled "Machine Learning Workflow and Hyperparameter Optimization".

Sub-sampling-based evaluation of alignment performance

To quantify how many matched samples are required for reliable harmonization between the source dataset and the destination dataset , we conducted a sub-sampling-based experiment. A matched (paired) sample is defined as the same biological sample profiled in both and . We randomly sampled k ∈ {0,1,2,3,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75, 80,85, 90,95,100,105,108} matched pairs without replacement and used these pairs to estimate an alignment mapping operator, called , that projects samples from the destination domain into the source domain (where the original machine learned model resides). The case k = 0 corresponds to the non-aligned baseline, where no mapping is learned and is the identity transformation. For each value of k, the sub-sampling procedure was repeated for 30 independent trials to account for variability induced by random selection of matched pairs. For each value of k and for each trial, the datasets are explicitly redefined: k-matched samples are randomly selected to form the matched subsets (used for alignment), while all remaining samples constitute the subsets used for training and testing. Consequently, increasing k increases the number of matched samples available for alignment, while reducing the size of the remaining data. Importantly, all methods, including the raw baseline, are recomputed on these k-specific data splits. Therefore, the raw performance curve does not represent a fixed baseline, but reflects performance under varying levels of matched data availability. In each trial, the mapping was fitted using only the selected subset of matched samples and subsequently applied to all destination samples, producing the aligned dataset .

Machine learning models were trained exclusively on the source dataset and evaluated on the transformed destination dataset . For each method and each value of k, we computed summary statistics (e.g. mean BA and standard deviation) between trials using only converged results, as some alignment methods may fail or produce undefined predictions for very small values of k as expected (e.g. k = 1).

We evaluated both alignment-based approaches (e.g. Optimal Transport, Bridge mapping, and Empirical Bayes harmonization) and baseline strategies that do not take advantage of the matched nature of same samples (e.g. the normalization pre-processing method). This design allows for direct comparison between methods that explicitly leverage matched samples and those that do not, across varying levels of pairing information. This experimental framework enables systematic characterization of how predictive performance depends on the number of matched samples while accounting for stochastic variability due to subsampling and model fitting. We compared the performances of different harmonization methods using linear mixed-effects models and ANOVA. The details are given in the Supplementary Information Section titled "Harmonization methods comparison".

Pre-processing and harmonization

We considered several harmonization strategies for comparison. In the current implementation, classifiers are trained on the source-domain data, while the destination-domain data are recomputed according to each transformation framework before evaluation. Thus, the main difference between harmonization methods lies in how the destination data are transformed toward the source feature space. Raw and normalized baselines are included to quantify the effect of pre-processing alone. All harmonization methods are applied after a mandatory pre-processing step, which includes median imputation and min–max scaling. Thus, harmonization is not performed independently of normalization, but always operates on preprocessed (normalized) feature matrices.

All pre-processing steps, alignment models, and optional feature-selection procedures were fitted independently within each method. When matched reference samples were unavailable, transformations were estimated from the available source and destination reference distributions, from unmatched datasets. Such mappings are expected to be less reliable than mappings estimated from true matched references, but provide a practical way to address domain shift in cross-cohort and multi-platform metabolomics studies, where technical variation can obscure biological signal (Yu et al., 2024). Indeed, this setting is intrinsically less constrained and assumes comparable biological variation across domains; violations of this assumption may lead to removal of true biological signal or spurious alignment.

We evaluated the following harmonization strategies to make and more comparable between domains.

1. Raw (unaltered control) This mode provides a lower-bound benchmark against which preprocessing and harmonization effects can be assessed. Models are trained on and evaluated on without imputation, scaling, or harmonization.

2. Normalization (within-domain pre-processing) This constitutes the main pre-processing baseline in our experiments and does not require matched pairs. Numeric features are median-imputed per feature and rescaled to [0,1] using min–max scaling (default normalization; z-score scaling is not used in this pipeline) within each domain, i.e. separately for and . The source imputer and scaler are fitted on the source training data and then applied to the source samples used for modeling. For the destination domain, when matched reference samples are available, the destination imputer and scaler are fitted on the matched destination reference set and then applied to destination samples; otherwise, they are fitted on the available destination dataset. No cross-domain information is used at this stage. Importantly, this normalization step is also the preliminary pre-processing step for all subsequent harmonization techniques. Therefore, OT, Bridge mapping, and Empirical Bayes harmonization are applied to normalized feature matrices rather than directly to the raw metabolite intensity scale.

3. Optimal Transport (OT) Optimal Transport provides a principled framework for aligning distributions across domains (Perrot et al., 2016). At the implementation level, we learn a linear mapping using the Python Optimal Transport library (Flamary et al., 2021). When matched reference samples are available, the mapping is learned from the matched destination references to the matched source references and then applied to destination samples. When matched references are absent, the available unmatched source and destination datasets are used as reference distributions to estimate the mapping. Models are trained on normalized and evaluated on , where denotes destination data after transformation by the learned Optimal Transport mapping into the source feature space.

4. Bridge mapping Bridge mapping directly learns a transformation from destination features to source features. When one-to-one matched reference pairs are available, we use paired linear mapping based on ordinary least squares (OLS), fitting a regression model that predicts source features from their corresponding destination features. This provides supervised alignment between domains when exact sample correspondences are available, consistent with regression-based domain adaptation frameworks (Cortes & Mohri, 2011). When explicitly selected, nearest-neighbor (NN) pseudo-pairs are constructed when true matched samples are not available or are unreliable. Specifically, for each sample in the destination dataset, we identify its k-nearest neighbors in the source dataset in the normalized feature space. Nearest neighbors are computed using Euclidean distance. The corresponding pseudo-target for each destination sample is defined as either the closest source sample (k = 1 or the average of the k-nearest neighbors (k> 1), providing a more stable estimate. An ordinary least squares (OLS) regression mapping is then learned from destination samples to these pseudo-targets. A pair is considered unreliable when true matched samples are unavailable, when matched samples contain missing features after pre-processing, fail quality checks, or when the number of matched samples is too small to reliably estimate a regression mapping. In such cases, proxy pairing via nearest neighbors is used. As an alternative to pairwise mapping, we use Correlation Alignment (CORAL) (Sun et al., 2017; Sun & Saenko, 2016), which performs distribution-level alignment by matching second-order statistics. CORAL is selected when no reliable pairs can be constructed (either true or pseudo-pairs), and alignment must rely solely on distribution-level correction. In this context, a meaningful neighborhood structure refers to the presence of stable cross-domain nearest-neighbor relationships. Stability is assessed by the consistency of neighbor identities across different values of k and by the magnitude of nearest-neighbor distances relative to the overall cross-domain distance distribution. When nearest-neighbor assignments remain largely unchanged and distances are consistently among the smallest cross-domain distances, pseudo-pairing is considered reliable. Conversely, when neighbor assignments vary substantially with k or nearest-neighbor distances are not appreciably smaller than typical cross-domain distances, pseudo-pairing is deemed unreliable and CORAL is used instead.

5. Empirical Bayes (EB) harmonization (ComBat-style) ComBat uses an Empirical Bayes framework to model and correct feature-wise location and scale differences across batches or domains (Johnson et al., 2007; Behdenna et al., 2023; Zhang & Parmigiani, 2020). In our implementation, the source and destination reference samples are concatenated and treated as two batches. Before the Empirical Bayes adjustment, the features are standardized using the concatenated reference matrix. The ComBat-style procedure then estimates domain-specific mean and variance adjustments and shrinks these estimates across features, providing stabilized correction parameters. When matched reference samples are available, the adjustment is estimated on the concatenated matched source and matched destination reference datasets. In the absence of matched references, the available unmatched source and destination datasets can be used as reference batches. The implementation saves harmonized source and destination reference tables, including standardized harmonized data and inverse-transformed versions on the original scale. However, for predictive evaluation, classifiers are trained on normalized source training data, while destination reference/test data are evaluated after EB-style harmonization. For additional unmatched destination samples, the current implementation applies an approximate mean-shift correction based on the difference between the original and harmonized destination reference means, rather than re-estimating a full ComBat model on those samples. No biological covariates are passed to the ComBat-style adjustment in the current pipeline. Therefore, the EB results should be interpreted as unsupervised domain/batch harmonization rather than covariate-preserving ComBat adjustment.

All these methods are part of the accompanying developed software that is freely available at GitHub.

Testing the software on external datasets

To test the robustness of the developed software, Metabolomics Workbench (UCSD Metabolomics, 2026) was used to find independently measured datasets that have comparable data. The criteria for selecting comparable included the presence of cases and controls, same species, tissue, and metabolites with identical names. The developed software was used to train ML models on one dataset and tested on another dataset.

Results

Selection of ICD-10 codes for analysis

After applying ICD-10 filtering criteria that excluded individuals with diagnoses in the 10 years prior to recruitment and required that the first diagnosis occur within 60 days after blood donation, we selected a single condition for further analysis: I50.9 (heart failure, unspecified). The dataset consisted of 12000 samples, of which 45 were I50.9 cases.

Sample and biomarker selection for perspective NMR measurements

To test machine learning models on new data, 295 plasma samples from the Estonian Biobank were chosen employing identical criteria for disease selection as in the previous phase. A total of 197 control samples and 98 samples with an I50.9 diagnosis were selected for analysis using NMR spectroscopy. To account for the anticipated variations in measurement values between different NMR platforms, 108 duplicate samples were added that had been previously measured on the original NMR spectrometer and included in dataset . Specifically, 74 duplicate control samples and 34 samples with an I50.9 diagnosis. The metabolites that could be reliably measured for dataset and used in the analysis were glucose, lactate, citrate, alanine, glutamine, histidine, isoleucine, leucine, valine, phenylalanine and tyrosine.

Unsupervised visualization of between-dataset structure

To visualize the global structure and potential distributional differences between the source () and destination () datasets, we performed PCA on the union of and after applying each transformation method under two conditions: no matched samples (k = 0) and full matching (k = 108). We then compared several common pre-processing choices with OT, BM and EB (see Fig. 1). The samples were color-coded by the dataset of origin ( vs. ), and the fraction of variance explained by each principal component is reported on the plot axes. PCA revealed a clear separation between and for raw, normalized and EB data with 0 matched samples but showed clustering for OT and BM.

Fig. 1.

PCA plots of datasets (blue triangles) and (red circles) for raw data - A, normalized data - B, OT with 0 matched samples - C, OT with 108 matched samples - D, BM with 0 matched samples - E, BM with 108 matched samples - F, EB with 0 matched samples - G, EB with 108 matched samples - H. The fraction of variance explained by each principal component is reported on the plot axes

Taken together, these unsupervised projections motivate the use of explicit harmonization methods in subsequent analyzes, as standard single-dataset transformations do not eliminate the between-dataset structure observed in and .

ML model testing using 0 matched samples

The machine learning models were trained on dataset and tested on without specifying any samples for matching, resulting in 295 unmatched samples for testing. After 5-fold cross validation the highest mean balanced accuracy scores for the training dataset were equal for OT and Bridge-mapped transformations using logistic regression algorithm and achieving BA scores of 0.78 and MCC of 0.03. Model testing had the highest mean balanced accuracy score for SVM, normalized data transformation, with scores for BA 0.64 and MCC 0.31 followed by SVM, Bridge-mapped data with scores for BA 0.61 and MCC 0.28. SHAP values for SVM were: glucose 0.36, lactate 0.09, citrate 0.05, alanine 0.23, glutamine 0.16, histidine 0.19, isoleucine 0.50, leucine 0.24, valine 0.57, phenylalanine 0.59, tyrosine 0.05. All results are shown as a table in a separate file S1.

ML model testing using 108 matched samples

The machine learning models were trained on dataset and tested on after using all 108 samples for matching and data transformation, resulting in 186 unmatched samples for testing. After 5-fold cross validation the highest mean balanced accuracy scores for training dataset were equal for OT and Bridge-mapped transformations using logistic regression algorithm and achieving BA scores of 0.78 and MCC of 0.09. Model testing had the highest mean balanced accuracy score for SVM, normalized data transformation, with scores for BA 0.72 and MCC 0.46 followed by LR, Empirical Bayes transformed data, with scores for BA 0.705 and MCC 0.39. SHAP values for SVM were: glucose 0.40, lactate 0.09, citrate 0.05, alanine 0.28, glutamine 0.17, histidine 0.20, isoleucine 0.54, leucine 0.27, valine 0.62, phenylalanine 0.68, tyrosine 0.05. All results are shown as a table in a separate file S2.

Harmonization assessment of matched samples

To assess the harmonization of matched samples, multiple methods were used. OT consistently outperformed all other harmonization methods achieving scores for Pearson correlation coefficients 0.89, Spearman rank correlations 0.86, paired t-tests 1, paired Wilcoxon signed-rank tests 0.61, ICC 0.89, CCC 0.89, RMSE 0.03, MAE 0.01, paired Cohen’s d effect size and Bland-Altman statistics . The complete list of results are given as a table in a separate file S5.

Determining the number of matched pairs required

We quantified how out-of-domain performance changes as a function of the number of matched pairs used to estimate an alignment by plotting the balanced accuracy (BA) of a logistic regression classifier against the number of matched pairs k (Fig. 2). For each value of k ≤ 108, a random subset of k matched pairs were drawn across 30 iterations and used to estimate a mapping that projects destination samples into the source domain. The special case k = 0 corresponds to the unaligned baseline (). The classifier was trained exclusively on the source dataset and evaluated on the transformed destination dataset , with BA used as the primary performance metric. For alignment methods that explicitly require matched pairs (e.g. OT, Bridge mapping, and ComBat), performance depends on both the quality of the estimated mapping and the number of matched samples available for its estimation. In contrast, the raw and independently normalized baselines do not use matched-pair information when deriving pre-processing parameters and therefore would be expected to yield invariant performance if evaluated on an identical test set. However, in the present experimental design, the subsets of matched pairs selected at each iteration alter the composition of the remaining samples available for evaluation. Consequently, some variation in BA is observed even for the raw and normalized baselines. This variation should not be interpreted as an effect of additional matched pairs on normalization itself, but rather as a consequence of differences in the evaluation samples and other stochastic elements of the analysis pipeline, including random sub-sampling, cross-validation splits, and model fitting. Across the full range of k, incorporating matched pairs generally improved predictive performance relative to the unaligned baseline, indicating that even a limited number of matched samples can provide useful information for harmonization. Since matched pairs were randomly sub-sampled (at a fixed seed RS=0), BA exhibited variability at each value of k, and monotonic improvements were not guaranteed. Nevertheless, the overall trend was positive, with performance stabilizing as k increased and the estimated mappings became more reliable. The highest BA was observed for EB with logistic regression when all 108 matched pairs were used (; Fig. 2), supporting the practical value of incorporating matched samples while also suggesting diminishing returns at larger values of k.

Fig. 2.

Mean balanced accuracy scores with 95% confidence intervals for logistic regression using different harmonization methods and a different number of matched samples (x-axis). The harmonization techniques shown in the plot are: light brown squares - raw data, red circles - normalized data, pink triangles - OT, red diamond - BM, blue asterisk - EB

The SHAP values for logistic regression using EB harmonization with 108 matched samples were: glucose 0.31, lactate 0.05, citrate 0.04, alanine 0.44, glutamine 0.32, histidine 0.39, isoleucine 0.49, leucine 0.31, valine 0.46, phenylalanine 0.93, tyrosine 0.16. These results and other metrics can be found as a table in a separate file S6.

Testing the software on external datasets

To confirm the validity the developed software, external datasets from Metabolomics Workbench were used. The first set of studies focused on human gestational diabetes – ST001948 (Metabolomics Workbench, 2026c) and ST003671 (Metabolomics Workbench, 2026b). Both studies used liquid chromatography-mass spectrometry (LC-MS) in their experiments and data acquisition. The two datasets had 9 common metabolites including 5-Hydroxyindoleacetic acid, Creatine, Creatinine, Glycocholic acid, Hippuric acid, Hypoxanthine, Uracil, Uric acid and Uridine. The models were trained on ST001948 (33 controls and 34 cases) and tested on ST003671 (20 controls, 32 cases). Figure 3 is a collection of PCA plots for raw data and transformed datasets. The best ML results for raw data were using SGD with BA 0.63 and MCC 0.26. After data harmonization, the best results were for normalized data using DT with BA 0.71 and MCC 0.44. The results are included as a table in a separate file S3.

Fig. 3.

PCA plots of Metabolomics workbench datasets on human gestational diabetes – ST001948 (blue triangles) and ST003671 (red circles) using different harmonization methods. A - Raw data, B - Normalized data, C - OT, D - BM and E - EB. The fraction of variance explained by each principal component is reported on the plot axes

The second set of external datasets focused on mouse diabetes models – ST000075 (Metabolomics Workbench, 2026) and ST001253 (Metabolomics Workbench, 2026d). LC-MS was used in both studies for experiments and data acquisition. The two studies had 12 common metabolites CAR 16:0, CAR 18:0, CAR 18:2, Cholesterol, Oleoyl-EA, PC 16:0/16:0, SM 18:1;O2/16:0, SM 38:1;O2, SM 40:2;O2, PG 18:0/18:1, SM 34:2;O2, SM 42:2;O2. The models were trained on ST000075 (41 controls and 30 cases) and tested on ST001253 (8 controls, 7 cases). Figure 4 is a collection of PCA plots for raw data and transformed datasets. The best ML results on raw data were for LR with BA 0.67 and MCC 0.37. For harmonized data, the best ML results were for normalized data using NB with BA 0.92 and MCC 0.87. The results are included as a table in a separate file S4.

Fig. 4.

PCA plots of Metabolomics workbench datasets on mouse diabetes models – ST000075 (blue triangles) and ST001253 (red circles) using different harmonization methods. A - Raw data, B - Normalized data, C - OT, D - BM and E - EB. The fraction of variance explained by each principal component is reported on the plot axes

Since none of the datasets contained any matched samples, all methods were applied using harmonization methods that did not require paired observations, that is, samples measured in both datasets.

Discussion

In this study, we used NMR metabolomics data from plasma samples from the Estonian Biobank to predict the near-term onset of cardiovascular disease I50.9, unspecified hear failure, focusing on individuals without a cardiovascular disease diagnosis in the 10 years preceding biobank enrollment and with a first diagnosis within 60 days after recruitment. This design was motivated by the hypothesis that subtle metabolic alterations might already be detectable before clinical diagnosis and could therefore provide information on imminent disease onset. Previous studies have reported pre-diagnostic metabolic changes in cancer and cardiovascular conditions, including alterations in amino acids, biogenic amines, trimethylamine N-oxide, and choline (Li et al., 2022; Stepien et al., 2021). Early detection of such metabolic signatures may support risk stratification, prevention, and earlier clinical intervention.

The main methodological challenge addressed in this work was the transfer of predictive models across NMR datasets acquired under different analytical and experimental conditions. Measurements performed on different NMR platforms, especially when samples have been stored for extended periods, can differ due to metabolite degradation, sample preparation effects, instrument-specific variation, and differences in absolute quantification procedures (Giskeødegård et al., 2019; Jaggard et al., 2021). To address these discrepancies, we evaluated several harmonization strategies, including within-domain normalization, Optimal Transport, Bridge mapping, and Empirical Bayes harmonization. These approaches differ in how they use matched samples, where each pair corresponds to the same biological sample measured in both domains. The duplicated samples measured on both platforms were therefore intentionally included as reference samples for calibration and alignment.

Our results show that matched samples are highly valuable for cross-domain prediction. Across most values of k, the methods using matched reference information outperformed the unaligned baseline, indicating that even partial correspondence between the source and destination datasets can improve transferability. However, performance depended not only on the number of matched samples but also on the particular subset selected. This variability was most pronounced at small k, where alignment was unstable and some methods produced outputs with high variability. In particular, at k = 1, the cross-domain transformation is not sufficiently determined and cannot be estimated reliably, illustrating that paired-sample-based harmonization requires sufficient calibration data to produce a stable and robust mapping.

As k increased, performance became more stable and generally improved. When all available matched samples were used (k = 108), the evaluated methods reached their highest performance, with Empirical Bayes harmonization achieving the best balanced accuracy, followed by normalization and Bridge-based approaches. These results suggest that, given sufficient matched reference data, platform- and storage-related discrepancies can be substantially mitigated, allowing predictive performance on the destination dataset to approach that observed in the source-domain setting.

A notable finding is the strong performance of simple within-domain normalization. Although normalization does not explicitly learn a paired mapping between domains, it remained competitive across values of k and approached the performance of more complex alignment strategies when many matched samples were available.

This suggests that a substantial fraction of the domain shift can be addressed by robust pre-processing alone. More complex alignment methods may provide additional gains when enough matched reference samples are available, but their advantage is not automatic and depends on the stability of the estimated transformation.

These findings highlight a central trade-off in harmonization: reducing distributional differences between domains does not necessarily improve predictive performance. Alignment methods can improve source–destination overlap, but they may also distort or attenuate features that are important for classification. This is particularly relevant when alignment is based primarily on marginal distributions rather than class-conditional structure (Li et al., 2020; Phan et al., 2024; Xiao et al., 2021). In the context of OT, for example, global distribution matching may weaken feature–label relationships if class-discriminative structure is not preserved.

This point is also illustrated by the PCA visualizations. OT and Bridge mapping can improve the apparent overlap between source and destination samples in PCA space, but improved visual overlap does not necessarily imply improved classification. PCA identifies directions of maximal variance without using class labels and therefore does not guarantee preservation of class separability (Jolliffe & Cadima, 2016; Lever et al., 2017). Thus, harmonization should not be evaluated solely by visual domain overlap, but by its downstream ability to preserve or enhance discriminative biological signal. Using various scores for harmonization assessment (including Pearson correlation coefficients, Spearman rank correlations, paired t-tests, paired Wilcoxon signed-rank tests, ICC, CCC, RMSE, MAE, paired Cohen’s d effect size and Bland-Altman statistics.) showed that the matched samples achieve great similarity using OT and Bridge methods.

A related concern is overcorrection. Batch-correction and harmonization methods are designed to remove systematic technical variation, but if technical and biological factors are confounded, they may also remove biologically meaningful differences (Yu et al., 2024). This is especially important in metabolomics, where subtle intensity shifts may carry information relevant to disease. Our results therefore support a cautious view: the best harmonization method is not necessarily the one that maximizes distributional similarity but the one that best preserves predictive structure.

To further assess generalizability, we evaluated the harmonization framework on independent external datasets from Metabolomics Workbench (UCSD Metabolomics, 2026), including human gestational diabetes and mouse models of diabetes. These datasets did not contain matched paired samples. In this setting, models were trained on one dataset and evaluated on the other. Even without direct sample correspondence, harmonization improved balanced accuracy and Matthews correlation coefficient compared to raw, unaligned evaluation. These results suggest that harmonization can improve cross-domain metabolomics prediction beyond the specific Estonian Biobank/NMR setting, although alignment that does not use matched samples remains less constrained and should be interpreted with appropriate caution.

The biological relevance of the predictive models is supported by the metabolites selected during model training. Several selected features have previously been associated with cardiovascular disease or heart failure. Elevated glucose reflects hyperglycemia and cardiac risk related to diabetes (Yoshida et al., 2025); lactate has prognostic value in acute heart failure (Bosso et al., 2020); and amino acids such as alanine, glutamine, and branched-chain amino acids have been implicated in heart failure progression and metabolic dysfunction (Zhou et al., 2021; Kozhevnikova et al., 2025; McGarrah & White, 2023). Additional metabolites, including phenylalanine, tyrosine, citrate, and histidine, have also been linked to altered cardiac function and disease states (Cheng et al., 2021; Nemutlu et al., 2015; Wang et al., 2012). The recurrence of biologically plausible metabolites supports the interpretability of machine-learning models and suggests that predictive performance was not driven solely by technical artifacts.

Summarizing, this study demonstrates that harmonization can improve cross-cohort metabolomics prediction, particularly when matched reference samples are available. Simple normalization provides a strong and robust baseline whereas more sophisticated approaches such as EB, OT, and BM become more effective as the number and representativeness of matched samples increase.

In all paired iterations, the harmonization method had a significant effect on every measured aspect of test-set performance. OT consistently and significantly outperformed all other strategies on balanced accuracy, MCC, sensitivity, PPV, NPV, ROC-AUC, and PR-AUC, with BM as a strong second. EB offered little advantage over simple normalization for ranking-based metrics, and raw data retained the highest specificity at the cost of substantially lower sensitivity and balanced accuracy. These results support clear practical recommendations: when several matched samples are available, OT is a strong candidate. Whereas, if unmatched samples dominate or are the only available ones, simple normalization should be preferred.

Limitations

Several limitations of this study should be noted. First, the prevalence of ICD–10 code I50.9 (unspecified heart failure) in the training dataset was relatively limited, which may have constrained the achievable predictive performance. Increasing the number of positive cases could improve model calibration and enhance generalization, particularly in cross-cohort settings where class imbalance can further amplify performance variability.

Second, the effectiveness of the harmonization methods was strongly dependent on the number of matched samples and the method. Although performance improved consistently with increasing k, BM exhibited poor performance at very low values of k. This highlights an inherent limitation of alignment approaches that rely on paired data: a minimum level of correspondence is required to estimate stable transformations for some harmonization methods.

Third, the evaluation on external datasets was conducted in settings where no matched samples were available between cohorts. Although this allowed us to demonstrate that data harmonization can improve predictive performance even in fully unpaired scenarios, it also limited our ability to directly assess how harmonization methods based on matched samples generalize beyond the primary datasets. Future studies should include external datasets with known matched samples or standardized reference materials, such as quality control samples, which would allow a comprehensive evaluation of alignment strategies across different experimental setups.

The current implementation does not employ a fully nested cross-validation framework. Hyperparameter tuning is performed using an internal cross-validation procedure on the training data, after which model performance is estimated using stratified cross-validation and independent testing on matched destination (DM) and independent destination (DNM) datasets. Consequently, the reported cross-validation performance estimates may be slightly optimistic. However, the principal conclusions of the study are based on model performance in external destination-domain datasets, which provide a substantially less biased estimate of generalizability.

Finally, although harmonization improved predictive performance, the relationship between harmonization and classification accuracy remains complex. Some methods that achieved strong distributional alignment did not consistently improve classification outcomes, reflecting a trade-off between reducing domain shift and preserving discriminative biological signal. Further work is needed to develop alignment strategies that explicitly account for class structure and to better understand the conditions under which harmonization leads to meaningful improvements in predictive performance.

Conclusions

In this study, we investigated cross-cohort prediction of ICD–10 code I50.9 using a large NMR metabolomics cohort for model development and an independent cohort acquired later on a distinct NMR platform for external validation. To address temporal and platform-related distributional shifts, we compared several harmonization strategies, including within-domain normalization, Optimal Transport, Bridge mapping, and Empirical Bayes correction, leveraging matched reference samples when available.

Overall, harmonization improved out-of-domain performance relative to the unaligned baseline, particularly when matched samples were available. However, simple within-domain normalization remained a strong and often competitive baseline, sometimes matching or exceeding more complex alignment methods. This suggests that reducing distributional differences is not sufficient on its own: harmonization must also preserve class-discriminative and biologically meaningful structure.

Our results therefore highlight a practical trade-off in cross-cohort metabolomics modeling. Advanced alignment methods can be beneficial when they are well constrained by adequate reference data, but may introduce distortions when matched samples are limited or when global distribution alignment conflicts with predictive signal. The additional external datasets further support this view, showing that the best strategy depends on the relationship between technical domain shift and biological variation, rather than on methodological complexity alone.

In summary, domain harmonization can improve metabolomics model transferability, but its benefits should be evaluated against simpler preprocessing baselines. Overall the results support a clear practical recommendation: when several matched samples (and dominantly over unmatched ones) are available, OT is a strong candidate. Whereas if unmatched samples dominate or are the only available ones, simple within domain normalization should be preferred.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

AO - Writing—original draft, review & editing, Visualization, Validation, Software, Resources, Project administration, Investigation, Data curation, Conceptualization. LG - Writing—review & editing, Data curation. AA - Writing—review & editing, Supervision, Resources, Project administration, Data Curation, Conceptualization. SD - Writing—original draft, review & editing, Supervision, Software, Visualization, Validation, Methodology, Conceptualization.

Data availability

The data used in this project are available for researchers after the approval of the application by the Scientific Advisory Committee of the Estonian Biobank at the University of Tartu. The data from Metabolomics Workbench is freely available from their website https://www.metabolomicsworkbench.org/.

Declarations

Competing interests

The authors declare they have no known competing financial or personal relationships that could have influenced the work reported in this paper.

Ethical approval

All participating subjects have signed an informed consent form and the privacy rights of human subjects have been observed. The activities of the EstBB are regulated by the Human Genes Research Act, which was adopted in 2000 specifically for the operations of the EstBB. Individual level data analysis in the EstBB was carried out under ethical approval 1.1-12/15 from the Estonian Committee on Bioethics and Human Research (Estonian Ministry of Social Affairs), using data according to release application 3-10/GI-9133 from the Estonian Biobank.