Amanida: an R package for meta-analysis of metabolomics non-integral data

Abstract

Summary

The combination, analysis and evaluation of different studies which try to answer or solve the same scientific question, also known as a meta-analysis, plays a crucial role in answering relevant clinical relevant questions. Unfortunately, metabolomics studies rarely disclose all the statistical information needed to perform a meta-analysis. Here, we present a meta-analysis approach using only the most reported statistical parameters in this field: P-value and fold-change. The P-values are combined via Fisher’s method and fold-changes by averaging, both weighted by the study size (n). The amanida package includes several visualization options: a volcano plot for quantitative results, a vote plot for total regulation behaviours (up/down regulations) for each compound, and a explore plot of the vote-counting results with the number of times a compound is found upregulated or downregulated. In this way, it is very easy to detect discrepancies between studies at a first glance.

Availability and implementation

Amanida code and documentation are at CRAN and https://github.com/mariallr/amanida.

Supplementary information

Supplementary data are available at Bioinformatics online.

1 Introduction

The widespread of metabolomics as a potential tool for clinical diagnosis has increased the systematic reviews and meta-analysis on this topic. Meta-analysis is the statistical combination in a single estimate for results from primary studies answering the same question, which is a common practice in medical research. Typical protocols for meta-analysis only consider one metric to conduct the analysis, whether statistical significance or relative change. They require raw data [Metaboanalyst’ (Chong et al., 2018)] or statistics parameters like standard error, standard deviation or variance [R package ‘meta’ (Balduzzi et al., 2019)], rarely disclosed in metabolomics studies (Lee et al., 2020; Tofte et al., 2020). This causes that in some cases about half of the studies on systematic reviews are not suitable to be included in the meta-analysis (Guasch-Ferré et al., 2016; Pang et al., 2021). Although some standardizations have been proposed for the chemical analysis of metabolomics experiments (sample preparation, experimental analysis, quality control, metabolite identification and data pre-processing) (Sumner et al., 2007), clinical metabolomics needs standardized statistical reporting (Mutter et al., 2019) including how to process meta-analysis. Also, metabolomics public repositories, such as MetaboLights (Haug et al., 2020), do not disclose the statistics data for all studies. On this matter, some approaches have been developed when the studies to be included for meta-analysis do not disclose statistical data, such as vote-counting (Bushman and Wang, 2009), a qualitative estimate that takes into account only the trend of the compound. Some omics have high established protocols for meta-analysis, i.e. microarray in genomics, although the last improvements these methods cannot be adapted to metabolomics as they require the same genes or probes between studies and/or mean per group with standard deviation (Huo et al., 2020; Marot et al., 2009), which is not the case for metabolomics, due to the different analytical techniques used. In this omics, standards protocols only recommend to disclose the overall result of the univariate analysis, P-value and fold-change, without the need of reporting deviation or other metrics (Viant et al., 2019).

Public meta-analysis tools can only by applied to data with standard deviation or directly to raw data (see Supplementary Table S1). Currently there is no available methodology to do a meta-analysis based on studies that only disclose overall results. Amanida addresses the issue of combining overall results to perform meta-analysis based on statistical significance (P-value), relative change (fold-change) and study size. This approach increases the power of meta-analysis in metabolomics where the relative change is as important as the statistical significance (Sinclair and Dudley, 2019; Tolstikov et al., 2020). Estimates are weighted by study size to give more value to big studies where results are less overfitted or spurious (Rattray et al., 2018). Amanida R package also includes the option of performing a qualitative vote-counting plot, as many metabolomics studies will only report the relative change trend, and in this case, only a systematic review can be performed.

2 Statistical information

For significance evaluation using the statistic result P-value, we use the weighted P-values combination (Yoon et al., 2021), which is a variant of Fisher’s method (Fisher, 1925). A gamma distribution is used to assign non-integral weights proportional to study size to each P-value (1).

$${\mathrm{p}}_{\mathrm{combined}}=\mathrm{ }{\mathrm{P}}_{\mathrm{\Gamma }\left(\mathrm{n}\mathrm{ },\mathrm{ }2\right)\mathrm{ }}\left({\sum }_{\mathrm{i}}^{\mathrm{k}}{\mathrm{F}}_{\mathrm{X};{\mathrm{df}}_{\mathrm{i}},\mathrm{ }2}^{-1}\left({\mathrm{p}}_{\mathrm{i}}\right)\right),\mathrm{ }{\mathrm{df}}_{\mathrm{i}}=\frac{{\mathrm{N}}_{\mathrm{i}}}{{\sum }_{\mathrm{i}}^{\mathrm{k}}\mathrm{N}}\mathrm{ }\mathrm{n}$$ (1)

The fold-change is logarithmically transformed (base 2) to reduce skewness due to methodology (Curran-Everett, 2018), so that the variability is more homogeneous, and the distribution of the sample mean is consistent with a normal distribution. The logarithmically transformed fold-change values are averaged with weighting by study size (2).

$${\mathrm{FC}}_{\mathrm{combined}}=2^{\frac{\sum _{\mathrm{i}=1}^{\mathrm{k}}{\log }_2\left({\mathrm{FC}}_{\mathrm{i}}\right)\times {\mathrm{N}}_{\mathrm{i}}}{\sum _{\mathrm{i}=1}^{\mathrm{k}}{\mathrm{N}}_{\mathrm{i}}}}$$ (2)

Missing data are ignored, and negative values fold-change which stands for inverse comparison (control/case), are reversed (-1/value).

A qualitative analysis of the data can be performed with a vote-counting approach. Vote-counting comprises the general behaviour of the metabolites per study. As previously described (Bushman and Wang, 2009), votes are assigned as follows: value of +1 for compounds up-regulated, value of -1 for down-regulated, and 0 if no change in the behaviour is reported. Then, the total vote per compound is obtained summing the votes.

3 Software description

The Amanida R package allows a meta-analysis of metabolomics data, combining the results of different studies addressing the same question. The user provides the input data through text files (in txt, csv or xlsx format) containing the following information: identifier (previously curated by the user), P-value, fold-change, study size (N) and reference. Then amanida computes the quantitative and qualitative meta-analysis. Results are disclosed in two tables, one for the quantitative meta-analysis, with the global P-value and fold-change obtained, and one for the qualitative meta-analysis, with the vote-counting and number of articles.

Results can be graphically inspected via different plots: the volcano_plot where P-value and fold-change are plotted in logarithmic scale (see Fig. 1), for which the user can select the cut-off thresholds, labelling the selected compounds with their identifiers; the vote_plot where the vote-counting results are plotted per each compound; and the explore_plot where the vote-counting results are plotted against the total number of articles in which each compound is reported as upregulated or downregulated (see Fig. 2). All analysis can be obtained in a completely automatic manner using amanida_report function. To illustrate the package we have used a dataset from a urinary metabolomics meta-analysis study of colorectal cancer (Mallafré et al., 2021).

Fig. 1.

Amanida volcano plot. Quantitative meta-analysis results with a cut-off of 0.05 for P-value and 3 for fold-change. Data obtained from (Mallafré et al., 2021)

Fig. 2.

Amanida explore plot. Vote-counting results plotted against total number of articles divided by trend. Data obtained from (Mallafré et al., 2021)

4 Validation

To evaluate the package, we selected a metabolomics meta-analysis with data disclosed (Lee et al., 2020) which describes the association of metabolites with lung cancer risk. We could only use the information of amino acids concentration disclosed in six studies to compare both methodologies, as it was the only fully available (Supplementary Table S2). We must mention that to include the original six studies as in Lee et al. from one study the numeric values have been extracted directly from box-plots (Pietzke et al., 2019). The meta-analysis used on the reviews compares the weighted means between groups using the fixed model or random model if there is high heterogeneity (I² > 40%) between studies. The principal difference is that the fixed model assumes the effect in all studies is the same and the random model assumes that the effects between studies vary according to a normal distribution, used for convention. When there is high heterogeneity between studies with random models the studies are weighted more equally, this gives higher weight to small studies which is an unwanted approximation in metabolomics. Regarding results, Lee et al. found three statistically significant compounds associated with lung cancer risk: Methionine (I² = 86.0%), Proline (I² = 87.1%) and Tryptophan (I² = 83.6%). Following the same procedure, we have repeated the meta-analysis using the ‘Meta’ R package (Balduzzi et al., 2019). We obtained significant results (see Supplementary Figs) for Proline [random model (I^2 = 87.1%)]. Differences in the results are due to the difference in the box-plot estimations from (Pietzke et al., 2019).

We applied the amanida approach to the same data as Lee et al. for amino acids, where 20 of the 21 amino acids achieve statistical significance (P-value combined < 0.05, Supplementary Table S3), however the fold-change combined in all cases is smaller than 1.5, far from the threshold of 2 to consider a biological change. This means it might be a pattern in the amino acids of lung cancer patients, but the biological effect is small.

Evaluation of meta-analysis results includes multiple metrics including the multiple steps of processing applied compound by compound, while amanida results are obtained straightforward in a detailed report. As mentioned before, amanida has the advantage to work with P-values and fold-change, metrics more disclosed in metabolomics studies than the compound concentrations with their deviations. Also, it increases the readability of the results, classical meta-analysis includes multiple parameters to consider, such as heterogeneity, ranks or multiple models to obtain relevant results. We must say that Lee et al. considered as good metabolites, those with extremely high heterogeneity, and that Cochrane recommends not to do meta-analysis when considerable variation of results, i.e. I² is 75–100% (Higgins et al., 2019). The validation study used, was the only one that the authors found that reported both mean differences and standard deviations along with P-values and fold-changes.

5 Limitations

Applying amanida meta-analysis directly to the statistical estimates reduces the combination accuracy since without the raw data is not possible to know how much dispersion there is or how many outliers are included. Another drawback is found when the studies only disclose the statistical information for the significant compounds whereas preventing the correct combination of all results. Good practices in metabolomics suggest using a minimum of 50 samples per group to obtain relevant results, in a meta-analysis, these small studies are not recommended to be included due to the variability introduced, as we can observe with Proenza et al.’s (2003) and Yue et al.’s (2018) studies which have 14 and 20 participants per group, respectively. A common criticism for all meta-analyses described in Cochrane Handbook is that they ‘combine apples with oranges’ (Higgins et al., 2019), which have been also said for metabolomics as there is a wide range of techniques for compound detection or extraction methods with different sensibility that are combined. This limitation can be restricted on the systematic search, selecting only studies with the same technique and protocol, but it will reduce substantially the number of studies to combine.

6 Summary

\Amanida has been developed to deal with two issues, the few data disclosed in metabolomics and attribute different weights for the studies according to sample size. Standards metabolomics protocols basically recommend to disclose the overall result of the analysis, P-value and fold-change, without the need of reporting their means or deviation or other metrics (used in classical meta-analysis). In amanida, both overall statistical results are weighted according to the sample size, where larger studies will have more importance. Classical meta-analysis does not measure the strength of the combined result, only shows if the data has a pattern. To look for the strength we combine the statistical significance (P-value) with the effect (fold-change), which measures the quantity of change between groups. Amanida is the first approach to a meta-analysis with non-integral data.