Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 May 7.
Published in final edited form as: Nutr Metab Cardiovasc Dis. 2020 Jan 21;30(5):777–787. doi: 10.1016/j.numecd.2020.01.004

Serum Metabolites Associate with Lipid Phenotypes Among Bogalusa Heart Study Participants

Xiaoying Gu a,b,1, Changwei Li c, Jiang He a, Shengxu Li a, Lydia A Bazzano a, Jason M Kinchen d, Wei Chen a, Hua He a, Dongfeng Gu b, Tanika N Kelly a
PMCID: PMC7524581  NIHMSID: NIHMS1551677  PMID: 32131987

Abstract

Background and aims:

Dyslipidemia has been identified as a major risk factor for cardiovascular disease. We aimed to identify metabolites and metabolite modules showing novel association with lipids among Bogalusa Heart Study (BHS) participants using untargeted metabolomics.

Methods and Results:

Untargeted ultrahigh performance liquid chromatography-tandem mass spectroscopy was used to quantify serum metabolites of 1 243 BHS participants (816 whites and 427 African-Americans). The association of single metabolites with lipids was assessed using multiple linear regression models to adjust for covariables. Weighted correlation network analysis was utilized to identify modules of co-abundant metabolites and examine their covariable adjusted correlations with lipids. All analyses were conducted according to race and using Bonferroni-corrected α-thresholds to determine statistical significance.

Thirteen metabolites with known biochemical identities showing novel association achieved Bonferroni-significance, p < 1.04×10−5, and showed consistent effect directions in both whites and African-Americans. Twelve were from lipid sub-pathways including fatty acid metabolism (arachidonoylcholine, dihomo-linolenoyl-choline, docosahexaenoylcholine, linoleoylcholine, oleoylcholine, palmitoylcholine, and stearoylcholine), monohydroxy fatty acids (2-hydroxybehenate, 2-hydroxypalmitate, and 2-hydroxystearate), and lysoplasmalogens [1-(1-enyl-oleoyl)-GPE (P-18:1) and 1-(1enyl-stearoyl)-GPE (P-18:0)]. The gamma-glutamylglutamine, peptide from the gamma-glutamyl amino acid sub-pathway, were also identified. In addition, four metabolite modules achieved Bonferroni-significance, p < 1.39×10−3, in both whites and African-Americans. These four modules were largely comprised of metabolites from lipid sub-pathways, with one module comprised of metabolites which were not identified in the single metabolite analyses.

Conclusion:

The current study identified 13 metabolites and 4 metabolite modules showing novel association with lipids, providing new insights into the physiological mechanisms regulating lipid levels.

Keywords: metabolomics, lipids, total cholesterol, low-density lipoprotein cholesterol, high-density lipoprotein cholesterol, triglyceride, weighted correlation network analysis (WGCNA)

INTRODUCTION

As a key contributor to the development of atherosclerotic cardiovascular disease, dyslipidemia has been identified as a leading risk factor for morbidity and mortality globally.1–4 Alterations in lipid metabolism may occur prior to observed dyslipidemia, as assessed by traditionally measured lipid parameters.5 Identification of metabolic abnormalities preceding the diagnosis of dyslipidemia could enhance efforts for the prevention of this condition and its known cardiovascular disease sequelae.

Lipids are influenced by the complex interactions of genomic and environmental factors. While numerous studies have successfully elucidated genetic and behavioral risk factors for dyslipidemia6–9, the physiological mechanisms linking them to serum lipids remain poorly defined. Because the human metabolome integrates the end-product of both endogenous and exogenous processes, its study may provide a powerful tool to better understand the biological pathways underlying sub-optimal lipid profiles.10 Metabolites such as α-tocopherol, lactate, and pyruvate have already been demonstrated to associate with lipids levels in animal11,12 and epidemiologic studies13,14. Long-chain and branched-chain fatty acids may also be involved in the elevation of lipid levels.15 Although these findings are promising, limitations of past works include the small sample sizes employed and the sole use of targeted metabolomics approaches which are restricted to specific metabolites from pathways with presumed biological relevance.16,17 Research exploring the relation of the human metabolome to lipid phenotypes using an agnostic metabolite measurement strategy has yet to be conducted.18 Work in this area may not only provide novel biological insights but could also yield important prognostic information, allowing for the early prevention of dyslipidemia and its cardiovascular disease sequelae.

In the current study, we employed untargeted metabolomics to identify novel serum metabolites and metabolite modules associated with total cholesterol (TC), low-density lipoprotein cholesterol (LDL-C), high-density lipoprotein cholesterol (HDL-C), and triglycerides (TG). Our cross-sectional analysis leveraged carefully collected information on metabolites, lipids, and important covariables among a large, biracial sample of 1 243 (816 whites and 427 African-Americans) Bogalusa Heart Study (BHS) participants who took part in the 2013–2016 study visit.

METHODS

Study Population

The BHS is a community-based long-term study investigating the natural history of cardiovascular disease among a biracial sample (65% white and 35% African-American) of residents from Bogalusa, Louisiana, begun in 1973 by Dr. Gerald Berenson. From 1973 to 2016, 7 surveys were conducted in children and adolescents aged 4 to 17 years, and 10 surveys were conducted among adults aged 18 to 51 years who had been examined previously as children. The current BHS cohort includes 1 298 participants born between 1959 and 1979 who were screened at least 2 times during childhood and 2 times during adulthood for cardiovascular disease risk factors. Data and specimens collected in the recent 2013 to 2016 follow-up visit were leveraged in the current cross-sectional study of these participants. Among the 1 298 participants eligible for inclusion, those without metabolite (n = 37), lipid (n = 8) or covariable (n=10) data were excluded. A total of 1 243 participants remained for the analysis (95.8% response rate).

Informed consents were obtained from all the Bogalusa Heart Study participants after detailed explanation of the study. The study was approved by the Institutional Review Board at Tulane University.

Measurement of Metabolites

Untargeted, ultrahigh performance liquid chromatography-tandem mass spectroscopy (UPLC-MS/MS) was conducted by Metabolon© using BHS serum samples that had been stored at −80°C since the 2013 to 2016 visit.19 Rigorous quality assurance was conducted during measurement of metabolites which included the use of blanks, blind duplicates (5% of the BHS samples), and standard biochemical compounds which were curated into every analyzed sample. Compounds were identified by comparison to library entries of purified standards or recurrent unknown entities. The library was maintained based on authenticated standards that contains the retention time/index (RI), mass to charge ratio (m/z), and chromatographic data (including MS/MS spectral data) on all molecules present in the library. Furthermore, biochemical identifications are based on three criteria: retention index within a narrow RI window of the proposed identification, accurate mass match to the library +/− 10 ppm, and the MS/MS forward and reverse scores between the experimental data and authentic standards. The quality control and curation processes were designed to ensure accurate and consistent identification of true chemical entities, and to remove those representing system artifacts, mis-assignments, and background noise (Supplementary Methods).20 A normalization step was performed to correct variation resulting from instrument inter-day tuning differences as this study spanned multiple days. Each compound was corrected in run-day blocks by registering the medians to equal one and normalizing each data point proportionately. Every run day block contains 36 samples randomly selected from all the samples of study participants. Untargeted metabolomics resulted in the detection and quantification of 1 466 metabolites, with their pathway information pre-specified by Metabolon© (Supplementary Table 1). These included 1 055 known biochemical compounds in pathways related to amino acids (n=201), carbohydrates (n=25), cofactors and vitamins (n=35), energy (n=9), lipids (n=435), nucleotides (n=42), peptides (n=52), and xenobiotics (n=256), and 18 known biochemical compounds whose pathways have yet to be determined (referred to as ‘partially characterized molecules’). An additional 393 unnamed compounds representing a distinct biochemical entity were also quantified. These metabolites were labeled with an “X” followed by numbers (e.g., X-12345) and may be identified upon the eventual acquisition of a matching purified standard (or via classical structural elucidation analyses). Our metabolite identification procedure included matching data to a library using three variables, mass-to-charge ratio (m/z), retention index (RI) and msn scan, followed by review by a human curator, yielding high confidence the metabolite identification calls. Pathway and sub-pathway information was derived from the literature or from internal expertise at Metabolon Inc. (Durham, NC). Metabolite identification levels, presented in the Supplementary Table 1 as numbers 1 through 4, were determined according to the metabolomics standards initiative.21 The unnamed compounds may be identified upon the eventual acquisition of a matching purified standard (or via classical structural analysis). Additional information on metabolite identification and relative quantification is also provided in Supplementary Table 1.

Prior to the statistical analysis, additional quality control and manipulation of the metabolite data was undertaken. Batch effects were assessed using principle components analysis with scaled data across all run-days and by selecting a random sample of two different run days, which revealed no evidence of clustering of metabolite data by run-days (Supplementary Figure 1). Data filtering included the exclusion of 213 metabolites that were missing or below the detection threshold in more than 80% of samples and 51 metabolites with a reliability coefficient <0.3 based on blind duplicate samples. Blind duplicate samples were from 64 BHS study participants randomly selected from all participants. Among the 1 202 metabolites passing quality control, 167 were missing or below the detection threshold in 50% to 80% of the samples. Similar to previous analyses22, these metabolites were analyzed as ordinal variables after categorization into one of three mutually exclusive groups: 1) missing or below-the-detection-limit; 2) below the median; or 3) greater than or equal to the median. The remaining 1 035 metabolites were analyzed as continuous variables, where the minimum observed value was imputed for metabolites with missing or below-the-detection-limit values.

Measurement of Study Covariables and Phenotypes

Covariable and phenotype data were collected following stringent protocols that have been employed consistently at each clinical study visit.23 Questionnaires were administered to obtain information on demographic characteristics (including age, gender, race, and education), lifestyle risk factors (including cigarette smoking and alcohol consumption), and personal medical history. Education was classified as ≥12 years or <12 years of education received. Smoking and drinking status was classified as current, former, or never smokers or drinkers. Anthropometric measures were obtained by trained staff with participants in light clothing without shoes. During each visit, body weight and height were measured twice to the nearest 0.1 kg and 0.1 cm, respectively. The mean values of height and weight were used to estimate body mass index (BMI), which was calculated as weight in kilograms divided by height in square meters.

Participants were instructed to fast for 12 hours prior to the blood sample collection. Serum TC, HDL-C and TG levels were assayed using an enzymatic procedure as part of a lipid panel (Laboratory Corporation of America, Burlington, NC, USA).24,25 LDL-C was calculated using the Friedewald equation (LDL-C = TC - HDL-C - TG/5) for those with TG less than 400 mg/dl.26

Statistical Analysis

Characteristics of study participants were presented as means and standard deviations or median and interquartile range for continuous variables and as percentages for categorical variables. Differences between white and African-American participants were examined using t tests for continuous variables and χ2 tests for categorical variables.

Association of single metabolites with lipid phenotypes

Prior to association analyses, TG values were log-transformed to normalize their distribution. Multiple linear regression models were used to analyze the associations between each metabolite and lipid phenotype after adjustment for age, gender, cigarette smoking (current smoker, ever-smoker, or never-smoker), drinking (current, ever, or never drinker), education (≥12 years or <12 years of education received), BMI, and lipid lowering medication. All analyses were performed according to race. A stringent Bonferroni correction for testing 1 202 metabolites was employed, using an α-threshold of 1.04×10−5 (0.05/1 202/4) to determine statistical significance. An identified association was considered novel if the metabolite had not been reported previously for association with lipid phenotypes and was not in a sub-pathway that had been reported previously for association with lipid phenotypes. Robustly identified metabolites were those that achieved significance after Bonferroni correction in the mutually exclusive race groups. Sensitivity analyses were performed by excluding the participants who took lipid lowering medication. To assess the potential clinical relevance of our findings, identified metabolites were tested for association with carotid intima-media thickness (CIMT), a lipid related subclinical measure of atherosclerosis, among the BHS participants. These statistical analyses were performed with PROC GLM procedure in SAS (version 9.4; SAS Institute, Cary, NC). Data visualization techniques utilized the plotrix package in R (version 3.4.1).

Associations of metabolite modules with lipid phenotypes

To identify clusters of highly correlated serum metabolites among BHS participants, weighted correlation network analysis (WGCNA) was utilized.27 Unlike principal component analysis, this unsupervised data dimension reduction technique allows for dependency between components, which may more accurately represent the related biological pathways of identified metabolites.27,28 A description of WGCNA and its application to metabolomics studies has been reported previously.27,29 Briefly, the metabolite network was constructed as an adjacency matrix based on the weighted pairwise-correlations of all metabolites.30 Modules, defined as densely interconnected metabolites, were then identified from the network using an unsupervised hierarchical clustering approach.31 For each module, an eigenmetabolite was generated. This measure represents the module’s first principle component and can be interpreted as its weighted average metabolite value. Because preliminary analyses revealed similar metabolite clustering across race groups, metabolite modules were constructed using metabolite data for the 1 202 metabolites passing quality control among all study participants. To determine which biological pathways were best represented by each module, the metabolites most strongly correlated with each module’s eigenmetabolite (r>0.70) were identified, and the sub-pathways representing those metabolites were used to label each module.

Adjusted lipid phenotype measures were created using the residual values generated by regressing each lipid phenotype on age, gender, cigarette smoking, drinking, education, BMI, and lipid lowering medication. The correlations between each module (eigenmetabolite) and the adjusted lipid phenotypes were then estimated, separately, according to race. To correct for testing 9 serum metabolite modules (eigenmetabolites), a Bonferroni corrected α-threshold of 1.39×10−3 (0.05/9/4) was employed. Similar to the single metabolite analyses, robustly identified metabolite modules were those that achieved significance after Bonferroni correction in the mutually exclusive race groups. These analyses were performed using the WGCNA package in R (version 3.4.1).

RESULTS

Characteristics of the BHS metabolomics study participants are shown in Table 1. On average, whites were older and had lower BMI compared to African-Americans. They were also more likely to drink and have a high-school diploma and less likely to smoke compared to African-Americans. African-American BHS participants tended to have better lipid profiles, with lower TC, LDL-C, and TG and higher HDL-C as compared to white BHS participants.

Table 1.

Characteristics of Bogalusa Heart Study Participants

Whites (n=816) African-Americans (n=427) p value
mean (SD) or % 95% CI mean (SD) or % 95% CI
Age, y 48.52 (5.05) 38.62–58.42 47.48 (5.63) 36.45–58.51 0.002
Male 41.19 41.16–41.22 37.47 37.42–37.52 0.054
Current cigarette smoking 17.77 17.74–17.80 23.65 23.61–23.69 0.041
Current alcohol drinking 57.97 57.94–58.00 51.29 51.24–51.34 <0.001
≥ High school education 56.37 56.34–56.40 35.36 35.31–35.41 <0.001
BMI, kg/m2 30.36 (6.84) 16.95–43.77 33.49 (8.93) 15.99–50.99 <0.001
Lipid lowering medication 15.44 15.42–15.46 11.94 11.91–11.97 0.094
TC, mg/dl 195.46 (39.37) 118.29–272.63 188.46 (42.06) 106.02–270.90 0.004
LDL-C, mg/dl 116.40 (34.03) 49.70–183.10 111.97 (38.08) 37.33–186.61 0.046
HDL-C, mg/dl 50.32 (16.32) 18.33–82.31 53.78 (16.25) 21.93–85.63 <0.001
TG, mg/dl, median (Q1-Q3) 115.00 (83.50–173.00) NA 96.00 (68.00–140.00) NA <0.001
Open in a new tab

Data are presented as mean (standard deviation) or median (interquartile range) for continuous variables and as percentage for categorical variables.

BMI, body mass index; CI, confidence interval; HDL-C, high-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; NA, not applicable; SD, standard deviation; TC, total cholesterol; TG, triglyceride.

Association of single metabolites with lipid levels

After multivariable adjustment, a total of 347 metabolites (335 in whites and 226 in African-Americans, respectively) were associated with the lipid phenotypes after correcting for multiple tests (p < 1.04×10−5). These included 193, 126, 103, and 234 metabolites in whites, and 134, 77, 46, and 150 metabolites in African-Americans that were associated with TC, LDL-C, HDL-C, and TG, respectively (p < 1.04×10−5; Figure 1A-H and Supplementary Table 2). Two-hundred eleven of these metabolites (128, 75, 34, and 143 for TC, LDL-C, HDL-C, and TG, respectively) achieved Bonferroni corrected significance in both race groups (Supplementary Table 2). As expected, the majority of the 211 robustly identified metabolites (81%) were from the lipid super-pathway (Table 3). In addition, we identified 15 from the amino acid super-pathway, 4 from the carbohydrate super-pathway, 5 from the cofactors and vitamins super-pathway, 1 from the nucleotide super-pathway, 4 from the peptide super-pathway, and 12 from unknown super-pathways. There was considerable overlap of associations across lipid phenotypes (Supplementary Figure 2), with 5 metabolites shared by all lipid phenotypes (TC, LDL-C, HDL-C, and TG), 37 shared by 3 lipid phenotypes (8 by TC, LDL-C, and HDL-C; 1 by TC, HDL-C, and TG; 28 by TC, LDL-C, and TG), 80 shared by 2 lipid phenotypes (31 by TC and LDL-C, 6 by TC and HDL-C, 36 by TC and TG, 2 by LDL-C and TG, 5 by HDL-C and TG), and 13, 1, 9, and 66 specifically associated with TC, LDL-C, HDL-C, and TG, respectively. Exact beta estimates, standard errors, and p-values, along with AIC values, for each metabolite tested according to race and lipid phenotype are presented in Supplementary Table 3. Sensitivity analysis excluding those individuals who took lipid lowering medication showed similar results (Supplementary Figure 3).

Figure 1.

Figure 1.

Open in a new tab

Volcano plots of effect sizes versus –log10 p values for all 1202 metabolite associations with total cholesterol (A), low-density lipoprotein cholesterol (C), high-density lipoprotein cholesterol (E), and triglyceride (G) among whites and total cholesterol (B), low-density lipoprotein cholesterol (D), high-density lipoprotein cholesterol (F), and triglyceride (H) among African-Americans.

Table 3.

Number of metabolites associated with total and lipoprotein cholesterol in each sub-pathway among Bogalusa Heart Study participants

Super-pathway Sub-pathway Number of metabolites
TC LDL-C HDL-C TG Any
Lipid Ceramides 12 12 1 4 13
Diacylglycerol 10 3 3 11 11
Endocannabinoid 1 1 1 1
Fatty Acid Metabolism (Acyl Choline) 7 3 7
Fatty Acid Metabolism(Acyl Carnitine) 7 4 5 7
Fatty Acid, Dihydroxy 1 1 1
Fatty Acid, Keto 1 1 1
Fatty Acid, Monohydroxy 3 2 2 1 3
Lysophospholipid 13 5 3 22 24
Lysoplasmalogen 2 2
Monoacylglycerol 8 11 11
Phosphatidylcholine (PC) 12 7 3 16 18
Phosphatidylethanolamine (PE) 11 11
Phosphatidylinositol (PI) 4 1 7 7
Phospholipid Metabolism 1 1 1
Plasmalogen 1 9 4 9
Polyunsaturated Fatty Acid (n3 and n6) 4 3 6
Sphingolipid Metabolism 33 34 4 9 36
Sterol 1 1 1 1
Amino Acid Glutamate Metabolism 1 1
Glutathione Metabolism 1 1
Glycine, Serine and Threonine Metabolism 2 2
Leucine, Isoleucine and Valine Metabolism 8 8
Methionine, Cysteine, SAM and Taurine Metabolism 1 1
Phenylalanine Metabolism 1 1
Tyrosine Metabolism 1 1
Carbohydrate Glycolysis, Gluconeogenesis, and Pyruvate Metabolism 3 3
Pentose Metabolism 1 1
Cofactors and Vitamins Tocopherol Metabolism 1 1 1 1
Vitamin A Metabolism 2 2 1 4
Nucleotide Purine Metabolism, (Hypo)Xanthine/Inosine containing 1 1
Peptide Dipeptide 1 1 1
Fibrinogen Cleavage Peptide 2 1 2
Gamma-glutamyl Amino Acid 1 1
Unknown Unknown 3 3 1 11 12
Open in a new tab

Among the 211 robustly identified metabolites, 25 showed novel association. These metabolites included 13 with known structural identities (Table 2) and 12 unnamed metabolites (Supplementary Table 2). Of the 13 named metabolites, 12 were identified from four lipid sub-pathways, including: 7 involved in fatty acid metabolism (arachidonoylcholine, dihomo-linolenoyl-choline, docosahexaenoylcholine, linoleoylcholine, oleoylcholine, palmitoylcholine, and stearoylcholine); 3 monohydroxy fatty acids (2-hydroxybehenate, 2-hydroxypalmitate, and 2-hydroxystearate); and 2 lysoplasmalogens [1-(1-enyl-oleoyl)-GPE (P-18:1) and 1-(1-enyl-stearoyl)-GPE (P-18:0)]. In addition, gamma-glutamylglutamine, one peptide from the gamma-glutamyl amino acid sub-pathway was also identified. Six of the novel metabolites were significantly associated with more than one lipid trait. For example, TC, LDL-C, and HDL-C demonstrated respective increase of 9.59, 5.95, and 3.13 mg/dl among whites (p = 2.84×10−12, 1.28×10−6, and 1.94×10−10, respectively) and 13.95, 9.15, and 3.69 mg/dl among African-Americans (p = 6.19×10−12, 8.39×10−7, and 1.19×10−6, respectively) for each standard deviation increase of 2-hydroxypalmitate. Sensitivity analysis with log transformed data for novel named metabolites achieving significance in both White and African-American Bogalusa Heart study participants showed similar results (Supplementary Table 4).

Table 2.

Named Metabolites Showing Novel Association and Achieving Significance in Both White and African-American Bogalusa Heart Study Participants

Super-Pathway Sub-Pathway Metabolite Lipid Trait Whites African-Americans
β (SE) p Value β (SE) p Value
Lipid Fatty Acid Metabolism (Acyl Choline) arachidonoylcholine TC 12.01 (1.47) 1.09×10−15 9.86 (1.78) 5.06×10−8
HDL-C 2.68 (0.54) 6.93×10−7 3.36 (0.66) 5.34×10−7
dihomo-linolenoyl-choline TC 9.64 (1.28) 1.17×10−13 11.08 (2.30) 1.97×10−6
docosahexaenoylcholine TC 9.86 (1.35) 6.44×10−13 11.46 (2.03) 3.01×10−8
linoleoylcholine TC 10.47 (1.36) 3.94×10−14 10.71 (2.04) 2.43×10−7
HDL-C 2.98 (0.49) 1.90×10−9 3.74 (0.76) 1.07×10−6
oleoylcholine TC 11.54 (1.37) 1.40×10−16 9.88 (2.13) 4.78×10−6
palmitoylcholine TC 11.65 (1.38) 1.32×10−16 10.59 (1.98) 1.45×10−7
HDL-C 2.73 (0.50) 7.58×10−8 3.52 (0.73) 2.37×10−6
stearoylcholine TC 10.94 (1.37) 4.43×10−15 9.99 (1.94) 3.77×10−7
Fatty acid, Monohydroxy 2-hydroxybehenate TC 10.74 (1.50) 2.09×10−12 8.95 (1.75) 4.76×10−7
TGa 0.20 (0.02) 7.19×10−22 0.12 (0.02) 1.84×10−9
2-hydroxypalmitate TC 9.59 (1.35) 2.84×10−12 13.95 (1.97) 6.19×10−12
LDL-C 5.95 (1.22) 1.28×10−6 9.15 (1.83) 8.39×10−7
HDL-C 3.13 (0.48) 1.94×10−10 3.69 (0.75) 1.19×10−6
2-hydroxystearate TC 8.48 (1.39) 1.54×10−9 12.92 (1.94) 8.04×10−11
LDL-C 5.63 (1.24) 6.46×10−6 8.62 (1.79) 2.15×10−6
HDL-C 3.22 (0.49) 1.24×10−10 4.33 (0.72) 4.09×10−9
Lysoplasmalogen 1-(1-enyl-oleoyl)-GPE (P-18:1) HDL-C 2.36 (0.50) 3.20×10−6 4.18 (0.71) 7.31×10−9
1-(1-enyl-stearoyl)-GPE (P-18:0) HDL-C 2.44 (0.50) 1.10×10−6 4.20 (0.74) 2.53×10−8
Peptide Gamma-Glutamyl Amino Acid gamma-glutamylglutamine TGa −0.10 (0.02) 1.07×10−6 −0.11 (0.02) 4.50×10−6
Open in a new tab
a

Change of log-transformed TG per standard deviation increase of metabolite.

HDL-C, high-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; TC, total cholesterol; TG, triglyceride

Associations of metabolite modules with lipid levels

The 9 metabolite modules identified among BHS study participants are depicted in Figure 2. Detailed information on the metabolites most strongly correlated with each module’s eigenmetabolite are displayed in Supplementary Table 5. A heatmap showing the pairwise correlations of metabolites across modules is displayed in Supplementary Figure 4. Four and 6 of the modules significantly associated with lipid phenotypes after correction for multiple tests (p < 1.39×10−3) in whites and African-Americans, respectively (Figure 2). Four modules consistently associated with at least one lipid phenotypes across race groups, including 3, 2, 2, and 2 for TC, LDL-C, HDL-C, and TG, respectively. Among them, module 2, comprising metabolites from fatty acid metabolism (acyl choline) and lysophospholipid sub-pathways, was significantly and positively associated with all four lipid phenotypes (p = 7×10−39 and p = 2×10−15 for TC, p = 2×10−16 and p = 2×10−5 for LDL-C, p = 2×10−7 and p = 8×10−6 for HDL-C, and p = 3×10−16 and p = 2×10−15 for TG in whites and African-Americans, respectively). Module 1 [representing metabolites from lysophospholipid, phosphatidylethanolamine (PE), phosphatidylcholine (PC), monoacylglycerol, and diacylglycerol sub-pathways], module 3 [representing metabolites from long chain fatty acid, polyunsaturated fatty acid (n3 and n6), monohydroxy fatty acid, branched fatty acid, medium chain fatty acid, and endocannabinoid sub-pathways], and module 4 [representing metabolites from sphingolipid metabolism sub-pathway] were significantly and positively correlated with TC (p = 2×10−41 and p = 2×10−13 for module 1, p = 3×10−6 and p = 0.002 for module 3, and p = 4×10−83 and p = 7×10−38 for module 4 in whites and African-Americans, respectively). Modules 1, 3, and 4 also had significant, positive correlations with one additional lipid phenotype across race groups, with module 1 correlated with TG (p = 1×10−154 and p = 2×10−92 in whites and African-Americans, respectively), module 3 correlated with HDL-C (p = 3×10−4 and p = 3×10−4 in whites and African-Americans, respectively), and module 4 correlated with LDL-C (p = 2×10−90 and p = 7×10−41 in whites and African-Americans, respectively).

Figure 2.

Figure 2.

Open in a new tab

Correlations of metabolite modules with lipid phenotypes among whites (A) and African-Americans (B). HDL-C, high-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; TC, total cholesterol; TG, triglyceride. Color should be used for figures in print.

DISCUSSION

In the first untargeted metabolomics study of lipid phenotypes, we identified 211 metabolites which robustly associated with TC, LDL-C, HDL-C, or TG in both white and African-American participants. Our findings included 25 metabolites showing novel association, 13 with known and 12 with unknown biochemical identities, and 186 metabolites (or corresponding sub-pathways) identified by previous animal or epidemiologic studies. As expected, the majority of associated metabolites (81%) were from lipid-related sub-pathways. Among the 13 metabolites with known biochemical identities showing novel association, 12 were from lipid sub-pathways that included fatty acid metabolism, monohydroxy fatty acids, and lysoplasmalogens. The gamma-glutamylglutamine metabolites, one peptide from the gamma-glutamyl amino acid sub-pathway, were also identified. In addition, there were four metabolite modules significantly associated with at least one lipid phenotype in both whites and African-Americans. These four modules were largely comprised of metabolites from lipid sub-pathways, and included one module comprised of metabolites which were not identified in the single metabolite analyses. An additional 124 and 15 metabolites were associated with lipid phenotypes exclusively in whites and African-Americans, respectively. These findings will require replication in independent study populations. In aggregate, findings from the current study provided promising evidence for a role of human metabolome in dyslipidemia.

Among the 13 metabolites with known biochemical identities showing novel association, 10 were from fatty acid related lipid sub-pathways. Such findings might be unsurprising given the known relevance of dietary fatty acids on lipid profiles.32–35 The 10 fatty acid related metabolites included 7 involved in fatty acid metabolism (arachidonoylcholine, dihomo-linolenoyl-choline, docosahexaenoylcholine, linoleoylcholine, oleoylcholine, palmitoloelycholine, palmitoylcholine, and stearoylcholine) and three monohydroxy fatty acid metabolites (2-hydroxybehenate, 2-hydroxypalmitate, and 2-hydroxystearate). Although this study is the first to report associations of these metabolites with lipids, unique biological insights can be derived from previous studies exploring their biological functions. For example, arachidonoylcholine was indicated to have cholinomimetic activity similar to that of nicotine, a biochemical with known effects on lipid profile.36,37 In addition, oleoylcholine, palmitoloelycholine, and palmitoylcholine levels in liver tissue of male mice were previously shown to be influenced by dietary intake of green tea38. Since green tea has also been shown to associate with lipid traits, our data support a potential mediating effect of these metabolites on the green tea-lipid relation39–41. The multivariable adjusted results showed that the associations of 2-hydroxybehenate and 2-hydroxystearate with CIMT were significant after correction for multiple tests (0.05/13=0.038) (Supplementary Table 6). CIMT showed an increase of 0.03 mm and decrease of 0.03 mm for each standard deviation increase of 2-hydroxybehenate and 2-hydroxystearate, respectively (p = 0.002 and 0.003, respectively). The effect direction of 2-hydroxybehenate was consistent with that identified in analyses of lipid metabolites. Because the remaining metabolites have not been described previously, future research is needed to better elucidate their potential function in lipid metabolism.

In addition to fatty acid related metabolites, two novel lipid metabolites from the lysoplasmalogen sub-pathway [1-(1-enyl-oleoyl)-GPE (P-18:1) and 1-(1-enyl-stearoyl)-GPE (P-18:0)] and one novel peptide metabolite from the gamma-glutamyl amino acid sub-pathway (gamma-glutamylglutamine) were associated with the lipid phenotypes. The 1-(1-enyl-oleoyl)-GPE (P-18:1) and 1-(1-enyl-stearoyl)-GPE (P-18:0) metabolites remain relatively unstudied. In contrast, gamma-glutamylglutamine was previously associated with cardiometabolic phenotypes, including type 2 diabetes and chronic kidney disease progression.42,43 Given the correlation of these traits with lipid phenotypes, we conducted post-hoc analyses to determine if the observed association between gamma-glutamylglutamine and TG remained after further adjustment for change in estimated glomerular filtration rate over time and fasting plasma glucose. Post-hoc analyses revealed similar effect sizes and p-values to the original analyses (data not shown), suggesting that the observed associations are independent of kidney function and dysglycemia. Given the relatively robust associations of these 3 metabolites with lipid phenotypes in both whites and African-Americans, further functional studies are needed to follow-up on these unique biological insights.

WGCNA analysis revealed four metabolite modules consistently associated with lipid phenotypes in whites and African-Americans. These modules were comprised of metabolites from lipid sub-pathways, and aside from module 3, the findings were driven by metabolites already identified in the single metabolite analyses. The unique metabolites identified in module 3 were from lipid sub-pathways that included endocannabinoids along with long chain, polyunsaturated, monohydroxy, branched, and medium chain fatty acids. Except for monohydroxy fatty acid, these sub-pathways have all been reported previously to influence dyslipidemia. For example, the endocannabinoid system is a physiologic signaling system that plays an important role in regulating lipid metabolism.44 Previous clinical trials demonstrated improvement of lipid profiles upon administration of cannabinoid receptor blockers.45,46 Omega-3 polyunsaturated fatty acids, containing predominantly eicosapentanoic acid and docosahexanoic acid, may have complementary biological effects such as down-regulation of hepatic lipogenesis and enhancement of very-low-density lipoprotein lipolysis.47 Similarly, metabolites from long chain, medium chain, and branched fatty acids sub-pathways were previously reported to play a role in the regulation of lipid profiles.15,48,49 Medium chain fatty acid was reported to down-regulate key lipid-sensing genes such as liver X receptor-alpha in human liver cells with steatosis, and have positive effects on adipose triglyceride lipase and hormone-sensitive lipase. These results suggested that medium chain fatty acid might reduce lipid accumulation by regulating key lipid-sensing genes.49 In summary, findings from WGCNA analyses highlight the relevance of network-based methods for revealing novel insights into the biological pathways underlying lipid phenotypes which may be missed by more traditional single-metabolite analyses.

The current study has several important strengths. To our knowledge, this study of lipid phenotypes with untargeted metabolomics provided unbiased interrogation of the human metabolite repertoire in relation to this complex trait beyond previous findings about metabolites associated with lipid phenotypes. In addition, measurement of metabolites was conducted using a stringent study protocol and rigorous quality control procedures. Furthermore, this study leveraged the rich resources of the BHS, taking advantage of carefully collected measures of serum lipids and all other study covariables. Finally, the consistency of our findings, replicating 192 metabolites or sub-pathways implicated by previous studies, provides empirical evidence of the robustness of our results. Despite these advantages, several limitations of this study should also be addressed. Given the cross-sectional nature of the study, it is difficult to make causal inferences about the relationship between the identified metabolites and lipid traits. Prospective studies are needed to determine whether the identified metabolites are etiologically relevant in the development of dyslipidemia. Furthermore, an external replication sample was not available for this study. However, by employing stringent Bonferroni correction for multiple testing and requiring consistent, robust associations across mutually exclusive groups of African-American and white BHS participants, type 1 error should be minimized.

In the first untargeted metabolomics study of serum lipids, we provide compelling evidence for 25 metabolites and 4 metabolite modules showing novel association with these complex phenotypes. Metabolites with known biochemical structures showing novel association were primarily from fatty acid related lipid sub-pathways, and also included two metabolites from the lysoplasmologen lipid sub-pathway and one gamma-glutamyl amino acid peptides. Further studies to assess the temporal relevance of these metabolites to lipid levels is warranted. Among the four metabolite modules identified, one was comprised of unique metabolites not identified in the single marker analyses, which were from lipid sub-pathways that included endocannabinoids along with long chain, polyunsaturated, monohydroxy, branched, and medium chain fatty acids.

Supplementary Material

1
2

Highlights.

  • We used untargeted metabolomics profiling to find novel metabolites associated with lipids.

  • We found 25 novel metabolites and 4 metabolite modules associated with lipids.

  • Novel metabolites were primarily from fatty acid related lipid sub-pathways.

  • The other three novel metabolites were from the lysoplasmologen and gamma-glutamyl amino acid peptides sub-pathway.

  • Highlighted the potential power of WGCNA to identify novel metabolites and biological pathways.

Acknowledgments

SOURCES OF FUNDING

This research was supported by the National Institute on Aging of the NIH under award numbers R01AG041200 and R21AG051914. Research reported in this publication was partially supported by the National Institute of General Medical Sciences of the NIH under award number P20GM109036.

Footnotes

CONFILICT OF INTEREST

J.K. is employed by Metabolon, Inc., and he contributed to the logistics and optimization of the untargeted metabolomics and the untargeted metabolomics data interpretation. Metabolon, Inc. was not involved in the design of the study, statistical analysis, or interpretation of the results. The other authors declared no conflict of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

REFERENCES

  • 1.Collaborators GRF. Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks, 1990–2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet. 2016;388:1659–1724. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Farzadfar F, Finucane MM, Danaei G, Pelizzari PM, Cowan MJ, Paciorek CJ, et al. National, regional, and global trends in serum total cholesterol since 1980: systematic analysis of health examination surveys and epidemiological studies with 321 country-years and 3.0 million participants. Lancet. 2011;377:578–586. [DOI] [PubMed] [Google Scholar]
  • 3.Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Blaha MJ, et al. Heart disease and stroke statistics−−2014 update: a report from the American Heart Association. Circulation. 2014;129:e28–e292. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Third report of the National Cholesterol Education Program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III) final report. Circulation. 2002;106:3143–3421. [PubMed] [Google Scholar]
  • 5.Davis PH, Dawson JD, Blecha MB, Mastbergen RK, Sonka M. Measurement of aortic intimal-medial thickness in adolescents and young adults. Ultrasound Med Biol. 2010;36:560–565. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Lu X, Peloso GM, Liu DJ, Wu Y, Zhang H, Zhou W, et al. Exome chip meta-analysis identifies novel loci and East Asian-specific coding variants that contribute to lipid levels and coronary artery disease. Nat Genet. 2017;49:1722–1730. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Global Lipids Genetics C, Willer CJ, Schmidt EM, Sengupta S, Peloso GM, Gustafsson S, et al. Discovery and refinement of loci associated with lipid levels. Nat Genet. 2013;45:1274–1283. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Do R, Willer CJ, Schmidt EM, Sengupta S, Gao C, Peloso GM, et al. Common variants associated with plasma triglycerides and risk for coronary artery disease. Nat Genet. 2013;45:1345–1352. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Bayram F, Kocer D, Gundogan K, Kaya A, Demir O, Coskun R, et al. Prevalence of dyslipidemia and associated risk factors in Turkish adults. J Clin Lipidol. 2014;8:206–216. [DOI] [PubMed] [Google Scholar]
  • 10.Mayr M. Metabolomics: ready for the prime time? Circ Cardiovasc Genet. 2008;1:58–65. [DOI] [PubMed] [Google Scholar]
  • 11.Miao H, Zhao YH, Vaziri ND, Tang DD, Chen H, Chen H, et al. Lipidomics biomarkers of diet-induced hyperlipidemia and its treatment with Poria cocos. J Agric Food Chem. 2016;64:969–979. [DOI] [PubMed] [Google Scholar]
  • 12.Kim DY, Kim J, Ham HJ, Choue R. Effects of d-alpha-tocopherol supplements on lipid metabolism in a high-fat diet-fed animal model. Nutr Res Pract. 2013;7:481–487. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Illison VK, Rondo PH, de Oliveira AM, D’Abronzo FH, Campos KF. The relationship between plasma alpha-tocopherol concentration and vitamin E intake in patients with type 2 diabetes mellitus. Int J Vitam Nutr Res. 2011;81:12–20. [DOI] [PubMed] [Google Scholar]
  • 14.Marshall MW, Iacono JM, Wheeler MA, Mackin JF, Canary JJ. Changes in lactate dehydrogenase, LDH isoenzymes, lactate, and pyruvate as a result of feeding low fat diets to healthy men and women. Metabolism. 1976;25:169–178. [DOI] [PubMed] [Google Scholar]
  • 15.Mika A, Stepnowski P, Kaska L, Proczko M, Wisniewski P, Sledzinski M, et al. A comprehensive study of serum odd- and branched-chain fatty acids in patients with excess weight. Obesity. 2016;24:1669–1676. [DOI] [PubMed] [Google Scholar]
  • 16.Menni C, Zierer J, Valdes AM, Spector TD. Mixing omics: combining genetics and metabolomics to study rheumatic diseases. Nat Rev Rheumatol. 2017;13:174–181. [DOI] [PubMed] [Google Scholar]
  • 17.Roberts LD, Souza AL, Gerszten RE, Clish CB. Targeted metabolomics. Curr Protoc Mol Biol. 2012;Chapter 30:Unit 30.32.31–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Vinayavekhin N, Saghatelian A. Untargeted metabolomics. Curr Protoc Mol Biol. 2010;Chapter 30:Unit 30.31.31–24. [DOI] [PubMed] [Google Scholar]
  • 19.Evans AM, DeHaven CD, Barrett T, Mitchell M, Milgram E. Integrated, nontargeted ultrahigh performance liquid chromatography/electrospray ionization tandem mass spectrometry platform for the identification and relative quantification of the small-molecule complement of biological systems. Anal Chem. 2009;81:6656–6667. [DOI] [PubMed] [Google Scholar]
  • 20.Dehaven CD, Evans AM, Dai H, Lawton KA. Organization of GC/MS and LC/MS metabolomics data into chemical libraries. J Cheminform. 2010;2:9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Sumner LW, Amberg A, Barrett D, Beale MH, Beger R, Daykin CA, et al. Proposed minimum reporting standards for chemical analysis Chemical Analysis Working Group (CAWG) Metabolomics Standards Initiative (MSI). Metabolomics. 2007;3:211–221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Zheng Y, Yu B, Alexander D, Mosley TH, Heiss G, Nettleton JA, et al. Metabolomics and incident hypertension among blacks: the atherosclerosis risk in communities study. Hypertension. 2013;62:398–403. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Foster TA, Berenson GS. Measurement error and reliability in four pediatric cross-sectional surveys of cardiovascular disease risk factor variables--the Bogalusa Heart Study. J Chronic Dis. 1987;40:13–21. [DOI] [PubMed] [Google Scholar]
  • 24.Allain CC, Poon LS, Chan CS, Richmond W, Fu PC. Enzymatic determination of total serum cholesterol. Clin Chem. 1974;20:470–475. [PubMed] [Google Scholar]
  • 25.Bucolo G, David H. Quantitative determination of serum triglycerides by the use of enzymes. Clin Chem. 1973;19:476–482. [PubMed] [Google Scholar]
  • 26.Friedewald WT, Levy RI, Fredrickson DS. Estimation of the concentration of low-density lipoprotein cholesterol in plasma, without use of the preparative ultracentrifuge. Clin Chem. 1972;18:499–502. [PubMed] [Google Scholar]
  • 27.Langfelder P, Horvath S. WGCNA: an R package for weighted correlation network analysis. BMC Bioinformatics. 2008;9:559. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Worley B, Powers R. Multivariate analysis in metabolomics. Curr Metabolomics. 2013;1:92–107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Pedersen HK, Gudmundsdottir V, Nielsen HB, Hyotylainen T, Nielsen T, Jensen BA, et al. Human gut microbes impact host serum metabolome and insulin sensitivity. Nature. 2016;535:376–381. [DOI] [PubMed] [Google Scholar]
  • 30.Yip AM, Horvath S. Gene network interconnectedness and the generalized topological overlap measure. BMC Bioinformatics. 2007;8:22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Langfelder P, Zhang B, Horvath S. Defining clusters from a hierarchical cluster tree: the Dynamic Tree Cut package for R. Bioinformatics. 2008;24:719–720. [DOI] [PubMed] [Google Scholar]
  • 32.Montoya MT, Porres A, Serrano S, Fruchart JC, Mata P, Gerique JA, et al. Fatty acid saturation of the diet and plasma lipid concentrations, lipoprotein particle concentrations, and cholesterol efflux capacity. Am J Clin Nutr. 2002;75:484–491. [DOI] [PubMed] [Google Scholar]
  • 33.Glatz JF, Luiken JJ, Bonen A. Membrane fatty acid transporters as regulators of lipid metabolism: implications for metabolic disease. Physiol Rev. 2010;90:367–417. [DOI] [PubMed] [Google Scholar]
  • 34.Richter M, Baumgartner J, Wentzel-Viljoen E, Smuts CM. Different dietary fatty acids are associated with blood lipids in healthy South African men and women: the PURE study. Int J Cardiol. 2014;172:368–374. [DOI] [PubMed] [Google Scholar]
  • 35.Eslick GD, Howe PR, Smith C, Priest R, Bensoussan A. Benefits of fish oil supplementation in hyperlipidemia: a systematic review and meta-analysis. Int J Cardiol. 2009;136:4–16. [DOI] [PubMed] [Google Scholar]
  • 36.Chattopadhyay K, Chattopadhyay BD. Effect of nicotine on lipid profile, peroxidation & antioxidant enzymes in female rats with restricted dietary protein. Indian J Med Res. 2008;127:571–576. [PubMed] [Google Scholar]
  • 37.Bezuglov VV, Zinchenko GN, Nikitina LA, Buznikov GA. [Arachidonoylcholine and N,N-dimethylaminoethyl arachidonate are new cholinergic compounds]. Russian Journal of Bioorganic Chemistry. 2001;27:200–203. [DOI] [PubMed] [Google Scholar]
  • 38.Luo T, Miranda-Garcia O, Adamson A, Hamilton-Reeves J, Sullivan DK, Kinchen JM, et al. Consumption of walnuts in combination with other whole foods produces physiologic, metabolic, and gene expression changes in obese C57BL/6J high-fat-fed male mice. J Nutr. 2016;146:1641–1650. [DOI] [PubMed] [Google Scholar]
  • 39.Hunter PM, Hegele RA. Functional foods and dietary supplements for the management of dyslipidaemia. Nat Rev Endocrinol. 2017;13:278–288. [DOI] [PubMed] [Google Scholar]
  • 40.Zheng XX, Xu YL, Li SH, Liu XX, Hui R, Huang XH. Green tea intake lowers fasting serum total and LDL cholesterol in adults: a meta-analysis of 14 randomized controlled trials. Am J Clin Nutr. 2011;94:601–610. [DOI] [PubMed] [Google Scholar]
  • 41.Koo SI, Noh SK. Green tea as inhibitor of the intestinal absorption of lipids: potential mechanism for its lipid-lowering effect. J Nutr Biochem. 2007;18:179–183. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Yu D, Moore SC, Matthews CE, Xiang YB, Zhang X, Gao YT, et al. Plasma metabolomic profiles in association with type 2 diabetes risk and prevalence in Chinese adults. Metabolomics. 2016;12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Shah VO, Townsend RR, Feldman HI, Pappan KL, Kensicki E, Vander Jagt DL. Plasma metabolomic profiles in different stages of CKD. Clin J Am Soc Nephrol. 2013;8:363–370. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Deedwania P. The endocannabinoid system and cardiometabolic risk: effects of CB1 receptor blockade on lipid metabolism. Int J Cardiol. 2009;131:305–312. [DOI] [PubMed] [Google Scholar]
  • 45.Pi-Sunyer FX, Aronne LJ, Heshmati HM, Devin J, Rosenstock J. Effect of rimonabant, a cannabinoid-1 receptor blocker, on weight and cardiometabolic risk factors in overweight or obese patients: RIO-North America: a randomized controlled trial. JAMA. 2006;295:761–775. [DOI] [PubMed] [Google Scholar]
  • 46.Despres JP, Golay A, Sjostrom L. Effects of rimonabant on metabolic risk factors in overweight patients with dyslipidemia. N Engl J Med. 2005;353:2121–2134. [DOI] [PubMed] [Google Scholar]
  • 47.Davidson MH, Benes LB. The future of n-3 polyunsaturated fatty acid therapy. Curr Opin Lipidol. 2016;27:570–578. [DOI] [PubMed] [Google Scholar]
  • 48.Nakamura MT, Yudell BE, Loor JJ. Regulation of energy metabolism by long-chain fatty acids. Prog Lipid Res. 2014;53:124–144. [DOI] [PubMed] [Google Scholar]
  • 49.Wang B, Fu J, Li L, Gong D, Wen X, Yu P, et al. Medium-chain fatty acid reduces lipid accumulation by regulating expression of lipid-sensing genes in human liver cells with steatosis. Int J Food Sci Nutr. 2016;67:288–297. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1551677-supplement-1.xlsx (861.4KB, xlsx)
2
NIHMS1551677-supplement-2.docx (6MB, docx)

RESOURCES