Abstract
Background:
Glyphosate is a widely used herbicide in global agriculture. Glyphosate and its primary environmental degradate, aminomethyl phosphonic acid (AMPA), have been shown to disrupt endocrine function and induce oxidative stress in in vitro and animal studies. To our knowledge, these relationships have not been previously characterized in epidemiological settings. Elevated urinary levels of glyphosate and AMPA may be indicative of health effects caused by previous exposure via multiple mechanisms including oxidative stress.
Methods:
Glyphosate and AMPA were measured in 347 urine samples collected between 16–20 weeks gestation and 24–28 weeks gestation from pregnant women in the PROTECT birth cohort. Urinary biomarkers of oxidative stress, comprising 8-isoprostane-prostaglandin-F2α (8-iso-PGF2α), its metabolite 2,3-dinor-5,6-dihydro-15-F2t-isoprostane (8-isoprostane metabolite) and prostaglandin-F2α (PGF2α), were also measured. Linear mixed effect models assessed the association between exposures and oxidative stress adjusting for maternal age, smoking status, alcohol consumption, household income and specific gravity. Potential nonlinear trends were also assessed using tertiles of glyphosate and AMPA exposure levels.
Results:
No significant differences in exposure or oxidative stress biomarker concentrations were observed between study visits. An interquartile range (IQR) increase in AMPA was associated with 9.5% (95% CI: 0.5%–19.3%) higher 8-iso-PGF2α metabolite concentrations. Significant linear trends were also identified when examining tertiles of exposure variables. Compared to the lowest exposure group, the second and third tertiles of AMPA were significantly associated with 12.8% (0.6%–26.5%) and 15.2% (1.8%–30.3%) higher 8-isoprostane metabolite, respectively. An IQR increase in glyphosate was suggestively associated with 4.7% (−0.9%–10.7%) higher 8-iso-PGF2α.
Conclusions:
Urinary concentrations of the main environmental degradate of glyphosate, AMPA, were associated with higher levels of certain oxidative stress biomarkers. Associations with glyphosate reflected similar trends, although findings were not as strong. Additional research is required to better characterize the association between glyphosate exposure and biomarkers of oxidative stress, as well as potential downstream health consequences.
Keywords: Glyphosate, AMPA, oxidative stress, birth cohort
1. INTRODUCTION
Glyphosate is the most widely used herbicide in the global agriculture industry. In the U.S. alone, its use has increased 250-fold since the 1970’s (Benbrook, 2016). The increase is largely credited to its use as the main active ingredient in the product “Roundup” and the introduction of genetically modified or “Roundup-ready” crops in 1996 (Dill et al., 2010). In 2000, following the expiration of its patent, glyphosate prices greatly diminished and led to its present-day use in nearly 90% of corn, soybean and cotton crops in the U.S. (Bonny, 2016; Duke, 2018). Glyphosate based herbicides (GBH) are additionally being sold under various names by other manufacturers. Two-thirds of the total volume of glyphosate applied in the U.S. from 1974 to 2014 has been applied in the past 10 years (Benbrook, 2016). The widespread use of glyphosate has led to its detection in soil, air and water sources in the environment (Battaglin et al., 2014; Chang et al., 2011). Beginning in 2023, Bayer will be replacing glyphosate-based products in the residential sector in the US with formulations containing alternative active ingredients. Unfortunately, other manufacturers may not follow suit, and Bayer plans no changes to the availability in the professional and agricultural markets.
The environmental half-life of glyphosate may range from a few days to 91 days (Borggaard and Gimsing, 2008; Vereecken, 2005). In the environment only, glyphosate is broken down into its primary environmental degradate, aminomethyl phosphonic acid (AMPA) following exposure to microorganisms found in soil and water (Chaufan et al., 2014; Shushkova et al., 2009; Singh and Singh, 2016; Zhang et al., 2015). Exposure to glyphosate and AMPA in animals and humans has led to their detection in feces and urine samples (Niemann et al., 2015; von Soosten et al., 2016). A previous study assessing urinary adjustment methods reported an average half-life of 7.25 hours (95% CI: 5.5–9 hours) for glyphosate in humans (Connolly et al., 2019). Similarly, a recent study that examined oral exposure to glyphosate reported a half-life of approximately 9.05 hours (Zoller et al., 2020). AMPA may exhibit more toxic properties compared to glyphosate in the environment, and has been shown to be more persistent with an environmental half-life ranging from 76 to 240 days (Sun et al., 2019; Torretta et al., 2018).
The primary function of glyphosate is to inhibit antioxidant enzyme activity (in particular the enzyme 5-enolpyruvylshikimate-3-phosphate synthase (EPSPS)) in the shikimate pathway causing a reduction in photosynthesis. Glyphosate decreases photosynthesis by elevating chlorophyll degradation, while AMPA accomplishes the same decrease in photosynthesis by disturbing chlorophyll biosynthesis. Both processes lead to increases in reactive oxygen species (ROS) which results in physiological dysfunction, cell death (Duke, 2018), and necrosis of foliage (Gomes et al., 2016). The absence of both the shikimate pathway and EPSPS in humans is widely viewed as the primary basis for the lack of acute glyphosate toxicity reported in mammals, amphibians and reptiles (Duke, 2018; McComb et al., 2008; Williams et al., 2012). However, a growing body of evidence suggests that there may be chronic effects of exposure to even low levels of glyphosate.
Recent in vitro and in vivo animal studies have shown there may be potential negative effects on reproductive health as a result of exposure to glyphosate and other GBH. For example, repeated exposure to low doses (5 mg/kg) of GBH (Bretmont Wipeout) negatively impacted fertility in male mice (Abarikwu et al., 2015). Male rats exposed to higher levels of glyphosate (50–100 mg/kg) have shown decreases in spermatid counts and increases in abnormal sperm morphology (Nardi et al., 2017). Meanwhile, Perego et al. (2017) detected potential endocrine disruption in cell cultures of cattle ovaries following exposure to varying doses of glyphosate and glyphosate in formulation (e.g. Roundup) (1, 10, and 300 mg/L), and noted that glyphosate in formulation had more potent effects than glyphosate alone (Perego et al., 2017). Further studies have shown that exposure to Roundup in rat (500 mg/kg), and crab (0.01 and 0.2 mg/L) animal models had an overall negative impact on ovary function (Canosa et al., 2018; De Almeida et al., 2018). Moreover, Roundup (2 mg/kg/day), administered either orally or subcutaneously, has been shown to negatively affect ovarian follicular dynamics and gene expression, as well as the overall proliferative activity of the ovaries and uterus in neonatal lambs (Alarcón et al., 2019).
Recent animal models have also provided supportive evidence that exposure to glyphosate and AMPA may induce oxidative stress. Specifically, glyphosate exposure during early stages of rat development has been shown to increase oxidative stress markers in the brain (Gallegos et al., 2018). Several studies have also reported negative effects on levels malondialdehyde, a biomarker of oxidative stress, in chicken and rat offspring following glyphosate exposure (Fathi et al., 2019; Turkmen et al., 2019). Increases in oxidative stress following glyphosate exposure has also been observed in both zebrafish (Velasques et al., 2016) and the human renal proximal tubule cell line (Gao et al., 2019).
Increases in oxidative stress related to GBH exposure may lead to adverse pregnancy outcomes in humans, such as preterm birth. Previously, average levels of 8-iso-PGF2α, a sensitive biomarker of oxidative stress, were associated with increased odds of spontaneous preterm birth (Ferguson et al., 2015). Earlier studies have indicated that placental oxidative stress may elicit increased risk of preeclampsia which may lead to preterm birth (Aouache et al., 2018). Additionally, increased oxidative stress during pregnancy has the potential to alter signaling changes in the cervix causing shortened or spontaneous labor (Sanders et al., 2015; Venkatesh et al., 2016). Increased levels of oxidative stress may also lead to disruptions in placental protein synthesis and nutrient transport during pregnancy causing fetal growth restriction that may lead to preterm birth (Burton et al., 2009; Jansson and Powell, 2007).
Despite evidence from in vitro and animal models suggesting glyphosate exposure associations with adverse health effects, few studies have evaluated its effect and/or potential mechanism of action in humans. Given this lack of evidence and the potential for oxidative stress to negatively impact pregnancy outcomes, further study characterizing the relationship between glyphosate exposure and oxidative stress is needed. Therefore, the aim of this study was to determine the associations between glyphosate and biomarkers of oxidative stress in pregnant women.
2. METHODS
2.1. Study Population
Participants included in this study were part of the PROTECT birth cohort, which examines the effects of environmental exposures on adverse pregnancy outcomes such as preterm birth. Information on study methods have been described previously (Cantonwine et al., 2014). In brief, study participants were comprised of pregnant women residing in the Northern karst region of Puerto Rico between 2012 to 2017. Women were recruited from one of seven hospitals and prenatal clinics at 14 +/− 2 weeks gestation. Women deemed eligible to participate in the study included those 18–40 years old, had an initial clinic visit prior to 20 weeks gestation, had no reported use of oral contraceptives within the 3 months prior to becoming pregnant, did not utilize in vitro fertilization to become pregnant, and had no other known medical or obstetric conditions. Women included in the study provided spot urine samples at three different time points during pregnancy (16–20, 20–24, and 24–28 weeks gestation). All participants provided demographic data at the first study visit. This study was approved by the research and ethics committees at the following locations: University of Michigan School of Public Health, University of Puerto Rico, Northeastern University, and all participating hospitals and clinics. Participants all provided their full informed consent prior to their participation in the study.
2.2. Urinary Biomarkers
Collection and analysis of urine samples in the PROTECT cohort have been described previously (Silver et al., 2021). In brief, urine samples collected at the first (Visit 1) and third study visit (Visit 3) were sent to NSF International (Ann Arbor, MI) for measurement of glyphosate and its primary environmental degradate, aminomethyl phosphonic acid (AMPA). Concentrations were assessed using gas chromatography tandem triple quadrupole mass spectrometry (GC-MS/MS) on an Agilent 7890B gas chromatograph coupled to an Agilent 7000C triple quadrupole mass spectrometer. A validated in-house method developed based on a method that had been previously described (Silver et al., 2021) was used for the assessment of glyphosate and AMPA in all samples.
Urine samples were also assessed for the following measures of oxidative stress using stable isotope dilution gas chromatography-negative ion chemical ionization-mass spectrometry: free 8-isoprostane-prostaglandin-F2α (8-iso-PGF2α), the main 8-iso-PGF2α metabolite (2,3-dinor-5,6-dihydro-15-F 2t -isoprostane), and prostaglandin-F2α (PGF2α) at the Eicosanoid Core Laboratory at Vanderbilt University Medical Center (Nashville, TN) as described previously (Cathey et al., 2021). A mathematical approach described previously was also used to evaluate absolute levels of 8-iso-PGF2α hypothesized to be attributed to chemical lipid peroxidation (absolute chemical lipid peroxidation: aCLP) and prostaglandin-endoperoxide synthases (absolute enzymatic lipid peroxidation; aPGHS) (Van’t Erve et al., 2016). This method examines the ratio of 8-iso-PGF2α and PGF2α to quantify contributions of chemical and enzymatic lipid peroxidation pathways to levels of 8-iso-PGF2α. These values are hypothesized to serve as proxies to assess the association with glyphosate and AMPA that is attributable to oxidative stress (chemical lipid peroxidation) and inflammation (enzymatic lipid peroxidation) (Van’t Erve et al., 2016).
2.3. Statistical Analyses
Descriptive statistics were used to assess the demographics of study participants. All biomarkers were log-normally distributed and were natural-log transformed for subsequent analyses. Measurements below the limit of detection (LOD) were replaced with the LOD divided by the square root of 2. Urinary dilution was controlled for using specific gravity. In calculations for descriptive tables, concentrations were corrected for specific gravity using the following formula: Pc = P[(SGm − 1) / (SGi − 1)], where Pc represents the corrected concentration (ng/mL), P is the original concentration, SGm is the median specific gravity among participants (1.02), and SGi is the specific gravity of a given participant (Meeker et al., 2009).
One-way ANOVA tests evaluated differences in mean exposure and outcome levels between study visits. Intraclass correlation coefficients (ICCs) were calculated to assess within-to between-individual variability of exposure measurements for participants with measurements at both time points. Spearman correlation coefficients were also calculated to assess the relationship among glyphosate and AMPA at each study visit. Linear mixed effect models with random intercepts for repeated correlated outcomes were fitted to assess associations between continuous exposure measures of glyphosate and AMPA, and measures of oxidative stress. In order to assess potential for nonlinear trends, exposure variables were also modeled as tertiles.
Given the high percentage of AMPA samples below the LOD (~50%), the median concentration of samples above the LOD was used as the cut-off point to create the following three-level ordinal variable: samples below the LOD (low), bottom 50% of samples above the LOD (medium), and top 50% of samples above the LOD (high). Similar sized tertiles were created for glyphosate, as well. An additional sensitivity analysis was done to assess variability of effect estimates using the following equally sized tertiles for glyphosate: bottom 33% of all samples (22% < LOD and 11% > LOD) (low), middle 33% of all samples (medium), and top 33% of all samples (high).
Linear models were also fit to assess associations at each study visit to identify potential windows of susceptibility. All models included raw glyphosate/AMPA concentrations (instead of specific gravity-corrected concentrations) and were adjusted for specific gravity and categorical forms of education, maternal age, smoking status, alcohol consumption, and household income based on a priori knowledge. In sensitivity analyses, we also tested other possible confounders (employment status, marital status, parity, and exposure to environmental tobacco smoke (ETS)) and included those which resulted in a change of the main effect estimate by at least 10%. This resulted in additional models which adjusted for the primary set of covariates plus parity and/or ETS.
Our initial study population consisted of 646 women for whom we had oxidative stress biomarker measures for at least 1 study visit. Of those, there were 227 women for whom we also had glyphosate and AMPA biomarker measurements for at least one study visit based on a previously described preliminary analysis among this cohort (Silver et al., 2021). Because of missing information on selected primary covariates, final statistical models included 205 women (151 total measures at visit 1 and 169 total measures at visit 3), while further adjusted models in sensitivity analyses included 165 women (122 total measures at visit 1 and 134 total measures at visit 3). All statistical analyses were conducted using R (version 3.5.1).
3. RESULTS
3.1. Demographics and Biomarker Distributions
Demographic statistics of 227 participants included in the study having complete exposure and outcome data are shown in Table 1. Women were typically under the age of 30 (88.5%), had obtained at least a bachelor’s degree (44.7%), lived in a household with income less than $30,000 a year (60.5%), indicated they had never smoked (83.6%), and reported never drinking (42.7%). Distributions of specific gravity-corrected urinary glyphosate and AMPA among these women are shown in Table 2. Non-corrected exposure distributions are provided in Supplemental Table 6. Approximately 22% of glyphosate concentrations fell below the LOD compared with 50% of AMPA concentrations. Distributions of oxidative stress biomarkers are shown in Supplemental Table 1. There were no significant differences detected between study visits for the exposure or oxidative stress biomarkers. ICCs were 0.36 (95% CI: 0.20 – 0.51) and 0.25 (95% CI: 0.06 – 0.43) for specific gravity-corrected glyphosate and AMPA, respectively. Specific gravity-corrected glyphosate and AMPA concentrations were significantly correlated at Visit 1 (Spearman p = 0.42; p < 0.0001) and at Visit 3 (Spearman p = 0.51; p < 0.0001) as previously reported in an earlier study in PROTECT (Silver et al., 2021).
Table 1.
Visit 1 | Visit 3 | |
---|---|---|
Covariates | (N=162) | (N=185) |
Maternal Age | ||
18–24 | 64 (39.5%) | 76 (41.1%) |
25–29 | 52 (32.1%) | 61 (33.0%) |
30–34 | 27 (16.7%) | 27 (14.6%) |
35–41 | 19 (11.7%) | 21 (11.4%) |
Education | ||
GED or less | 34 (21.0%) | 39 (21.1%) |
Some college | 53 (32.7%) | 66 (35.7%) |
Bachelors or greater | 73 (45.1%) | 78 (42.2%) |
Missing | 2 (1.2%) | 2 (1.1%) |
Parity | ||
0 | 61 (37.7%) | 73 (39.5%) |
1 | 54 (33.3%) | 59 (31.9%) |
2 or more | 16 (9.9%) | 21 (11.4%) |
Missing | 31 (19.1%) | 32 (17.3%) |
Household Income | ||
$10,000 or less | 49 (30.2%) | 57 (30.8%) |
$10,000 – $29,999 | 49 (30.2%) | 55 (29.7%) |
$30,000 – $49,999 | 33 (20.4%) | 37 (20.0%) |
$50,000 or greater | 22 (13.6%) | 22 (11.9%) |
Missing | 9 (5.6%) | 14 (7.6%) |
Marital Status | ||
Single | 30 (18.5%) | 33 (17.8%) |
Married | 85 (52.5%) | 103 (55.7%) |
Cohabitating | 46 (28.4%) | 48 (25.9%) |
Missing | 1 (0.6%) | 1 (0.5%) |
Smoking Status | ||
Never | 134 (82.7%) | 156 (84.3%) |
Ever | 20 (12.3%) | 24 (13.0%) |
Current | 7 (4.3%) | 4 (2.2%) |
Missing | 1 (0.6%) | 1 (0.5%) |
Alcohol Use | ||
Never | 67 (41.4%) | 81 (43.8%) |
Yes (before pregnancy) | 85 (52.5%) | 91 (49.2%) |
Yes (currently) | 8 (4.9%) | 11 (5.9%) |
Missing | 2 (1.2%) | 2 (1.1%) |
Maternal ETS | ||
Never | 143 (88.3%) | 165 (89.2) |
One hour or less | 9 (5.6%) | 6 (3.2%) |
Greater than one hour | 7 (4.3%) | 9 (4.9%) |
Missing | 3 (1.9%) | 5 (2.7%) |
Employment Status | ||
Yes | 58 (35.8%) | 69 (37.3%) |
No | 103 (63.6%) | 115 (62.2%) |
Missing | 1 (0.6%) | 1 (0.5%) |
Table 2.
n | n<LOD | %<LOD | min | 25th | 50th | 75th | 90th | 95th | Max | GM | GSD | P-value | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
AMPA | Total | 347 | 172 | 50% | 0.09 | 0.17 | 0.26 | 0.49 | 0.85 | 1.05 | 10.61 | 0.30 | 2.08 | 0.17 |
V1 | 162 | 78 | 48% | 0.09 | 0.17 | 0.27 | 0.51 | 0.83 | 1.11 | 10.61 | 0.31 | 2.18 | ||
V3 | 185 | 94 | 51% | 0.09 | 0.18 | 0.24 | 0.47 | 0.84 | 1.01 | 1.77 | 0.29 | 1.98 | ||
GLYPH | Total | 347 | 77 | 22% | 0.09 | 0.29 | 0.50 | 0.79 | 1.23 | 1.63 | 2.66 | 0.49 | 2.02 | 0.86 |
V1 | 162 | 36 | 22% | 0.12 | 0.30 | 0.53 | 0.77 | 1.19 | 1.54 | 2.05 | 0.49 | 1.99 | ||
V3 | 185 | 41 | 22% | 0.09 | 0.28 | 0.48 | 0.83 | 1.30 | 1.67 | 2.66 | 0.48 | 2.06 |
3.2. Mixed Effect Models
An interquartile range (IQR) increase in AMPA was significantly associated with a 6.71% (95% CI: 1.51% – 12.17%) increase in concentrations of the main 8-iso-PGF2α metabolite after adjustment for maternal education, age, smoking status, alcohol consumption, household income and specific gravity (Table 3). Additional adjustment based on a 10% change in the main effect led to the inclusion of parity and maternal exposure to ETS, and yielded similar results (Effect estimate: 7.01%; 95% CI: 0.89% – 13.50%) (Supplemental Table 5). When tertiles (i.e., three level ordinal variable as described in the Methods section due to proportion of non-detects greater than 33%) for AMPA concentrations were examined, the association between AMPA and the main 8-iso-PGF2α metabolite remained statistically significant. Compared to the lowest exposure tertile, the second and third tertiles of AMPA were significantly associated with 12.85% (95% CI: 0.63%–26.55%) and 15.20% (95% CI: 1.83%–30.32%) higher levels of the main 8-iso-PGF2α metabolite, respectively (Supplementary Table 3).
Table 3.
8-ISO-PGF2 | ISOM | PGF2a | ||||
---|---|---|---|---|---|---|
% Change (95% CI) | p | % Change (95% CI) | p | % Change (95% CI) | p | |
GLYPH * | 4.73 (−0.88 – 10.67) | 0.10 | 2.31 (−8.08 – 8.01) | 0.41 | 6.12 (−3.70 – 16.95) | 0.23 |
AMPA * | 2.30 (−2.89 – 7.76) | 0.39 | 6.71 (1.51 – 12.17) | 0.01 | 1.75 (−7.24 – 11.62) | 0.71 |
aCLP | aPGHS | |||||
% Change (95% CI) | p | % Change (95% CI) | p | |||
GLYPH * | 6.32 (−2.09 – 15.46) | 0.15 | 71.38 (−15.61 – 248.05) | 0.14 | ||
AMPA * | 2.20 (−5.48 – 10.50) | 0.59 | 22.04 (−37.67 – 138.96) | 0.56 |
Exposure variables modeled as continuous and adjusted for education, maternal age, smoking status, alcohol consumption, household income and specific gravity.
When examining each study visit individually, an IQR increase in AMPA was significantly associated with 9.03% (95% CI: 0.55% – 18.21%) higher levels of the main 8-iso-PGF2α metabolite during the third study visit. Associations from the first study visit were lower in magnitude and were not statistically significant (Figure 2). A similar positive trend was detected among AMPA and 8-isoprostane, particularly at Visit 3, although associations were not as strong and did not reach statistical significance (Figure 1). Following additional adjustment for parity and maternal exposure to ETS, results failed to reach statistical significance at both study visits for AMPA and 8-isoprostane and its main metabolite (Supplemental Figures 1 & 2).
For urinary glyphosate, an IQR increase was suggestively associated with a 4.7% (−0.9%–10.7%) increase in 8-iso-PGF2α (Table 3). Results following additional adjustment for parity and maternal exposure to ETS also did not reach statistical significance (Supplemental Table 5). Regardless of method used to identify cut-off points or set of covariates included in the model, tertiles of glyphosate exposure were not found to be significantly associated with oxidative stress biomarkers (Supplemental Tables 2 & 4). When examining glyphosate at Visit 3 as a continuous variable, an IQR increase of glyphosate was associated with a 7.23% (95% CI: 0.11% – 14.86%) increase in 8-iso-PGF2α (Figure 1) and a 10.90% (95% CI: 0.25% – 22.69%) increase in aCLP (results not shown). Although estimates remained positive for glyphosate and 8-iso-PGF2α and aCLP, they were attenuated towards the null following additional adjustment for parity and/or maternal ETS (Supplemental Figure 1). Findings for the mathematically derived measures of oxidative stress hypothesized to distinguish between oxidative stress and inflammation pathways were greater in magnitude than other measures of oxidative stress, but did not reach statistical significance (Table 3).
4. DISCUSSION
In this prospective cohort study among pregnant women in Puerto Rico, we examined associations between repeated measures of urinary glyphosate and its main environmental degradate, AMPA, with respect to biomarkers of lipid oxidative stress, including 8-iso-PGF2α, its main metabolite, and fractions derived from the 8-iso-PGF2α/PGF2α ratio. Results from this study identified a positive association between AMPA and the main metabolite of 8-iso-PGF2α. Previously, Dorjgochoo et al. (2012) suggested that the 8-iso-PGF2α metabolite may be a more sensitive biomarker of oxidative stress compared to 8-iso-PGF2α itself, given its stronger associations with antioxidant use and vitamin supplementation in a cross-sectional study of healthy women in the Shanghai Women’s Health Study (n = 845) (Dorjgochoo et al., 2012). Further, the source of unmetabolized 8-iso-PGF2α in urine is likely to be conditional on the disease under study and could reflect kidney rather than systemic production, as is the case with classic prostaglandins (Patrono and FitzGerald, 1997). In the present study, a positive monotonic trend was also identified in an analysis of AMPA tertiles in relation to the 8-iso-PGF2α metabolite. Estimates were strongest for samples collected at 24–28 weeks of gestation (Visit 3), while samples collected at 16–20 weeks of gestation (Visit 1) were close to the null. Visit specific results were attenuated following additional covariate adjustment. Although estimates followed similar trends among AMPA and the parent compound 8-iso-PGF2α, they were lower in magnitude and did not reach statistical significance.
Few human studies have measured both glyphosate and AMPA exposure to date. A previous pregnancy study (11–38 weeks of gestation; n = 71) detected a higher range of glyphosate concentrations in urine (0.5–7.2 ng/mL) compared to the present study (0.09–2.66 ng/mL) (Parvez et al., 2018). A sub-cohort of older adults from the Rancho Bernardo Study (RBS) of Healthy Aging, a prospective cohort study based in Southern California, reported a lower geometric mean (GM) of specific gravity-corrected glyphosate (0.31 ng/mL) and a similar GM of specific gravity-corrected AMPA (0.285 ng/mL) in urine samples compared to the present study (glyphosate = 0.49 ng/mL; AMPA = 0.30 ng/mL) (Mills et al., 2017). Recently, a sub-cohort of The Infant Development and the Environment Study (TIDES) (>13 weeks of gestation; n = 94) reported median specific gravity-corrected concentrations of 0.22 and 0.14 ng/mL, respectively (Lesseur et al., 2021), which were lower than those seen in our study (glyphosate: 0.50 ng/mL; AMPA: 0.26 ng/mL).
To our knowledge this was the first human study to explore the association between AMPA and measures of oxidative stress. Evidence from in vitro and animal models supports a complex association between AMPA and oxidative stress. A study examining spined toad hatchlings found an association between AMPA and decreased levels of oxidative stress markers known to alter homeostasis if disrupted (e.g. superoxide dismutase, glutathione peroxidase, and catalase) in a non-monotonic manner (Cheron et al., 2022). An earlier study in mussels displayed similar findings with glyphosate, AMPA and a mixture of both (Matozzo et al., 2019). Despite these findings, our study found that with increasing categorical levels of AMPA, there was a monotonic increase in the 8-iso-PGF2α metabolite. Further study is needed to better understand the dose response relationship between AMPA and biomarkers of oxidative stress in humans.
Recent work has shown that increased levels of oxidative stress in pregnant mothers is associated with increased odds of preterm birth (Eick et al., 2020; Ferguson et al., 2015). Glyphosate and AMPA have also been found to be associated with increased odds of preterm birth, particularly with exposure later in pregnancy (Silver et al., 2021). Similarly, in the present study AMPA was found to be more strongly associated with biomarkers of oxidative stress during the third study visit compared to earlier in pregnancy. Taken together, there may be suggestive evidence indicating a potential window of susceptibility present later in pregnancy. Although findings in our study were attenuated following additional adjustment, further study is needed to better characterize timing of glyphosate and AMPA exposure in pregnancy, and their relationship to oxidative stress and adverse pregnancy outcomes.
4.1. Limitations
There are several limitations to the analyses conducted in the present study. To begin, results may not be generalizable to individuals outside the study population, such as non-pregnant women, children, men, those with preexisting conditions and pregnant women not included in the study. In addition, the ratio method used to distinguish between hypothesized levels of chemical and enzymatic lipid peroxidation may not serve as an accurate depiction of true differences among oxidative stress and inflammation. Our small sample size also presents a significant limitation to the study, though this was primarily an exploratory study aimed at providing introductory evidence of an association between glyphosate and oxidative stress biomarkers. Future work will assess glyphosate and AMPA concentrations in a larger set of urine samples in order to substantiate our current findings. There is also the possibility of chance findings given the number of statistical comparisons that were conducted. Additionally, AMPA is an environmental degradate of glyphosate, as well as several other potential co-pollutants (e.g. amino-polyphosphates) making it difficult to distinguish the proportion of AMPA present as a result of glyphosate and other alternative sources of exposure (Grandcoin et al., 2017). However, the strength of the correlations observed between urinary glyphosate and AMPA concentrations suggest that glyphosate may be the primary source of AMPA in the environment leading to exposure among our study participants.
4.2. Strengths
Despite these limitations, the present study is the first to identify associations among glyphosate and AMPA in relation to measures of oxidative stress during pregnancy. Associations found between AMPA and certain measures of oxidative stress increases the need for further study given its level of persistence in the environment, numerous co-pollutant sources, and lack of information regarding its effect on adverse health outcomes in humans. The inclusion of two time points of measurement for both glyphosate and AMPA also allowed for greater statistical power in estimating associations, and to potentially identify windows of susceptibility; an important factor to be considered given the short half-life of glyphosate. Lastly, we explored mathematically derived measures of oxidative stress hypothesized to distinguish between oxidative stress and inflammation pathways in order to more fully study the associations between oxidative stress and exposure to glyphosate and AMPA during pregnancy.
4.3. Conclusion
The present study provides evidence of a potential association between urinary AMPA and certain measures of oxidative stress, particularly during the third visit (24–28 weeks gestation), indicative of a potential window of susceptibility during development. Glyphosate followed similar trends, although estimates may have been attenuated given its shorter half-life. Provided the widespread use of glyphosate, multiple sources of AMPA, the latter’s persistence within the environment, and potential for adverse effects during pregnancy, further research is needed to characterize these associations within PROTECT and other cohorts.
Supplementary Material
HIGHLIGHTS.
AMPA was positively associated with the main metabolite of 8-iso-PGF2α
AMPA was more strongly associated with 8-isoprostane metabolite at the third visit
AMPA tertiles were linearly associated with the 8-isoprostane metabolite
ACKNOWLEDGMENTS
The authors thank the nurses and research staff who participated in cohort recruitment and follow up, as well as the Federally Qualified Health Centers in Puerto Rico that facilitated participant recruitment, including Morovis Community Health Center, Prymed in Ciales, Camuy Health Services, Inc., and the Delta OBGyn Group in Manati, as well as the Manati Medical Center and the Metro Pavia Hospital in Arecibo. Funding for this work was supported by the National Institutes of Health/National Institute of Environmental Health Sciences (grants P42ES017198, P50ES026049, P30ES017885, and R01ES032203) and by the Environmental influences on Child Health Outcomes (ECHO) program, Office of The Director, National Institutes of Health (grant UH3OD023251). The funding sources had no involvement in the study design, collection/analysis/interpretation of the data, the writing of this manuscript, or the decision to submit this manuscript for publication.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Competing Interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
CRediT authorship contribution statement
J. Eaton: formal analysis, investigation, methodology, visualization, writing - original draft, writing - review and editing
A. Cathey: supervision, writing - review and editing
J. Fernandez: supervision, writing - review and editing
D. Watkins: supervision, writing - review and editing
M. Silver: writing – review and editing
G. Milne: data curation, writing – review and editing
Z. Rosario: data curation, resources
C. Vélez-Vega: funding acquisition
A. Alshawabkeh: funding acquisition, project administration
J. Cordero: funding acquisition
J. Meeker: conceptualization, funding acquisition, project administration, supervision, writing - review and editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
REFERENCES
- Abarikwu SO, Akiri OF, Durojaiye MA, Adenike A, 2015. Combined effects of repeated administration of Bretmont Wipeout (glyphosate) and Ultrazin (atrazine) on testosterone, oxidative stress and sperm quality of Wistar rats. Toxicol. Mech. Methods 25, 70–80. 10.3109/15376516.2014.989349 [DOI] [PubMed] [Google Scholar]
- Alarcón R, Ingaramo PI, Rivera OE, Dioguardi GH, Repetti MR, Demonte LD, Milesi MM, Varayoud J, Muñoz-de-Toro M, Luque EH, 2019. Neonatal exposure to a glyphosate-based herbicide alters the histofunctional differentiation of the ovaries and uterus in lambs. Mol. Cell. Endocrinol 482, 45–56. 10.1016/j.mce.2018.12.007 [DOI] [PubMed] [Google Scholar]
- Aouache R, Biquard L, Vaiman D, Miralles F, 2018. Oxidative stress in preeclampsia and placental diseases. Int. J. Mol. Sci 19. 10.3390/ijms19051496 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Battaglin WA, Meyer MT, Kuivila KM, Dietze JE, 2014. Glyphosate and its degradation product AMPA occur frequently and widely in U.S. soils, surface water, groundwater, and precipitation. J. Am. Water Resour. Assoc 50, 275–290. 10.1111/jawr.12159 [DOI] [Google Scholar]
- Benbrook CM, 2016. Trends in glyphosate herbicide use in the United States and globally. Environ. Sci. Eur 28, 3. 10.1186/s12302-016-0070-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bonny S, 2016. Genetically Modified Herbicide-Tolerant Crops, Weeds, and Herbicides: Overview and Impact. Environ. Manage 57, 31–48. 10.1007/s00267-015-0589-7 [DOI] [PubMed] [Google Scholar]
- Borggaard OK, Gimsing AL, 2008. Fate of glyphosate in soil and the possibility of leaching to ground and surface waters: a review. Pest Manag. Sci 64, 441–456. 10.1002/ps.1512 [DOI] [PubMed] [Google Scholar]
- Burton GJ, Yung HW, Cindrova-Davies T, Charnock-Jones DS, 2009. Placental Endoplasmic Reticulum Stress and Oxidative Stress in the Pathophysiology of Unexplained Intrauterine Growth Restriction and Early Onset Preeclampsia. Placenta 30, 43–48. 10.1016/j.placenta.2008.11.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Canosa IS, Silveyra GR, Avigliano L, Medesani DA, Rodríguez EM, 2018. Ovarian growth impairment after chronic exposure to Roundup Ultramax® in the estuarine crab Neohelice granulata. Environ. Sci. Pollut. Res 25, 1568–1575. 10.1007/s11356-017-0581-2 [DOI] [PubMed] [Google Scholar]
- Cantonwine DE, Cordero JF, Rivera-González LO, Anzalota Del Toro LV, Ferguson KK, Mukherjee B, Calafat AM, Crespo N, Jiménez-Vélez B, Padilla IY, Alshawabkeh AN, Meeker JD, 2014. Urinary phthalate metabolite concentrations among pregnant women in Northern Puerto Rico: Distribution, temporal variability, and predictors. Environ. Int 62, 1–11. 10.1016/j.envint.2013.09.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cathey AL, Eaton JL, Ashrap P, Watkins DJ, Rosario ZY, Vélez Vega C, Alshawabkeh AN, Cordero JF, Mukherjee B, Meeker JD, 2021. Individual and joint effects of phthalate metabolites on biomarkers of oxidative stress among pregnant women in Puerto Rico. Environ. Int 154. 10.1016/j.envint.2021.106565 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chang F. chih, Simcik MF, Capel PD, 2011. Occurrence and fate of the herbicide glyphosate and its degradate aminomethylphosphonic acid in the atmosphere. Environ. Toxicol. Chem 30, 548–555. 10.1002/etc.431 [DOI] [PubMed] [Google Scholar]
- Chaufan G, Coalova I, Del Carmen Ríos De Molina M, 2014. Glyphosate commercial formulation causes cytotoxicity, oxidative effects, and apoptosis on human cells: Differences with its active ingredient. Int. J. Toxicol 33, 29–38. 10.1177/1091581813517906 [DOI] [PubMed] [Google Scholar]
- Cheron M, Costantini D, Angelier F, Ribout C, Brischoux F, 2022. Aminomethylphosphonic acid (AMPA) alters oxidative status during embryonic development in an amphibian species. Chemosphere 287. 10.1016/j.chemosphere.2021.131882 [DOI] [PubMed] [Google Scholar]
- Connolly A, Jones K, Basinas I, Galea KS, Kenny L, McGowan P, Coggins MA, 2019. Exploring the half-life of glyphosate in human urine samples. Int. J. Hyg. Environ. Health 222, 205–210. 10.1016/j.ijheh.2018.09.004 [DOI] [PubMed] [Google Scholar]
- De Almeida LKS, Pletschke BI, Frost CL, 2018. Moderate levels of glyphosate and its formulations vary in their cytotoxicity and genotoxicity in a whole blood model and in human cell lines with different estrogen receptor status. 3 Biotech 8, 1–15. 10.1007/s13205-018-1464-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dill GM, Sammons RD, Feng PCC, Kohn F, Kretzmer K, Mehrsheikh A, Bleeke M, Honegger JL, Farmer D, Wright D, Haupfear EA, 2010. Glyphosate: Discovery, Development, Applications, and Properties. Glyphosate Resist. Crop. Weeds, Wiley Online Books 10.1002/9780470634394.ch1 [DOI] [Google Scholar]
- Dorjgochoo T, Gao YT, Chow WH, Shu XO, Yang G, Cai Q, Rothman N, Cai H, Li H, Deng X, Franke A, Roberts LJ, Milne G, Zheng W, Dai Q, 2012. Major metabolite of F2-isoprostane in urine may be a more sensitive biomarker of oxidative stress than isoprostane itself. Am. J. Clin. Nutr 96, 405–414. 10.3945/ajcn.112.034918 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Duke SO, 2018. The history and current status of glyphosate. Pest Manag. Sci 74, 1027–1034. 10.1002/ps.4652 [DOI] [PubMed] [Google Scholar]
- Eick SM, Ferguson KK, Milne GL, Rios-McConnell R, Vélez-Vega C, Rosario Z, Alshawabkeh A, Cordero JF, Meeker JD, 2020. Repeated measures of urinary oxidative stress biomarkers and preterm birth in Puerto Rico. Free Radic. Biol. Med 146, 299–305. 10.1016/j.freeradbiomed.2019.11.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fathi MA, Abdelghani E, Shen D, Ren X, Dai P, Li Z, Tang Q, Li Y, Li C, 2019. Effect of in ovo glyphosate injection on embryonic development, serum biochemistry, antioxidant status and histopathological changes in newly hatched chicks. J. Anim. Physiol. Anim. Nutr. (Berl) 103, 1776–1784. 10.1111/jpn.13181 [DOI] [PubMed] [Google Scholar]
- Ferguson KK, McElrath TF, Chen Y-H, Loch-Caruso R, Mukherjee B, Meeker JD, 2015. Repeated measures of urinary oxidative stress biomarkers during pregnancy and preterm birth. Am. J. Obstet. Gynecol 212, 208.e1–8. 10.1016/j.ajog.2014.08.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gallegos CE, Baier CJ, Bartos M, Bras C, Domínguez S, Mónaco N, Gumilar F, Giménez MS, Minetti A, 2018. Perinatal Glyphosate-Based Herbicide Exposure in Rats Alters Brain Antioxidant Status, Glutamate and Acetylcholine Metabolism and Affects Recognition Memory. Neurotox. Res 34, 363–374. 10.1007/s12640-018-9894-2 [DOI] [PubMed] [Google Scholar]
- Gao H, Chen J, Ding F, Chou X, Zhang X, Wan Y, Hu J, Wu Q, 2019. Activation of the N-methyl-d-aspartate receptor is involved in glyphosate-induced renal proximal tubule cell apoptosis. J. Appl. Toxicol 39, 1096–1107. 10.1002/jat.3795 [DOI] [PubMed] [Google Scholar]
- Gomes MP, Le Manac’h SG, Maccario S, Labrecque M, Lucotte M, Juneau P, 2016. Differential effects of glyphosate and aminomethylphosphonic acid (AMPA) on photosynthesis and chlorophyll metabolism in willow plants. Pestic. Biochem. Physiol 130, 65–70. 10.1016/j.pestbp.2015.11.010 [DOI] [PubMed] [Google Scholar]
- Grandcoin A, Piel S, Baurès E, 2017. AminoMethylPhosphonic acid (AMPA) in natural waters: Its sources, behavior and environmental fate. Water Res. 117, 187–197. 10.1016/j.watres.2017.03.055 [DOI] [PubMed] [Google Scholar]
- Jansson T, Powell TL, 2007. Role of the placenta in fetal programming: underlying mechanisms and potential interventional approaches. Clin. Sci 113, 1–13. 10.1042/CS20060339 [DOI] [PubMed] [Google Scholar]
- Lesseur C, Pirrotte P, Pathak KV, Manservisi F, Mandrioli D, Belpoggi F, Panzacchi S, Li Q, Barrett ES, Nguyen RHN, Sathyanarayana S, Swan SH, Chen J, 2021. Maternal urinary levels of glyphosate during pregnancy and anogenital distance in newborns in a US multicenter pregnancy cohort. Environ. Pollut 280, 117002. 10.1016/j.envpol.2021.117002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Matozzo V, Munari M, Masiero L, Finos L, Marin MG, 2019. Ecotoxicological hazard of a mixture of glyphosate and aminomethylphosphonic acid to the mussel Mytilus galloprovincialis (Lamarck 1819). Sci. Rep 9, 1–9. 10.1038/s41598-019-50607-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- McComb BC, Curtis L, Chambers CL, Newton M, Bentson K, 2008. Acute toxic hazard evaluations of glyphosate herbicide on terrestrial vertebrates of the oregon coast range. Environ. Sci. Pollut. Res 15, 266–272. 10.1065/espr2007.07.437 [DOI] [PubMed] [Google Scholar]
- Meeker JD, Hu H, Cantonwine DE, Lamadrid-Figueroa H, Calafat AM, Ettinger AS, Hernandez-Avila M, Loch-Caruso R, Téllez-Rojo MM, 2009. Urinary phthalate metabolites in relation to preterm birth in Mexico City. Environ. Health Perspect 117, 1587–1592. 10.1289/ehp.0800522 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mills PJ, Kania-Korwel I, Fagan J, McEvoy LK, Laughlin GA, Barrett-Connor E, 2017. Excretion of the herbicide glyphosate in older adults between 1993 and 2016. JAMA - J. Am. Med. Assoc 318, 1610–1611. 10.1001/jama.2017.11726 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nardi J, Moras PB, Koeppe C, Dallegrave E, Leal MB, Rossato-Grando LG, 2017. Prepubertal subchronic exposure to soy milk and glyphosate leads to endocrine disruption. Food Chem. Toxicol 100, 247–252. 10.1016/j.fct.2016.12.030 [DOI] [PubMed] [Google Scholar]
- Niemann L, Sieke C, Pfeil R, Solecki R, 2015. A critical review of glyphosate findings in human urine samples and comparison with the exposure of operators and consumers. J. fur Verbraucherschutz und Leb 10, 3–12. 10.1007/s00003-014-0927-3 [DOI] [Google Scholar]
- Parvez S, Gerona RR, Proctor C, Friesen M, Ashby JL, Reiter JL, Lui Z, Winchester PD, 2018. Glyphosate exposure in pregnancy and shortened gestational length: A prospective Indiana birth cohort study. Environ. Heal. A Glob. Access Sci. Source 17, 1–12. 10.1186/s12940-018-0367-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Patrono C, FitzGerald GA, 1997. Isoprostanes: potential markers of oxidant stress in atherothrombotic disease. Arterioscler. Thromb. Vasc. Biol 17, 2309–2315. 10.1161/01.atv.17.11.2309 [DOI] [PubMed] [Google Scholar]
- Perego MC, Schutz LF, Caloni F, Cortinovis C, Albonico M, Spicer LJ, 2017. Evidence for direct effects of glyphosate on ovarian function: glyphosate influences steroidogenesis and proliferation of bovine granulosa but not theca cells in vitro. J. Appl. Toxicol 37, 692–698. 10.1002/jat.3417 [DOI] [PubMed] [Google Scholar]
- Sanders T, Liu Y-M, Tchounwou PB, 2015. Cytotoxic, genotoxic, and neurotoxic effects of Mg, Pb, and Fe on pheochromocytoma (PC-12) cells. Environ. Toxicol 30, 1445–1458. 10.1002/tox.22014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shushkova TV, Vasil’eva GK, Ermakova IT, Leont’evskiĭ AA, 2009. [Sorption and microbial degradation of glyphosphate in soil suspensions]. Prikl. Biokhim. Mikrobiol 45, 664–669. [PubMed] [Google Scholar]
- Silver MK, Fernandez J, Tang J, McDade A, Sabino J, Rosario Z, Vega CV, Alshawabkeh A, Cordero JF, Meeker JD, 2021. Prenatal exposure to glyphosate and its environmental degradate, aminomethylphosphonic acid (Ampa), and preterm birth: A nested case-control study in the protect cohort (puerto rico). Environ. Health Perspect 129, 1–11. 10.1289/EHP7295 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Singh B, Singh K, 2016. Microbial degradation of herbicides. Crit. Rev. Microbiol 42, 245–261. 10.3109/1040841X.2014.929564 [DOI] [PubMed] [Google Scholar]
- Sun M, Li H, Jaisi DP, 2019. Degradation of glyphosate and bioavailability of phosphorus derived from glyphosate in a soil-water system. Water Res. 163, 114840. 10.1016/j.watres.2019.07.007 [DOI] [PubMed] [Google Scholar]
- Torretta V, Katsoyiannis IA, Viotti P, Rada EC, 2018. Critical review of the effects of glyphosate exposure to the environment and humans through the food supply chain. Sustain. 10, 1–20. 10.3390/su10040950 [DOI] [Google Scholar]
- Turkmen R, Birdane YO, Demirel HH, Yavuz H, Kabu M, Ince S, 2019. Antioxidant and cytoprotective effects of N-acetylcysteine against subchronic oral glyphosate-based herbicide-induced oxidative stress in rats. Environ. Sci. Pollut. Res 26, 11427–11437. 10.1007/s11356-019-04585-5 [DOI] [PubMed] [Google Scholar]
- Van’t Erve TJ, Lih FB, Jelsema C, Deterding LJ, Eling TE, Mason RP, Kadiiska MB, 2016. Reinterpreting the best biomarker of oxidative stress: The 8-iso-prostaglandin F2α/prostaglandin F2α ratio shows complex origins of lipid peroxidation biomarkers in animal models. Free Radic. Biol. Med 95, 65–73. 10.1016/j.freeradbiomed.2016.03.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Velasques RR, Sandrini JZ, Da Rosa CE, 2016. Roundup® in Zebrafish: Effects on Oxidative Status and Gene Expression. Zebrafish 13, 432–441. 10.1089/zeb.2016.1259 [DOI] [PubMed] [Google Scholar]
- Venkatesh KK, Cantonwine DE, Ferguson K, Arjona M, Meeker JD, McElrath TF, 2016. Inflammatory and oxidative stress markers associated with decreased cervical length in pregnancy. Am. J. Reprod. Immunol 76, 376–382. 10.1111/aji.12545 [DOI] [PubMed] [Google Scholar]
- Vereecken H, 2005. Mobility and leaching of glyphosate: a review. Pest Manag. Sci 61, 1139–1151. 10.1002/ps.1122 [DOI] [PubMed] [Google Scholar]
- von Soosten D, Meyer U, Hüther L, Dänicke S, Lahrssen-Wiederholt M, Schafft H, Spolders M, Breves G, 2016. Excretion pathways and ruminal disappearance of glyphosate and its degradation product aminomethylphosphonic acid in dairy cows. J. Dairy Sci 99, 5318–5324. 10.3168/jds.2015-10585 [DOI] [PubMed] [Google Scholar]
- Williams AL, Watson RE, Desesso JM, 2012. Developmental and reproductive outcomes in humans and animals after glyphosate exposure: A critical analysis. J. Toxicol. Environ. Heal. - Part B Crit. Rev 15, 39–96. 10.1080/10937404.2012.632361 [DOI] [PubMed] [Google Scholar]
- Zhang C, Hu X, Luo J, Wu Z, Wang L, Li B, Wang Y, Sun G, 2015. Degradation dynamics of glyphosate in different types of citrus orchard soils in China. Molecules 20, 1161–1175. 10.3390/molecules20011161 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zoller O, Rhyn P, Zarn JA, Dudler V, 2020. Urine glyphosate level as a quantitative biomarker of oral exposure. Int. J. Hyg. Environ. Health 228, 113526. 10.1016/j.ijheh.2020.113526 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.