Skip to main content
International Journal of Environmental Research and Public Health logoLink to International Journal of Environmental Research and Public Health
. 2022 Feb 17;19(4):2273. doi: 10.3390/ijerph19042273

Using Metrics of a Mixture Effect and Nutrition from an Observational Study for Consideration towards Causal Inference

Chris Gennings 1,*, Katherine Svensson 2, Alicja Wolk 3,4, Christian Lindh 5, Hannu Kiviranta 6, Carl-Gustaf Bornehag 1,2
Editors: Danielle J Carlin, Cynthia V Rider
PMCID: PMC8872366  PMID: 35206461

Abstract

Environmental exposures to a myriad of chemicals are associated with adverse health effects in humans, while good nutrition is associated with improved health. Single chemical in vivo and in vitro studies demonstrate causal links between the chemicals and outcomes, but such studies do not represent human exposure to environmental mixtures. One way of summarizing the effect of the joint action of chemical mixtures is through an empirically weighted index using weighted quantile sum (WQS) regression. My Nutrition Index (MNI) is a metric of overall dietary nutrition based on guideline values, including for pregnant women. Our objective is to demonstrate the use of an index as a metric for more causally linking human exposure to health outcomes using observational data. We use both a WQS index of 26 endocrine-disrupting chemicals (EDCs) and MNI using data from the SELMA pregnancy cohort to conduct causal inference using g-computation with counterfactuals for assumed either reduced prenatal EDC exposures or improved prenatal nutrition. Reducing the EDC exposure using the WQS index as a metric or improving dietary nutrition using MNI as a metric, the counterfactuals in a causal inference with one SD change indicate significant improvement in cognitive function. Evaluation of such a strategy may support decision makers for risk management of EDCs and individual choices for improving dietary nutrition.

Keywords: WQS regression, endocrine disruptors, nutritional status, g-computation

1. Introduction

Evaluating the potential impact of environmental and dietary exposures on human health is challenged by using observational data. Estimation of effects from observational studies using regression methods is based on covariate-adjusted models, which may have bias due to confounders, collinearity among exposures, effect modification, and are generally considered to be association models rather than causal models [1]. Robins’ generalized methods (g methods) allow investigators to use observational data to estimate parameters that would be obtained in a perfectly randomized controlled trial [2,3,4]. Under certain assumptions, these estimates can be interpreted towards causality. g-computation is one way of estimating marginal quantities (i.e., not conditional on other covariates) of exposures using causal inference with the advantage of efficiency (i.e., small standard error) and stable estimates [5,6]. In short, g-computation is conducted using an initial flexible model of Y on exposures and covariates and constructing counterfactuals with predictions from the initial regression coefficients. Estimates and confidence intervals on marginal effects of exposure on Y are determined from Monte Carlo simulation and a bootstrap ensemble step. A g-computation analysis allows us to ask, “What would be the improvement in a human health outcome (Y) if environmental risk factors are reduced and exposure to beneficial (protective) factors are improved?”. This will be examined, with focus on reducing prenatal exposure to mixtures of environmental chemicals and improving dietary nutrition, along with the resulting improvement in children´s cognitive outcomes.

Recognizing that human exposure includes multiple environmental chemicals and dietary nutrition is based on dozens of nutrients, we propose the use of indices as metrics of exposure to use in g-computation. We use biomonitoring data and an empirically weighted index of multiple chemicals related to an important health outcome as a metric of a mixture effect (i.e., the joint action) of environmental exposures. We use a subject-specific nutrition index, My Nutrition Index (MNI; [7,8,9]), based on published recommended nutritional guidelines as a metric of overall dietary nutrition that incorporates dozens of macro- and micro-nutrients.

Our objective is to combine the estimation of a mixture effect of environmental chemicals of concern and the effect due to good nutrition as measured by MNI, with the construction of counterfactuals in g-computation, for a step towards causal inference. Using such a step, researchers can visualize “what if” exposure of the selected chemicals were reduced to a specific level as measured by the mixture index, and what the benefit would be in terms of the selected health-based outcome. On the other hand, researchers may ask what the benefit would be in terms of the selected health-based outcome with improved nutrition. We demonstrate the approach by extending the work of Tanner et al. [10], linking prenatal exposure to a mixture of chemicals to seven-year IQ by incorporating sex-specific weights, as they demonstrated a different effect for boys and girls. Using a WQS sex-stratified interaction model results in a single index with sex-specific weights and slope parameters. We also further adjust the model for nutrition using MNI. Finally, we conjecture how to use the resulting counterfactual to address issues around what would it take to reduce exposure to the specified level and what would be the score of MNI to improve the health outcome.

2. Methods

2.1. Study Population and Cognitive Assessment

Detailed recruitment and sample collection procedures have been previously described for the Swedish Environmental Longitudinal Mother and Child, Asthma and Allergy (SELMA) pregnancy cohort [10,11]. Participants provided written consent and the study was reviewed and approved by the Ethical Review Board (Uppsala, Sweden) (Dnr 2007/062 and Dnr 2017/177). Trained psychologists evaluated the cognitive function of children at 7 years of age using the Wechsler Intelligence Scale for Children, 4th edition, full scale IQ [10,12,13]. The current analysis includes 678 mother–child pairs with complete data on all exposures (EDC prenatal mixture of 26 chemicals), prenatal food frequency questionnaires, outcome (7-year IQ) and confounding variables (child sex; maternal nutrition index, energy, age at birth, pre-pregnancy weight, education level, Ravens IQ, and smoking status).

2.2. Collection of Prenatal Blood and Urine Samples

First-morning void urine and non-fasting blood samples were collected from mothers during their first prenatal visit (median 10-week gestation). Detailed analytical methods for all exposures are provided elsewhere [10]. Samples were stored at −20 °C (urine) and −70 °C (serum) at the Laboratory of Occupational and Environmental Medicine at Lund University, Lund, Sweden. The urine samples were analyzed by liquid chromatography tandem mass spectrometry (LC-MS/MS) to quantify 24 urinary analytes. Urinary analyte concentrations were adjusted for urine dilution by creatinine (i.e., adjusted analyte level = [analyte]/[creatinine]). Eight PFAS were quantified in serum using LC-MS/MS. Plasma was analyzed for 22 persistent organic pollutants (chlorinated or brominated) at the National Institute for Health and Welfare, Finland. Overall, 54 analytes were measured in urine, serum, and plasma. Following Tanner et al. [10], we summed analytes for DEHP and DINP using molar sums. DDT and its metabolite were also summed, as were 10 PCB congeners. We generally limited the analysis to compounds detectable in at least 75% of samples (Table A1), with the exception of summed variables.

2.3. My Nutrition Index (MNI) and Covariates

Data for maternal characteristics were collected at study entry for education, age, weight, and IQ using the shortened Ravens Standard Progressive Matrices [14,15,16]. Self-administered food frequency questionnaires were assessed in mid-pregnancy and linked to daily intake of nutrients, including energy levels [7]. The My Nutrition Index (MNI) was calculated using the nutrient data from food frequency questionnaires during pregnancy [7]. Children were characterized by sex, birthweight, and prematurity (<37 weeks’ gestation at birth). Although prematurity and birthweight may impact child IQ, we did not adjust for these variables since they may be mediators. We selected covariates for inclusion in the regression models using a directed acyclic graph (DAG; Figure A1).

2.4. Statistical Analysis

WQS regression consists of two steps: (i) estimation of a weighted index of standardized (i.e., quantiles) concentrations using a nonlinear model in an ensemble step where the final index is averaged across results from bootstrap samples of observations [17] or random subsets of components [18,19]; followed by (ii) a test for significance of the regression coefficient associated with the weighted index. The two steps are generally conducted in random splits of the data (e.g., 40% for weight estimation and 60% for hypothesis testing) and may be conducted in multiple random splits to address generalizability of the results [10,20]. The strength of the ensemble step is an increase in the number of correctly identified agents of concern compared to the use of a single nonlinear model in the estimation of the index [17]. This is analogous to forgoing parsimony for model prediction with correlated components. An example is some shrinkage methods are notorious for selecting one from a set of correlated components arbitrarily [21]. Similarly, using WQS regression in a single bootstrap sample may weight fewer components with nonnegligible weights compared to the final index averaged across all bootstrap samples. We used the gWQS R package version 3.0.4 for analysis [22].

We recently incorporated the capability of estimating the WQS index in the presence of interaction with either a continuous variable (e.g., BMI) or a categorical variable (e.g., sex) in the gWQS R package. The generalized linear model is parameterized to include a regression coefficient for the weighted index as a linear term and in the interaction term. For example, these terms may be β0+β1WQS+β2x+β12xWQS, when x is an indicator for a binary variable (e.g., sex). When x = 0, these terms would be β0+β1WQS, and when x = 1, they would be (β0+β2)+(β1+β12)WQS. Therefore, β2 is the change in the intercept due to the binary variable, β12 is the change in the slope of WQS due to the variable, and β1+β12 is the slope for the category when x = 1. This parameterization is for a WQS stratified interaction model.

The WQS stratified interaction model was used to estimate an empirical weighted index for the overall mixture effect of the 26 prenatal EDCs on 7-year IQ, adjusted by covariates [10]. To improve generalizability of the results, we evaluated 100 repeated holdout datasets where each training set (randomly selected 40% of the sample) was used to estimate a weighted index and each holdout dataset was used to estimate the association between the weighted index and outcome. As a sex-specific effect was noted by Tanner et al. [10], we used a WQS stratified interaction model which allowed for sex-specific weights (i.e., using 52 weights for the 26 chemicals scored into quantiles, 26 for boys and 26 for girls) and estimated in the presence of a potential interaction between the weighted index and sex. This parameterization not only allowed for sex-specific weights, but also sex-specific regression coefficients [23,24].

2.5. G-Computation

As is true for ordinary regression assumptions, we assume the WQS regression model is correctly specified with no unmeasured confounding, and that exposures predate the response. Further, there are primarily three assumptions, called identifiability conditions, for g-computation to estimate unbiased exposure effects [25]: (i) under consistency, well defined true exposures correspond to the measured exposures in the data; (ii) for (conditional) exchangeability, the probability of every exposure depends only on the covariates; and (iii) the positivity assumption assumes there is a nonzero probability that exposure could occur in all subgroups. In our data, exposures are measured using validated biomonitoring data from urine and serum samples. Exchangeability is addressed using covariates in the model that are linked to exposures. Based on temporality of the measurements and outcome, the exposures do not depend on IQ values. For the positivity assumption, exposure of the EDC mixtures is for all SELMA pregnant women, with detection rates to individual chemicals generally near 100% (Table A1).

Using the constructed WQSsc index (i.e., the WQS index centered and scaled), we conducted g-computation resulting in estimation of the mixture effect under counterfactuals such as exposure is at the mean WQS (i.e., WQSsc = 0) versus WQSsc values of −1 (i.e., one standard deviation below the mean exposure as measured by the WQS index). The comparison is the marginal direct mixture effect using the WQS index as a measure of change in the mixture effect of EDC exposures related to 7-year IQ. We used the gcomp function in the Risk Communicator R package on CRAN.

For further consideration, we conducted a similar g-computation analysis using My Nutrition Index. The counterfactual considered the direct effect of typical prenatal nutrition on 7-year IQ (i.e., as measured at the mean of MNI) compared to improved prenatal nutrition one standard deviation above the mean, i.e., improved nutrition as measured by MNI.

Finally, we compared different strategies for reducing exposure to the EDC mixtures using the WQS index as the weighted metric, where those most associated with a decline in IQ are more highly weighted. First, we decomposed the components of the WQS index into two categories: Persistent (i.e., PFAS: PFOA, PFAS, PFNA, PFDA, PFUnDA, PFHxS; and Persistent Chlorinated: HCB, Nonachlor, DDT and DDE, and the sum of PCBs) and Non-persistent (i.e., Phenols: triclosan, BPA, BPF, BFS; plasticizers: MEP, MBP, MBzP, DEHP, DINP, MHiDP, MCiNP, MOiNCH, DPHP; and other short-lived: TCP, PBA, 2OHPH). The weighted sum of each category (that together summed to 100%) was calculated per subject and displayed graphically. We used the counterfactual of reducing exposure where the WQS index is one standard deviation below the mean of the index. We considered two hypothetical scenarios: (1) What percent of the non-persistent chemicals would have to be reduced to achieve the goal of reducing exposure to one SD below the mean; (2) would completely eliminating the plasticizers be enough to achieve the goal? We calculated the revised value of the index under both scenarios for every subject and calculated a histogram of the WQS index with a reference line at the target value.

3. Results

Summary statistics for characterizing the mother–child pairs are provided in Table 1 Both urinary and serum matrices were used to measure 26 chemicals (41 analytes) in first-trimester maternal biological samples including from phenols, plasticizers (phthalate and non-phthalate), short-lived chemicals (organic flame retardant, OP pesticide, pyrethroid pesticide, a PAH), PFASs, and persistent chlorinated chemicals (including DDT/DDE and PCBs). Most were detected in 100% of the prenatal samples (Table A1).

Table 1.

Summary statistics of population characteristics using the SELMA pregnancy cohort (N = 678). (*) The WQS index is derived from a WQS sex-stratified interaction model of 26 EDCs associated with child IQ at 7 years of age, adjusted by covariates.

Mean SD
Exposure WQS index associated with 7-year IQ
(sex-stratified, decile-scaled) *
2.24 0.50
Maternal characteristics Graduated college n (%) 467 (69)
My Nutrition Index (MNI) 66.8 14.0
Energy (kcals) 1895 545
Age at birth (years) 31.3 4.6
Weight in 1st trimester of pregnancy (kg) 68.8 13.5
IQ (Raven) 114.8 14.9
Parity 1.8 0.86
Smoked in 1st trimester pregnancy n (%) 74 (11)
Creatinine (mmol/L) 10.4 4.7
Child characteristics Female n (%) 346 (51)
Premature birth n (%) 25 (3.7)
Full Scale WISC IQ at 7 years 99.9 12.7

We used a sex-stratified interaction WQS regression model which allowed for sex-specific weights and beta coefficients. The interaction between WQS and sex was borderline significant (Table 2; with 95% confidence interval based on the 100 holdout datasets: (−1.28, 5.13), with only 9% negative estimates). In such a parameterization, the beta associated with WQS is the slope for the reference group (boys), and the slope for the comparison group (girls) is the sum of the betas associated with the WQS and the interaction term. The weighted index was negative (estimated: −2.13) and significant for boys with 95% confidence interval (−4.27, −0.36) (Table 2). This indicates that as prenatal exposure to the 26 chemicals increases, as measured by the WQS index, there is a decline in seven-year IQ in boys. However, the slope for girls was diminished and not significant (estimated: −2.13 + 1.98 = −0.15; with 95% CI: (−2.32, 1.93)). Interpreting these beta coefficients in terms of unit changes in the WQS index (i.e., a decile-change in the WQS index) is complicated due to the infinite ways such a change could be achieved with changes in exposures to 26 chemicals; we subsequently address two scenarios in the framework of g-computation. In terms of the relationship with covariates in the same model (Table 2), there was a significant positive association between seven-year IQ and maternal IQ, maternal education, and the MNI; there was a significant negative association for maternal pre-pregnancy weight.

Table 2.

Parameter estimates (mean, standard error, 2.5 percentile, 97.5 percentile) from WQS sex-stratified interaction regression across 100 holdout datasets. The slope associated with WQS is for males; the interaction between WQS and sex is the difference in slopes between boys and girls.

Parameter Estimate Std. Error 2.5% 97.5%
(Intercept) 88.700 4.530 80.700 96.700
WQS −2.130 1.110 −4.270 −0.359
Female −0.622 3.190 −5.740 5.630
MNI 0.073 0.027 0.018 0.118
Energy 0.000 0.001 −0.001 0.001
Mom Age (at birth) −0.158 0.089 −0.324 0.004
Mom Weight −0.098 0.029 −0.156 −0.040
Mom Educ 4.790 0.862 3.090 6.470
Mom IQ 0.158 0.026 0.104 0.205
Smoker −2.100 1.320 −4.420 0.536
WQS:Female 1.980 1.640 −1.280 5.130

The distribution of estimated weights across the repeated training sets (Figure 1) identified chemicals of concern from each of the chemical groups. The chemicals with highest weights in boys were BPF, TCP, MBzP, PFOA, triclosan, MOiNCH, MEP, MCINP, PFHxS, PBA, BPA, and PFOS. In girls, the weights were generally lower, but DPHP, MEP, BPF, BPS, PFHxS, PFUnDA, MBzP, MOiNCH, OHPH, TCP, and the PCB sum had mean weights above the guideline cutoff of 1/52 = 0.019 (where 52 is the total number of weights, 26 for boys and 26 for girls).

Figure 1.

Figure 1

Figure 1

Estimated weight distribution in a WQS stratified regression model for 26 prenatal chemicals and 7-year IQ, using 100 repeated holdout validation datasets, for (A) boys; (B) girls; and (C) a divergent plot comparing the mean estimated sex-specific weights. Notes (A,B) Bars correspond to the right axis and indicate the percent of times a chemical exceeded the concern threshold in 100 repeated holdouts. Data points, boxplots, and diamonds correspond to the left axis. Data points indicate weights for each of the 100 holdouts. Box plots show 25th, 50th, and 75th percentiles, and whiskers show 10th and 90th percentiles of weights for the 100 holdouts. Closed diamonds show mean weights for the 100 holdouts; (C) The dotted lines represent the threshold guideline from the equi-weighted index (i.e., 1/(2c)), where c is the number of components.

We then used g-computation for marginal estimation of a causal link between the mixture effect from prenatal exposures as measured by the sex-stratified WQS index and 7-year IQ. Conditioning on the average weights used in the WQS index in the stratified interaction repeated holdout model, the mean IQ at seven years calculated at the average index was 98 (bootstrap 95% CI: 97, 100) for boys and was 102 (95% CI: 100, 103) for girls. In comparison, we constructed a counterfactual that prenatal exposure to the mixture as measured by the WQS index was decreased and fixed at one SD below the mean. In this counterfactual, the estimated IQ at seven years was 101 (95% CI: 99, 103) for boys and was 103 (95% CI: 101, 105) for girls. Thus, the marginal direct effect of the mixture had a mean difference in IQ of 2.2 (95% CI: 0.97, 3.4) for boys and 1.4 (95% CI: −0.13, 3.0) for girls. Such a change in the mean IQ with reduced exposure to the mixture of these EDCs could be evaluated as part of a health impact assessment; that is, is the counterfactual a meaningful goal for new regulatory management?

Suppose we set a “policy” to reduce the WQS index to one SD below its mean which is equivalent to the stratified interaction WQS index having a value less than 1.74 (Figure 2A), to reduce the adverse impact of the EDCs on important developmental markers (e.g., seven-year IQ). Recall, the WQS index is a stratified weighted index based on deciles from the population. To reduce the index using the pre-specified quantile designations with fixed weights would require the concentrations of many of the components be reduced. Of course, some of the components are environmentally persistent (i.e., the PFAS and persistent chlorinated, Figure 1) and cannot be readily reduced. In the SELMA cohort, roughly 26% of the WQS index for individuals is due to persistent compounds (Figure 2B). To reduce the WQS index below the target value without a change in the persistent chemicals, the remaining chemicals would have to be reduced to a point where their part of the index was 30% of the observed value (Figure 2C), where 99% of the subjects would have an adjusted WQS value below the suggested target of 1.74 (i.e., one SD below the mean). In comparison, we considered the scenario where the plasticizers were eliminated. The distribution of the adjusted WQS Index (Figure 2D), consisting only of the persistent chemicals, the phenols, and the other short-lived chemicals, is not an adequate adjustment as only 75% of the subjects fall below the suggested target. However, eliminating both the plasticizers and the other short-lived chemicals (Figure 2E) or the plasticizers and the phenols (Figure 2F) results in more than 95% of the subjects with values below the suggested target. These hypothetical scenarios demonstrate how the metric of prenatal exposure to 26 EDCs related to seven-year IQ, in a causal framework, can be used to suggest reduction strategies in environmental chemicals leading to potential improvements in public health.

Figure 2.

Figure 2

Figure 2

(A) Histogram of the stratified interaction WQS index in the SELMA cohort with reference line at selected target value (i.e., one SD below the WQS mean), (B) percentage of the WQS index per subject due to persistent chemicals (i.e., PFAS and persistent chlorinated), phenols, plasticizers, and other short-lived chemicals; (C) the distribution of the WQS index due to the persistent and 30% of the non-persistent chemicals, i.e., a 70% cut to non-persistent chemicals; the distribution of the WQS index (D) eliminating the plasticizers, (E) eliminating the plasticizers and the other short-lived compounds, and (F) eliminating the plasticizers and the phenols.

In comparison, improving dietary nutrition is based on human behavior and choices, not necessarily regulatory policy changes. The MNI provides a metric for individuals to use to improve their dietary nutrition. We used g-computation for marginal estimation of a causal link between prenatal nutrition as measured by MNI and seven-year IQ. The mean IQ at seven years calculated at the average MNI (i.e., mean = 66.8; Table 1) was 99.9% (bootstrap 95% CI: 99, 101). In comparison, we constructed a counterfactual that prenatal nutrition improved one SD, to an MNI value of 80.8. The mean IQ increased to 101 (bootstrap 95% CI: 100, 102) which is a marginal direct effect of 1.2 points in IQ with 95% CI (0.3, 2.1). Although these results are on average nutrient intake and for a population, dietary choices are made by individuals based on food preferences and access. Use of a metric that measures how nutritious daily dietary choices are may provide a strategy for individuals to improve their nutrition.

4. Discussion

Our focus is on the use of counterfactuals to compare “what is” to “what if” using indices of adverse effects of EDCs and the beneficial effect of good nutrition during pregnancy on neurodevelopment (i.e., seven-year IQ). The “what is” condition is what is typical as measured at the average; here, the “what if” condition is based on a standard deviation change in the corresponding metric in the direction of improved childhood seven-year IQ. We extended the work of Tanner et al. (2020) to include a stratified interaction WQS index as the metric of the mixture effect from 26 EDC prenatal exposures. In this model, the sex-stratified interaction was borderline significant, indicating the negative association was stronger in boys than girls, while allowing for sex-specific weights in the index. We adjusted the model for nutrition using the MNI as a metric of overall nutrition which was significantly related to improved IQ. Qualitatively, the prenatal exposure to EDCs, particularly related to changes in seven-year IQ described here (adjusted for prenatal nutrition), are similar to those compounds determined to be of concern by Tanner et al. [10]. Only MCINP differ between Tanner et al., 2020 and the current study. We used the stratified interaction WQS index in a causal framework using g-computation to address counterfactuals for exposure reduction to a mixture of 26 EDCs and the MNI as a metric for prenatal nutrition in a counterfactual linked to seven-year IQ.

Other studies have focused on the “what if” question with g methods using single chemical exposure levels. For example, Garcia et al. [26] focused on air pollutant interventions as measured by NO2 or PM2.5 and childhood asthma incidence. They asked, “How would the incidence rate of asthma in our participants change if we could modify their exposure to regional NO2 (or PM2.5)?” Instead of focusing on beta coefficients relating conditional incidence rate ratios for a one-unit change in air pollution exposure, they presented a population intervention measure that estimates asthma incidence rates had exposure been, for example, no higher than 20 ppb NO2. Their motivation, similar to ours, is to move beyond the report of point estimates to potentially improve the translation of the study to policymakers. That is, instead of only interpreting the results of the study in terms of the significance of a beta coefficient, the g methods which use an index to represent multi-dimensional components (e.g., environmental chemicals for the WQS index and dozens of nutrients for the MNI) allow researchers and decision makers to take an additional step using the metric to provide targets for changes. We considered several scenarios for reducing the WQS index to a target level—i.e., an overall cut to all non-persistent chemicals and cuts based on chemical classes. Persistent chemicals such as PFAS (called the “forever chemicals”) should also be considered for reduction strategies; however, their remediation is often ineffective but is attracting intensive research seeking effective technologies for their removal from the environment [27,28]. The illustration indicated that a severe cut is necessary to reduce environmental exposures to the “what if” target value. A cut of 70% of the non-persistent chemicals would no doubt be difficult to achieve, and removing all plasticizers alone is not enough to reach the target value. Use of a weighted index in such counterfactuals may complement current strategies for risk management which generally focus on single chemicals, even though risk increases when mixtures are considered. Further, chemical combinations are generally considered based on convenient groupings (e.g., assuming additivity) and not based on human exposure patterns or unintentional mixtures [29].

Biomonitoring data has consistently demonstrated that human exposures to environmental chemicals are from multiple chemical classes across the lifespan [30]. Of particular concern are exposures to vulnerable populations, such as pregnant women, as environmental exposures during critical periods of human development may increase risk to adverse health and developmental outcomes (e.g., [31,32]). There remains the challenge of how best to evaluate the impact of such exposures to mixtures of chemicals that typically have complex correlation patterns. Braun et al. [33] proposed three broad questions to focus analysis of complex mixtures in epidemiological studies: (i) What are the potential health impacts of individual agents? (ii) What is the interaction among agents? (iii) What are the health effects of cumulative exposures to multiple agents? They outlined different analysis strategies depending on the research question.

Our focus herein is focusing on the third question regarding cumulative exposures to multiple agents. It is in line with the concept of a mixture effect from the toxicology literature where relevant environmental exposures may result in the phenomenon of “something from nothing” [34]. Environmental chemicals may be at exposures well below an effect level, but joint action of the components may produce significant effects. For example, consideration of the joint action may include a wide range of chemical classes that act along an adverse outcome pathway and are not restricted to chemicals with the same mechanism of action. Ignoring joint action of mixtures may lead to significant underestimation of risk. Thus, an important research strategy is constructing a metric for a mixture effect and identifying the contribution of individual components to the effect using human observational data.

In addition, there is a computational advantage in regression models to reduce multiple components that are highly correlated to a single metric, thereby reducing concern of multicollinearity where regression coefficients have inflated variances [35], and the reversal paradox [36] where regression coefficients have opposite signs due to correlation between components. One such strategy for estimating a metric for a mixture effect is WQS regression (e.g., [17,19,20,37]) which results in an empirically weighted index of quantiles of the mixture components. Components of concern are identified by non-negligible weights.

Another important multi-dimensional exposure source to human health is dietary nutrients. Optimal levels of many dietary nutrients (including calories, dietary fat, protein, carbohydrates, vitamins, and minerals) depend on personal characteristics (for example, age, height, weight, sex, activity level, medical conditions, behavior, and dietary choices). Therefore, the perfect diet for one person may be very different for another, meaning that there is no one-size-fits-all optimal diet. We developed the MNI, which accounts for these personal characteristics in evaluating the nutritional value of an individual’s dietary intake based on published guideline values for dozens of macro- and micronutrients and dietary components. It is a metric of how close each nutrient and dietary component is to guideline values based on the appropriateness of the response for the characteristics of the subject (e.g., age, height, weight, sex, activity level, behavior). It assigns higher scores for nutrient concentrations that fall within the published dietary guidelines’ recommended concentration range and assigns lower scores if intake for a given nutrient deviates from this optimal range (i.e., deficient or excess intake). It provides an overall index score ranging from 0 to 100, with higher scores reflecting a more nutritious diet. Thus, a perfect MNI score would be obtained if adequate intake of all nutrients is met. Our focus on nutrition is to address a potential positive public health message about the benefits of a balanced diet based on an individual’s characteristics and dietary choices. The MNI is comprised of dozens of nutrients and food components [7,9] and provides a metric for subject-specific overall nutrition. An area of ongoing inquiry is whether individuals will use a metric such as MNI to improve their dietary choices.

The strategy of using an index in g methods as a metric of exposures to multiple environmental chemicals assumes a single index in the direction of adverse effects. It may be the case that some components in a selected mixture have positive associations and others negative associations with the selected outcome [17]. For example, during some developmental periods, metals may be either nutrients (essential elements) or toxins (e.g., [38]). An advantage of estimating weights in a nonlinear model in the first step of WQS regression is that the analyst may constrain the estimation in a single direction, one at a time, thereby further improving the ill-conditioning in the data due to complex correlations. The analysis strategy may then incorporate the resulting index with positive association (i.e., a metric for a positive mixture effect) and an index with negative association (i.e., a metric for a negative mixture effect) in a final model to elucidate the response surface of the toxins and nutrients together and improve the interpretability of the data through the two indices. In the case of prenatal EDCs and cognitive development, the association is only in the adverse direction.

This approach of using both positive and negative indices in the same model is in contrast to the approach taken in quantile g-computation where the objective is to measure the overall effect of the mixture [39]. The measure of the overall effect is based on the sum of all regression coefficients due to quantiled exposures in the generalized linear model, which is then also included in a g-computation framework. This is not the same as a mixture effect where components have joint effects [40].

Prospective cohort studies, such as SELMA, are second only to randomized studies in producing unbiased resulting inference [1]. However, there are ethical concerns for randomized exposure of pregnant women to potentially toxic chemicals or poor diets, so observational data is used to study prenatal human exposures and developmental effects. Although observational in nature, the prospective cohort study design has an important advantage over other observational designs in that exposure measurements precede the development of health/developmental outcomes, allowing the temporal sequence of the relation to be more firmly established and minimizing the risk of recall bias [1]. The g methods are then useful under the identifiability conditions for unbiased counterfactual comparisons.

As with all studies, our study has limitations. Although we controlled for many important confounders and we claim that the identifiability conditions hold, there may still be residual confounders due to other variables that we were unable to collect. Under the assumptions of identifiability, we claim causality; with the potential for residual confounders, our strategy is perhaps better framed as a step towards causality. The results are limited to the exposure patterns of a Swedish pregnancy cohort with a non-diverse racial profile. Further, the impact of using a weighted index in a causal inference has not yet been fully explored.

5. Conclusions

In conclusion, we found that in a population-based pregnancy cohort, early prenatal exposure to a mixture of EDCs is associated with lower levels of cognitive functioning at age seven. This adverse association is stronger in boys with sex-specific chemical effects. However, there remains a nutritional positive association as measured by My Nutrition Index. We used a flexible model to support a causal inference with counterfactuals related to two indices measured during pregnancy and their impact on seven-year IQ: reduced exposure to the mixture as measured by the stratified interaction WQS index and to improved prenatal dietary nutrition as measured by MNI. Several hypothetical scenarios were considered to demonstrate how a metric of exposure could be used to suggest reduction strategies in environmental chemicals. Further evaluation of such a strategy may support decision makers for risk management of mixtures of EDCs. In addition, the My Nutrition Index provides a personalized metric of nutritional value that individuals may use to guide their dietary choices to improve their dietary habits.

Appendix A

Table A1.

EDCs measured in first-trimester maternal biological samples (ng/mL) (N = 678).

Matrix Chemical Type Compound (Further Description) Abbreviation LOD/LOQ a % ≥LOD GM GSD WQS f
Urine Phenols 2,4,4′-trichloro-2′-hydroxydiphenyl ether Triclosan 0.100 92 1.34 9.9 x
bisphenol A BPA 0.050 100 1.52 2.4 x
4,4-bisphenol F (BPA replacement analogue) BPF 0.024 92 0.16 5.3 x
bisphenol S (BPA replacement analogue) BPS 0.009 98 0.07 3.0 x
Plasticizers (Phthalate & non-phthalate) monoethyl phthalate MEP 0.010 100 63.6 3.0 x
monobutyl phthalate MBP 0.100 100 67.6 2.2 x
monobenzyl phthalate MBzP 0.040 100 15.5 3.0 x
mono(2-ethylhexyl) phthalate MEHP 0.100 100 - -
mono(2-ethyl-5-hydroxyhexyl) phthalate MEHHP 0.020 100 - -
mono(2-ethyl-5-oxohexyl) phthalate MEOHP 0.030 100 - -
mono(2-ethyl-5-carboxypentyl) phthalate MECPP 0.020 100 - -
di-(2-ethylhexyl) phthalate (parent compound) DEHP b - - 63.6 2.4 x
mono(hydroxy-iso-nonyl) phthalate MHiNP 0.020 100 - -
mono(oxo-iso-nonyl) phthalate MOiNP 0.010 100 - -
mono(carboxy-iso-octyl) phthalate MCiOP 0.020 100 - -
diisononyl phthalate (parent compound) DINP c - - 42.4 2.6 x
monohydroxyisodecyl phthalate MHiDP 0.031 100 1.24 2.7 x
monocarboxyisononyl phthalate MCiNP 0.031 100 0.68 2.4 x
2-4-methyl-7-oxyooctyl-oxycarbonyl-cyclohexane carboxylic acid (phthalate replacement) MOiNCH 0.023 99 0.30 4.0 x
diphenylphosphate (organophosphate flame retardant) DPHP 0.042 100 1.34 2.5 x
Other Short-Lived 3,5,6-trichloro-2-pyridinol (organophosphate pesticide) TCP 0.035 100 1.25 2.5 x
3-phenoxybenzoic acid (pyrethroid pesticide) PBA 0.017 99 0.16 2.7 x
2-hydroxyphenanthrene (polycyclic aromatic hydrocarbon) 2OHPH 0.003 100 0.20 2.3 x
Serum Perfluoro-alkyl Substances (PFAS) perfluorooctanoic acid PFOA 0.020 100 1.58 1.8 x
perfluorooctane sulfonate PFOS 0.060 100 5.37 1.7 x
perfluorononanoic acid PFNA 0.010 100 0.53 1.7 x
perfluorodecanoic acid PFDA 0.020 100 0.26 1.6 x
perfluoroundecanoic acid PFUnDA 0.020 99 0.22 1.9 x
perfluorohexanesulfonic acid PFHxS 0.030 100 1.32 1.8 x
Plasma Persistent Chlorinated hexachlorobenzene HCB 0.005 100 0.05 1.4 x
trans-nonachlor Nonachlor 0.005 78 0.01 2.7 x
dichlorodiphenyltrichloroethane alone DDTa 0.015 99 - -
dichlorodiphenyldichloroethylene DDE 0.040 8 - -
total dichlorodiphenyltrichloroethane DDT d - - 0.18 2.0 x
polychlorinated biphenyl 74 PCB 74 0.005 73 - -
polychlorinated biphenyl 99 PCB 99 0.005 81 - -
polychlorinated biphenyl 118 PCB 118 0.005 99 - -
polychlorinated biphenyl 138 PCB 138 0.005 100 - -
polychlorinated biphenyl 153 PCB 153 0.005 100 - -
polychlorinated biphenyl 156 PCB 156 0.005 90 - -
polychlorinated biphenyl 170 PCB 170 0.005 100 - -
polychlorinated biphenyl 180 PCB 180 0.005 100 - -
polychlorinated biphenyl 183 PCB 183 0.005 76 - -
polychlorinated biphenyl 187 PCB 187 0.005 98 - -
total polychlorinated biphenyls PCB e - - 0.37 1.7 x

Abbreviations: GM, geometric mean; GSD, geometric standard deviation; LOD, limit of detection. Notes: Values < LOD retained the machine read value for urine and serum compounds, values <LOQ were substituted with LOQ/2 for plasma compounds. a LOD reported for all urine and serum compounds, LOQ reported for plasma compounds. b Molar sum of metabolites: mono-2-ethylhexyl, mono(2-ethyl-5-hydroxyhexyl), mono(2-carboxymethylhexyl), mono(2-ethyl-5-oxohexyl), and mono(2-ethyl-5-carboxypentyl) phthalates. c Molar sum of metabolites: mono(hydroxyisononyl), mono(oxoisononyl), and mono(carboxyisooctyl) phthalates. d Sum of DDT and its metabolite dichlorodiphenyldichloroethylene. e Sum of PCB congeners 74, 99, 118, 138, 153, 156, 170, 180, 183, 187. f Included in WQS analysis.

Figure A1.

Figure A1

Assumed directed acyclic graph (DAG) for prenatal EDCs and child IQ.

Author Contributions

Conceptualization/formal statistical analysis, C.G.; formal laboratory analysis, C.L. and H.K.; data curation, A.W.; writing—original draft preparation, C.G.; writing—review and editing, K.S., C.-G.B., A.W., C.L. and H.K.; cohort project administration, C.-G.B.; funding acquisition, C.G. and C.-G.B. All authors have read and agreed to the published version of the manuscript.

Funding

We gratefully acknowledge support from the National Institutes of Health (T32ES007334, R01ES028811, P30ES023515, U2CES026555) for biostatistical methods development for environmental mixtures and EDC Mix Risk (#634880) from the European Union’s Horizon 2020 Research and Innovation Program for analytical samples.

Institutional Review Board Statement

Participants provided written consent and the study was reviewed and approved by the Ethical Review Board (Uppsala, Sweden) (Dnr 2007/062 and Dnr 2017/177).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

According to the Ethical Review Board decision and obtained personal consent, data on participating children or their mothers can not be made freely available since they constitute clinical data subject to secrecy in accordance with the Swedish Public Access to Information and Secrecy Act [OSL 2009:400]. Unique combinations of clinical data could make a study participant identifiable, and consequently, a review of secrecy may result in restrictions regarding data availability.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Maki K.C., Slavin J.L., Rains T.M., Kris-Etherton P.M. Limitations of observational evidence: Implications for evidence-based dietary recommendations. Adv. Nutr. 2014;5:7–15. doi: 10.3945/an.113.004929. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Robins J.M., Hernan M.A., Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology. 2000;11:550–560. doi: 10.1097/00001648-200009000-00011. [DOI] [PubMed] [Google Scholar]
  • 3.Snowden J.M., Rose S., Mortimer K.M. Implementation of g-computation on a simulated data set: Demonstration of a causal inference technique. Am. J. Epidemiol. 2011;173:731–738. doi: 10.1093/aje/kwq472. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Naimi A.I., Cole S.R., Cole E.H. An introduction to g methods. Int. J. Epidemiol. 2017;46:756–762. doi: 10.1093/ije/dyw323. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Daniel R.M., Cousens S.N., De Stavola B.L., Kenward M.G., Sterne J.A.C. Methods for dealing with time-dependent confounding. Stat. Med. 2013;32:1584–1618. doi: 10.1002/sim.5686. [DOI] [PubMed] [Google Scholar]
  • 6.Wang A., Nianogo R.A., Arah O.A. g-computation of average treatment effects on the treated and the untreated. BMC Med. Res. Methodol. 2017;17:1–5. doi: 10.1186/s12874-016-0282-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Gennings C., Wolk A., Hakansson N., Lindh C., Bornehag C.G. Contrasting prenatal nutrition and environmental exposures in association with birth weight and cognitive function in children at 7 years. BMJ Nutr. Prev. Health. 2020;3:162. doi: 10.1136/bmjnph-2020-000099. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Gennings C., Manning L., Keefer L. Evaulation of My Nutrition Index in an IBD Patient Population. J. Nutr. Med. Diet Care. 2021;7:49. [Google Scholar]
  • 9.Busgang S.A., Malin A.J., Gennings C. My Nutrition Index: A method for measuring optimal daily nutrient intake. BMC Nutr. 2022. in press . [DOI] [PMC free article] [PubMed]
  • 10.Tanner E.M., Hallerback M.U., Wikstrom S., Lindh C., Kiviranta H., Gennings C., Bornehag C.G. Early prenatal exposure to suspected endocrine disruptor mixtures is associated with lower IQ at age seven. Environ. Int. 2020;134:105185. doi: 10.1016/j.envint.2019.105185. [DOI] [PubMed] [Google Scholar]
  • 11.Bornehag C.G., Moniruzzaman S., Larsson M., Lindstrom C.B., Hasselgren M., Bodin A., Kobyletzkic L.B.V., Carlstedt F., Lundin F., Nånberg E., et al. The SELMA study: A birth cohort study in Sweden following more than 2000 mother-child pairs. Paediatr. Perinat. Epidemiol. 2012;26:456–467. doi: 10.1111/j.1365-3016.2012.01314.x. [DOI] [PubMed] [Google Scholar]
  • 12.Wechsler D. Wechsler Intellignece Scale in Children. 4th ed. The Psychological Corporation; San Antonio, TX, USA: 2003. [Google Scholar]
  • 13.Marinopoulou M., Billstedt E., Lin P.I., Hallerback M., Bornehag C.G. Number of words at age 2.5 years is associated with intellectual functioning at age 7 years in the SELMA study. Acta Paediatr. 2021;110:2134–2141. doi: 10.1111/apa.15835. [DOI] [PubMed] [Google Scholar]
  • 14.Van der Elst W., Ouwehand C., van Rijn P., Lee N., Van Boxtel M., Jolles J. The shortened Raven Standard Progressive Matrices: Item response theory-based psychometric analyses and normative. Assessment. 2013;20:48–59. doi: 10.1177/1073191111415999. [DOI] [PubMed] [Google Scholar]
  • 15.Bouma A., Mulder J., Lindeboom J. Neuropsychologische Diagnostiek: Handboek [Neuropsychological Assessment: Manual] Swets & Zeitlinger; Lisse, The Netherlands: 1996. [Google Scholar]
  • 16.Raven J., Raven J., Court J. Manual for Raven’s Progressive Matrices and Vocabulary. Oxford Psychologists Press; Oxford, UK: 1998. [Google Scholar]
  • 17.Carrico C., Gennings C., Wheeler D.C., Factor-Litvak P. Characterization of weighted quantile sum regression for highly correlated data in a risk analysis setting. J. Agric. Biol. Environ. Stat. 2015;20:100–120. doi: 10.1007/s13253-014-0180-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Bello G. Ph.D. Thesis. Virginia Commonwealth University-Graduate School at VCU Scholars Compass; Richmond, VA, USA: 2014. Application and Extension of Weighted Quantile Sum Regresion for the Development of a Clinical Risk Prediction Tool. [Google Scholar]
  • 19.Curtin P., Kellogg J.J., Cech N.B., Gennings C. A random subset implementation of weighted quantile sum (WQSRS) regression for analysis of high-dimensional mixtures. Commun. Stat.-Simul. Comput. 2019;50:1119–1134. doi: 10.1080/03610918.2019.1577971. [DOI] [Google Scholar]
  • 20.Tanner E.M., Bornehag C.G., Gennings C. Repeated holdout validation for weighted quantile sum regression. MethodsX. 2019;6:2855–2860. doi: 10.1016/j.mex.2019.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Zou H., Hastie T. Regularization and variable selection via the elastic net. J. R. Soc. Ser. B. 2005;67:301–320. doi: 10.1111/j.1467-9868.2005.00503.x. [DOI] [Google Scholar]
  • 22.Renzetti S., Curtin P., Just A.C., Bello G., Gennings C. gWQS: Generalized Weighted Quantile Sum Regression. R Package, Version 2.0.0. 2019. [(accessed on 1 June 2021)]. Available online: https://rdrr.io/cran/gWQS/
  • 23.Cowell W., Colicino E., Levin-Schwartz Y., Enlow M.B., Amarasiriwardena C., Andra S.S., Wright R.J. Prenatal metal mixtures and sex-specific infant negative affectivity. Env. Epidemiol. 2021;5:e147. doi: 10.1097/EE9.0000000000000147. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Svensson K., Tanner E., Gennings C., Lindh C., Kiviranta H., Wikstrom S., Bornehag C.G. Prenatal exposures to mixtures of endocrine disrupting chemicals and children’s weight trajectory up to age 5.5 in the SELMA study. Sci. Rep. 2021;11:11036. doi: 10.1038/s41598-021-89846-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Chatton A., Borgne F.L., Leyrat C., Gillaizeau F., Rousseau C., Barbin L., Laplaud D., Léger M., Giraudeau B., Foucher Y. g-computation, propensity score-based methods, and targeted maximum likelihood estimator for causal inference with different covariates sets: A comparative simulation study. Sci. Rep. 2020;10:9219. doi: 10.1038/s41598-020-65917-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Garcia E., Urman R., Berhane K., McConnell R., Gilliland F. Effects of policy-driven hypothetical air pollutant interventions on childhood asthma incidence in southern California. Proc. Natl. Acad. Sci. USA. 2019;116:15883–15888. doi: 10.1073/pnas.1815678116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Dhore R., Murthy G.S. Per/polyfluoroalkyl substances production, applications and environmental impacts. Bioresour. Technol. 2021;341:125808. doi: 10.1016/j.biortech.2021.125808. [DOI] [PubMed] [Google Scholar]
  • 28.Zhang Z., Sarkar D., Biswas J.K., Datta R. Biodegradation of per- and polyfluoroalkyl substances (PFAS): A review. Bioresour. Technol. 2022;344:126223. doi: 10.1016/j.biortech.2021.126223. [DOI] [PubMed] [Google Scholar]
  • 29.Ruden C. Future chemical risk management: Accounting for combination effects and assessing chemicals in groups. SOU 2019:45. [(accessed on 7 December 2021)];Swedish Government Official Reports. 2019 Ministry Publications Series. Available online: https://www.government.se/4adb1a/contentassets/ee36b3e49c354bb8967f3a33a2d5ca50/future-chemical-risk-management---accounting-for-combination-effects-and-assessing-chemicals-in-groups-sou-201945.
  • 30.Woodruff T.J., Zota A.R., Schwartz J.M. Environmental chemicals in pregnant women in the United States: NHANES 2003–2004. Environ. Health Perspect. 2011;119:878–885. doi: 10.1289/ehp.1002727. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Levin-Schwartz Y., Gennings C., Schnaas L., del Carmen Hernandez Chavez M., Bellinger D.C., Tellez-Rojo M.M., Baccarelli A.A., Wright R.O. Time-varying associations between prenatal metal mixtures and rapid visual processing in children. Environ. Health. 2019;18:92. doi: 10.1186/s12940-019-0526-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Bornehag C.G., Engdahl E., Hallerback M.U., Wikstrom S., Lindh C., Ruegg J., Tanner E., Gennings C. Prenatal exposure to bisphenols and cognitive function in children at 7 years of age in the Swedish SELMA study. Environ. Int. 2021;150:106433. doi: 10.1016/j.envint.2021.106433. [DOI] [PubMed] [Google Scholar]
  • 33.Braun J.M., Gennings C., Hauser R., Webster T.F. What Can Epidemiological Studies Tell Us about the Impact of Chemical Mixtures on Human Health? Environ. Health Perspect. 2016;124:A6–A9. doi: 10.1289/ehp.1510569. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Silva E., Rajapakse N., Kortenkamp A. Something from “nothing”--eight weak estrogenic chemicals combined at concentrations below NOECs produce significant mixture effects. Environ. Sci. Technol. 2002;36:1751–1756. doi: 10.1021/es0101227. [DOI] [PubMed] [Google Scholar]
  • 35.Neter J., Kutner M., Nachtsheim C., Wasserman W. Applied Linear Statistical Models. 4th ed. Irwin; Chicago, IL, USA: 1996. [Google Scholar]
  • 36.Tu Y., Gunnell D., Gilthorpe M. Simpson’s Paradox, Lord’s Paradox, and Suppression Effects are the same phenomenon--the reversal paradox. Emerg. Themes. Epidemiol. 2008;5:2. doi: 10.1186/1742-7622-5-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Czarnota J., Gennings C., Wheeler D.C. Assessment of weighted quantile sum regression for modeling chemical mixtures and cancer risk. Cancer Inform. 2015;14:159–171. doi: 10.4137/CIN.S17295. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Arora M., Reichenberg A., Willfors C., Austin C., Gennings C., Berggren S., Lichtenstein P., Anckarsäter H., Tammimies K., Bolte S. Fetal and postnatal metal dysregulation in autism. Nat. Commun. 2017;8:1–10. doi: 10.1038/ncomms15493. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Keil A.P., Buckley J.P., O’Brien K.M., Ferguson K.K., Zhao S., White A.J. A Quantile-Based g-Computation Approach to Addressing the Effects of Exposure Mixtures. Environ. Health Perspect. 2020;128:47004. doi: 10.1289/EHP5838. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Gennings C. Comment on “A Quantile-Based g-Computation Approach to Addressing the Effects of Exposure Mixtures”. Environ. Health Perspect. 2021;129:38001. doi: 10.1289/EHP8739. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

According to the Ethical Review Board decision and obtained personal consent, data on participating children or their mothers can not be made freely available since they constitute clinical data subject to secrecy in accordance with the Swedish Public Access to Information and Secrecy Act [OSL 2009:400]. Unique combinations of clinical data could make a study participant identifiable, and consequently, a review of secrecy may result in restrictions regarding data availability.


Articles from International Journal of Environmental Research and Public Health are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES