Skip to main content
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2018 Jul 13.
Published in final edited form as: J Toxicol Environ Health A. 2017 Jul 13;80(9):502–512. doi: 10.1080/15287394.2017.1330581

Prevalence and associated demographic characteristics of exposure to multiple metals and their species in human populations: the United States NHANES, 2007–2012

Youn K Shim 1, Michael D Lewin 1, Patricia Ruiz 1, June E Eichner 2, Moiz M Mumtaz 1
PMCID: PMC5693367  NIHMSID: NIHMS916014  PMID: 28703686

Abstract

Lead (Pb), cadmium (Cd), mercury (Hg), and arsenic (As) are among the top 10 pollutants of global public health concern. Studies have shown that exposures to these metals produce severe adverse effects. However the mechanisms underlying these effects, particularly joint toxicities, are poorly understood in humans. The objective of this investigation was to identify and characterize prevalent combinations of these metals and their species in the U.S. National Health and Nutrition Examination Survey (NHANES) population to provide background data for future studies of potential metal interactions. Exposure to each metal was defined as urine or blood levels ≥ respective medians of the NHANES 2007–2012 participants ≥6 years (n=7408). Adjusted-odds ratios (adj-OR) and 95% confidence intervals (CI) were determined for demographic factors (age, gender, and race/ethnicity). All models included two additional covariates, cotinine and body mass index (BMI). Species-specific analysis was also conducted for As and Hg including iAs (urinary arsenous acid and/or arsenic acid), met-iAs (urinary monomethylarsonic acid and/or dimethylarsinic acid), and oHg (blood methyl-mercury and/or ethyl-mercury). For combinations of As and Hg species, age- and gender-specific prevalence was determined among NHANES 2011–2012 participants (n=2342). Approximately 49.3% of the U.S. population were exposed to a combination of three or more metals. The most prevalent unique specific combinations were Pb/Cd/Hg/As, Pb/Cd/Hg, and Pb/Cd. Age was consistently associated with these combinations: adj-ORs ranged from 10.9 (Pb/Cd) to 11.2 (Pb/Cd/Hg/As). Race/ethnicity was significant for Pb/Cd/Hg/As. Among women of reproductive age, frequency of oHg/iAs/met-iAS and oHg/met-iAs was 22.9 and 40.3%, respectively. These findings of metals and their species in human population may help prioritize efforts to assess joint toxicities and their impact on public health.

Keywords: Prevalence of multiple metal exposure, exposure patterns, NHANES

Introduction

Lead (Pb), cadmium (Cd), mercury (Hg), and the metalloid, arsenic (As), which are referred to as metals from here on, are 4 metals among the top 10 chemicals of major public health global concern [World Health Organization (WHO 2010)]. These metals contaminate the air, surface waters, and soil through natural and human actions (Jarup 2003; Scheckel et al, 2013; Kah et al, 2012; Bradham et al, 2016; Diamond et al, 2016; Tagne-Fotso et al, 2016) and may enter the human body singly or in combinations from the same or different sources. These ubiquitous and persistent metals exert some common mechanisms of toxicity, such as production of oxidative stress (Jomova and Valko 2011; Bridges and Zalups, 2017), and release of stress proteins and interference with the functions of essential metals (Goyer 1997; Davidson et al. 2015). Exposures to multiple metals demonstrated enhanced toxicities compared to single metals, even at low dose levels (Wang and Fowler 2008). In humans, adverse effects of the individual metals have been widely studied (Sweet and Zelikoff, 2001; Ratcliffe et al 1996, Bernstam and Nriagu 2000; ATSDR 2015b), however their potential effects in combinations and associated effect-dose levels have not been well examined, particularly in higher combinations. Such studies are resource and time intensive because the available data are sparse.

One of the challenging issues in investigating potential adverse health effects of multiple chemical exposure are the uncertainties involved. In order to understand potential health impacts from multiple metals and associated dose levels, one needs to account for many parameters, including routes and durations of exposure, bioavailability, pharmacokinetics, metabolism, and half-lives. Given the complexities in calculating internal doses, for future research, it is necessary to prioritize metal combinations that are prevalent in human populations [National Institute of Environmental Health Sciences (NIEHS) 2012].

Exposures to individual chemicals in the non-institutionalized U.S. civil population have been characterized, mainly using data from the biomonitoring program, the National Health and Nutrition Examination Survey (NHANES). This program includes limited assessment of the sources or routes of exposure, but identifies individual, rather than combinations of, environmental chemicals of significance in blood and urine specimens [Centers for Disease Control and Prevention (CDC) 2009]. Recent studies showed that the NHANES data may also be used for further research on co-exposure to multiple chemicals in the U.S. population (Qian et al. 2015; Sobus et al. 2015; Woodruff et al. 2011). The objective of this investigation was to examine co-exposure to multiple metals and their species in the NHANES population to provide background data for future studies of potential metal interactions. The specific aims of our study were to (1) estimate the prevalence of all possible unique combinations of the 4 metals Pb, Cd, Hg, and As detected at or above the population median levels in blood and urine specimens of the 2007–2012 NHANES participants and identify most prevalent combinations, (2) identify the population profiles (i.e., age, gender, race), and (3) determine unique species combinations of blood organic-Hg and urinary inorganic-related As species.

Methods and Materials

Study population and Metal analytes

Briefly, NHANES is a nationally representative cross-sectional survey conducted by the National Center for Health Statistics, CDC. The survey aims to assess the health and nutritional status of the civilian, non-institutionalized U.S. population using a complex multi-stage probability sample design (NHANES 2015). NHANES collects information through a combination of interviews and physical examinations. Blood and urine specimens are collected for lab tests.

Various environmental chemical analytes were determined in subsamples of NHANES participants. Blood metals were measured in participants aged one year and older, whereas urine metals were determined only in subsamples of participants aged 6 years and older. Participants in three NHANES cycles (2007–2008–2009–2010, and 2011–2012) who were tested for both blood and urinary Pb, Cd, Hg as well as urinary As were included. NHANES measured As in urine but not in blood, because of its short half-life in blood. The cycles were combined to improve statistical power and obtain more reliable estimates.

Data on blood As and Hg species were available for only the 2011–2012 cycle. Urinary As species included arsenous acid, arsenic acid, monomethylarsenic acid (MMA), and dimethylarsinic acid (DMA); blood Hg species included methyl-Hg and ethyl-Hg. The lab methods used to measure the 4 metals (Pb, Cd, Hg, and As) and As and Hg species in blood and urine specimens have been described in NHANES (2015). NHANES urine samples are ‘spot’ samples that are collected one time at the time of physical examination (NHANES 2015; Pleil and Sobus, 2016). To adjust for variations in urinary dilution associated with changes in hydration status regression-based creatinine adjustment was performed for all urinary metal levels (Cole et al. 1995; Thompson et al. 1990).

Statistical analysis

All statistical analyses were conducted using SAS Version 9.3 survey procedures (SAS Institute Inc., Cary, NC, 2010) and SAS-callable SUDAAN Version 11.0 (Research Triangle Park, NC). SUDAAN’s Descript procedure was used for estimation of percentiles for subdomains and for estimation of geometric means (GM) and confidence intervals (CI). SURVEYFREQ, SURVEYREG, and SURVEYLOGISTIC procedures were used for weighted % and means, linear regressions, and logistic regressions, respectively. NHANES assigned each participant a sample weight to account for his or her probability of selection as well as for non-response (Johnson et al. 2013). Subsample weights, modified for combined cycles, and design variables were used to account for NHANES’s complex sample design. Taylor series linearization methods were used for variance estimation.

For all analytes, the sample % participants was presented with a concentration above the limit of detection (LOD). For each analyte with a detection rate above 50%, median and GM concentrations were also calculated with their respective 95% CI. The medians were calculated using data from all 3 cycles, except for blood methyl-Hg for which only the 2011–2012 data were available. When the concentration of an analyte was below the LOD, a value assigned by NHANES (i.e., LOD divided by the square root of 2) was employed.

Our main analyses included two parts. The first part involved the 4 metals and the second part was an advances analysis of the As and Hg species data. In the first part of the analysis that included 2007–2012 NHANES participants, a dichotomous variable was created separately for each of the urinary total As, blood and urinary Pb, blood and urinary Cd, and blood and urinary total Hg, indicating whether a person’s metal concentrations were greater than or equal to the respective medians of the entire study population. All possible unique combinations of the 4 metals (i.e., None, As, Cd, Hg, Pb, As/Cd, As/Hg, As/Pb, Cd/Hg, Cd/Pb, Hg/Pb, As/Cd/Hg, As/Cd/Pb, As/Hg/Pb, Cd/Hg/Pb, and As/Cd/Hg/Pb) were identified, where each metal concentration was at or above the population median in urine or blood, or both. Then the prevalence for each of the unique combinations of the metals was calculated. In addition, as a summary measure, the set of two metals that are most common across the unique combinations was also identified. For example, we estimated the prevalence of the two-metal set, As and Cd, was estimated by summing the prevalence estimates of all unique combinations containing As and Cd (i.e., As/Cd, As/Cd/Hg, As/Cd/Pb, and As/Cd/Hg/Pb). In the same manner, the prevalence of the two-metal set, As and Hg, was determined by summing the prevalence estimates of As/Hg, As/Cd/Hg, As/Hg/Pb, and As/Cd/Hg/Pb. Similarly, the set of three metals that are most commonly detected together across the unique combinations was identified.

To examine the demographic factors associated with the three most prevalent unique combinations, adjusted odds ratios (adj-OR) and their 95% CIs were calculated by employing logistic regression analyses where the odds of having the combination was regressed on the covariates. The controls consisted of NHANES participants who had no or only one of the 4 metals detected at or above the population median levels. The covariates of interest included age (in 10 year increments), gender, and race/ethnicity (Mexican-Americans, Non-Hispanic Whites, Non-Hispanic Blacks, and Others). Serum cotinine concentration (<0.015, 0.015–9.999, ≥10 ng/mL) and body mass index (BMI) were included in all models as potential confounders. These two potential confounders were selected because of the availability of objective quantifiable measurements and their known associations with both the metals and demographic variables (Chiba and Masironi 1992; CDC 2016; Padilla et al 2010). Serum cotinine has been established as a valid biomarker for active tobacco smoking (≥10 ng/mL) (Pirkle et al. 1996) and secondhand tobacco smoking (<10 ng/mL) (Benowitz 1996; Pirkle et al. 1996). Non-exposure to tobacco smoke was defined as <LOD (i.e., <0.015 ng/mL). For participants 21 years and older, BMI was calculated as weight in kg divided by height-squared in m2. For participants younger than 21 years, CDC BMI-for-Age Growth Chart coefficients were used for BMI calculation (CDC 2002). BMI was categorized into underweight, normal, overweight, and obese. For participants 21 years and older, the corresponding cut points were <18.5, 18.5–24.9, and 25–29.9, and ≥30. For participants younger than 21 years, the corresponding cut points were the <5th, 5th–<85th, 85th–<95th, and ≥95th percentiles obtained from the age- and gender -specific growth charts.

For the second part of the analysis that included only the 2011–2012 cycle participants, a dichotomous variable was created for each of the urinary inorganic-related As species (arsenous acid, arsenic acid, MMA, and DMA) and blood oHg species (methyl-Hg and ethyl-Hg), indicating whether an individual’s concentration was greater than or equal to the respective LOD values. LOD values were selected for cut points, because stable estimates of the population medians could not be obtained for several species (e.g., urinary arsenous acid and arsenic acid) due to a small number of participants with a level ≥ LOD as shown in Table 1. The As species was grouped into iAs (urinary arsenous acid and arsenic acid) and met-iAS (urinary MMA and DMA). The Hg species consisted of oHg (blood methyl-Hg and ethyl-Hg). Unique combinations of iAs, met-iAs, and oHg were presented, where each species group had at least one species detected at or above the respective LOD level. The gender- and age-prevalence estimates were obtained by categorizing the ages into 3 groups (6–14, 15–44, and ≥45 years). Women of the reproductive age group (15–44 years) were considered a vulnerable subgroup, since these metals cross the placental barrier (Baranowska 1995; Rudge et al. 2009; Piasek et al, 2016) and can also be detected in breast milk (Ettinger et al. 2014; Garcia-Esquinas et al. 2011).

Table 1.

Median and geometric mean concentrations of As, Cd, Pb, and Hg in urine and blood for the U.S. population 6 years and older, NHANES 2007–2012.

Chemical analytes LOD level a Sample N Detection rate Weighted % Weighted Concentrationb


2007–2010 2011–2012 GM (SE) Median (95% CI)
Urinec (μg/L)

 As, total 0.74 1.25 7969 98.5 9.78 (0.29) 7.91 (7.44, 8.44)
  Arsenous acid 1.2 0.48 7964 13.0 - -
  Arsenic acid 1.0 0.87 7945 2.1 - -
  Arsenobetaine 0.4 1.19 7963 54.2 1.80 (0.06) 1.20 (1.11, 1.28)
  Arsenocholine 0.6 0.28 7964 2.6 - -
  Dimethylarsinic acid 1.7 1.8 7964 82.2 4.27 (0.07) 3.92 (3.80, 4.07)
  Monomethylarsonic acid 0.9 0.89 7964 31.2 - -
  Trimethylarsine oxide 1.0 0.25 7964 1.3 - -
 Cd 0.042 0.56 7979 85.8 0.21 (0.00) 0.20 (0.19, 0.21)
 Hg 0.08 0.05 8006 94.3 0.47 (0.01) 0.44 (0.42, 0.47)
 Pb 0.1 0.08 7979 97.1 0.52 (0.01) 0.50 (0.49, 0.52)

Blood (μg/L)

 Cd 0.2 0.16 7603 69.8 0.31 (0.00) 0.27 (0.26, 0.28)
 Hg, total 0.33 0.16 7603 85.2 0.81 (0.03) 0.76 (0.72, 0.81)
  Inorganic 0.35 0.27 7586 19.1 - -
  Ethyld - 0.16 2376 3.6 - -
  Methyld - 0.12 2376 84.9 0.50 (0.03) 0.49 (0.42, 0.57)
 Pb (μg/dL) 0.25 0.25 7603 99.5 1.10 (0.02) 1.07 (1.03, 1.11)

Abbreviations: As, arsenic; Cd, cadmium; Pb, lead; Hg, mercury; GM, geometric mean; SE, standard error; 95% CI, 95% confidence interval.

a

NHANES survey cycles 2007–2008 and 2009–2010 had the same LOD levels.

b

Weighted geometric mean and median concentrations were calculated for each analyte with a detection rate above 50%. When the concentration of an analyte was below the limit-of-detection (LOD), we used a value assigned by NHANES (i.e., LOD divided by the square root of 2).

c

All urinary concentrations were adjusted per gram of urine creatinine using regression methods (Thompson et al. 1990).

d

NHANES measured ethyl-Hg and methyl-Hg in blood only in the NHANES 2011–2012 cycle.

Results

Prevalence of Pb, Cd, Hg, and As combinations

Most NHANES participants 6 years and older in 2007–2012 exhibited detectable levels of the 4 metals in urine (85.8 – 98.5%) and blood specimens (69.8 – 99.5%) (Table 1). The weighted creatinine-adjusted median values for As, Cd, Hg, and Pb in urine specimens were 7.91 μg/L, 0.2 μg/L, 0.44 μg/L, and 0.5 μg/L, respectively. The weighted median values in blood specimens were 0.27 μg/L for Cd, 0.76 μg/L for Hg, and 1.07 μg/dL for Pb (Table 1).

In 8.4% of the U.S. population 6 years and older, none of the 4 metals were detected in urine or blood at or above the respective population medians. Among the remaining participants, 16.2% exhibited only a single metal, 26.1% two metals, and 49.3% 3 or 4 metals (Table 2). The most common unique combination among all 16 possible combinations of 4 metals was the quaternary combination of As/Cd/Hg/Pb (22.1%), followed by the ternary combination of Cd/Hg/Pb (10.6%) and binary combination of Cd/Pb (8.4%) (Table 2). To summarize the findings, the set of two metals most commonly detected was also examined together across the unique combinations. In 45.5 % of the population, Cd and Pb were detected together in various unique forms [i.e., 8.4 (Cd/Pb) + 4.4 (As/Cd/Pb) + 10.6 (Cd/Hg/Pb) + 22.1(As/Cd/Hg/Pb)], while only 8.4% of the population displayed the unique Cd/Pb combination. The set of three metals most commonly detected together was Cd, Hg, and Pb; 32.7% of the population had the three metals in various unique forms [i.e., 10.6 (Cd/Hg/Pb) + 22.1 (As/Cd/Hg/Pb)] versus 10.6% had the unique Cd/Hg/Pb combination.

Table 2.

Specific unique combinations of As, Cd, Pb, and Hg detected at or above the respective median concentrations in urine or blood among the U.S. population 6 years and older, NHANES 2007–2012.

Metal combinationa Sample Nb Prevalencec

Weighted % (95% CI)
None 590 8.4 (7.0–9.7)
As 206 2.7 (2.2–3.1)
Cd 320 4.7 (3.9–5.6)
Pb 347 3.6 (3.1–4.2)
Hg 333 5.2 (4.6–5.8)
As/Cd 119 1.4 (1.0–1.7)
As/Hg 369 5.0 (4.3–5.8)
As/Pb 236 2.2 (1.8–2.7)
Cd/Hg 317 5.4 (4.6–6.3)
Cd/Pb 632 8.4 (7.3–9.5)
Pb/Hg 294 3.7 (3.1–4.3)
As/Cd/Hg 449 6.5 (5.5–7.4)
As/Cd/Pb 381 4.4 (3.7–5.1)
As/Hg/Pb 448 5.7 (4.9–6.5)
Cd/Hg/Pb 696 10.6 (9.3–11.9)
As/Cd/Hg/Pb 1671 22.1 (20.3–23.9)

Abbreviation: As, arsenic; Cd, cadmium; Pb, lead; Hg, mercury; 95% CI, 95% confidence interval.

a

As and Hg represent total As and total Hg.

b

All participants (n=7408) were tested for urinary and blood Cd, Pb, and Hg as well as urinary As.

c

Detected in blood and/or urine specimens.

Demographic characteristics of the most prevalent metal combinations

The participants who had one of the three most prevalent unique combinations (Pb/Cd, Pb/Cd/Hg, and Pb/Cd/Hg/As) were older (weighted GM age: 51.6, 52.8, and 53.7 years for Pb/Cd, Pb/Cd/Hg, and Pb/Cd/Hg/As, respectively) than the control group which consisted of participants who did not exhibit any or only one metal at or above the respective median concentration (weighted GM age: 22.9 years) (Table 3). These individuals were also more likely to be an active tobacco smoker as indicated by a serum cotinine level ≥ 10 ng/mL (weighted %: 40.1, 37.6, and 23.8% for Pb/Cd, Pb/Cd/Hg, and Pb/Cd/Hg/As, respectively) than controls (weighted %: 9.3%). In the multivariate logistic regression analyses, age was the most consistent independent risk factor among the demographic variables (age, gender, and race/ethnicity), for all three unique combinations (Figure 1). The adj-ORs for age in 10 year increments were 10.9 for Pb/Cd, 11.1 for Pb/Cd/Hg, and 11.2 for Pb/Cd/Hg/As, after taking into account the effects of gender, race/ethnicity, cotinine, and BMI. Race/ethnicity was a significant risk factor for the Pb/Cd/Hg/As combination. Compared to Non-Hispanic Whites, the odds of displaying the quaternary combination was 1.8-fold higher for Non-Hispanic Blacks, 2.6-fold for Mexican Americans, and 5.6-fold for “Other” race/ethnicity group that includes Asian ethnicity, after adjusting for the effects of all other covariates. Non-Hispanic Blacks also presented a significantly increased odds of having the Pb/Cd/Hg combination, while Mexican-Americans a significantly increased odds of the Pb/Cd combination, compared to non-Hispanic Whites. Males were less likely to display the Pb/Cd/Hg combination, but there was no significant gender effect for the other two prevalent combinations.

Table 3.

Demographic characteristics associated with most prevalent unique combinations of As, Cd, Pb, and Hg at or above the respective population medians among U.S. population 6 years and older, NHANES 2007–2012.

Characteristics None or one metal Pb/Cd Pb/Cd/Hg Pb/Cd/Hg/As

Na (Weighted %) Na (Weighted %) Na (Weighted %) Na (Weighted %)
Gender
 Male 910 (52.0) 340 (55.4) 330 (48.9) 813 (49.8)
 Female 841 (48.0) 274 (44.6) 345 (51.1) 819 (50.2)
Age, years
 Weighted GM±SE 22.9 ± 0.37 51.6 ± 0.83 52.8 ± 0.70 53.7 ± 0.43
Race/Ethnicity
 Non-Hispanic Blacks 398 (22.7) 132 (21.5) 166 (24.6) 340 (20.8)
 Mexican Americans 360 (20.6) 112 (18.2) 75 (11.1) 233 (14.3)
 Others 256 (14.6) 54 (8.8) 65 (9.6) 479 (29.4)
 Non-Hispanic Whites 737 (42.1) 316 (51.5) 369 (54.7) 580 (35.5)
Serum cotinine (ng/mL)b
 ≥10 163 (9.3) 246 (40.1) 254 (37.6) 389 (23.8)
 0.015–9.999 1143 (65.3) 281 (45.8) 298 (44.1) 863 (52.9)
 <0.015 445 (25.4) 87 (14.2) 123 (18.2) 380 (23.3)
BMI
 Obese 564 (32.2) 221 (36.0) 219 (32.4) 496 (30.4)
 Overweight 400 (22.8) 186 (30.3) 234 (34.7) 574 (35.2)
 Normal/Underweight 787 (44.9) 207 (33.7) 222 (32.9) 562 (34.4)

Abbreviation: As, arsenic; Cd, cadmium; Pb, lead; Hg, mercury; GM, geometric mean; SE, standard error; BMI, Body mass index.

a

Sample N: None or one metal=1751, Pb/Cd = 614, Pb/Cd/Hg = 675, Pb/Cd/Hg/As = 1632.

b

Serum cotinine levels ≥10 ng/mL indicate active tobacco smoking (Pirkle et al. 1996) and <10 ng/mL indicate second hand tobacco smoking (Benowitz 1996; Pirkle et al. 1996). Non-exposure to tobacco smoke defined as <LOD (i.e., 0.015 ng/mL).

Figure 1.

Figure 1

Demographic factors associated with most prevalent unique combinations of As, Cd, Pb, and Hg among U.S. population 6 years and older, NHANES 2007–2012.

Abbreviation: As, arsenic; Cd, cadmium; Pb, lead; Hg, mercury; OR, odds ratio; 95% CI, 95% confidential interval; SE, standard error; BMI, Body mass index. Age in 10 year increments. Serum cotinine levels ≥ 10 ng/mL indicate active tobacco smoking (Pirkle et al. 1996) and <10 ng/mL indicate second hand tobacco smoking (Benowitz 1996; Pirkle et al. 1996). We defined non-exposure to tobacco smoke as <LOD (i.e., 0.015 ng/mL).

Prevalence of oHg, iAs, and met-iAs combinations

Our analysis of the species combinations included the 2011–2012 participants (sample n=2342). The most prevalent combination of the species at a detectable level was the binary combination of oHg/met-iAs (42.1%), followed by the ternary combination of oHg/met-iAs/iAs (24.7%) (data not shown). The prevalence was lower for the binary combinations of iAs/met-iAs (3.4%) and oHg/iAs (0.5%) (data not shown). The age- and gender-specific prevalence of unique combinations of oHg, met-iAs, and iAs is illustrated in Figure 2. Among males, the ternary combination of oHg/met-iAs/iAs was detectable at or above the LOD levels in 22.6% of the 6–14 years group (sample n=239), reaching a peak (38.4%) in the 15–44 years group (sample n=482), then declined to 22.9% in the 45 years or older group (sample n=450). In females, the prevalence of oHg/met-iAs/iAs peaked (25.7%) in the 6–14 years group (sample n=242), followed by 22.9% in the reproductive age group (15–44 years, sample n=477), and 15.3% in the 45 years or older group (sample n=452). In contrast, the prevalence of the binary combination of oHg/met-iAs rose with age in both females (26.0, 40.3, and 53.% for 6–14, 15–44, and >44 years, respectively) and males (24.7, 34.2, and 50.%, for 6–14, 15–44, and >44 years, respectively).

Figure 2.

Figure 2

Unique combinations of blood organic Hg and urinary inorganic-related As species measured at or above the respective detection limits among U.S. population 6 years and older: NHANES 2011–2012.

iAs, urinary arsenous acid and/or arsenic acid; met-iAs, urinary monomethylarsonic acid and/or dimethylarsinic acid; oHg, blood methyl-Hg and/or ethyl-Hg.

Discussion

Individuals may be exposed to metals singly or in combination simultaneously, depending upon their environment. Subjects may also be exposed sequentially to the individual metals. Our study found that an estimated 49.3% of the U.S. population 6 years and older in 2007–2012 had a ternary or quaternary combination of Pb, Cd, Hg, and As, detected in their blood or urine at or above the population median concentrations. These 4 metals are often examined individually by biomonitoring programs in many countries because of their well-recognized biological and physicochemical properties; however, their joint exposure profiles have not been widely monitored. Our study identified that the most prevalent unique combination of the 4 metals in the U.S. population was Pb/Cd/Hg/As (22.1%), followed by Pb/Cd/Hg (10.6%) and Pb/Cd (8.4%).

Our findings provide baseline information for prioritizing efforts in assessing potential adverse health effects and the effect-doses of the metal combinations. The idea of prioritizing chemicals and their mixtures to reduce a complex problem to a more manageable one has been used effectively for over two decades. These efforts resulted in identifying mixtures of concern based upon frequency of occurrences in various environmental media and in completed exposure pathways (Fay and Mumtaz 1996; Toccalino et al. 2012). The identified mixtures were employed for experimental toxicity testing to fill data gaps. In the current analysis data on combinations of 4 metals commonly detected are provided for in the U.S. population.

These 4 metals of major public health concern (WHO 2010) are also among the top 7 hazardous substances that pose human health threats at Superfund sites in the United States [Agency for Toxic Substances and Disease Registry (ATSDR) 2015a]. Although data are limited regarding their combination effects in humans, some investigators suggested complex interactions between these metals. de Burbure et al. (2006) noted Hg co-exposure exacerbated the effect of Cd on urinary homovanillic acid in children, one of the end-products of dopamine metabolism, whereas Pb co-exposure appeared to antagonize the effect of Hg on the dopaminergic marker. In a study of 6–11 years old U.S. girls who participated in NHANES III, higher blood Pb was associated with lower levels of inhibin B, a marker of follicular development, with the strongest effect seen in the presence of high concentrations of both blood Pb and urinary Cd (Gollenberg et al. 2010).

Assessing health effects of exposure to chemical combinations is a challenging area, hampered by a paucity of relevant data as well as the complexities involved. Often multiple mechanisms act with such exposures, sometimes simultaneously or sequentially in the overall expression of toxicity. The nature and severity of toxicity may change as a function of combinations of chemicals and/or their ratios (e.g., chemical/chemical and chemical/metabolites) (ATSDR 2015b). Similarly, variability in toxicity exists between individuals due to polymorphisms and differences in metabolism (ATSDR 2015b). Most available information of human toxicity for chemical combinations has been derived from animal experimental studies and many of these investigations were not designed to actually quantify chemical interactions that contribute to greater than expected joint toxicity of a given mixture (ATSDR 2004). To achieve this, a weight of evidence methodology based on pair-wise comparison was developed (Mumtaz and Durkin 1992; ATSDR 2004). The binary studies have been particularly useful in understanding both mechanisms of toxicity and interactions as a function of dose and chemical combinations that play a critical role in toxicity of multiple chemicals.

Our study identified unique ternary and quaternary combinations of the 4 metals, and the binary and ternary combinations of their species. These results may be utilized in designing future experimental toxicity studies that might enable one to move away from dependence on pair-wise comparison methodology.

Our observations also confirmed age as the strongest independent demographic factor associated with all three most common combinations (i.e., Pb/Cd/Hg/As, Pb/Cd/Hg, and Pb/Cd). The strength of associations was similar between the three combinations (adj-ORs, 10.9–11.1 for each 10 year increment). Previous investigators reported nonlinear relationships between age and individual metals, including a quadratic relationship with gender-adjusted blood Hg concentrations among NHANES participants (Caldwell et al. 2009). A general population study in Belgium reported similar nonlinear and nonmonotonic relationships between age and urinary Cd concentrations (Chaumont et al. 2013). Ruiz et al (2010) suggested that life stage-related anatomical and physiological changes may play an important role in the absorption and excretion of Cd. Our observations on the race/ethnicity differences in the odds of having metal combinations may be partly explained by differences in exposure, such as diet (deCastro et al. 2014; Martorell et al. 2011). For the Pb/Cd/Hg/As combination, subgroup analyses by age suggested potential interaction effects between age and gender, as well as between age and race/ethnicity, although the interaction terms were not significant (data not shown). If the interaction effect is confirmed, this information might help to identify specific subgroups for further assessment of exposure sources and potential adverse health effects.

Our study is the first to report the prevalence of As and Hg species combinations in the U.S. general population. Our results showed that 22.9% of females in the reproductive age group (15–44 years) exhibited the combination of oHg/met-iAs/iAs at detectable levels in 2011–2012. The prevalence was even higher (25.7%) in younger females (6–14 years). These metals may be transferred from mother to fetus (Baranowska 1995; Rudge et al. 2009). Hence, further studies are needed to understand the potential adverse health effects and effect-dose relationships between mother and infant (Ettinger et al. 2014; Garcia-Esquinas et al. 2011). In the U.S. population, iAs was frequently detected together with met-iAs (i.e., MMA and/or DMA), as a binary (met-iAs/As, 3.4%) or ternary (oHg/met-iAs/iAs) combination (24.7%), but was rarely detected alone (0.1%) or as oHg/iAs (0.5%). In contrast, met-iAs occurred alone (6.6%) more frequently or as oHg/met-iAs (42.1%) without detectable iAs. iAs is rapidly metabolized to MMA and DMA in the body, which may partly explain our findings. It is also possible that the presence of met-iAS without detectable iAs may be partly explained by direct intake of met-iAs from exogenous sources as described previously (Aylward et al. 2014; deCastro et al. 2014).

Our study has several strengths and limitations. A large number of participants (n=7408) was included by combining three NHANES cycles, affording stable estimates of the prevalence. To our knowledge, this is the first study to report the age- and gender-specific prevalence of exposure to As and Hg species combinations in the U.S. general population. Our study is, however, limited by the small number of chemicals considered. Development of new statistical methods is urgently needed to simultaneously examine a large number of chemicals and non-chemical agents (e.g., radiation) and to interpret the results in a biologically meaningful way. It was not possible to examine age-cohort effect because of the limited number of age-cohort combinations in the data used. Further, other potential confounders (e.g., diet and occupation) were not included in examining demographic factors associated with prevalent combinations. Future studies examining exposure risk factors need to consider using most reliable methods for exposure measurements, such as long-term average daily intake for dietary factors. The population median concentrations were employed as the cut points to provide baseline information for the population. Future investigations assessing potential adverse health effects of metal combinations need to take steps to establish exposure levels that are sufficient to cause health concerns.

Conclusions

Data for metals (Pb, Cd, Hg, and As) exposure reported by NHANES 2007–2012 were analyzed. Approximately in 50% of the population 6 years or older a ternary or quaternary combination of the 4 metals was detected in their blood or urine specimens at or above the respective population median levels. The most prevalent unique combination was Pb/Cd/Hg/As, followed by Pb/Cd/Hg, and Pb/Cd. Age was confirmed as the most consistent independent demographic factor associated with exposure to these combinations, after adjusting for the effects of other demographic variables (gender and race/ethnicity) and two selected confounders (serum cotinine and BMI). Compared to Non-Hispanic Whites, all other race/ethnicity groups were associated with a significantly increased risk of Pb/Cd/Hg/As combination.

The prevalence of inorganic-related As and organic Hg species combinations is noteworthy. In addition, 63% of females of reproductive age (15–44 years) were exposed to the specific unique combination of oHg/iAs/met-iAs (23%) or oHg/met-iAs (40%). The prevalence of oHg/iAs/met-iAs was higher in younger females aged 6–14 years (25.7%) than other age groups. Our findings fill a data gap by providing baseline data on exposure patterns of multiple metal combinations and their species that demonstrated high relevance to public health in the U.S. population. These baseline data might help (1) prioritize efforts in assessing potential adverse health effects, (2) establish exposure levels that are sufficient to cause health concerns, (3) identify exposure sources, and (4) implement preventive measures.

Acknowledgments

The authors thank Drs. Kathleen Caldwell, Mary Mortensen, and Anne Sowell for their helpful comments on this paper. This study was supported by the Agency for Toxic Substances and Disease Registry. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Agency for Toxic Substances and Disease Registry.

Footnotes

Competing Financial Interests:

The authors declare that they have no actual or potential competing financial interests.

References

  1. ATSDR (Agency for Toxic Substances and Disease Registry) [accessed 5 December 2015];The ATSDR 2015 Substance Priority List. 2015a Available: http://www.atsdr.cdc.gov/spl/
  2. ATSDR (Agency for Toxic Substances and Disease Registry) [accessed 5 December 2015];Interaction profile for: arsenic, cadmium, chromium, and lead. 2004 Available: http://www.atsdr.cdc.gov/interactionprofiles/IP-metals1/ip04.pdf.
  3. ATSDR (Agency for Toxic Substances and Disease Registry) [accessed 22 January 2016];Addendum to the Toxicological Profile for Arsenic. 2015b Available: http://www.atsdr.cdc.gov/toxprofiles/Arsenic_addendum.pdf.
  4. Aylward LL, Ramasamy S, Hays SM, Schoeny R, Kirman CR. Evaluation of urinary speciated arsenic in NHANES: issues in interpretation in the context of potential inorganic arsenic exposure. Regul Toxicol Pharmacol. 2014;69:49–54. doi: 10.1016/j.yrtph.2014.02.011. [DOI] [PubMed] [Google Scholar]
  5. Baranowska I. Lead and cadmium in human placentas and maternal and neonatal blood (in a heavily polluted area) measured by graphite furnace atomic absorption spectrometry. Occup Environ Med. 1995;52:229–232. doi: 10.1136/oem.52.4.229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Benowitz NL. Cotinine as a biomarker of environmental tobacco smoke exposure. Epidemiol Rev. 1996;18:188–204. doi: 10.1093/oxfordjournals.epirev.a017925. [DOI] [PubMed] [Google Scholar]
  7. Bernstam L, Nriagu J. Molecular Aspects of Arsenic Stress. J Toxicol Environ Health B Crit Rev. 2000;3:293–322. doi: 10.1080/109374000436355. [DOI] [PubMed] [Google Scholar]
  8. Bradham KD, Green W, Hayes H, Nelson C, Alava P, Misenheimer J, Diamond JL, Thayer WC, Thomas DJ. Estimating Relative Bioavailability of Soil Lead in the Mouse. J Toxicol Environ Health A. 2016;79(24):1179–1182. doi: 10.1080/15287394.2016.1221789. [DOI] [PubMed] [Google Scholar]
  9. Bridges CC, Zalups RK. The Aging Kidney and the Nephrotoxic Effects of Mercury. J Toxicol Environ Health B Crit Rev. 2017;20(2):55–80. doi: 10.1080/10937404.2016.1243501. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Caldwell KL, Mortensen ME, Jones RL, Caudill SP, Osterloh JD. Total blood mercury concentrations in the U.S. Population: 1999–2006. Int J Hyg Environ Health. 2009;212:588–598. doi: 10.1016/j.ijheh.2009.04.004. [DOI] [PubMed] [Google Scholar]
  11. CDC (Centers for Disease Control and Prevention) [accessed 5 December 2015];2000 CDC Growth Charts for the United States: Methods and Development. 2002 Available: http://www.cdc.gov/growthcharts/2000growthchart-us.pdf.
  12. CDC (Centers for Disease Control and Prevention) Fourth Report on Human Exposure to Environmental Chemicals, 2009. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention; [accessed 5 December 2015]. Available: http://www.cdc.gov/exposurereport/ [Google Scholar]
  13. CDC (Centers for Disease Control and Prevention) [accessed 30 December 2016];Burden of Tobacco Use in the U.S. 2016 Available: https://www.cdc.gov/tobacco/campaign/tips/resources/data/cigarette-smoking-in-united-states.html.
  14. Chaumont A, Voisin C, Deumer G, Haufroid V, Annesi-Maesano I, Roels H, Thijs L, Staessen J, Bernard A. Associations of urinary cadmium with age and urinary proteins: Further evidence of physiological variations unrelated to metal accumulation and toxicity. Environ Health Perspect. 2013;121:1047–1053. doi: 10.1289/ehp.1306607. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Cole M, Craft AW, Parker L, Bell S, Seviour JA, McGill AC, Dale G. Urinary creatinine adjusted references ranges for homovanillic and vanyllylmandelic acid in children and adults. Clinica Chimica Acta. 1995;236:19–32. doi: 10.1016/0009-8981(95)06031-1. [DOI] [PubMed] [Google Scholar]
  16. Chiba M, Masironi R. Toxic and trace elements in tobacco and tobacco smoke. Bull World Health Org. 1992;70:269–275. [PMC free article] [PubMed] [Google Scholar]
  17. Davidson T, Ke Q, Costa M. Selected molecular mechanisms of metal toxicity and carcinogenicity. In: Nordberg GF, Fowler BA, Nordberg M, editors. Handbook on the Toxicology of Metals. San Diego: Elsevier Academic Press; 2015. pp. 173–196. [Google Scholar]
  18. de Burbure C, Buchet JP, Leroyer A, Nisse C, Haguenoer JM, Mutti A, Smerhovsky Z, Cikrt M, Trzcinka-Ochocka M, Razniewska G, Jakubowski M, Bernard A. Renal and Neurologic Effects of Cadmium, Lead, Mercury, and Arsenic in Children: Evidence of Early Effects and Multiple Interactions at Environmental Exposure Levels. Environ Health Perspect. 2006;114:584–590. doi: 10.1289/ehp.8202. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. deCastro BR, Caldwell KL, Jones RL, Blount BC, Pan Y, Ward C, Mortensen ME. Dietary Sources of Methylated Arsenic Species in Urine of the United States Population, NHANES 2003–2010. PLoS One. 2014;9:e108098. doi: 10.1371/journal.pone.0108098. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Diamond GL, Bradham KD, Brattin WJ, Burgess M, Griffin S, Hawkins CA, Juhasz AL, Klotzbach JM, Nelson C, Lowney YW, Scheckel KG, Thomas DJ. Predicting Oral Relative Bioavailability of Arsenic in Soil from in Vitro Bioaccessibility. J Toxicol Environ Health A. 2016;79:165–173. doi: 10.1080/15287394.2015.1134038. [DOI] [PubMed] [Google Scholar]
  21. Ettinger AS, Roy A, Amarasiriwardena CJ, Smith D, Lupoli N, Mercado-Garcia A, Lamadrid-Figueroa H, Tellez-Rojo MM, Hu H, Hernandez-Avila M. Maternal Blood, Plasma, and Breast Milk Lead: Lactational Transfer and Contribution to Infant Exposure. Environ Health Perspect. 2014;122:87–92. doi: 10.1289/ehp.1307187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Fay RM, Mumtaz MM. Development of a priority list of chemical mixtures occurring at 1188 hazardous waste sites, using the HazDat database. Food Chem Toxicol. 1996;34:1163–1165. doi: 10.1016/s0278-6915(97)00090-2. [DOI] [PubMed] [Google Scholar]
  23. Garcia-Esquinas E, Perez-Gomez B, Fernandez MA, Perez-Meixeira AM, Gil E, de Paz C, Iriso A, Sanz JC, Astray J, Cisneros M, de Santos A, Asensio A, Garcia-Sagredo JM, Garcia JF, Vioque J, Pollan M, Lopez-Abente G, Gonzalez MJ, Martinez M, Bohigas PA, Pastor R, Aragones N. Mercury, Lead and Cadmium in Human Milk in Relation to Diet, Lifestyle Habits and Sociodemographic Variables in Madrid (Spain) Chemosphere. 2011;85:268–276. doi: 10.1016/j.chemosphere.2011.05.029. [DOI] [PubMed] [Google Scholar]
  24. Gollenberg AL, Hediger ML, Lee PA, Himes JH, Louis GM. Association between lead and cadmium and reproductive hormones in peripubertal U.S. girls. Environ Health Perspect. 2010;118:1782–1787. doi: 10.1289/ehp.1001943. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Goyer RA. Toxic and essential metal interactions. Annu Rev Nutr. 1997;17:37–50. doi: 10.1146/annurev.nutr.17.1.37. [DOI] [PubMed] [Google Scholar]
  26. Jarup L. Hazards of heavy metal contamination. Br Med Bull. 2003;68:167–182. doi: 10.1093/bmb/ldg032. [DOI] [PubMed] [Google Scholar]
  27. Johnson CL, Paulose-Ram R, Ogden CL, Carroll MD, Kruszon-Moran D, Dohrmann SM, Curtin LR. National Health and Nutrition Examination Survey: Analytic Guidelines, 1999–2010. Vital Health Stat. 2013;2(161):1–24. [PubMed] [Google Scholar]
  28. Jomova K, Valko M. Advances in metal-induced oxidative stress and human disease. Toxicology. 2011;283:65–87. doi: 10.1016/j.tox.2011.03.001. [DOI] [PubMed] [Google Scholar]
  29. Kah M, Levy L, Brown C. Potential for Effects of Land Contamination on Human Health. 1. The Case of Cadmium. J Toxicol Environ Health B Crit Rev. 2012;15:348–363. doi: 10.1080/10937404.2012.705107. [DOI] [PubMed] [Google Scholar]
  30. Martorell I, Perelló G, Martí-Cid R, Llobet JM, Castell V, Domingo JL. Human exposure to arsenic, cadmium, mercury, and lead from foods in Catalonia, Spain: Temporal trend. Biol Trace Elem Res. 2011;142:309–322. doi: 10.1007/s12011-010-8787-x. [DOI] [PubMed] [Google Scholar]
  31. Mumtaz MM, Durkin PR. A weight of evidence scheme for assessing interactions in chemical mixtures. Toxicol Indus Health. 1992;8:377–406. [PubMed] [Google Scholar]
  32. NHANES (National Health and Nutrition Examination Survey) [accessed 5 December 2015];Questionnaires, Datasets, and Related Documentation. 2015 Available: http://www.cdc.gov/nchs/nhanes/nhanes_questionnaires.htm.
  33. NIEHS (National Institute of Environmental Health Sciences) Advancing Research on Mixtures: New Perspectives and Approaches for Predicting Adverse Human Health Effects. [accessed 5 December 2015];Workshop Summary. 2012 Available: http://www.niehs.nih.gov/about/events/pastmtg/2011/mixtures/pdf_workshop_report_508.pdf.
  34. Padilla MA, Elobeid M, Ruden DM, Allison DB. An examination of the association of selected toxic metals with total and central obesity indices: NHANES 99-02. Int J Environ Res Public Health. 2010;7:3332–3347. doi: 10.3390/ijerph7093332. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Pirkle JL, Flegal KM, Bernert JT, Brody DJ, Etzel RA, Maurer KR. Exposure of the U.S. population to environmental tobacco smoke: the Third National Health and Nutrition Examination Survey, 1988–1991. J Am Med Assoc. 1996;275:1233–1240. [PubMed] [Google Scholar]
  36. Qian H, Chen M, Kransler KM, Zaleski RT. Assessment of chemical coexposure patterns based upon phthalate biomonitoring data within the 2007/2008 national health and nutrition examination survey. J Expo Sci Environ Epidemiol. 2015;25:249–255. doi: 10.1038/jes.2014.24. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Piasek M, Jurasovic J, Sekovanic A, Brajenovic N, Brcic Karaconji I, Mikolic A, Grgec AS, Stasenko S. Placental Cadmium as an Additional Noninvasive Bioindicator of Active Maternal Tobacco Smoking. J Toxicol Environ Health A. 2016;79:443–446. doi: 10.1080/15287394.2016.1165640. [DOI] [PubMed] [Google Scholar]
  38. Pleil JD, Sobus JR. Estimating Central Tendency from a Single Spot Measure: A Closed-Form Solution for Lognormally Distributed Biomarker Data for Risk Assessment at the Individual Level. J Toxicol Environ Health A. 2016;79:837–847. doi: 10.1080/15287394.2016.1193108. [DOI] [PubMed] [Google Scholar]
  39. Ratcliffe HE, Swanson GM, Fischer LJ. Human Exposure to Mercury: A Critical Assessment of the Evidence of Adverse Health Effects. J Toxicol Environ Health A. 1996;49:221–270. doi: 10.1080/713851079. [DOI] [PubMed] [Google Scholar]
  40. Rudge CV, Rollin HB, Nogueira CM, Thomassen Y, Rudge MC, Odland JO. The placenta as a barrier for toxic and essential elements in paired maternal and cord blood samples of South African delivering women. J Environ Monit. 2009;11:1322–1330. doi: 10.1039/b903805a. [DOI] [PubMed] [Google Scholar]
  41. Ruiz P, Mumtaz M, Osterloh J, Fisher J, Fowler BA. Interpreting NHANES biomonitoring data, cadmium. Toxicol Lett. 2010;198:44–48. doi: 10.1016/j.toxlet.2010.04.022. [DOI] [PubMed] [Google Scholar]
  42. Scheckel KG, Diamond GL, Burgess MF, Klotzbach JM, Maddaloni M, Miller BW, Partridge CR, Serda SM. Amending Soils with Phosphate as Means to Mitigate Soil Lead Hazard: A Critical Review of the State of the Science. J Toxicol Environ Health B Crit Rev. 2013;16:337–380. doi: 10.1080/10937404.2013.825216. [DOI] [PubMed] [Google Scholar]
  43. Sobus JR, DeWoskin RS, Tan YM, Pleil JD, Phillips MB, George BJ, Christensen K, Schreinemachers DM, Williams MA, Hubal EA, Edwards SW. Uses of NHANES Biomarker Data for Chemical Risk Assessment: Trends, Challenges, and Opportunities. Environ Health Perspect. 2015;123:919–927. doi: 10.1289/ehp.1409177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Sweet LI, Zelikoff JT. Toxicology and Immunotoxicology of Mercury: A Comparative Review in Fish and Humans. J Toxicol Environ Health B Crit Rev. 2001;4:161–205. doi: 10.1080/109374001300339809. [DOI] [PubMed] [Google Scholar]
  45. Tagne-Fotso R, Leroyer A, Howsam M, Dehon B, Richeval C, Nisse C. Current Sources of Lead Exposure and Their Relative Contributions to the Blood Lead Levels in the General Adult Population of Northern France: The Imepoge Study, 2008–2010. J Toxicol Environ Health A. 2016;79:245–265. doi: 10.1080/15287394.2016.1149131. [DOI] [PubMed] [Google Scholar]
  46. Thompson SG, Barlow RD, Wald NJ, Van Vunakis H. How should urinary cotinine concentrations be adjusted for urinary creatinine concentration? Clin Chim Acta. 1990;187:289–295. doi: 10.1016/0009-8981(90)90114-8. [DOI] [PubMed] [Google Scholar]
  47. Toccalino PL, Norman JE, Scott JC. Chemical mixtures in untreated water from public-supply wells in the U.S.--occurrence, composition, and potential toxicity. Sci Total Environ. 2012;431:262–270. doi: 10.1016/j.scitotenv.2012.05.044. [DOI] [PubMed] [Google Scholar]
  48. Wang G, Fowler BA. Roles of biomarkers in evaluating interactions among mixtures of lead, cadmium and arsenic. Toxicol Appl Pharmacol. 2008;233:92–99. doi: 10.1016/j.taap.2008.01.017. [DOI] [PubMed] [Google Scholar]
  49. WHO (World Health Organization) [accessed 5 December 2015];Assessment of combined exposures to multiple chemicals: Report of a WHO/IPCS International workshop on aggregate/cumulative risk assessment. 2009 Available: http://www.who.int/ipcs/methods/harmonization/areas/workshopreportdocument7.pdf?ua=1.
  50. WHO (World Health Organization) [accessed 5 December 2015];Preventing disease through healthy environment: Action is needed on chemicals of major public health concern. 2010 Available: http://www.who.int/ipcs/features/chemicals_concern/en/
  51. Woodruff TJ, Zota AR, Schwartz JM. Environmental chemicals in pregnant women in the United States: NHANES 2003–2004. Environ Health Perspect. 2011;119:878–885. doi: 10.1289/ehp.1002727. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES