Skip to main content
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2014 Aug 22;33(10):2386–2398. doi: 10.1002/etc.2675

Sensitivity of ecological soil-screening levels for metals to exposure model parameterization and toxicity reference values

Bradley E Sample †,*, Anne Fairbrother , Ashley Kaiser , Sheryl Law §, Bill Adams ||
PMCID: PMC4282090  PMID: 24944000

Abstract

Ecological soil-screening levels (Eco-SSLs) were developed by the United States Environmental Protection Agency (USEPA) for the purposes of setting conservative soil screening values that can be used to eliminate the need for further ecological assessment for specific analytes at a given site. Ecological soil-screening levels for wildlife represent a simplified dietary exposure model solved in terms of soil concentrations to produce exposure equal to a no-observed-adverse-effect toxicity reference value (TRV). Sensitivity analyses were performed for 6 avian and mammalian model species, and 16 metals/metalloids for which Eco-SSLs have been developed. The relative influence of model parameters was expressed as the absolute value of the range of variation observed in the resulting soil concentration when exposure is equal to the TRV. Rank analysis of variance was used to identify parameters with greatest influence on model output. For both birds and mammals, soil ingestion displayed the broadest overall range (variability), although TRVs consistently had the greatest influence on calculated soil concentrations; bioavailability in food was consistently the least influential parameter, although an important site-specific variable. Relative importance of parameters differed by trophic group. Soil ingestion ranked 2nd for carnivores and herbivores, but was 4th for invertivores. Different patterns were exhibited, depending on which parameter, trophic group, and analyte combination was considered. The approach for TRV selection was also examined in detail, with Cu as the representative analyte. The underlying assumption that generic body-weight–normalized TRVs can be used to derive protective levels for any species is not supported by the data. Whereas the use of site-, species-, and analyte-specific exposure parameters is recommended to reduce variation in exposure estimates (soil protection level), improvement of TRVs is more problematic. Environ Toxicol Chem 2014;33:2386–2398.

Keywords: Soil screening, Ecological risk assessment, Metal, Wildlife toxicology, Toxicity reference values, Sensitivity analysis, Exposure modeling

INTRODUCTION

Ecological soil-screening levels (Eco-SSLs) for wildlife are soil contaminant concentrations associated with an exposure dose equivalent to a no-observed-adverse-effect level (NOAEL; for the most sensitive endpoint examined 1). Ecological soil-screening levels intentionally employ multiple conservative assumptions and represent soil concentrations believed to be protective of most birds and mammals directly ingesting contaminated soil and consuming biota that live in that soil 1. The purpose of deriving Eco-SSLs is to set conservative soil screening values that can be used to eliminate the need for further ecological assessment for specific analytes at a given site. The form and parameterization of the exposure model used to calculate Eco-SSLs is simplistic and conservative, making it suitable for screening purposes, but it may be overly conservative in some cases. In particular, Eco-SSLs, as published, are not suitable for higher-tier risk evaluation, nor for application as remedial goals 1.

The basic form of the model for calculating soil concentrations protective of wildlife is:

graphic file with name etc0033-2386-m1.jpg

where the dietary exposure concentration is the amount of a contaminant the species of concern can consume in its diet that is equal to an acceptable toxicity level. Biota uptake is the proportion of the contaminant in the soil that is taken up into the dietary item; it is equal to 1 when estimating exposure from direct ingestion of contaminated soil and may be greater or less than 1 for any of the dietary food items. Calculation of the exposure concentration is accomplished by setting it equal to the experimentally derived dietary concentration at the toxicity reference value (TRV) such that the hazard quotient (HQ) = 1:

graphic file with name etc0033-2386-m2.jpg

where HQj is the hazard quotient for contaminant (j) (unitless); TRVj is the toxicity reference value for contaminant (j) (mg/kg/d); Soilj is the concentration of contaminant (j) in soil (mg/kg dry wt); N is the number of different biota types in the diet; Bij is the concentration of contaminant (j) in diet type (i) (mg/kg dry wt); Pi is the proportion of diet type (i) in diet; Ps is the soil ingestion as proportion of diet; FIR is the food ingestion rate (kg food [dry wt]/kg body wt [dry wt]/d); AFij is the absorbed fraction of contaminant (j) from biota type (i); AFsj is the absorbed fraction of contaminant (j) from soil (s); and AUF is the area use factor.

Note that the model is solved in this manner to allow estimation of Bij using multiple modeling approaches (soil-biota bioaccumulation factors, diet-biota bioaccumulation factors, and log-linear bioaccumulation models).

Sample et al. 2 concluded that the structure of existing models for wildlife dietary exposure estimation are sufficiently robust for use in determining exposure limits for screening-level assessments, conducting higher-tier risk evaluations, and developing soil remedial goals. Sample et al. 2 recommended that exposure parameters (i.e., diet composition, bioavailability, bioaccessibility, Ps, and AUF) be developed on a site-, species-, and analyte-specific basis to achieve more realistic estimates to the extent that the resources for their derivation are justified by the value added to the risk assessment or remedial process.

However, derivation of exposure estimates that result in an NOAEL (or the lowest acceptable effect level) depends on the accuracy of the TRVs used to quantify the acceptable magnitude of adverse effect for the chemical of interest in the taxonomic class of concern. Although TRVs are critical components of ecological risk assessments, few standardized, consensus-based values are consistently adopted for wildlife 3. The United States Environmental Protection Agency (USEPA) Eco-SSL TRV derivation procedure converts dietary no- and lowest-observed-adverse-effect concentrations (NOAECs and LOAECs, respectively; mg/kg-diet) reported in the literature into dose metrics (mg/kg-body wt/d), then translates them back to exposure concentrations for surrogate species that can be used in risk-based decisions. Such an approach results in small-bodied animals always being most at risk because of the influence of higher ingestion rates per body weight, and it fails to account for interspecies differences in uptake, metabolism, or excretion processes. Other factors that influence TRV determinations include selection of key toxicity studies, the selection of the acceptable consensus-based effect level (e.g., EC20, NOAEC, LOAEC), and the application and magnitude of uncertainty factors 3,4.

The extent of what and how much additional data should be collected should be considered before designing and implementing studies to collect such data. To do this, the relative influence of the exposure parameters and TRV selection on the estimation of soil concentrations that are appropriately protective for wildlife needs to be quantified. We have undertaken a quantitative sensitivity analysis of the wildlife model used to calculate soil screening levels, using EPA's Eco-SSLs as examples. The objectives of the present study include: investigating the relative sensitivity of parameters in the fully expanded Eco-SSL wildlife model, determining whether conversion to a dose metric for TRVs adds uncertainty to the final value, and providing recommendations for data development and model parameterization to support higher-tier risk evaluation and derivation of remedial goals to protect site-specific wildlife receptors.

METHODS

Sensitivity analysis for fully expanded wildlife Eco-SSL model

To represent taxonomic (bird vs mammal) and trophic group (carnivore vs herbivore vs invertivore) variation, sensitivity analyses were performed for all 6 model species for which Eco-SSLs have been developed (i.e., meadow vole [Microtus pennsylvanicus], short-tailed shrew [Blarina brevicauda], long-tailed weasel [Mustela frenata], mourning dove [Zenaida macroura], American woodcock [Scolopax minor], and red-tailed hawk [Buteo jamaicensis]). This allowed for an evaluation of how model parameterization reflective of different life histories affects Eco-SSL output. Sensitivity analyses were restricted to the 16 metals/metalloids for which Eco-SSLs have been developed (Sb, As, Ba, Be, Cd, trivalent chromium [Cr(III)], hexavalent chromium [Cr(VI)], Co, Cu, Pb, Mn, Ni, Se, Ag, V, and Zn). Aluminum and Fe were not included in the present analysis because although Eco-SSL documents have been developed, numeric Eco-SSL values have not. Diet compositions were simplified in accordance with the Eco-SSLs; vole and dove consumed 100% plant; shrew and woodcock consumed 100% earthworms; and weasel and hawk consumed 100% small mammals.

Model parameterization

Distributions were developed, and model sensitivity was evaluated for 6 model parameters: TRV, food ingestion rate (FIR), soil ingestion rate, bioaccumulation, and the absorbed fractions of contaminant from soil and from food.

Toxicity reference value

Distributions for NOAEL TRVs were developed based on values from each Eco-SSL report on a dose basis (mg/kg body wt/d). The NOAEL distribution was based on growth and reproduction if the final TRV for the Eco-SSL were based on the geometric mean of these 2 endpoints. If the TRV selected for the final Eco-SSL was based on the highest bounded NOAEL below the lowest bounded lowest observed adverse effect level (LOAEL) for reproduction, growth, or mortality, then the NOAEL distribution for the analysis was based on all growth, reproduction, and mortality NOAELs. The NOAEL for the cobalt Eco-SSL for birds was based on the geometric mean of growth NOAELs. Therefore, the NOAEL distribution for cobalt in birds was restricted to growth NOAELs. The NOAEL values were used as reported in each Eco-SSL report. Sensitivity analyses were not performed for metals lacking NOAEL TRVs (Sb, Ba, Be, Cr[VI], and Ag for birds and Ag for mammals). Summary statistics for the distribution of NOAEL TRVs for birds and mammals for the 16 metal/metalloid Eco-SSLs, with the effect groups they represent, are presented in Supplemental Data, Table S1.

Food ingestion rate

The FIR for each wildlife species in the Eco-SSLs was based on data for typical and high end consumption values from multiple published references 5, with the FIR used to calculate the Eco-SSLs being the mean of high-end values. The “high-end” value approximates the 90th percentile, being either the mean + 1.282 × standard deviation or mean × 1.25 (if the standard deviation was not available).

For the purpose of this sensitivity analysis for the present study, an overall FIR distribution for each of the receptors was represented by the minimum low-end FIR and the maximum high-end FIR from all studies reported for each species. Mean and high-end FIR values were used as reported by the USEPA 5. Because low-end FIR values were not reported, the FIR distribution in each study was assumed to be symmetrical, and calculated as (mean FIR) − ([high-end FIR] − [mean FIR]). All values used to derive the FIR distributions are presented in Supplemental Data, Table S2.

Soil ingestion rate

The soil ingestion rate values used for the Eco-SSLs represent the 90th percentile of the Ps distribution for each species based on a probabilistic implementation of the soil ingestion model from Beyer et al. 6. Species-specific distributions for Ps consisted of the 5th through 95th percentile ranges reported by the USEPA 5. Because 5th percentile Ps values were negative for all species, the lower limit of each distribution was truncated to 0. Values used to derive soil ingestion rate distributions are presented in Supplemental Data, Table S3.

Bioaccumulation

The Eco-SSL derivations of estimated concentrations of contaminants in diet items (Bij) rely on analyte diet item-specific, log-linear regression models (if available) or median bioaccumulation factors (BAF; ratio of concentration in tissue to the concentration in soil). Variation in bioaccumulation or uptake models was evaluated using the 90% prediction intervals for bioaccumulation regression models (if uptake was based on regression models) or distributions for BAFs (if uptake was based on BAFs). Because they were unavailable in USEPA 5, distributional data were extracted from the source literature or were developed specifically for this sensitivity analysis in the present study. Minimum and maximum BAFs were obtained from Bechtel-Jacobs 7, Sample et al. 8, and Sample et al. 9 for plants, earthworms, and small mammals, respectively. Upper and lower 95% prediction limits on regression-based bioaccumulation estimates were calculated based on Zar 10, using data from the source literature. Analyte-specific bioaccumulation model parameters for terrestrial plants, earthworms, and small mammals are presented in Supplemental Data, Tables S4 through S6.

The following multiple analyte- or receptor-specific considerations specific to the bioaccumulation models were integrated into the sensitivity analysis. Bioaccumulation models for Sb and Be in terrestrial plants were scarce in the literature. Regression model parameters necessary to calculate 90% prediction intervals were not reported by the USEPA 5 and therefore were generated based on the presented plant uptake data. Because only total Cr bioaccumulation data were available, the same bioaccumulation models for total Cr were applied to both Cr(III) and Cr(VI). A BAF of 1 was assumed for Sb into earthworms 5. No variation was employed in this sensitivity analysis, because this BAF was a point estimate. The USEPA 5 determined that the available data could not accurately predict Ni bioaccumulation into earthworms, and therefore Eco-SSLs for avian and mammalian invertivores were not developed. Therefore, no sensitivity analyses for bioaccumulation of Ni for invertivores were performed. Ecological soil-screening levels for long-tailed weasel and red-tailed hawk for Sb, Ba, and Be employed diet-biota uptake factors (as point estimates) from Baes et al. 11. The USEPA 5 applied these diet-biota uptake factors to output from bioaccumulation models for earthworms to estimate concentrations in tissues of small mammals. Therefore, variation estimates for these analytes in small mammal tissues are a function of the bioaccumulation models for earthworms.

Absorbed fractions from soil and diet

The Eco-SSLs assume that analytes in soil and diet are 100% bioavailable. However, an increasing body of literature indicates that bioavailability frequently is less than 100% (e.g., Koch and Reimer 12). For this sensitivity analysis, results of in vitro bioaccessibility analyses (the fraction of the analyte extractable from its matrix into the gastrointestinal tract that is available for absorption 1214) was assumed to be a suitable metric to represent bioavailability.

Data from 33 bioaccessibility studies were extracted for 22 analytes and included in a bioaccessibility dataset (see Supplemental Data Table S7). Parameters included the reported percentage bioaccessibility, analytes, taxonomic class and species, extraction method, the medium (i.e., soil/sediment, biota–plant, or biota–animal), and other laboratory study method details. Data for all digestion phases were pooled, and summary statistics were calculated by analyte and general medium, separately for birds and mammals. The summary statistics are presented in Supplemental Data Table S8. Because bioaccessibility may theoretically range up to 100%, maximum soil and food bioaccessibility was assumed to be 100%, regardless of the maximum reported in published literature.

For this sensitivity analysis, mammalian bioaccessibility data were used if avian data were lacking. Plant and animal tissue bioaccessibility data were not used interchangeably. No sensitivity analyses were conducted if plant tissue or animal tissue bioaccessibility data were unavailable.

Bioaccessibility data were available for both total Cr (soil and plant tissue for both birds and mammals) and Cr (III) (soil and mammals only). Total Cr results were used for all sensitivity analyses for Cr (VI) and for plant tissue analyses for Cr (III).

Analyses

The relative sensitivity of the fully expanded wildlife Eco-SSL exposure model to variations in model parameters was evaluated using nominal-range sensitivity analyses 15. The Eco-SSL exposure model was iteratively solved by varying values for each input parameter for the minimum, median (or mean), and maximum values of their respective ranges, while holding the value for all other parameters constant and equal to that used to calculate the Eco-SSL. The “goal-seek” function in Microsoft Excel was used to solve each model iteration to identify the soil concentration resulting from the specific set of parameter values. The relative influence of each parameter was then expressed as the absolute value of the range of variation observed in the resulting soil concentration that results in exposure equal to the TRV (comparable to the Eco-SSL; Table 1). Model parameters with the greatest influence on model output were identified as those with the broadest calculated soil concentration range, and parameters with little influence were defined as producing limited variation in calculated soil concentrations.

Table 1.

Summary of the range of soil concentrations (mg/kg), and associated ranks, resulting from sensitivity analysis of the wildlife Eco-SSL model for metalsa

Analyte Species TRV FIR Ps B Soil bioavailability Food bioavailability
Range Rank Range Rank Range Rank Range Rank Range Rank Range Rank
Antimony Short-tailed shrew 11 334 5 0.84 4 0.01 2.5 1 0.01 2.5
Antimony Meadow vole 564 835 5 12 3 11 2 19 4 10.3 1
Antimony Long-tailed weasel 201 815 5 29 4 5 3 1 4.17 2
Arsenic American woodcock 350 5 620 6 171 4 52 2 158.84 3 11.94 1
Arsenic Mourning dove 449 6 116 3 257 5 84 2 239.6 4 14.9 1
Arsenic Red tailed hawk 7301 4 6806 3 58 304 6 262 2 40 002.25 5 27.33 1
Arsenic Short-tailed shrew 2875 6 206 5 30 2 134 4 26.42 1 48.25 3
Arsenic Meadow vole 5248 6 193 4 160 3 369 5 144.1 1 148.1 2
Arsenic Long-tailed weasel 5497 6 1050 3 4578 5 65 2 3712.14 4 7.81 1
Barium Short-tailed shrew 7807 5 6266 3 787 2 6352 4 641.57 1
Barium Meadow vole 12 002 6 3548 3 743 2 8126 5 616.6 1 5129.1 4
Barium Long-tailed weasel 34 767 2 53 998 3 576 540 5 490 1 162 888 4
Beryllium Short-tailed shrew 27 3 104 5 26 2 83 4 21.92 1
Beryllium Meadow vole 24 3 33 4 4 2 89 5 3.5 1
Beryllium Long-tailed weasel 72 2 535 3 1746 5 59 1 1243.86 4
Cadmium American woodcock 11 5 18 6 0.02 1.5 7.1 4 0.02 1.5 1.86 3
Cadmium Mourning dove 361 6 69 4 77 5 49 2 62 3 20.4 1
Cadmium Red-tailed hawk 5877 4 4401 3 37 858 6 264 2 8735.31 5 65.16 1
Cadmium Short-tailed shrew 1442 6 1.6 4 1.5 3.9 5 0 1.5 0.93 3
Cadmium Meadow vole 189 158 6 138 4 60 2 225 5 51.2 1 133.4 3
Cadmium Long-tailed weasel 100 544 6 703 5 548 4 85 2 379.24 3 31.21 1
Chromium III American woodcock 3567 5 308 4 16 2 66 3 12.98 1
Chromium III Mourning dove 10 490 6 135 3 274 5 65 2 209 4 22.6 1
Chromium III Red-tailed hawk 152 456 5 6116 4 1957 3 896 1 1516.5 2
Chromium III Short-tailed shrew 25 205 5 105 3 4 2 224 4 3.14 1
Chromium III Meadow vole 277 103 6 423 3 322 2 462 5 265.2 1 439 4
Chromium III Long-tailed weasel 265 067 5 1489 4 227 2 284 3 181.03 1
Chromium VI Short-tailed shrew 3503 5 403 3 16 2 863 4 12.84 1
Chromium VI Meadow vole 38 508 6 1630 3 1239 2 1779 5 1120 1 1689.8 4
Chromium VI Long-tailed weasel 32 863 5 6651 4 1681 3 1157 1 1544.99 2
Cobalt American woodcock 348 5 1449 6 181 4 109 2 155.73 3 12.23 1
Cobalt Mourning dove 766 3 475 2 5145 5 35 1 3161 4
Cobalt Red-tailed hawk 2832 4 5574 6 605 3 3463 5 505.92 2 175.77 1
Cobalt Short-tailed shrew 2562 6 706 5 67 3 475 4 54.61 2 46.77 1
Cobalt Meadow vole 23 581 5 2393 2 9401 4 543 1 7826.1 3
Cobalt Long-tailed weasel 3405 6 1918 5 228 3 1135 4 185.5 2 61.33 1
Copper American woodcock 1632 6 325 5 10 2 111 4 8.8 1 11.71 3
Copper Mourning dove 8525 6 210 3 365 5 129 2 336 4 76.3 1
Copper Red-tailed hawk 117 439 4 12 467 3 1.E + 08 6 563 2 183 361 5 150.08 1
Copper Short-tailed shrew 424 030 6 150 4 3 2 832 5 2.85 1 28.7 3
Copper Meadow vole 2E + 07 6 1757 3 6098 5 1690 2 5567.2 4 726 1
Copper Long-tailed weasel 8 638 612 6 5687 3 149 071 5 543 2 29 226.2 4 163.58 1
Lead American woodcock 2701 6 203 5 5 2 40 4 4.76 1 8.3 3
Lead Mourning dove 7141 6 93 3 453 5 59 2 431 4 16.2 1
Lead Red-tailed hawk 94 389 6 4532 5 4439 4 645 2 4230.28 3 136.36 1
Lead Short-tailed shrew 136 549 6 240 4 7 2 492 5 5.68 1 218.98 3
Lead Meadow vole 1E + 06 6 1607 2 11 876 5 1610 3 107 32.6 4 432.2 1
Lead Long-tailed weasel 563 211 6 4219 5 2420 4 687 2 2228.2 3 289.09 1
Manganese American woodcock 28 441 3 55 323 5 59 552 6 2107 2 35 352.5 4 155.53 1
Manganese Mourning dove 26 458 6 7492 4 8053 5 4282 2 6889.3 3 1671.8 1
Manganese Red-tailed hawk 400 950 5 410 609 6 190 700 4 36 849 2 159 990 3 3848.29 1
Manganese Short-tailed shrew 247 505 6 15 919 5 7010 4 6766 3 5895.56 2 568.54 1
Manganese Meadow vole 205 005 6 5964 3 2410 2 10 075 5 2034.9 1 6409.3 4
Manganese Long-tailed weasel 241 635 6 36 931 5 14 007 4 4035 2 11 764.3 3 453.71 1
Nickel Mourning dove 5172 6 389 3 2114 5 234 2 2006.6 4 42.9 1
Nickel Red-tailed hawk 71 504 3 19 952 2 130 100 5 1401 1 104 592 4
Nickel Meadow vole 60 448 6 445 2 727 5 586 3 649 4 206.9 1
Nickel Long-tailed weasel 34 316 5 1381 4 311 3 199 1 272 2
Selenium American woodcock 371 6 31 5 0.4 2 2.9 4 0.17 1 1.43 3
Selenium Mourning dove 149 5 3.4 3 0.6 2 7.8 4 0.3 1
Selenium Red-tailed hawk 13 580 6 810 5 739 4 107 3 66.97 2 29.24 1
Selenium Short-tailed shrew 322 6 3.3 5 0.03 2 2.0 4 0.01 1 1.05 3
Selenium Meadow vole 192 5 2.7 3 0.2 2 19 4 0.1 1
Selenium Long-tailed weasel 2463 6 84 5 1.2 2 14 4 0.47 1 6.31 3
Silver American woodcock 50 3 0.42 1 57 4 37.62 2
Silver Mourning dove 121 2 700 3 16 1
Silver Red-tailed hawk 5887 3 13 522 4 938 2 63.72 1
Silver Short-tailed shrew 42 2 0.26 1 928 4 301.43 3
Silver Meadow vole 1685 2 3591 3 1016 1
Silver Long-tailed weasel 5832 3 10 779 4 1023 2 88.66 1
Vanadium American woodcock 2233 5 91 4 32 3 3.4 1 28.94 2
Vanadium Mourning dove 3602 5 22 2 363 4 1.1 1 273.1 3
Vanadium Red-tailed hawk 40 247 5 882 4 672 3 28 1 614.23 2
Vanadium Short-tailed shrew 1 441 718 5 846 4 222 2 495 3 193.87 1
Vanadium Meadow vole 6 728 429 5 1454 2 8693 4 387 1 7866 3
Vanadium Long-tailed weasel 3 017 802 5 3425 4 2119 3 148 1 1929.16 2
Zinc American woodcock 10307 5 12 226 6 4.3 2 593 4 3.53 1 464.99 3
Zinc Mourning dove 19 565 6 2587 5 1364 2 2070 4 1268.2 1 1530.6 3
Zinc Red-tailed hawk 360 500 4 205 678 3 3E + 19 6 2470 2 1.2E + 07 5 1888.36 1
Zinc Short-tailed shrew 232 370 6 3164 5 2.0 2 1840 4 1.38 1 1376.95 3
Zinc Meadow vole 636 198 6 12 838 3 5059 2 20 997 5 4263.8 1 16 192.3 4
Zinc Long-tailed weasel 440 039 5 79 091 3 2E + 12 6 2879 2 205 148 4 2292.2 1
Minimum 11 1 0 0 0 1
Median 16 572 828 500 327 305 111
Maximum 1.72E + 07 4.11E + 05 3.24E + 19 3.68E + 04 1.24E + 07 1.62E + 04
a

Note that higher rank values correspond to higher importance. Analytes lacking ecological soil-screening levels (Eco-SSLs) (antimony, barium, beryllium, and hexavakent chromium for birds; nickel for woodcock and shrew) are not shown.

TRV = toxicity reference value; FIR = food ingestion rate; Ps = soil ingestion rate; B = bioaccumulation or uptake.

Soil concentration ranges were rank-transformed for each parameter within each analyte–receptor combination (Table 1) because of non-normal distribution as indicated by the Shapiro-Wilk's test. Analyses of variance (PROC GLM, SAS 16) were performed on the rank-transformed soil concentration ranges to determine: which parameters had the greatest influence on model output when all analytes are combined, and whether the pattern of model sensitivity differed among analytes, receptor class or trophic group. Interactions among categories were also investigated. If a significant analysis of variance f-test was obtained, Student-Newman-Kuels multiple comparison tests were employed to identify which parameters differed (significant at p 0.05). Supplemental Data, Figures S1 through S16, display the relative variability of calculated soil concentrations for each analyte and receptor in response to varying parameter values, in addition to the published Eco-SSL value and the analyte-specific background soil concentration range from the USEPA 5.

RESULTS

Although published wildlife Eco-SSL values exceed the maximum background soil concentration from the USEPA 5 for most metal–receptor combinations (Supplemental Data, Figures S1–S16), a substantial portion of Eco-SSL values fall within or below their analyte-specific background soil concentration range. American woodcock and short-tailed shrew exhibited the greatest number of ecological soil-screening levels falling within the background soil concentration range (n = 6; Cd, Cr[III], Cu, Pb, Se, and Zn). Antimony for shrews and V for woodcock have Eco-SSLs that fall below the minimum background concentration. Ecological soil-screening levels for mourning dove fell within the background soil concentration range for Cr(III), Pb, Se, and Zn. Ecological soil-screening levels were within the range of background for meadow voles (Se), long-tailed weasels (Se), and red-tailed hawks (V).

Soil concentration ranges estimated by varying model parameters as part of the sensitivity analysis varied widely among analytes, receptors, and parameters (Table 1; Supplemental Data, Figures S.1–S.16). Soil ingestion (Ps) displayed the broadest range (0 mg/kg to over 1019 mg/kg) and bioavailability in food the narrowest (0–1.6 mg/kg × 104 mg/kg). However, TRV consistently had the greatest influence on calculated soil concentrations, regardless of how the data were grouped (Tables1 and 2). FIR, soil ingestion rate, bioaccumulation, and bioavailability in soil were the next most influential model parameters, with their relative degree of influence varying by taxa, trophic group, and analyte. Bioavailability in food was consistently the least influential parameter, perhaps because of data limitations and not necessarily because of the parameter's importance. The relative importance of bioavailability in food and soil (to a lesser degree) also may be affected by their narrower distributions. Whereas both bioavailability parameters are constrained between 0% and 100%, most of the other parameters may vary over multiple orders of magnitude.

Table 2.

Results of ANOVA of rank-transformed ranges from ecological soil-screening level sensitivity analysis - ANOVA of parametersa

p value Parameter
TRV FIR Ps B AFs AFi
All data combined <0.0001 A B C D E F
 Taxa Birds <0.0001 A B B C C D
Mammals <0.0001 A B C BC D D
 Trophic group Carnivores <0.0001 A B B D C E
Herbivores <0.0001 A BC B BC C C
Invertivores <0.0001 A B D C E D
 Analyte Sb 0.0083 A AB B B B
As 0.0002 A AB AB BC BC C
Ba 0.69 A A A A A A
Be 0.6 A A A A A
Cd 0.0016 A AB BC BC BC C
CrIII 0.0002 A B B B B B
CrIV 0.0171 A AB AB AB B AB
Co 0.001 A A A AB AB B
Cu 0.0004 A BC AB BC BC C
Mn <0.0001 A A AB BC BC C
Pb <0.0001 A B B BC BC C
Ni 0.0011 A ABC A BC AB C
Se <0.0001 A B C B D C
Ag 0.64 A A A A
V <0.0001 A B B C C
Zn 0.011 A AB AB AB B B
a

Parameters with the same letter are not statistically different.

TRV = toxicity reference value; FIR = food ingestion rate; Ps = soil ingestion rate; B = bioaccumulation or uptake; AFs = assimilated fraction from soil; AFi = assimilated fraction from food.

Model parameters produced comparable responses among birds and mammals when all data were pooled; TRV and FIR were the most important parameters and bioavailability in food the least (Table 2). In addition, no statistically significant interactions were found between model parameter and receptor class for any analyte when analyses were performed by analyte.

In contrast, relative importance of parameters differed by trophic group. Soil ingestion was ranked as the 2nd most important parameter for carnivores and herbivores, but it was the 4th most important for invertivores (Table 2). Statistically significant interactions for trophic group and model parameters were also observed for Co, Cu, Pb, V, and Zn (Figure 1). Reciprocal relationships were evident for groups of parameters. For example, as FIR, bioaccumulation, and food bioavailability increased in importance, soil ingestion and soil bioavailability decreased. Soil ingestion and soil bioavailability exhibited greater importance for herbivores and carnivores for Cu and Pb and dominated exposure of carnivores and zinc (Figure 1). These extreme soil concentration estimates occur when soil ingestion is minimized in the model and all exposure therefore must be derived from the diet.

Figure 1.

Figure 1

Mean ranks of estimated soil concentration distributions for analytes displaying statistically significant interactions between model parameters and receptor trophic groups. Note that higher rank values correspond to higher importance. TRV = toxicity reference value; FIR = food ingestion rate.

To better understand the basis for the interactions observed for the bioaccumulation parameter, earthworm, plant, and small mammal Eco-SSL bioaccumulation models (presented in Supplemental Data Tables, S4–S6) were implemented for all analytes over the range of Eco-SSL soil concentrations for the 6 receptor species (Figure 2). These figures display how estimated tissue concentrations vary with soil concentration under each model. Patterns of mean ranks of estimated soil concentrations for trophic groups (Figure 1) generally correspond to the relative rates of bioaccumulation for their respective food types (Figure 2). Low bioaccumulation rates corresponded with lower importance of the FIR and bioaccumulation, and greater importance of Ps. In contrast, Ps was less important, and FIR and bioaccumulation were more important, when bioaccumulation rates were high. For example, the amount of bioaccumulation of Co into earthworms and small mammals greatly exceeds that observed for plant tissues. Similarly, the bioaccumulation into earthworms of Cu, Pb, and V to a lesser degree greatly exceeds that for plants and small mammals, indicating that bioaccumulation for these metals is of greater importance for invertivores than for either herbivores or carnivores. In contrast, Zn bioaccumulation into small mammal tissue as a function of soil concentration is essentially flat and much less than that for either earthworms or plants. The extreme soil concentration estimates for Zn in carnivores when soil ingestion is zero are a consequence of this shallow slope. Elevated (but not statistically significant) importance of the soil ingestion parameter associated with low bioaccumulation rates into small mammals (Figure 2) is also evident for As, Ba, Be, and Cd (Supplemental Data, Figures S2–S5).

Figure 2.

Figure 2

Variation of modeled plant, earthworm, and small mammal tissue concentrations relative to soil concentrations for ecological soil-screening level metals.

These relationships are a function of modeled bioaccumulation rates expressed in simplified diets. Whether similar patterns would be observed for exposure based on measured, field-collected food items, or on a more complex diet representative of field conditions is unclear.

Further analysis of TRVs

The sensitivity analysis shows that the calculation of a soil value for wildlife is highly dependent on TRV selection, and it can vary from 10-fold to 10 000-fold, depending on the analyte. We evaluated the approach for TRV selection in more detail; specifically addressing whether a dose metric (rather than a dietary concentration) provides a more accurate method of cross-species extrapolation. Livestock and wildlife studies generally report toxicity data based on food concentration (mg/kg food), which then is converted to an average daily dose per body weight (mg/kg-body wt/d). The dose for the target species then needs to be converted back to a food concentration to calculate a protective dietary exposure concentration and a final soil value. Because information on food consumption rates as a function of body weight or toxicity frequently are not provided in study reports, extrapolations to and from dose-based metrics adds uncertainty and variability to the results.

We extracted dose-based NOAELs for the growth endpoint for avian and mammalian species from toxicity data tables for the TRVs in the Cu Eco-SSL 17. Food concentrations for each study were derived from the reported body weight and FIR, or if they were unavailable, values were selected from the literature used for the Eco-SSL derivation. We then compared the derived food concentration with the reported food concentration in the study and the NOAEC reported in the diet for studies with other species (Tables3 and 4). The underlying assumption that generic body weight-normalized NOAELs can be used to derive protective levels for any species, such as is done in derivation of the Eco-SSLs, is not supported by the data.

Table 3.

Calculated food concentrations that are no-observable-adverse-effect-concentration (NOAEC; mg/kg-food) based on doses from avian species

Test species Reference NOAEC food (measured) Body weight (kg) Ingestion rate (kg or L per d)a NOAEC dose (mg/kg/d) (calculated) NOAEC food concentration (calculated) (mg/kg food)
Chicken1 Chicken2 Chick Duck 1 Duck 2 Turkey Quail
Chicken1 (Gallus domesticus) 44 150 1.1 0.06 7.7 150 80 93 7 186 197 56
Chicken2 (Gallus domesticus) 45 40 0.24 0.02b 3.8 75 40 46 4 92 98 28
Chick (immature) (Gallus domesticus) 46 570 0.53 0.04b 47 915 490 570 46 1133 1203 344
Duck1 (Anas platyrhynchos) Fosterc 9.9 0.24 0.24 10 199 106 124 10 246 261 75
Duck2 (Anas platyrhynchos) 47 100 2.6b 0.11b 4.2 81 43 50 4 100 106 30
Turkey (Meleagris gallopavo) 48 60 3.1 0.12b 2.3 46 24 28 2 56 60 17
Japanese quail (Coturnix japonica) 49 600 0.18 0.02 82 1598 855 995 80 1978 2101 600
a

Duck1 study was a water exposure; all others were food exposures.

b

Value not given in the study; used value from ecological soil-screening level.

c

S.D. Foster, 1999, Master's thesis, Colorado State University, Fort Collins, CO, USA.

A = avian; B = mammalia.

Table 4.

Calculated food concentrations that are no observable adverse effect concentration (NOAEC; mg/kg-food) based on doses from mammalian species

Test organism Reference NOAEC (study data) Body weight (kg) Ingestion rate (kg or L per d)a NOAEC dose (mg/kg/d) (calculated) NOAEC food concentration (calculated) (mg/kg food)
Shrew Mouse Rat 1 Rat 2 Guinea pig Rabbit 1 Rabbit 2 Mink Pig 1 Pig 2 Pony Cattle
Common shrew (Sorex araneus) 50 2.1 0.01 0.001b 36 2 1 3 3 3 6 4 3 10 9 12 10
Mouse (Mus musculus) 51 1000 0.02b 0.003 4.5 1500 1000 1810 2125 2020 4024 2697 2436 6835 6356 8636 7388
Rat 1 (Rattus norvegicus) 51 25 0.35 0.029a 0.17 21 14 25 29 28 56 37 34 94 88 119 102
Rat 2 (Rattus norvegicus) 52 1000 0.17 0.012 1.2 706 471 852 1000 951 1894 1269 1146 3217 2991 4064 3477
Guinea pig (Cavia porcellus) 53 20 0.66 0.049b 0.11 15 10 18 21 20 40 27 24 68 63 85 73
Rabbit 1 (Oryctolagus cuniculus) 54 450 2.2 0.082 0.16 168 112 202 238 226 450 302 272 764 711 966 826
Rabbit 2 (Oryctolagus cuniculus) 55 498 1.6 0.089 1.5 277 185 334 392 373 743 498 450 1262 1174 1595 1364
Mink (Mustela vision) 56 166 1.9 0.117b 0.62 102 68 123 145 138 274 184 166 466 433 589 503
Pig 1 (Sus scrofa) 57 199 108 2.37 0.10 44 29 53 62 59 117 79 71 199 185 251 215
Pig 2 (Sus scrofa) 58 560 100b 2.36 0.13 132 88 160 187 178 355 238 215 602 560 761 651
Shetland pony (Equus caballus) 59 2088 152 2.64 0.24 363 242 438 514 488 973 652 589 1653 1537 2088 1786
Cattle (Bos taurus) 60 40 430 8.73 0.02 8 5 10 12 11 22 15 13 37 34 47 40
a

Mouse study was a water exposure; all others were food exposures.

b

Value not given in the study; used value from ecological soil-screening levels.

First, measured NOAEC food concentrations vary dramatically depending on the test organism. For example, measured dietary NOAECs among the bird species tested ranged from a low of 9.9 mg/kg food for ducks (Anas platyrhynchos) to a high of 600 mg/kg food in Japanese quail (Coturnix japonica) (Table 3). Mammals had a similar range of responses, with NOAECs varying from 2.1 mg/kg food for the common shrew (Sorex araneus) to over 2000 for the Shetland pony (Equus caballus) (Table 4). It is unlikely that some of the variation is because of differences in test protocols, the variation is not reduced when dietary thresholds are converted to a dose based on body weights (Tables3 and 4). In addition, the rank order of species sensitivities changes, depending on whether they are ordered by dietary threshold or by dose threshold (Table 5). Examining Table 5, obviously ranking by dose is equivalent to ranking by body weight, whereas ranking by diet tends to group by phylogeny (with some exceptions attributable to differences in test protocols).

Table 5.

Rank order of species sensitivity, shown for both dietary thresholds (mg/kg-food) and dose (mg/kg-body wt)

Rank
Diet Dose Δ
Avian test species
 Chick (immature) (Gallus domesticus) 6 6 0
 Chicken1 (Gallus domesticus) 5 4 1
 Chicken2 (Gallus domesticus) 2 2 0
 Duck1 (Anas platyrhynchos) 1 5 4
 Duck2 (Anas platyrhynchos) 4 3 1
 Turkey (Meleagris gallopavo) 3 1 2
 Japanese quail (Coturnix japonica) 7 7 0
Mammalian test organism
 Common shrew (Sorex araneus) 1 12 11
 Mouse (Mus musculus) 10 11 1
 Rat 2 (Rattus norvegicus) 11 9 2
 Rat 1 (Rattus norvegicus) 3 6 3
 Guinea pig (Cavia porcellus) 2 3 1
 Rabbit 1 (Oryctolagus cuniculus) 7 5 2
 Rabbit 2 (Oryctolagus cuniculus) 8 10 2
 Mink (Mustela vision) 5 8 3
 Pig 1 (Sus scrofa) 6 2 4
 Pig 2 (Sus scrofa) 9 4 5
 Shetland pony (Equus caballus) 12 7 5
 Cattle (Bos taurus) 4 1 3

Given the differences in sensitivity ranking between food-based and dose-based threshold values, that calculating a dietary threshold NOAEC for 1 species using the dose-based threshold from another results in a value that differs from what was actually measured for that species is not surprising. Tables 3 and 4 show the food concentration for each species that would be a threshold NOAEC if calculated using the dose for each of the other species. For example, the column labeled “Turkey” (Meleagris gallopavo) shows the calculated dietary NOAEC for the turkey using the dose-based NOAEC for the species shown in each row. The dietary NOAEC that was reported in the literature study was 60 mg/kg food. Using the dose from the chicken1 (Gallus domesticus) study, however, the dietary NOAEC would be calculated as 197 mg/kg food. Using the dose from the duck1 study instead, the dietary NOAEC would be calculated as 261 mg/kg food. From the data in Table 3, it is evident that using the dose-based NOAEC from any of the other tested species would result in a food concentration threshold that would not be protective for the turkey, even though the measured dietary NOAECs for chicken2 and duck1 were considered protective. Conversely, if the turkey were used to derive safe food concentrations for all other species (reading across the row labeled “Turkey” in Table 3), all other species would have dietary NOAECs that are much lower than their measured values. This could result in an incorrect conclusion of risk for these species or a cleanup target that would be much lower than needed, resulting in unnecessary cost and habitat disruption. A review of Table 4 shows comparable results for mammals.

In summary, the test species that is used to calculate a threshold dose affects the final dietary concentration and risk-based cleanup value for the target species. Selecting the most sensitive species based on calculated dose thresholds does not guarantee that all species will be adequately protected. This is because the underlying assumption that all of the variation among species is attributable to differences in body weight, which, in turn, leads to differences in FIR and therefore species-specific food concentrations, is not accurate. If that assumption were accurate, we would assume by inference that if there are multiple studies with the same species, the effective threshold dose (i.e., on a body wt basis) should be the same in all of the studies. To test this assumption, we again examined the Cu toxicity data set but this time limited it to only 1-d-old chicks (Gallus gallus) exposed to copper sulfate (copper (II) sulfate pentahydrate) for a 21- to 28-d duration, for the growth endpoint (Supplemental Data, Table S9) to reduce variability associated with age differences, form of the metal, or exposure duration.

Based on the 8 data records available, the NOAEC based on food concentration ranged from 40 mg/kg food to 341 mg/kg food and LOAECs range from 80 mg/kg food to 500 mg/kg food. The study with the lowest NOAEC of 40 also had the lowest LOAEC at 80 mg/kg food, well below the next lowest LOAEC of 250 mg/kg food derived from any other study, regardless of exposure duration or age of test subjects. No information was provided in the tables to explain why this 1 study differed significantly from the others. Converting the NOAEC data to NOAEL dose did not substantially reduce the variability in toxicity thresholds derived from the various studies. The ratio of the NOAEC to the NOAEL ranged from 0.8 to 15.6, suggesting that either the FIR or body weight were variable among the studies. This difference disappears when the FIR and body weight were assumed based on literature data, rather than study-specific data, because the FIR:body weight ratio becomes a constant. However, this is not a solution to the problem, because multiplying the concentration by a constant ratio does not standardize the dose among tests for the same species. If the theory of similarity of dose is to be supported, using measured body weight and FIR should overcome this difficulty, because all animals should respond similarly at the same ingested dose, with different studies having variable FIRs. The calculated doses ranged from 1.9 to 36.6 from the 1-d-old chick studies (with 1 outlier at 103 mg/kg-body wt/d). Studies that reported both body weight and FIR resulted in doses that ranged from 1.9 to 29.5. More driving variables are present than differences in body size.

The selection of the final TRV using a dose-based metric will rely on a different study than when the selection is made on a food concentration basis. When looking at all of the chicken studies for copper sulfate (copper (II) sulfate pentahydrate; growth endpoint) regardless of age or duration (Supplemental Data, Table S9), the lowest food-based NOAEC is at 40 mg/kg food with its corresponding LOAEC at 80 mg/kg food. The next lowest LOAEC is 350 mg/kg food. The highest bounded NOAEC below this LOAEC is 250 mg/kg food, from this same study, and multiple other studies also report a NOAEC of 250 mg/kg food. Conversely, disregarding the study with the extremely low effect levels the lowest dose-based LOAEC is 14.3 mg/kg-body weight/d, which corresponds to a test concentration of 250 mg/kg food. A different study produced the highest bounded dose-based NOAEC below this value (13.3 mg/kg body wt/d), also based on a study with test concentration of 250 mg/kg food. All of the remaining dose-based NOAECs lower than 13.3 mg/kg body weight/d were unbounded, so they did not meet the criteria for selection of a TRV.

Looking at all these data on a concentration basis, it appears that 1-d-old chicks are overall more sensitive to Cu than are older birds, because of higher FIR. However, when the data are examined on a dose basis, the older birds appear to have 1 study with a lower dose. Using the geometric mean of doses of studies grouped by age and test duration would result in a more appropriate conclusion regarding the greater sensitivity of younger birds. Having a large enough data set to make this type of adjustment is unusual, however, leaving the risk assessor with no way of knowing whether the calculated dose is affected by incorrect body weight or FIR values. Thus, calculating threshold doses from studies that only reported food concentrations, particularly if body weights or FIRs were not measured, adds uncertainty to selection of TRVs.

Therefore, given that the conversion to a dose metric does not standardize the toxicity results among tests conducted with the same species, there is no reason to believe that it would do so among different species. If the purpose is to achieve uniformity of results among multiple studies for a single species, and to make adjustments for differences in FIR as a function of body weight, the example in the present study makes clear that the study data are not sufficiently robust to support such a calculation. Selection of an appropriate TRV is highly dependent on test species, age of test animals, duration of study, type of metal salt, and so forth. The variability in interspecific extrapolations illustrated in Tables3 and 4 is further compounded by differences in absorption, distribution, metabolism, and excretion rates that are not related to body weight. Although the use of concentration-based NOEACs will have similar problems with cross-species absorption, distribution, metabolism, and excretion differences, they circumvent the added variability of incorrect assumptions about body weights and FIRs of test species and species of concern. Table 4 illustrates that dietary Cu concentrations protective of the growth endpoint based on studies with 9 different mammal species, range from 2.1 mg/kg food to 2088 mg/kg food whereas concentrations calculated using dose conversions range from 0.02 mg/kg food to 36 mg/kg food, thus increasing the variability and failing to achieve a meaningful reduction in uncertainty.

DISCUSSION

Newman et al. 18 stated that knowledge concerning the relative importance of exposure pathways for metals can be used to focus both data collection and modeling efforts. Identification of the most critical exposure pathways and parameters helps identify key data gaps that may be addressed through focused data collection, either in the field, in the laboratory, or from published literature. As new information, whether literature based or site specific, is acquired and implemented, the relative importance of pathways and parameters can change.

Our analysis for the present study identified extensive variability in the relative influence of parameters used in wildlife exposure models, such as those used for the Eco-SSL. Although the TRV consistently displayed the greatest influence, other parameters such as FIR, soil ingestion rate, and bioaccumulation also displayed significant influence, depending on the receptors and analyte considered. Dietary and soil bioavailability displayed the least influence. This may, however, be an artifact of limited sample size or a function of distributions constrained to range from 0% to 100%, and not necessarily indicative of the importance of these parameters. Results of the sensitivity analyses, however, may differ depending on whether they are applied to rather simple models for screening (such as the Eco-SSLs) or for more detailed models used to develop site-specific risk estimates or cleanup values.

The relative influence of the TRV may be over represented in our analysis because TRVs are highly variable and can vary over a much wider range than can the exposure parameters in the Eco-SSL model. For example, the breadth of NOAEL distributions varied from 0.8 times the Eco-SSL NOAEL (for Be in mammals) to over 41 000 times the Eco-SSL NOAEL (for Sb in mammals; Supplemental Data, Table S1). Most (75%) analyte–receptor combinations displayed NOAEL ranges that were more than 11-fold greater than the Eco-SSL NOAEL. Therefore, accuracy in determining the TRV (such as using ED20 effect levels as opposed to the less precise NOAEL and LOAEL) is highly compelling. However, setting that aside, the next most variable and important exposure parameters are bioaccumulation and food ingestion, with bioaccumulation varying from 0 times to 12.5 times the Eco-SSL value (depending on the analyte and food type) and food ingestion varying from a low of 0.71 times the Eco-SSL value for the meadow vole to 1.57 times the Eco-SSL FIR for the long-tailed weasel. The remaining parameters, soil ingestion (Ps), and both bioavailability parameters by definition are constrained to vary between 0% and 100%. This wide disparity between the distributions of the exposure and effects parameters likely accounts for the calculated importance of the TRV in this analysis.

Nonetheless, the relative influence of the breadth of the distribution for each model parameter on estimated soil concentrations is partially displayed in the summary statistics for the range of estimated soil concentrations (Table 1). Parameters that are constrained to vary between 0% and 100% (soil ingestion and both bioavailability parameters) generally have lower median estimated soil concentration ranges as compared with those that do not. The divergence of maximum ranges from this pattern (narrower estimated soil concentration distributions associated with parameters that are constrained to vary between 0% and 100%) is likely a function of the interactions between parameters (e.g., bioaccumulation and soil ingestion) expanding the range of estimated soil concentrations.

The peer review panel for the USEPA's draft Ecological Soil Screening Level (Eco-SSL) Guidance document recommended that sensitivity analyses be conducted to identify the model parameters that are most important in determining the final Eco-SSL value 19. An initial sensitivity analysis on a subset of analytes (including Cr, Co, and Sb) using draft parameter values found that the TRV was the most influential value, followed by soil–plant or soil–earthworm BAFs 20. Contrary to the current analysis, food and soil ingestion rates were consistently among the least influential parameters 20. However, the importance of soil ingestion varied by chemical, with analytes displaying greater bioaccumulation potential having lower importance, and analytes displaying lower bioaccumulation potential having greater importance. This corresponds with our results. Variation in the slope and intercept had comparatively little influence on the final Eco-SSL value for log-linear regression bioaccumulation models.

Surprisingly few examples of sensitivity analyses on wildlife exposure models such as that used in the Eco-SSLs are reported in the literature. What is reported, however, is generally consistent with results from our analyses. For example, Arcadis 21 reports risk estimates to be most sensitive to the TRV selected. The concentration of contaminants in prey was the 2nd most important, and incidental soil ingestion rate and bioavailability (in gut or sediment) were less influential parameters.

Hope 22 reported that the 3 parameters with greatest influence on polychlorinated biphenyl (PCB) exposure of mink (Mustela vision) assumed to forage in both terrestrial and aquatic habitats were soil PCB concentration (73% of variation), AUF (9.6% of variation), and Ps (3.1% of variation); dietary bioavailability of PCBs accounted for only 0.1% of variation. In contrast, Moore et al. 23 found that exposure of mink and kingfisher (Ceryle alcyon) to Hg and PCBs in a strictly aquatic habitat was most sensitive to food ingestion (as represented by metabolic rate and energy content of prey), accounting for 68% to 85% of variation, respectively. Because the models of both Hope 22 and Moore et al. 23 are based on field-collected data, and the Hope 22 model included AUF, the results obtained may not be strictly comparable to the present study.

Nominal-range sensitivity analyses were used by the USEPA 24 to determine the relative influence of exposure parameters for the spotted sandpiper (Actitis macularia), tundra swan (Cygnus columbianus), and vagrant shrews (Sorex vagrans) as part of the baseline ecological risk assessment for the Coeur d'Alene Basin in Idaho, USA. Parameters with the greatest influence on exposure for each species, ordered by decreasing importance, were: Pb concentrations in soil–sediment, Pb concentrations in aquatic invertebrates, Ps, body weight, and bioavailability for spotted sandpipers; soil ingestion, Pb concentrations in aquatic plants, Pb concentrations in soil–sediment, body weight, and bioavailability for tundra swan; and arthropod BAF, Pb concentrations in soil–sediment, Ps, bioavailability, and body weight for vagrant shrews. Because distributions for all of the influential parameters (except for soil ingestion by spotted sandpipers) were based on site-specific data, although variability in risk estimates for these species was considerable, uncertainty was considered to be low.

In an analysis performed to support the development of the Eco-SSLs, Menzie-Cura and TN&A 25 concluded that bioaccumulation by plants or soil invertebrates was the most important exposure route for Cd, Cu, Pb, Ni, Se, and Zn for herbivores and invertivores, respectively. In contrast, incidental soil ingestion was most important for Al, Fe, and V. Similar to our results, they identify soil ingestion as a more important pathway for herbivores, and bioaccumulation was proportionally more important for invertivores.

The relative importance of soil ingestion also may vary as a function of life history characteristics, with soil ingestion being of greater importance among ground feeding species as compared with more arboreal- or aerial-feeding species. Specific data to address this question, however, are currently lacking, because virtually all species for which soil ingestion data are available forage primarily or exclusively on the ground 6.

Although the focus of our analysis was on Eco-SSLs for metals, likely our results are generally applicable to Eco-SSLs for organics (i.e., TRVs are likely the most important parameter followed by FIR; relative bioaccumulation rates into plants, invertebrates, or small mammals likely influence the relative importance of the soil ingestion and food ingestion pathways, and so forth). However, until similar sensitivity analyses are performed for Eco-SSLs for organic contaminants, the specific degree to which they are comparable will be unknown.

TRVs for higher tiered assessments or soil cleanup values

Unlike screening level risk assessments, soil cleanup values must be specific to locally present wildlife species. These cleanup values should be closely tied to risk thresholds for locally important species to be cost effective and to minimize habitat destruction during a cleanup. This means the risk assessor needs to move from general estimates of effect thresholds to species-specific values, necessitating some method of extrapolating from tested species to nontested species of concern. For most metals, at least a small data set is available on toxicity for domesticated livestock (including poultry), which can be used as a starting point 26,27. Nevertheless, extrapolating from livestock or standard laboratory test species to the wide variety of wildlife species remains a challenge. Mineau et al. 28,29 developed a set of scaling factors for extrapolating threshold doses for pesticides among bird species based on acute mortality (LD50) data, but these may not be applicable to chronic responses nor to metals that have significantly different modes of action.

Extrapolation of acute toxicity thresholds from tested to nontested species can be generated using the USEPA's web-based interspecies correlation estimation (ICE) model 30. This model presents the correlation of the sensitivity of pairs of aquatic or terrestrial species based on a meta-analysis of all available toxicity data for those 2 species. The interspecies correlation coefficient can then be used to estimate the toxicity of a new chemical as long as data are available for 1 of the species in the pair. This model is constrained to those species for which there are some toxicity data and has been validated only for acute (LD50) studies. The model is most useful as a screening tool, because it can be used to accurately predict acute exposure (the hazard level at the 5th percentile of the species sensitivity distribution) 3133.

The primary limitations on the development of a chronic ICE or other interspecies extrapolation models for chronic data are the lack of readily available chronic toxicity data for a sufficiently large number of species, the high number of potentially relevant chronic endpoints, and the lack of standardization among chronic toxicity study designs. For example, Luttik et al. 34 found a large number of unbounded NOECs (e.g., >50 mg/kg body wt) and relatively few bounded values. There is no simple solution for the lack of toxicological data relevant to a species of interest. Development of interspecies correlation relationships for chronic data similar to ICE or Mineau et al. 29 from acute data appears to be the most promising, although currently no robust compilation of chronic toxicity data to populate such a database is available.

Uncertainty around the interspecies correlation can be reduced by increasing phylogenic similarity and focusing on chemical groups with similar modes of action 35. Refining the analysis of chronic data by using the same endpoint from each of the reproduction studies, rather than the most sensitive endpoint, may be possible regardless of similarity across studies. For those species for which no correlation coefficients are available, estimates may be made using correlation coefficients from species with close phylogenetic relationships. However, conflicting conclusions have been drawn about the degree that phylogeny plays in interspecies differences in sensitivity. Clark et al. 36 and Raimondo et al. 35 found that similarities in sensitivity were directly related to phylogeny for metals. In contrast, Baril et al. 37, Mineau et al. 29, Joermann 38, and Schafer and Brunton 39 concluded that toxicological susceptibility does not always follow phylogenetic lines, at least for cholinesterase-inhibiting pesticides.

Alternatively, Allard et al. 4 recommend the use of species sensitivity distributions (SSD) to select an appropriately conservative threshold value when data are available for a sufficient number of species (generally, 8–10 species from different families are required for development of an SSD with acceptable uncertainty bounds around the 5th percentile value 40). However, Allard et al. acknowledge that having sufficient data for vertebrate wildlife species from which to generate an SSD is rare and instead suggest the development of a composite dose–response function using data from all tested species. A benchmark dose 41 then can be derived from this curve that would be protective of all species represented and would be more robust than single-species extrapolations. The acceptability of such an approach is still highly theoretical, but it has potential, particularly for developing toxicity thresholds for chronic exposures.

CONCLUSIONS

The analysis in the present study indicates clearly that output of the Eco-SSL wildlife exposure model is significantly influenced by the parameter values selected, with the relative influence of each parameter varying by receptor and analyte. This variation in the influence of model parameters potentially complicates data collection to support decision-making. For this reason, and as suggested by Sample et al. 2, parameters to support wildlife exposure estimation should be developed on a site-, species-, and analyte-specific basis to the extent that the expense for their derivation is justified by the value they add. The analysis conducted in the present study has shown that food and soil ingestion rates have the greatest influence on the estimation of exposure of wildlife. Therefore, we recommend the use of the allometric equations developed by Nagy 42 to standardize this input across all risk assessments. Using species-specific soil ingestion rates measured by Beyer et al. 6, or generating these data on a site-specific basis, would provide a greater degree of site- and species-specific representativeness and reduce associated uncertainty. For example, small mammals, and sometimes ground feeding birds, are regularly collected for tissue analysis to support food-web modeling. Removal and analysis of gastrointestinal tract contents to determine soil ingestion rates could be conducted before performing analyses of chemical residues in tissues. Our analysis clearly shows that the Eco-SSLs, especially those for invertivorous birds and mammals, are conservative for some metals and when exceeded should be supplemented with site-specific data before selecting soil cleanup values and performing remedial actions.

Although no clear guidance for selecting TRVs suitable for developing soil cleanup values is available from regulatory agencies 3, multiple viable options for data based, site-specific selections have been made in the present study and elsewhere [4,43]. Although several of the approaches necessitate time-consuming literature searches and data retrieval, the number of species and feeding guilds for which TRVs are required for site cleanup is less than that for initial screening. Given that this analysis indicates that the TRV has a great influence on final risk estimates, the added expense of derivation of species-appropriate TRVs would be offset by the reduced uncertainty when making decisions about the nature and extent of contaminated soil cleanup.

Acknowledgments

Support for this research was provided by Rio Tinto.

Supporting Information

All Supplemental Data may be found in the online version of this article.

Table S1

(16 KB XLS).

etc0033-2386-SD1.pdf (7.1MB, pdf)
Table S2

(14 KB XLS).

etc0033-2386-SD2.xlsx (15.6KB, xlsx)
Table S3

(12 KB XLS).

etc0033-2386-SD3.xlsx (14KB, xlsx)
Table S4

(16 KB XLS).

etc0033-2386-SD4.xlsx (11.5KB, xlsx)
Table S5

(15 KB XLS).

etc0033-2386-SD5.xlsx (15.3KB, xlsx)
Table S6

(13 KB XLS).

etc0033-2386-SD6.xlsx (14.7KB, xlsx)
Table S7

(94 KB XLS).

etc0033-2386-SD7.xlsx (12.9KB, xlsx)
Table S8

(23 KB XLS).

etc0033-2386-SD8.xlsx (91.5KB, xlsx)
Table S9

(47 KB XLS).

etc0033-2386-SD9.xlsx (22.6KB, xlsx)
Figures S1 – S16

(7.5 MB PDF).

etc0033-2386-SD10.xlsx (46.4KB, xlsx)
etc0033-2386-SD11.docx (87KB, docx)

REFERENCES

  1. US Environmental Protection Agency. Washington, DC: Office of Solid Waste and Emergency Response; 2005. Guidance for developing ecological soil screening levels; pp. 7–55. OSWER Directive 9285. [Google Scholar]
  2. Sample BE, Schlekat C, Spurgeon DJ, Menzie C, Rauscher J, Adams B. Recommendations to improve wildlife exposure estimation for development of soil screening and cleanup values. Int Environ Assess Manage. 2014;10:372–387. doi: 10.1002/ieam.1482. [DOI] [PubMed] [Google Scholar]
  3. Mayfield DB, Fairbrother A. Efforts to standardize wildlife toxicity values remain unrealized. Int Environ Assess Manage. 2013;9:114–123. doi: 10.1002/ieam.1357. [DOI] [PubMed] [Google Scholar]
  4. Allard P, Fairbrother A, Hope BK, Hull RN, Johnson MS, Kapustka L, Mann G, McDonald B, Sample BE. Recommendations for the development and application of wildlife toxicity reference values. Int Environ Assess Manage. 2010;6:28–37. doi: 10.1897/IEAM_2009-010.1. [DOI] [PubMed] [Google Scholar]
  5. US Environmental Protection Agency. Washington, DC: Office of Solid Waste and Emergency Response; 2007. Guidance for developing ecological soil screening levels (Eco-SSLs) pp. 7–55. OSWER Directive 9285. [Google Scholar]
  6. Beyer WN, Conner E, Gerould S. Estimates of soil ingestion by wildlife. J Wildl Manage. 1994;58:375–382. [Google Scholar]
  7. Bechtel-Jacobs Company LLC. Oak Ridge, TN, USA: Oak Ridge National Laboratory; 1998. Empirical models for the uptake of inorganic chemicals from soil by plants. BJC/OR-133. BJC/OR-133. [Google Scholar]
  8. Sample BE, Beauchamp JJ, Efroymson R, Suter GW, II, Ashwood T. Development and Validation of Bioaccumulation Models for Earthworms. Oak Ridge, TN, USA: Oak Ridge National Laboratory; 1998. ES/ER/TM-220. [Google Scholar]
  9. Sample BE, Beauchamp JJ, Efroymson R, Suter GW, II, Ashwood T. Development and Validation of Bioaccumulation Models for Small Mammals. Oak Ridge, TN, USA: Oak Ridge National Laboratory; 1998. ES/ER/TM-219. [Google Scholar]
  10. Zar JH. Biostatistical Analysis. 2nd Ed. Englewood Cliffs, NJ, USA: Prentiss-Hall, Inc; 1984. [Google Scholar]
  11. Baes CF, Sharp RD, Sjoreen AL, Shor RW. A Review and Analysis of Parameters for Assessing Transport of Environmentally Released Radionuclides Through Agriculture. Oak Ridge, TN, USA: Oak Ridge National Laboratory; 1984. ORNL-5786. [Google Scholar]
  12. Koch I, Reimer K. Bioaccessibility extractions for contaminant risk assessments. In: Pawliszyn J, Le XC, Li XF, Lee HK, editors. Comprehensive Sampling and Sample Preparation. Vol. 3. Oxford, UK: Elsevier, Academic Press; 2012. pp. 487–507. [Google Scholar]
  13. Ruby MV, Davis A, Schoof R, Eberle S, Sellstone C. Estimation of lead and arsenic bioavailability using a physiologically based extraction test. Environ Sci Technol. 1996;30:422–430. [Google Scholar]
  14. National Research Council. Bioavailability of Contaminants in Soils and Sediments: Processes, Tools, and Applications. Washington, DC: National Academies Press; 2003. [cited 2013 May 5]. Available from: http://www.nap.edu/openbook/0309086256/html/ [Google Scholar]
  15. Cullen AC, Frey HC. Probabilistic Techniques in Exposure Assessment. New York, NY, USA: Plenum Press; 1999. [Google Scholar]
  16. SAS Institute, Inc. Cary, NC, USA: 2004. SAS/STAT User's Guide. Version 9.1. Vol. 6. [Google Scholar]
  17. US Environmental Protection Agency. Washington, DC, USA: Office of Solid Waste and Emergency Response; 2007. Ecological soil screening levels for copper. OSWER Directive 9285. [Google Scholar]
  18. Newman MC, Diamond GL, Menzie C, Moya J, Nriagu J. Washington, DC: US Environmental Protection Agency; 2004. Issue paper on metal exposure assessment. [cited 26 August 2013]. Available from: www.epa.gov/raf/publications/pdfs/exposurefinal81904.pdf. [Google Scholar]
  19. Versar. Washington, DC, USA: US Environmental Protection Agency, ORD/Office of Science Policy; 2000. Peer review workshop report on ecological soil screening level (Eco-SSL) guidance document. [cited 2013 August 26]. Available from: www.epa.gov/oswer/riskassessment/ecorisk/pdf/peerrev.pdf. [Google Scholar]
  20. Sample BE. Sensitivity analysis of the wildlife Eco-SSL models. 2001. Presentation at 2001 SRA Meeting, Seattle, WA, USA. December, 2001. [cited 26 August 2013]. Available from: http://www.riskworld.com/abstract/2001/SRAam01/ab01aa278.htm.
  21. Arcadis. Kingston, TN, USA: Tennessee Valley Authority Ash Recovery Project; 2012. Wildlife dietary exposure models. [cited 26 August 2013]. Available from: http://www.tva.com/kingston/admin_record/pdf/NTC/NTC83/Appendix%20I%20BERA/Appendix_W_Wildlife_Dietary_Exposure_Models_Report.pdf. [Google Scholar]
  22. Hope BK. Assessment of risk to terrestrial receptors using uncertainty analysis. Human Ecol Risk Assess. 1999;5:145–170. [Google Scholar]
  23. Moore DRJ, Sample BE, Suter GW, II, Parkhurst BR, Teed RS. A probabilistic risk assessment of the effects of methylmercury and PCBs on mink and kingfishers along East Fork Poplar Creek, Oak Ridge, Tennessee. Environ Toxicol Chem. 1999;18:2941–2953. [Google Scholar]
  24. US Environmental Protection Agency. Seattle, WA: Coeur d'Alene Basin Remedial Investigation/Feasibility Study; 2001. Final ecological risk assessment. [Google Scholar]
  25. Menzie-Cura & Associates, Inc., TN&A, Inc. Washington, DC: US Environmental Protection Agency, National Center for Environmental Assessment; 2000. Development of a framework to address bioavailability issues in wildlife. EPA/630/P-02/003A. [Google Scholar]
  26. National Research Council (NRC) Mineral Tolerance of Domestic Animals. Washington, DC, USA: The National Academies Press; 1980. [cited 26 August 2013]. Available from: http://www.nap.edu/catalog.php?record_id=25. [Google Scholar]
  27. National Research Council (NRC) Mineral Tolerance of Animals. 2nd. Washington, DC: The National Academies Press; 2005. p. 497. [cited 26 August 2013]. Available from: http://www.nap.edu/catalog/11309.html. [Google Scholar]
  28. Mineau P, Collins BT, Baril A. On the use of scaling factors to improve interspecies extrapolation of acute toxicity in birds. Regul Toxicol Pharmacol. 1996;24:24–29. doi: 10.1006/rtph.1996.0061. [DOI] [PubMed] [Google Scholar]
  29. Mineau P, Baril A, Collins BT, Duffe J, Joerman G, Luttik R. Pesticide acute toxicity reference values for birds. Rev Environ Contam Toxicol. 2001;170:13–74. [PubMed] [Google Scholar]
  30. Raimondo S, Mineau P, Barron MG. Estimation of chemical toxicity in wildlife species using interspecies correlation models. Environ Sci Technol. 2007;41:5888–5894. doi: 10.1021/es070359o. [DOI] [PubMed] [Google Scholar]
  31. Dyer SD, Versteeg DJ, Belanger SE, Chaney JG, Mayer FL. Interspecies correlation estimates predict protective environmental concentrations. Environ Sci Technol. 2008;40:3102–3111. doi: 10.1021/es051738p. [DOI] [PubMed] [Google Scholar]
  32. Awkermann J, Raimondo S, Barron MG. Development of species sensitivity distributions for using wildlife using interspecies toxicity correlation models. Environ Sci Technol. 2008;42:3447–3452. doi: 10.1021/es702861u. [DOI] [PubMed] [Google Scholar]
  33. Awkermann J, Raimondo S, Barron MG. Estimation of wildlife hazard levels using interspecies correlation models and standard laboratory rodent toxicity data. J Toxicol Environ Health, Part A. 2009:1604–1609. doi: 10.1080/15287390903232491. [DOI] [PubMed] [Google Scholar]
  34. Luttik R, Mineau P, Roelofs W. A review of interspecies toxicity extrapolation in birds and mammals and a proposal for long-term toxicity data. Ecotoxicology. 2005;14:817–832. doi: 10.1007/s10646-005-0030-8. [DOI] [PubMed] [Google Scholar]
  35. Raimondo S, Jackson CR, Barron MG. Influence of taxonomic relatedness and chemical mode of action in acute interspecies estimation models for aquatic species. Environ Sci Technol. 2010;44:7711–7716. doi: 10.1021/es101630b. [DOI] [PubMed] [Google Scholar]
  36. Clark J, Ortego LS, Fairbrother A. Sources of variability in plant toxicity testing. Chemosphere. 2004;57:1599–1612. doi: 10.1016/j.chemosphere.2004.07.044. [DOI] [PubMed] [Google Scholar]
  37. Baril A, Jobin B, Mineau P, Collins BT. A consideration of inter-species variability in the use of the median lethal dose (LD50) in avian risk assessment. 1994. Tech Rep Series 216. Canadian Wildlife Service Headquarters, Hull, Quebec.
  38. Joermann G. Comparative toxicity of pesticides to birds. Nachrichtenblatt des Deutschen Pflanzenschutzdienstes. 1991;43:275–279. [Google Scholar]
  39. Schafer EW, Jr, Brunton RB. Indicator bird species for toxicity determinations: Is the technique usable in test method development. In: Beck JR, editor. Vertebrate pest control and management materials. Philadelphia, PA: American Society for Testing and Materials; 1979. pp. 157–168. [Google Scholar]
  40. Wheeler JR, Grist EPM, Leung KMY, Morritta D, Crane M. Species sensitivity distributions: Data and model choice. Marine Poll Bull. 2002;45:192–202. doi: 10.1016/s0025-326x(01)00327-7. [DOI] [PubMed] [Google Scholar]
  41. Moore DRJ, Caux P-Y. Estimating low toxic effects. Environ Toxicol Chem. 1997;16:794–801. [Google Scholar]
  42. Nagy KA. Food requirements of wild animals: Predictive equations for free-living mammals, reptiles, and birds. Nutr Abs Rev Series B. 2001;71:21R–31R. [Google Scholar]
  43. Mayfield DB, Johnson MS, Burris JA, Fairbrother A. Furthering the derivation of predictive wildlife toxicity reference values for use in soil cleanup decisions. Integr Environ Assess Manage. 2014;10:358–371. doi: 10.1002/ieam.1474. [DOI] [PubMed] [Google Scholar]
  44. Hoda AHA, Maha MM. Potency of copper as a growth promoter in broiler chickens. Vet Med J (Giza) 1995;43:77–85. [Google Scholar]
  45. McGhee F, Greger CR, Couch JR. Copper and iron toxicity. Poult Sci. 1965;44:310–312. doi: 10.3382/ps.0440310. [DOI] [PubMed] [Google Scholar]
  46. Mehring AL, Brumbaugh JH, Sutherland AJ, Titus HW. Tolerance of growing chickens for dietary copper. Poult Sci. 1960;39:713–719. [Google Scholar]
  47. King JO. The feeding of copper sulphate to ducklings. Br Poult Sci. 1975;16:409–411. doi: 10.1080/00071667508416205. [DOI] [PubMed] [Google Scholar]
  48. Kashani AB, Samie H, Emerick RJ, Carlson CW. Effect of copper with three levels of sulfur containing amino acides in diets for turkeys. Poult Sci. 1986;65:1754–1759. doi: 10.3382/ps.0651754. [DOI] [PubMed] [Google Scholar]
  49. Yannakopoulos AL, Tserveni-Gousi AS, Zervas G. Effect of dietary copper sulfate on the performance of growing Japanese quail. J Agric Sci. 1990;115:291–293. [Google Scholar]
  50. Dodds-Smith ME, Johnson MS, Thompson DJ. Trace metal accumulation by the shrew Sorex araneus. I. Total body burden, growth, and mortality. Ecotoxicol Environ Saf. 1992;24:102–117. doi: 10.1016/0147-6513(92)90039-6. [DOI] [PubMed] [Google Scholar]
  51. Hebert CD, Elwell MR, Travlos GS, Fitz CJ, Bucher JR. Subchronic toxicity of cupric sulfate administered in drinking water and feed to rats and mice. Fundam Appl Toxicol. 1993;21:461–475. doi: 10.1006/faat.1993.1122. [DOI] [PubMed] [Google Scholar]
  52. Uthus EO. High dietary arsenic exacerbates copper deprivation in rats. J Trace Elem Exp Med. 2001;14:43–55. [Google Scholar]
  53. Arthur D. Interrelationships of molybdenum and copper in the diet of the guinea pig. J Nutr. 1965;87:69–76. doi: 10.1093/jn/87.1.69. [DOI] [PubMed] [Google Scholar]
  54. Bassuny SM. The effect of copper sulfate supplement on rabbit performance under Egyptian conditions. J Appl Rabbit Res. 1991;14:93–97. [Google Scholar]
  55. Grobner MA, Cheeke PR, Patton NM. Effect of dietary copper and oxytetracycline on growth and mortality of weanling rabbits. J Appl Rabbit Res. 1986;9:46–53. [Google Scholar]
  56. Brandt A. Effect of dietary copper and zinc on the haemotology of male pastel mink kits: A pilot investigation. Scientifur. 1983;7:61–65. [Google Scholar]
  57. Ward TL, Watkins KL, Southern LL. Interactive effects of sodium zeolite a and copper in growing swine growth and bone and tissue mineral concentrations. J Anim Sci. 1991;69:726–733. doi: 10.2527/1991.692726x. [DOI] [PubMed] [Google Scholar]
  58. Allcroft R, Burns KN, Lewis G. The effects of high levels of copper in rations for pigs. Vet Rec. 1961;73:714. [Google Scholar]
  59. Smith JD, Jordan RM, Nelson ML. Tolerance of ponies to high levels of dietary copper. J Anim Sci. 1975;41:1645–1649. doi: 10.2527/jas1975.4161645x. [DOI] [PubMed] [Google Scholar]
  60. Engle TE, Spears JW. Performance, carcass characteristics and lipid metabolism in growing and finishing simmental steers fed varying concentrations of copper. J Anim Sci. 2001;79:2920–2925. doi: 10.2527/2001.79112920x. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Table S1

(16 KB XLS).

etc0033-2386-SD1.pdf (7.1MB, pdf)
Table S2

(14 KB XLS).

etc0033-2386-SD2.xlsx (15.6KB, xlsx)
Table S3

(12 KB XLS).

etc0033-2386-SD3.xlsx (14KB, xlsx)
Table S4

(16 KB XLS).

etc0033-2386-SD4.xlsx (11.5KB, xlsx)
Table S5

(15 KB XLS).

etc0033-2386-SD5.xlsx (15.3KB, xlsx)
Table S6

(13 KB XLS).

etc0033-2386-SD6.xlsx (14.7KB, xlsx)
Table S7

(94 KB XLS).

etc0033-2386-SD7.xlsx (12.9KB, xlsx)
Table S8

(23 KB XLS).

etc0033-2386-SD8.xlsx (91.5KB, xlsx)
Table S9

(47 KB XLS).

etc0033-2386-SD9.xlsx (22.6KB, xlsx)
Figures S1 – S16

(7.5 MB PDF).

etc0033-2386-SD10.xlsx (46.4KB, xlsx)
etc0033-2386-SD11.docx (87KB, docx)

Articles from Environmental Toxicology and Chemistry / Setac are provided here courtesy of Wiley

RESOURCES