Abstract
Background
Dichlorodiphenyltrichloroethane (DDT) continues to be used for control of infectious diseases in several countries. In-utero exposure to DDT and dichlorodiphenyldichloroethylene (DDE) has been associated with developmental and cognitive impairment among children. We examined this association in an historical cohort in which the level of exposure was greater than in previous studies.
Methods
The association of in-utero DDT and DDE exposure with infant and child neurodevelopment was examined in approximately 1100 subjects in the Collaborative Perinatal Project, a prospective birth cohort enrolling pregnant women from 12 study centers in the U.S. from 1959 to 1965. Maternal DDT and DDE concentrations were measured in archived serum specimens. Infant mental and motor development was assessed at age 8 months using the Bayley Scales of Infant Development, and child cognitive development was assessed at age 7 years using the Wechsler Intelligence Scale for Children.
Results
Although levels of both DDT and DDE were relatively high in this population (median DDT concentration, 8.9 µg/L; DDE, 24.5 µg/L), neither was related to Mental or Psychomotor Development scores on the Bayley Scales or to Full-Scale IQ at 7 years of age. Categorical analyses showed no evidence of dose-response for either maternal DDT or DDE, and estimates of the association between continuous measures of exposure and neurodevelopment were indistinguishable from 0.
Conclusions
Adverse associations were not observed between maternal serum DDT and DDE concentrations and offspring neurodevelopment at 8 months or 7 years of age in this cohort.
Dichlorodiphenyltrichloroethane (DDT) is an organochlorine compound that was widely used as an insecticide until its ban by most industrialized countries in the late 20th century. The insecticide is composed primarily of the isomer p,p’-DDT (1,1,1-trichloro-2,2-bis(p-chlorophenyl)ethane) (hereafter, DDT), which degrades or is metabolized to p,p’-DDE (1,1-dichloro-2,2-bis(p-chlorophenyl)ethylene) (hereafter, DDE).1 Humans are exposed to DDT and DDE transplacentally, during breastfeeding, and through contaminated food sources.
DDT is insecticidal through toxic effects on the nervous system, and is still used in countries where malaria vector control is needed, mostly in sub-Saharan Africa. Some studies have reported adverse effects on human health in relation to DDT and DDE exposure, including potential effects on the developing human nervous system.2–6 Overall, however, this association has been examined in relatively few populations, and DDT and DDE concentrations in these studies are generally much lower than in populations where DDT is still in use.
To this end, our study was designed to evaluate the association of DDT and DDE levels in maternal third-trimester serum with child performance on the Bayley Scales of Infant Development at 8 months of age and cognitive testing at 7 years of age using the Weschler Intelligence Scale for Children. Data are from a subset of mother-infant pairs in the Collaborative Perinatal Project, a prospective cohort study that enrolled pregnant women during a time of peak DDT use in the United States.1
Methods
Study population
The Collaborative Perinatal Project was a prospective cohort study designed to identify determinants of neurodevelopmental deficits in children.7 Pregnant women were recruited between 1959 and 1965 from 12 U.S. study centers in Baltimore, Boston, Buffalo, Memphis, Minneapolis, New Orleans, Philadelphia, Portland, Providence, and Richmond, and from 2 centers in New York; nearly all were prenatal clinics at university hospitals. The method of systematic sampling used to select subjects varied across center, as described previously.8 Women were ineligible if they were incarcerated, if they were planning to leave the area or to give up the child for adoption, or if they gave birth on the day they were recruited into the study. Of the 55,908 pregnant women recruited into the study, 4% were lost to follow-up before delivery. Among women not lost to follow-up at delivery, approximately 80% had been recruited after 20 weeks of gestation; all had at least one prenatal visit recorded, and 9161 had more than one pregnancy in the study. 71% of the liveborn children in the Collaborative Perinatal Project were followed to age 7 years.
Data collection
Participant data were collected at each prenatal visit, at delivery, and when the child was 24 hours, 4 months, 8 months, and 1, 3, 4, 7, and 8 years of age. Beginning at recruitment, nonfasting maternal blood was collected every 8 weeks during pregnancy.9 Also at recruitment, extensive data were collected on each woman’s reproductive and medical history, as well as demographic and lifestyle information. The socioeconomic index calculated for subjects was the mean of three percentile scores (for education, occupation, and family income), where education was that of the head of the household, occupation was that of the head of the household or the chief wage earner, and the score used to calculate the percentile for an occupation was based on the percentiles of education and income among those with the same occupation. Additional demographic, health, and lifestyle data were collected during follow-up, and developmentally appropriate tests were administered to infants and children to assess various aspects of cognitive development – including when the children were 8 months and 7 years of age, as described below.
Outcome measures
When infants were 8 months of age, personnel trained by Nancy Bayley administered a research version of the Bayley Scales of Infant Development to assess infant mental and psychomotor development. The Bayley Mental Development Index assesses age-appropriate cognitive, language, and social development. The research version of the Bayley Mental Development Index included a 106-item scale (at 8 months, 83 items were administered), which differed slightly from the published Mental Development Index10; the latter included an additional 12 items, and 1 item from the research version was not included in the published Mental Development Index. The Bayley Psychomotor Development Index assesses various aspects of fine and gross motor coordination, and consisted of 43 items (at 8 months, infants completed 37 items). Most infants (93%) were tested between 7.5 and 9 months of age. Seven children (0.6 percent) were tested before 7.5 months; the remaining 6.4 percent were tested after 9 months and before 10 months of age. Consistent testing protocols were ensured by regular visits by Nancy Bayley to monitor testing procedures, and by assessments of inter-rater reliability across centers.11 Scoring of each Scale of Infant Development was checked twice, once locally and a second time at the National Institute of Mental Health. Both the Mental Development Index and the Psychomotor Development Index showed good internal reliability (split-half reliabilities, r=0.89 and 0.90, respectively).12
When the child was 7 years of age, trained psychologists administered seven of the eleven components of the 1949 Wechsler Intelligence Scale for Children (Information, Comprehension, Vocabulary, Digit Span, Picture Arrangement, Block Design, and Coding); the Full-Scale IQ was based on the results from these seven components. At the time of testing, the median age of the children was 7.0 years (quartiles, 6.9, 7.2), and the median grade in school was close to second. Two IQ tests were administered three months apart to a subset of study children by examiners from different study centers; correlation between the scores was high (r=0.85, n=416).13
Selection criteria and sampling scheme
We selected mother-child pairs based on their participation in the Collaborative Perinatal Project during pregnancy (baseline) and at 8 months and 7 years – the ages at which the neurodevelopmental outcomes of interest were assessed. To be eligible, children had to be singleton and with a 3 ml third-trimester maternal serum specimen. We sampled 1,256 mother-infant pairs at random from the 43,628 eligible children. Only one pregnancy per mother was eligible for selection. As additional assurance of power, we also selected at random: (1) 197 children whose Psychomotor Development Index was one or more standard deviations below or above the mean and (2) an additional 207 children whose 7-year Full-Scale Wechsler Intelligence Scale for Children IQ score was one or more standard deviations below or above the mean. By increasing the probability of inclusion for subjects with extreme values (outcome-dependent sampling), study power was increased relative to a study of equivalent size with simple random sampling.14
We sampled subjects based on their Psychomotor Development Index rather than their Mental Development Index because at the time our study was planned, the extant literature reported adverse associations between maternal PCB concentrations and Psychomotor as opposed to Mental Development Indices. Of the 1,453 observations selected for the Bayley Scales of Infant Development analyses (selected at random, n=1,256; Psychomotor Development sample, n=197), we excluded 3 duplicate observations (selected both in the random and Psychomotor Development samples), 232 of those selected at random who were missing Bayley Scales of Infant Development data 60 who were missing DDE values, 3 who were missing DDT values, and 13 who were missing covariate data for maternal education. This left 1142 mother-infant pairs in the Bayley Scales of Infant Development analyses. Of the 1463 observations selected for the 7-year IQ analyses (1,256 at random; IQ tail sample, n=207), we excluded 10 duplicate observations (selected both in the random and IQ samples), 314 of those selected at random who were missing IQ data, 86 who were missing DDE values, 3 who were missing DDT values, and 9 who were missing covariate data for maternal education, leaving 1041 mother-child pairs in the IQ analyses. The institutional review board of the National Institute of Environmental Health Sciences approved the study protocol.
Measurement of maternal serum DDT and DDE concentrations
Because more serum samples were available from the third trimester than for any other period, we analyzed third-trimester samples. Serum samples had been stored at −20°C in glass with no recorded thaw. In 1997–1999, the 3-ml serum samples were analyzed for DDT, DDE, 11 specific PCB congeners, and 7 other organochlorines at the Centers for Disease Control and Prevention using solid-phase extraction, followed by dual-column gas chromatographic separation with electron capture detection.15 All mothers had DDE levels greater than the limit of detection (0.61 µg/L), and all but one had DDT levels above the limit of detection (0.66 µg/L). For the mother with the low DDT level, the measured concentration reported by the laboratory was used in the analyses. Each analytic batch consisted of 10 specimens, where 1 of the specimens was an aliquot from a single large pool of serum to allow for an independent calculation of a coefficient of variation. The 9 remaining samples were individual serum specimens from mothers participating in the Collaborative Perinatal Project. As a means of ensuring comparability across sampling strata—owing to our use of outcome-dependent sampling to select mother-child pairs for the analysis—all batches that included a specimen for a subject selected by Bayley or IQ score also had at least one specimen for a subject selected at random. This was done to reduce the potential for differential bias in DDT and DDE concentrations with regard to the outcomes of interest (i.e. within-batch error was equally likely to affect specimens regardless of their selection probability). The between-batch coefficient of variation was 22.1% for DDE (mean= 11.10 µg/L) and 19.0% for DDT (mean= 29.56 µg/L). Serum cholesterol and triglycerides were also measured by the Centers for Disease Control and Prevention using standard enzymatic assays.
Data analyses
Our independent variables of interest were maternal serum DDT and DDE concentration, which were modeled on a “wet-weight” basis. We created categories for DDE concentration with cutpoints <15, 15–29.9, 30–44.9, 45–59.9 and ≥60 µg/L to be comparable with previous analyses of related data.16 Category cutpoints for DDT were chosen a priori to include a reasonable proportion of participants in the upper category, with the remaining categories of equal width. We also fit models where DDT or DDE concentration was modeled as an untransformed continuous variable in µg/L, and expressed as the change in outcome score (points) for each 5- and 15-µg/L increase in maternal DDT or DDE concentration, respectively. Potential confounding factors were selected using directed acyclic graphs.17,18 A single directed acyclic graph was created for both 8-month (Bayley Scales of Infant Development) and 7-year (IQ) outcomes, as we regarded the causal structure of these models to be similar. The minimally sufficient set of adjustment variables were study center, maternal education, race, and maternal age. Center and race (white/African-American/other) were entered in models as categorical variables, and maternal age and education were continuous variables (in years). In addition, as a secondary analysis, we examined bivariate associations of potential confounders with organochlorine concentrations and with child IQ, as a means of empirically identifying potential confounding factors in the Collaborative Perinatal Project. Results are shown first for our primary model (confounders selected using directed acyclic graphs), then for the primary model plus maternal triglyceride and total cholesterol concentrations, and finally for the primary model with the potential confounders selected empirically. Models were also fit using lipid-standardized measures of DDT and DDE (i.e. organochlorine concentrations expressed on a per-lipid basis in ng/g lipid); results using lipid-standardized measures are reported in text. To account for the outcome-dependent sampling design, regression models employed weights that were the inverse of the sampling probabilities.19 Models were fit using the GLM procedure in SAS (Version 9.2; SAS Institute, Inc., Cary, NC).
We assessed the impact of missing data on our results through multiple imputation, imputing missing outcome data at 8 months and 7 years of age, and also missing covariate data. The imputation model included maternal IQ, 4-year child IQ score, socioeconomic index, and maternal polychlorinated biphenyl congener #153 concentration, in addition to all dependent and independent variables in the regression analyses. Ten datasets were imputed and analyzed using the MI and MIANALYZE procedures in SAS (Version 9.2; SAS Institute, Inc., Cary, NC, USA). We present results for continuous measures of DDT and DDE in relation to Bayley Scales of Infant Development and IQ using both the imputed data and the complete-case data.
Finally, we estimated the dose-response relations of maternal DDT and DDE with the neurodevelopmental outcomes using a generalized additive model (GAM). This model was implemented in SAS 9.2 using the GAM procedure, specifying a smoothing spline with two degrees of freedom. This semiparametic generalized additive model model allowed us to adjust parametrically for the same covariates used in the linear analyses, as well as to estimate the association between exposure and outcome nonparametically.
Results
Sample characteristics
Overall, characteristics of the selected families were similar to the entire Collaborative Perinatal Project cohort (Table 1). Most women in our sample were between 18 and 25 years of age at the time of their child’s birth, and nearly half of all women had had two or more previous pregnancies. Slightly fewer than half of women reported being a current smoker, and approximately 55% of women reported not finishing high school. Data collected in the neonatal period showed that the great majority did not breastfeed their child.
Table 1.
Characteristics of mothers and children in the present study subsamples and those in the overall Collaborative Perinatal Project cohort.
| Characteristic | Bayley (n=1142) (%)a |
7-year IQ (n=1041) (%)a |
Entire Cohort (n=44,099) (%)a |
|---|---|---|---|
| Mothers and families | |||
| Mother’s age at child’s birth (years) | |||
| <18 | 8 | 8 | 10 |
| 18–24.9 | 51 | 51 | 50 |
| 25–29.9 | 22 | 20 | 21 |
| 30+ | 19 | 21 | 18 |
| Race | |||
| African American | 48 | 49 | 47 |
| White | 46 | 48 | 46 |
| Other | 6 | 3 | 7 |
| Previous pregnancies | |||
| 0 | 32 | 30 | 31 |
| 1 | 23 | 23 | 23 |
| 2+ | 45 | 47 | 46 |
| Current smoker | |||
| Yes | 45 | 45 | 47 |
| No | 55 | 55 | 54 |
| Socioeconomic indexb | |||
| <4 | 37 | 35 | 39 |
| 4–5.9 | 33 | 33 | 31 |
| 6+ | 31 | 32 | 30 |
| Education (years) | |||
| < high school | 55 | 55 | 57 |
| high school | 32 | 30 | 31 |
| > high school | 13 | 14 | 12 |
| Child | |||
| Sex | |||
| Boy | 49 | 51 | 51 |
| Girl | 51 | 49 | 49 |
| Breastfed > 1 day | |||
| Yes | 14 | 17 | 16 |
| No | 86 | 83 | 84 |
Percents may not sum to 100 because of rounding.
See Methods section of text for definition of socioeconomic index.
DDT and DDE concentrations
Table 2 describes maternal DDT and DDE concentrations (and their ratio) on both a wet-weight (µg/L) and per-lipid (µg/g lipid) basis for purposes of comparison across studies. Median wet-weight DDT concentration was 8.9 µg/L, and ranged from 0.4 to 91.8 µg/L, while DDE concentration was greater (median 24.5 µg/L, range 3.1 to 178.1 µg/L). The median DDT-to-DDE ratio was 0.37. Maternal DDT and DDE concentrations were strongly correlated (Spearman r=0.72).
Table 2.
DDT and DDE concentrations in mothers with complete data in the combineda subsamples (n=1393).
| DDT measure | No.b | Mean | Min | P10 | P25 | P50 | P75 | P90 | Max |
|---|---|---|---|---|---|---|---|---|---|
| Wet-weight (µg/liter) | |||||||||
| pp-DDT | 1393 | 11.10 | 0.36 | 4.65 | 6.27 | 8.90 | 13.67 | 19.95 | 91.84 |
| pp-DDE | 1393 | 29.56 | 3.14 | 12.12 | 16.81 | 24.50 | 36.00 | 52.07 | 178.06 |
| DDT:DDE | 1393 | 0.41 | 0.05 | 0.22 | 0.28 | 0.37 | 0.48 | 0.64 | 2.38 |
| Per lipid basis (µg/g lipid) | |||||||||
| pp-DDT | 1392 | 1.41 | 0.11 | 0.56 | 0.78 | 1.10 | 1.73 | 2.57 | 12.82 |
| pp-DDE | 1392 | 3.75 | 0.50 | 1.58 | 2.09 | 3.00 | 4.49 | 6.86 | 25.31 |
| DDT:DDE | 1392 | 0.41 | 0.05 | 0.22 | 0.28 | 0.37 | 0.48 | 0.64 | 2.38 |
The combined sample includes the random, Bayley, and IQ subsamples.
Numbers for maternal concentrations on a per-lipid basis were smaller due to a missing lipid value.
Bayley Scales of Infant Development and IQ
In our study overall (random, Psychomotor Development Index, and IQ samples), scores on the Bayley Mental Development Index averaged 86.5 (standard deviation= 15.0) and scores on the Bayley Psychomotor Development Index averaged 87.4 (standard deviation= 18.2). Wechsler Intelligence Scale for Children IQ scores at 7 years averaged 96.2, with a standard deviation of 16.6. The standard deviation of scores was larger in the overall study than in the random sample only, owing to oversampling of low and high scores. Bayley Mental Development Index and Bayley Psychomotor Development Index scores were correlated with each other (Pearson r=0.59), but only weakly correlated with IQ scores at 7 years (Pearson r=0.17 and 0.18, respectively).
Organochlorine concentrations, child IQ, and potential confounding variables
Table 3 presents bivariate associations of potential confounding variables with organochlorine concentrations and with child IQ. Wet-weight DDT and DDE concentrations were highest among women who were African American, had a low socioeconomic index, had less than a high school education, and did not breastfeed for more than 1 day. These patterns of association are consistent with predictors of DDT and DDE exposure in another cohort of women assessed during a similar time period.20 Younger, non-white, multiparous, and currently smoking women were more likely to have children with lower IQ scores at age 7 years, as were women who did not complete high school or who had a low socioeconomic index. Women with higher concentrations of total cholesterol and triglycerides were more likely to have higher wet-weight DDT and DDE concentrations, as well as to have children with higher IQ scores (Table 3). Based on these results, we chose to adjust for socioeconomic index (continuous), parity (categorical), maternal smoking at registration (yes/no), breastfeeding status (breastfed more than 1 day, yes/no), and maternal lipid concentration, in addition to study center, maternal race, education, and age in secondary models.
Table 3.
Relation of maternal, child, and family characteristics to maternal DDT and DDE concentrations and to child IQ at age 7 years (n=1041).a
| DDT µg/liter | DDE µg/liter | 7-year IQ | |||||
|---|---|---|---|---|---|---|---|
| Characteristic | No.b | Mean | (95% CI) | Mean | (95% CI) | Mean | (95% CI) |
| Mothers and families | |||||||
| Mother’s age at child’s birth (years) | |||||||
| <18 | 83 | 12.5 | (10.9 to 14.2) | 30.6 | (26.4 to 34.9) | 88.9 | (85.3 to 92.5) |
| 18–24.9 | 526 | 10.9 | (10.2 to 11.6) | 29.7 | (28.0 to 31.4) | 96.5 | (95.1 to 98.0) |
| 25–29.9 | 213 | 11.5 | (10.4 to 12.5) | 30.9 | (28.2 to 33.6) | 95.3 | (93.0 to 97.6) |
| 30+ | 219 | 11.6 | (10.5 to 12.6) | 29.5 | (26.9 to 32.1) | 97.1 | (94.8 to 99.3) |
| Race | |||||||
| African American | 508 | 14.5 | (13.9 to 15.1) | 36.4 | (34.7 to 38.0) | 87.4 | (86.2 to 88.7) |
| White | 498 | 7.9 | (7.2 to 8.5) | 23.2 | (21.5 to 24.8) | 105.2 | (104.0 to 106.5) |
| Other | 35 | 11.9 | (9.6 to 14.1) | 32.8 | (26.8 to 38.8) | 85.2 | (80.5 to 89.9) |
| Previous pregnancies | |||||||
| 0 | 311 | 10.4 | (9.6 to 11.3) | 30.6 | (28.4 to 32.8) | 97.9 | (96.0 to 99.8) |
| 1 | 240 | 11.0 | (10.0 to 12.0) | 30.0 | (27.4 to 32.5) | 97.5 | (95.4 to 99.7) |
| 2+ | 489 | 12.0 | (11.3 to 12.6) | 29.7 | (27.9 to 31.4) | 93.6 | (92.1 to 95.1) |
| Total cholesterol (µg/L) | |||||||
| <200 | 264 | 10.8 | (9.9 to 11.7) | 27.6 | (25.3 to 30.0) | 91.3 | (89.3 to 93.3) |
| 200–299 | 613 | 11.2 | (10.5 to 11.8) | 29.4 | (27.9 to 31.0) | 96.9 | (95.6 to 98.3) |
| 300+ | 163 | 12.5 | (11.3 to 13.7) | 35.7 | (32.7 to 38.7) | 98.9 | (96.3 to 101.5) |
| Triglyceride (µg/L) | |||||||
| <150 | 227 | 10.2 | (9.3 to 11.2) | 27.9 | (25.3 to 30.4) | 94.1 | (91.8 to 96.3) |
| 150–249 | 547 | 11.4 | (10.8 to 12.1) | 30.0 | (28.4 to 31.7) | 95.7 | (94.2 to 97.1) |
| 250+ | 266 | 11.8 | (10.9 to 12.7) | 31.4 | (29.1 to 33.8) | 97.5 | (95.4 to 99.6) |
| Current smoker | |||||||
| Yes | 466 | 10.8 | (10.1 to 11.5) | 30.3 | (28.5 to 32.1) | 95.5 | (93.9 to 97.0) |
| No | 570 | 11.6 | (11.0 to 12.2) | 29.7 | (28.1 to 31.3) | 96.1 | (94.7 to 97.5) |
| Socioeconomic indexc | |||||||
| <4 | 366 | 13.6 | (12.8 to 14.4) | 33.4 | (31.4 to 35.4) | 87.5 | (86.0 to 89.1) |
| 4–5.9 | 342 | 11.0 | (10.2 to 11.8) | 29.4 | (27.3 to 31.5) | 94.4 | (92.8 to 96.1) |
| 6+ | 333 | 9.0 | (8.2 to 9.8) | 26.7 | (24.6 to 28.9) | 106.4 | (104.7 to 108.0) |
| Education (years) | |||||||
| < high school | 577 | 12.5 | (11.9 to 13.1) | 31.0 | (29.4 to 32.6) | 89.8 | (88.5 to 91.0) |
| high school | 314 | 10.1 | (9.2 to 10.9) | 29.1 | (26.8 to 31.3) | 98.8 | (97.1 to 100.5) |
| > high school | 150 | 9.0 | (7.8 to 10.2) | 28.1 | (25.0 to 31.2 | 112.3 | (109.9 to 114.7) |
| Child | |||||||
| Sex | |||||||
| Boy | 528 | 11.2 | (10.6 to 11.9) | 30.2 | (28.6 to 31.9) | 96.4 | (94.9 to 97.8) |
| Girl | 513 | 11.3 | (10.7 to 12.0) | 29.7 | (28.0 to 31.4) | 95.1 | (93.7 to 96.6) |
| Breastfed > 1 day | |||||||
| Yes | 161 | 8.3 | (7.1 to 9.5) | 24.6 | (21.6 to 27.6) | 109.7 | (107.2 to 112.2) |
| No | 798 | 13.7 | (12.1 to 15.3) | 32.4 | (28.2 to 36.6) | 93.4 | (90.0 to 96.8) |
Means and 95% confidence intervals estimated with sampling weights.
Data missing for some characteristics for some study participants.
See Methods section of text for definition of socioeconomic index.
CI indicates confidence interval.
Maternal DDT and DDE concentrations and Bayley Scales of Infant Development
The findings for DDT and DDE in relation to Bayley Scales of Infant Development scores were not suggestive of an association (Table 4), despite a fair degree of precision. Adjusted means for both the Mental Development Index and Psychomotor Development Index showed a lack of dose-response in relation to maternal DDT concentration, as indicated by the summary beta coefficients for the Mental Development Index of 0.1 (95% CI= −0.5 to 0.8), and for the Psychomotor Development Index of 0.7 (−0.1 to 1.5). Results for DDE also showed a lack of dose-response, and estimates of linear trend were not different from 0. Adjustment for maternal cholesterol and triglyceride concentrations did not materially affect our results, nor did the addition of the potential confounders selected empirically (Table 4). Results using lipid-standardized DDT and DDE concentrations were similarly null (data not shown). Regression estimates of the DDT/DDE and Bayley Scales of Infant Development association using the imputed dataset were not appreciably different from those estimated based on the “complete case” dataset (Table 4).
Table 4.
Adjusted associations between categorical and continuous measures of maternal serum DDT/DDE concentrations and cognitive development in the Collaborative Perinatal Project.
| 8-Month Bayley Mental Development Index |
8-Month Bayley Psychomotor Development Index |
7-Year Wechsler Intelligence Scale for Children IQ |
|||||||
|---|---|---|---|---|---|---|---|---|---|
| No. | Mean | (95% CI) | No. | Mean | (95% CI) | No. | Mean | (95% CI) | |
| DDT (µg/L) | |||||||||
| Categoricala | |||||||||
| <5a | 148 | 85.7 | (83.0 to 88.5) | 149 | 85.9 | (82.5 to 89.3) | 141 | 91.6 | (88.9 to 94.3) |
| 5– 9.9 | 516 | 86.9 | (85.2 to 88.7) | 517 | 88.4 | (86.2 to 90.7) | 449 | 92.9 | (90.9 to 94.9) |
| 10– 14.9 | 241 | 87.2 | (85.0 to 89.5) | 241 | 87.7 | (84.9 to 90.5) | 226 | 93.5 | (91.2 to 95.7) |
| 15– 19.9 | 126 | 86.5 | (83.6 to 89.5) | 126 | 88.7 | (85.0 to 92.3) | 120 | 94.1 | (91.2 to 97.0) |
| 20+ | 109 | 85.0 | (81.8 to 88.1) | 109 | 88.2 | (84.2 to 92.1) | 105 | 93.0 | (89.9 to 96.0) |
| βa (per 5 µg/L DDT) | 1140 | 0.1 | (−0.5 to 0.8) | 1142 | 0.7 | (−0.1 to 1.5) | 1041 | 0.3 | (−0.3 to 0.9) |
| βb (per 5 µg/L DDT) | 1139 | 0.1 | (−0.6 to 0.7) | 1141 | 0.7 | (−0.2 to 1.5) | 1040 | 0.2 | (−0.4 to 0.8) |
| βc (per 5 µg/L DDT) | 1052 | 0.0 | (−0.7 to 0.7) | 1053 | 0.5 | (−0.4 to 1.3) | 947 | 0.5 | (−0.2 to 1.1) |
| βd (per 5 µg/L DDT) | 1450 | −0.1 | (−0.8 to 0.6) | 1450 | 0.6 | (−0.2 to 1.4) | 1453 | 0.0 | (−0.6 to 0.6) |
| DDE (µg/L) | |||||||||
| Categoricala | |||||||||
| <15a | 209 | 85.2 | (82.8 to 87.6) | 210 | 84.7 | (81.7 to 87.7) | 195 | 91.7 | (89.3 to 94.2) |
| 15– 29.9 | 514 | 87.4 | (85.6 to 89.1) | 515 | 89.0 | (86.8 to 91.3) | 468 | 94.0 | (92.1 to 95.9) |
| 30– 44.9 | 247 | 86.4 | (84.2 to 88.6) | 247 | 88.3 | (85.6 to 91.1) | 216 | 92.0 | (89.7 to 94.3) |
| 45– 59.9 | 103 | 85.8 | (82.6 to 88.9) | 103 | 86.7 | (82.8 to 90.6) | 88 | 95.5 | (92.3 to 98.6) |
| 60+ | 67 | 87.1 | (83.3 to 90.8) | 67 | 88.9 | (84.1 to 93.6) | 74 | 90.5 | (87.1 to 93.9) |
| βa (per 15 µg/L DDE) | 1140 | 0.2 | (−0.6 to 0.9) | 1142 | 0.7 | (−0.2 to 1.6) | 1041 | −0.3 | (−0.9 to 0.4) |
| βb (per 15 µg/L DDE) | 1139 | 0.1 | (−0.7 to 0.9) | 1141 | 0.6 | (−0.4 to 1.5) | 1040 | −0.4 | (−1.1 to 0.3) |
| βc (per 15 µg/L DDE) | 1052 | 0.0 | (−0.8 to 0.7) | 1053 | 0.2 | (−0.7 to 1.2) | 947 | 0.0 | (−0.7 to 0.7) |
| βd (per 15 µg/L DDE) | 1450 | 0.0 | (−0.7 to 0.8) | 1450 | 0.6 | (−0.4 to 1.6) | 1453 | −0.4 | (−1.1 to 0.3) |
Adjusted for categorical variables study center and maternal race, and continuous variables maternal age and education in years (primary model).
Primary model + serum cholesterol and serum triglyceride concentration.
Primary model + serum cholesterol, serum triglyceride concentration, socioeconomic index (continuous), parity (categorical) to maternal smoking at registration (yes/no), breastfeeding status (breastfed more than 1 day, yes/no)
Coefficient for primary model based on imputed dataset.
Maternal DDT and DDE concentrations and child IQ
We found no evidence of an association between maternal serum DDT and DDE concentrations and 7-year child IQ (Table 4). Specifically, the estimated change in IQ for a 5 µg/L increase in DDT was 0.3 points (95% CI= −0.3 to 0.9), and for a 15 µg/L increase in DDE, −0.3 points (−0.9 to 0.4). Results were essentially unchanged after additional adjustment for maternal serum cholesterol and triglyceride concentrations and after the addition of socioeconomic index, parity, maternal smoking, and breastfeeding status (Table 4). When DDT/DDE concentrations were expressed on a per-lipid basis, results were similarly null (data not shown). Estimates of the association between DDT/DDE and 7-year IQ following multiple imputation were also not different from 0 (Table 4).
Nonlinear analyses
Results of the semiparametric spline models confirmed the results of the categorical and linear trend analyses, indicating a lack of dose-response regardless of exposure or outcome (Figure 1). There was no evidence of nonlinearity.
Figure 1.
Semiparametric spline functions, with 2 degrees of freedom, illustrating the relation between maternal serum DDT and DDE concentrations and (A–B) Mental Development Index (MDI) and (C–D) Psychomotor Development Index (PDI) scores on the Bayley Scales of Infant Development (BSID) and (E–F) the Wechsler Intelligence Scale for Children (WISC), adjusted for the categorical variables of study center and maternal race and for the continuous variables of maternal age and education, in years (n≈1100).
Discussion
We observed no association between either DDT or DDE measured in maternal serum during the third trimester of pregnancy and child development scores (Bayley Scales of Infant Development at 8 months and IQ measured at 7 years of age).
Findings from previous epidemiologic studies of DDT and cognitive development vary greatly, and potential explanations for heterogeneity in results across studies are not readily apparent. Study populations with the highest maternal DDE concentrations and the largest number of participants show null results, while study populations with lower concentrations and fewer participants tend to report adverse or mixed results with regard to Bayley Scales of Infant Development and IQ outcomes (Figure 2). In our large sample (n=1142), we found no association between maternal DDE concentration and Mental Development Index or Psychomotor Development Index at 8 months, and no association with child IQ at age 7 years. Similarly the North Carolina Infant Feeding Study (n=802) found no association between cord DDE concentrations and scores on the Bayley Scales of Infant Development assessed at 6, 12, 18, or 24 months,21,22 or McCarthy Scales of Children’s Abilities at ages 3, 4, or 5 years.23 Median maternal DDE concentrations in both studies were high relative to most other studies (24.5 ng/ml in the Collaborative Perinatal Project and 12.6 ng/ml in the North Carolina Infant Feeding Study). A smaller (n=330) study of Mexican-American families living in the Salinas Valley (the CHAMACOS cohort) also had relatively high maternal DDE concentrations (median DDE, 12.8 ng/ml), and results for DDE were generally null: maternal concentrations were associated with poorer performance on the Bayley Psychomotor Development Index at 6 months but not at 12 or 24 months of age. Additionally, maternal DDE concentrations were not associated with Bayley Mental Development Index at 6, 12, or 24 months of age.2
Figure 2.
Selected characteristics of studies on DDE/DDT and Bayley Scales of Infant Development and IQ. For studies with outcomes reported at multiple time points, we report largest number of subjects. To compute the median, studies reporting cord serum or lipid-adjusted maternal concentrations were converted to maternal serum wet-weight equivalent using conversion factors reported by Needham and colleagues.28 For some studies, we used the geometric mean as an estimate of the median. IQ outcomes included the Wechsler Intelligence Scale for Children and the McCarthy Scales of Children’s Abilities.
Fewer studies have evaluated maternal DDT concentrations in relation to Bayley Scales of Infant Development or IQ outcomes (Figure 2). In our study of archived serum specimens, the median maternal DDT concentration was 8.9 ng/ml. Among the two other studies evaluating similar outcomes (Figure 2), concentrations were lower by more than an order of magnitude, and sample sizes were considerably less, yet adverse associations between DDT and neurodevelopment were observed. In Menorca, cord DDT concentrations were associated with poorer performance on the General Cognitive, Memory, and Verbal areas of the McCarthy Scales of Children’s Abilities,4 and in the CHAMACOS cohort, higher maternal DDT concentrations were associated with lower scores on the Bayley Mental Development Index at 12 and 24 months (but not at 6 months), and lower scores on the Bayley Psychomotor Development Index at 6 and 12 months, but not at 24 months of age.2
One possible explanation for the lack of adverse associations among studies with high DDT and DDE concentrations relative to other studies is that the association between DDT and DDE and neurodevelopment is nonlinear. Adverse associations might be more pronounced (or only observable) at lower concentrations of these organochlorines. Semiparametric modeling of the dose-response relation in the Collaborative Perinatal Project does not appear to support this hypothesis. However, we had relatively limited power to detect associations at very low concentrations.
The summary of studies shown in Figure 2 also suggests that contemporary studies are more likely to observe an adverse or mixed association for DDT/DDE and cognitive development, compared with studies based on samples collected decades ago. If we assume that in-utero exposure to increased concentrations of DDT/DDE does not cause adverse cognitive development in the child, DDT measures in contemporary studies may be serving as surrogate biomarkers for other lipophilic compounds that may be neurotoxic, such as polybrominated diphenyl ethers.24
The limitations of the present study merit discussion. Due to loss to follow-up, 16% of Bayley Scales of Infant Development and 21% of IQ data were missing, and we assumed these data to be missing at random (MAR). With this assumption and the relatively low proportion of missing data, we assumed that the degree of selection bias would be minimal, a conclusion supported by simulation.25 Nevertheless, we used multiple imputation to impute missing outcome data at 8 months and 7 years of age. Imputed results were comparable to the complete-case results, and inferences were not affected. An additional limitation is the use of an omnibus test of intelligence in the Collaborative Perinatal Project, rather than domain-specific tests of neurobehavioral function at age 7 years. If adverse associations are present only for more specific aspects of neurocognitive functioning, our study would not have been able to detect them.
In addition, several forms of exposure misclassification may have biased our results towards the null. Although we reported a higher between-batch coefficient of variation for DDE (22.1%) and DDT (19.0%) compared with contemporary studies of DDT/DDE concentrations, our measures of variation are consistent with another study using serum specimens collected during the 1950’s, which estimated within-batch coefficients of variation of 26.9% for DDE and 22.6 for DDT.26 Furthermore, error in measurement of DDT and DDE concentrations should be considered in the context of the large between-subject variability in DDT and DDE concentrations, which, in our population, was 69% for DDT and 66% for DDE.
A second but related issue concerns the use of maternal serum as a biomarker for in-utero exposure. For instance, if third-trimester maternal serum concentrations of DDT or DDE are less optimal biomarkers of exposure than cord serum, maternal milk, or samples collected earlier in pregnancy, this would also attenuate our measures of association. However, the degree of attenuation is likely to be small because maternal serum concentrations of DDE are strongly (and positively) associated across trimesters of pregnancy in the Collaborative Perinatal Project (Pearson r=0.82 or above),27 and maternal serum DDE samples taken around delivery are strongly related to cord serum and maternal milk specimen concentrations.28,29
Finally, although DDE is considered to be relatively resistant to degradation in a variety of situations,30 few data exist concerning the impact of long-term frozen storage on DDT and DDE concentrations in human serum. To the extent that degradation of organochlorine concentrations occurred in the archived specimens used in the Collaborative Perinatal Project, the effect of degradation would likely bias our results towards the null. However, the degree of potential misclassification due to degradation is likely minimal, as some studies suggest that DDE (at least) resists degradation in frozen storage. For instance, breastmilk samples stored at −20 degrees C in 1972 showed no evidence of degradation when DDE concentrations were analyzed after 15 and 25 years of storage.31,32 Similarly, lipid concentrations appear to be unaffected by long-term frozen storage, and degradation is thus unlikely to have biased our estimates of association.33,34
Our study has several strengths. First, the Collabrative Perinatal Project resource allowed a well-powered study that included approximately 1100 families for whom extensive outcome and covariate data were available. Second, our study had enhanced statistical power because (1) we oversampled children at the tails of the Bayley Psychomotor Development Index and IQ distributions and (2) we had a greater range of maternal DDT and DDE concentrations compared with other studies of DDT/DDE and neurodevelopment. In addition, we were able to assess neurodevelopment at multiple points during development, in particular after school entry when IQ tests tend to have greater predictive validity for aspects of learning such as school achievement.35 Furthermore, measures of neurodevelopment underwent strict quality control in the Collaborative Perinatal Project, and were of high quality. Collaborative Perinatal Project investigators undertook several validity studies of the collected data: in one study of data collected during pregnancy and the first postnatal year, less than 0.1% of data were found to be erroneous.7
We examined maternal DDT and DDE concentrations in relation to neurodevelopment at several time points during infancy and childhood. Exposure to DDT in this population was substantial, as was the distribution of exposure, making it well suited to address the question of whether maternal exposure to DDT affects infant neurodevelopment. Results suggest that DDT does not have adverse effects on infant and child neurodevelopment.
Acknowledgments
Support for this project was provided by the Intramural Research Programs of the National Institute of Environmental Health Sciences and National Institute of Child Health and Human Development, NIH, and the Centers for Disease Control and Prevention, Department of Health and Human Services.
We acknowledge Xuguang Guo of Westat for SAS programming assistance and Susan Edelstein of Image Associates for creating the illustrations in this manuscript. We also thank Richard Canfield of Cornell University for his comments on an earlier draft of our manuscript.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.ATSDR. Toxicological Profile for DDT, DDE, DDD. Atlanta, GA: Agency for Toxic Substances and Disease Registry; 2002. [PubMed] [Google Scholar]
- 2.Eskenazi B, Marks AR, Bradman A, Fenster L, Johnson C, Barr DB, Jewell NP. In utero exposure to dichlorodiphenyltrichloroethane (DDT) and dichlorodiphenyldichloroethylene (DDE) and neurodevelopment among young Mexican American children. Pediatrics. 2006;118(1):233–241. doi: 10.1542/peds.2005-3117. [DOI] [PubMed] [Google Scholar]
- 3.Ribas-Fito N, Cardo E, Sala M, Eulalia de Muga M, Mazon C, Verdu A, Kogevinas M, Grimalt JO, Sunyer J. Breastfeeding, exposure to organochlorine compounds, and neurodevelopment in infants. Pediatrics. 2003;111(5 Pt 1):e580–e585. doi: 10.1542/peds.111.5.e580. [DOI] [PubMed] [Google Scholar]
- 4.Ribas-Fito N, Torrent M, Carrizo D, Munoz-Ortiz L, Julvez J, Grimalt JO, Sunyer J. In utero exposure to background concentrations of DDT and cognitive functioning among preschoolers. Am J Epidemiol. 2006;164(10):955–962. doi: 10.1093/aje/kwj299. [DOI] [PubMed] [Google Scholar]
- 5.Torres-Sanchez L, Rothenberg SJ, Schnaas L, Cebrian ME, Osorio E, Del Carmen Hernandez M, Garcia-Hernandez RM, Del Rio-Garcia C, Wolff MS, Lopez-Carrillo L. In utero p,p'-DDE exposure and infant neurodevelopment: a perinatal cohort in Mexico. Environ Health Perspect. 2007;115(3):435–439. doi: 10.1289/ehp.9566. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Torres-Sanchez L, Schnaas L, Cebrian ME, Hernandez Mdel C, Valencia EO, Garcia Hernandez RM, Lopez-Carrillo L. Prenatal dichlorodiphenyldichloroethylene (DDE) exposure and neurodevelopment: a follow-up from 12 to 30 months of age. Neurotoxicology. 2009;30(6):1162–1165. doi: 10.1016/j.neuro.2009.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Niswander KR, Gordon M. The Women and Their Pregnancies. Washington, DC: U.S. Government Printing Office, U.S. Department of Health, Education, and Welfare; 1972. [Google Scholar]
- 8.Longnecker MP, Klebanoff MA, Brock JW, Zhou H, Gray KA, Needham LL, Wilcox AJ. Maternal serum level of 1,1-dichloro-2,2-bis(p-chlorophenyl)ethylene and risk of cryptorchidism, hypospadias, and polythelia among male offspring. Am J Epidemiol. 2002;155(4):313–322. doi: 10.1093/aje/155.4.313. [DOI] [PubMed] [Google Scholar]
- 9.Longnecker MP, Bernstein L, Bird CL, Yancey AK, Peterson JC. Measurement of organochlorine levels in postprandial serum or in blood collected in serum separator tubes. Cancer Epidemiol Biomarkers Prev. 1996;5(9):753–755. [PubMed] [Google Scholar]
- 10.Bayley N. Manual for the Bayley Scales of Infant Development. New York: Psychological Corporation; 1969. [Google Scholar]
- 11.Broman SH, Nichols PL, Kennedy WA. Preschool IQ: Prenatal and Early Developmental Correlates. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1975. [Google Scholar]
- 12.Werner EE, Bayley N. The reliability of Bayley's Revised Scale of Mental and Motor Development during the first year of life. Child Development. 1966;37(1):39–50. [Google Scholar]
- 13.Broman S. The Collaborative Perinatal Project: An Overview. In: Mednick SA, Harway M, Finello KM, editors. Handbook of longitudinal research. Vol. 1. New York: Praeger; 1984. pp. 185–215. [Google Scholar]
- 14.Zhou H, Chen J, Rissanen TH, Korrick SA, Hu H, Salonen JT, Longnecker MP. Outcome-dependent sampling: an efficient sampling and inference procedure for studies with a continuous outcome. Epidemiology. 2007;18(4):461–468. doi: 10.1097/EDE.0b013e31806462d3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Brock JW, Burse VW, Ashley DL, Najam AR, Green VE, Korver MP, Powell MK, Hodge CC, Needham LL. An improved analysis for chlorinated pesticides and polychlorinated biphenyls (PCBs) in human and bovine sera using solid-phase extraction. J Anal Toxicol. 1996;20(7):528–536. doi: 10.1093/jat/20.7.528. [DOI] [PubMed] [Google Scholar]
- 16.Longnecker MP, Klebanoff MA, Zhou H, Brock JW. Association between maternal serum concentration of the DDT metabolite DDE and preterm and small-for-gestational-age babies at birth. Lancet. 2001;358(9276):110–114. doi: 10.1016/S0140-6736(01)05329-6. [DOI] [PubMed] [Google Scholar]
- 17.Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology. 1999;10(1):37–48. [PubMed] [Google Scholar]
- 18.Hernan MA, Hernandez-Diaz S, Werler MM, Mitchell AA. Causal knowledge as a prerequisite for confounding evaluation: an application to birth defects epidemiology. Am J Epidemiol. 2002;155(2):176–184. doi: 10.1093/aje/155.2.176. [DOI] [PubMed] [Google Scholar]
- 19.Horvitz DG, Thompson DJ. A generalization of sampling without replacement from a finite universe. J Amer Stat Assoc. 1952;47:663–685. [Google Scholar]
- 20.James RA, Hertz-Picciotto I, Willman E, Keller JA, Charles MJ. Determinants of serum polychlorinated biphenyls and organochlorine pesticides measured in women from the child health and development study cohort, 1963–1967. Environ Health Perspect. 2002;110(7):617–624. doi: 10.1289/ehp.02110617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Gladen BC, Rogan WJ, Hardy P, Thullen J, Tingelstad J, Tully M. Development after exposure to polychlorinated biphenyls and dichlorodiphenyl dichloroethene transplacentally and through human milk. J Pediatr. 1988;113(6):991–995. doi: 10.1016/s0022-3476(88)80569-9. [DOI] [PubMed] [Google Scholar]
- 22.Rogan WJ, Gladen BC. PCBs, DDE, and child development at 18 and 24 months. Ann Epidemiol. 1991;1(5):407–413. doi: 10.1016/1047-2797(91)90010-a. [DOI] [PubMed] [Google Scholar]
- 23.Gladen BC, Rogan WJ. Effects of perinatal polychlorinated biphenyls and dichlorodiphenyl dichloroethene on later development. J Pediatr. 1991;119(1 Pt 1):58–63. doi: 10.1016/s0022-3476(05)81039-x. [DOI] [PubMed] [Google Scholar]
- 24.Herbstman JB, Sjodin A, Kurzon M, Lederman SA, Jones RS, Rauh V, Needham LL, Tang D, Niedzwiecki M, Wang RY, Perera F. Prenatal exposure to PBDEs and neurodevelopment. Environ Health Perspect. 2010;118(5):712–719. doi: 10.1289/ehp.0901340. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Kristman VL, Manno M, Cote P. Methods to account for attrition in longitudinal data: do they work? A simulation study. Eur J Epidemiol. 2005;20(8):657–662. doi: 10.1007/s10654-005-7919-7. [DOI] [PubMed] [Google Scholar]
- 26.Jusko TA, Koepsell TD, Baker RJ, Greenfield TA, Willman EJ, Charles MJ, Teplin SW, Checkoway H, Hertz-Picciotto I. Maternal DDT exposures in relation to fetal and 5-year growth. Epidemiology. 2006;17(6):692–700. doi: 10.1097/01.ede.0000232226.06807.90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Longnecker MP, Klebanoff MA, Gladen BC, Berendes HW. Serial levels of serum organochlorines during pregnancy and postpartum. Arch Environ Health. 1999;54(2):110–114. doi: 10.1080/00039899909602244. [DOI] [PubMed] [Google Scholar]
- 28.Needham LL, Grandjean P, Heinzow B, Jorgensen PJ, Nielsen F, Patterson DG, Sjodin A, Turner WE, Weihe P. Partition of Environmental Chemicals between Maternal and Fetal Blood and Tissues. Environ Sci Technol. 2011;45(3):1121–1126. doi: 10.1021/es1019614. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Rogan WJ, Gladen BC, McKinney JD, Carreras N, Hardy P, Thullen J, Tingelstad J, Tully M. Polychlorinated biphenyls (PCBs) and dichlorodiphenyl dichloroethene (DDE) in human milk: effects of maternal factors and previous lactation. Am J Public Health. 1986;76(2):172–177. doi: 10.2105/ajph.76.2.172. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Quensen JF, 3rd, Mueller SA, Jain MK, Tiedje JM. Reductive dechlorination of DDE to DDMU in marine sediment microcosms. Science. 1998;280(5364):722–724. doi: 10.1126/science.280.5364.722. [DOI] [PubMed] [Google Scholar]
- 31.Lunden A, Noren K. Polychlorinated naphthalenes and other organochlorine contaminants in Swedish human milk, 1972–1992. Arch Environ Contam Toxicol. 1998;34(4):414–423. doi: 10.1007/s002449900338. [DOI] [PubMed] [Google Scholar]
- 32.Noren K. Changes in the levels of organochlorine pesticides, polychlorinated-biphenyls, dibenzo-para-dioxins and dibenzofurans in human-milk from Stockholm, 1972–1985. Chemosphere. 1988;17(1):39–49. [Google Scholar]
- 33.Kuchmak M, Taylor L, Olansky AS. Suitability of frozen and lyophilized reference sera for cholesterol and triglyceride determinations. Clin Chim Acta. 1982;120(2):261–271. doi: 10.1016/0009-8981(82)90163-2. [DOI] [PubMed] [Google Scholar]
- 34.Matthan NR, Ip B, Resteghini N, Ausman LM, Lichtenstein AH. Long-term fatty acid stability in human serum cholesteryl ester, triglyceride, and phospholipid fractions. J Lipid Res. 2010;51(9):2826–2832. doi: 10.1194/jlr.D007534. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Sattler JM. Assessment of children : cognitive foundations. 5th ed. San Diego: J.M. Sattler; 2008. [Google Scholar]