Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Sep 1.
Published in final edited form as: Environ Res. 2021 Nov 3;204(Pt C):112325. doi: 10.1016/j.envres.2021.112325

Time after a peak-pesticide use period and neurobehavior among ecuadorian children and adolescents: The ESPINA study

C Espinosa da Silva a, S Gahagan b, J Suarez-Torres c, D Lopez-Paredes c, H Checkoway a,d, JR Suarez-Lopez a,*
PMCID: PMC9138759  NIHMSID: NIHMS1802602  PMID: 34740618

Abstract

Background:

Limited evidence exists regarding transient neurobehavioral alterations associated with episodic pesticide exposures or agricultural pesticide spray periods. We previously observed that children examined soon after a pesticide spray period (the Mother’s Day flower harvest [MDH]) had lower neurobehavioral performance than children examined later. The present study builds on our previous work by incorporating longitudinal analyses from childhood through adolescence.

Methods:

We examined participants in agricultural communities in Ecuador (ESPINA study) during three periods: July–August 2008 (N = 313, 4-9-year-olds); April 2016 (N = 330, 11–17-year-olds); July–October 2016 (N = 535, 11–17-year-olds). Participants were examined primarily during a period of low floricultural production. Neurobehavior was assessed using the NEPSY-II (domains: Attention/Inhibitory Control, Language, Memory/Learning, Visuospatial Processing, and Social Perception). Linear regression and generalized linear mixed models were used to examine cross-sectional and longitudinal associations between examination date (days) after the MDH and neurobehavioral outcomes, adjusting for demographic, anthropometric, and socio-economic variables.

Results:

Participants were examined between 63 and 171 days after the MDH. Mean neurobehavioral domain scores ranged from 1.0 to 17.0 (SDrange = 2.1–3.1) in 2008 and 1.0 to 15.5 (SDrange = 2.0–2.3) in 2016. In cross-sectional analyses (2016 only; N = 523), we found significant or borderline positive associations between time after the MDH and Attention/Inhibitory Control (difference/10 days [β] = 0.22 points [95% CI = 0.03, 0.41]) and Language (β = 0.16 points [95% CI = −0.03, 0.34]). We also observed positive, longitudinal associations (2008–2016) with Attention/Inhibitory Control (β = 0.19 points [95% CI = 0.04, 0.34]) through 112 days after the harvest and Visuospatial Processing (β = 3.56, β-quadratic = −0.19 [95% CI: −0.29, −0.09]) through 92 days.

Conclusions:

Children examined sooner after the harvest had lower neurobehavioral performance compared to children examined later, suggesting that peak pesticide spray seasons may transiently affect neurobehavior followed by recovery during low pesticide-use periods. Reduction of pesticide exposure potential for children during peak pesticide-use periods is advised.

Keywords: Pesticides, Organophosphate, Neurobehavior, Ecuador, Children, Adolescents

1. Introduction

Agriculture is frequently associated with periods of heightened production and concomitant pesticide use. As a result, seasonal variations in pesticide use may be associated with increased potential for pesticide exposure among agricultural workers (Crane et al., 2013; Krenz et al., 2015; Peiris-John et al., 2005; Quandt et al., 2015; Singleton et al., 2015; Strelitz et al., 2014) and non-worker adults and children living near crops (Crane et al., 2013; Galea et al., 2015; Suarez-Lopez et al., 2017; Thompson et al., 2014).

A large body of evidence has described adverse neurotoxic effects of prenatal and childhood pesticide exposure, including motor and cognitive deficits, and lower attention, inhibitory control, memory among others (Hernandez et al., 2016; Kofman et al., 2006; Marks et al., 2010; Muñoz-Quezada et al., 2013; Suarez-Lopez et al., 2017). Insecticides, such as organophosphates, are neurotoxins which directly induce toxicity to neurons and glia and inhibit the metabolism of acetylcholine through inhibition of acetylcholinesterase (AChE) and butylcholinesterase (Abou-Donia, 2003; Aldridge et al., 2005; Qiao et al., 2003; Slotkin, 2004).

Limited human evidence exists regarding cyclical alterations on mental health processes associated with episodic pesticide exposures or agricultural pesticide spray periods. Experiments conducted with rats and zebrafish found that transient organophosphate insecticide exposure was associated with decreased neurobehavioral performance which improved over time upon removal of the exposure (Levin et al., 2003; Maurissen et al., 2000; Middlemore-Risher et al., 2010). These findings are comparable to results reported in four studies in humans. Studies in Egypt among adolescent agricultural workers found that seasonal pesticide exposures were associated with altered neurological symptoms and neurobehavior which then recovered many weeks after the end of the pesticide spray season (Khan et al., 2014; Rohlman et al., 2016). Improvement in visuomotor performance and short-term verbal memory over time has also been described among pesticide-intoxicated adults in Nicaragua (Delgado et al., 2004). In our previous work with the study of Secondary Exposure to Pesticides among Children and Adolescents (ESPINA: Estudio de la Exposición Secundaria a Plaguicidas en Niños y Adolescentes [Spanish]), we observed evidence of transient alterations in neurobehavioral performance associated with a peak pesticide spray period (the Mother’s Day flower harvest [MDH] in May) among children 4–9 years of age living in agricultural settings in Ecuador. We found that children examined sooner after the harvest had greater cholinesterase inhibitor exposures and lower performance on tests of attention, inhibitory control, visuospatial processing, and sensorimotor function compared to children examined later during a low flower production period (Suarez-Lopez et al., 2017). It was noted that differences in performance were apparent until ~84 days after the harvest, which coincides with the time needed for AChE activity to return to pre-exposure levels after inhibition by organophosphate pesticides (Mason, 2000).

The objective of this study was to evaluate whether the cyclical (temporal) alterations in neurobehavioral performance associated with pesticide spray seasons observed in childhood continued to be observed in adolescence among participants of the ESPINA study. Based on the epidemiological and experimental findings previously described, we hypothesized that participants examined earlier after the end of a peak pesticide spray season would have lower neurobehavioral scores compared to those examined later.

2. Methods

2.1. Study population, setting, and design

The ESPINA study aims to evaluate the effects of pesticide exposures on neurobehavioral development among children and adolescents living in Pedro Moncayo County, Pichincha province, Ecuador. Pedro Moncayo County has a prominent floricultural industry where flower plantations have reported using more than 20 different insecticides (including organophosphates, carbamates, neonicotinoids, and pyrethroids) and over 50 types of fungicides (Grandjean et al., 2006; Suarez-Lopez et al., 2018) that are applied using hand sprayers by agricultural workers.

ESPINA study data used in the present analyses were collected during three examination periods (July–August 2008, April 2016, and July- –October 2016) in schools across the five parishes in Pedro Moncayo County: Malchinguí, Tocachi, La Esperanza, Tabacundo, and Tupigachi. In 2008, we examined 313 boys and girls aged 4–9 years during a low pesticide-use period (July through August, Fig. 1). Most participants (73%) were recruited from the 2004 Survey of Access and Demand of Health Services in Pedro Moncayo County, which collected information of 18,187 residents (including complete anthropometric information of 922 children) and was representative of the local population of Pedro Moncayo County (Suarez-Lopez et al., 2012). The 2004 survey aimed to sample all residents of Pedro Moncayo County and used residential addresses and home visits to collect data on demographic, socio-economic, occupational and health information from all household members. In 2008, we were able to recontact 419 of the 922 children from the 2004 survey. Most losses were due to unlocatable residential addresses or because the child had moved to a different residence. A total of 124 new volunteers also registered to participate and were recruited through community announcements provided by community leaders, governing councils, and word-of-mouth. After a pre-survey of parents, children were eligible to participate if they met the following inclusion criteria: A) cohabitation with a floricultural worker for at least one year or B) never cohabited with an agricultural worker, never lived in a house where agricultural pesticides were stored, and had no previous direct contact with pesticides. Multiple children were allowed to participate per household. Two hundred and sixty-six (63%) children from the 2004 sample and 86 (69%) new volunteers were eligible, of whom 259 and 84 children agreed to participate in the study, respectively. Twenty-seven participants (of whom 47% cohabitated with a floricultural worker) did not arrive for their examination appointments, 1 child did not assent to be examined, and 2 children refused their examinations.

Fig. 1.

Fig. 1.

Open in a new tab

Participant flow chart. *311 participants examined in both examinations in 2016.

Compared to participants from the 2004 survey, ESPINA 2008 participants had comparable distributions of sex, but lower prevalence of stunting (24% vs. 37%) and lower prevalence of cohabitation with floricultural workers (55% vs. 63%). The latter was by design as the intention was to have a balanced distribution of children who lived with floricultural workers and children who did not live with agricultural workers (Suarez-Lopez et al., 2012). In 2016, we examined 554 boys and girls aged 11–17 years across different agricultural periods including April (n = 330) and July through October (n = 535, Fig. 1). Some participants had been examined in 2008 and received a follow-up examination (n = 238), while others were new participants (n = 316). New participants were recruited using the home update and presurvey in 2016 in preparation for the 2016–2017 Pedro Moncayo County Community Survey (formerly known as the Survey of Access and Demand of Health Services in Pedro Moncayo County). Participants examined in ESPINA 2016 did not have statistically significant differences in sex distribution, race, years of parental education or hemoglobin concentration compared to participants from the 2008 survey who were not examined in 2016; however, they had lower height-for-age z-scores (−1.5 vs. −1.2, p = 0.01). Additional details of participant recruitment in 2016 have been published previously (Suarez-Lopez et al., 2019).

To maximize the number of participants included in our analyses, we imputed missing data for parental education in 2008 (n = 17) and 2016 (n = 10), as well as residential distance to the nearest flower plantation in 2008 (n = 3). Among the children who were examined in 2008 but were missing parental education data in 2016 (n = 5), data for parental education were imputed using the 2008 data for maternal and paternal education. For participants with missing paternal and maternal education data in 2008 (n = 17) and missing parental education in 2016 (n = 5), a random imputation was conducted for each variable based on a normal distribution of the variable during the respective examination period. Among participants missing residential distance to the nearest flower plantation in 2008 (n = 3), an imputation was conducted using a random selection of values generated from a random normal distribution based on the ESPINA mean and standard deviations values. Since a small number of participants reported being White (n = 4) or Black (n = 2), we grouped these 6 participants in the mestizo (mix of White and Indigenous) category to improve model stability when adjusting for race. The present analyses include 309 observations from July–August 2008 (3 observations were excluded due to missing neurobehavioral data and 1 due to missing covariate data, Fig. 1), 319 observations from April 2016 (7 observations excluded due to missing neurobehavioral data and 4 due to missing covariate data), and 523 observations from July–October 2016 (8 observations excluded due to missing neurobehavioral data and 4 due to missing covariate data).

We collected informed consent from adult participants (aged 18 years or older) and parents, as well as parental permission of participation and informed assent of child participants. This study was approved by the Institutional Review Boards of the University of California San Diego, the University of Minnesota, Universidad San Francisco de Quito, and the Ministry of Public Health of Ecuador.

2.2. Examination date after the Mother’s day flower harvest

In the floricultural industry in Pedro Moncayo County, flower production and concomitant pesticide use fluctuate according to the demand for flowers for certain holidays (i.e., Thanksgiving, Christmas, Valentine’s Day, and Mother’s Day). This results in heightened flower production and pesticide-use periods from October to April followed by periods of low flower production and pesticide-use from May to September. To assess seasonal effects on the outcomes of interest, we used time in days after the end of the MDH in which participants were assessed. Among participants examined in 2008 of the ESPINA study, we previously observed that time after the MDH was an important construct of pesticide exposure as it was positively associated with AChE activity, particularly among children living near floricultural crops (Suarez-Lopez et al., 2018). These findings provided compelling information that the pesticide exposure levels of children examined sooner after the harvest were greater than those of children examined later during the low production period. The main exposure construct of this study is time after the end of the MDH, which was calculated by subtracting the approximate end date of the MDH (i.e., 00:00am on May 8, 2008 and 00:00am on May 5, 2016) from the date and start time of the participant’s examination.

2.3. Neurobehavioral assessments

Neurobehavioral performance was measured using the NEPSY-II test (NCS Pearson, San Antonio, TX) (Kemp and Korkman, 2010). Neurobehavioral assessments were conducted in seven schools in Pedro Moncayo County. During the examinations of July–August 2008 and July–October 2016, trained psychologists blinded to participant exposure status assessed participants in 13 subtests across five domains: 1) Attention and Inhibitory Control (also known as Attention and Executive Functioning, subtests: statue [assessed in 2008 only], auditory attention & response set, inhibition); 2) Language (subtests: comprehension of instructions, speeded naming); 3) Memory and Learning (subtests: narrative memory [assessed in 2008 only], immediate and delayed memory for faces); 4) Visuospatial Processing (subtests: design copying, geometric puzzles); and 5) Social Perception (only assessed in 2016, subtest: affect recognition). Two subtests required translation into Spanish using terminology appropriate for the local population (auditory attention and response set and comprehension of instructions). The translation was approved by NCS Pearson. Participants were examined alone and in a quiet room by the examiner. Participants examined in April 2016 were only assessed for Attention and Inhibitory Control.

We used the NEPSY scaled scores for each subtest to assess performance, which are age-standardized values based on a national sample of children in the United States (Korkman et al., 2007). Scaled scores for the NEPSY subtest were calculated using the NEPSY-II scoring assistant software (NCS Pearson, Inc., San Antonio, TX) and higher scores indicate better performance. A detailed description of subtest scoring has been published elsewhere (Kemp and Korkman, 2010; Suarez-Lopez et al., 2013, 2017). Domain scores were used as measures of neurobehavioral performance and were calculated by averaging one primary scaled score from all subtests within each domain. For subtests that included either correct and error components (i.e., auditory attention and response set) or time and error components (i.e., inhibition, speeded naming, visuomotor precision), the combined scaled scores representing the combination of both components were used as primary scaled scores. Affect recognition was the only subtest in the Social Perception domain and, as such, the Social Perception domain is equivalent to the affect recognition scaled score. In this analysis, the sample sizes varied across domains because the NEPSY-II subtests were developed for different age ranges and were implemented only among participants in the appropriate age ranges.

2.4. Other measures

Parental education was calculated as the mean number of years of education from both parents. Participant weight was measured using a digital scale (Tanita model 0108 MC; Corporation of America, Arlington Heights, IL, USA) and height was measured to the nearest 1 mm using a height board, based on the World Health Organization’s (WHO) recommended procedures (World Health Organization, 2008). Height-for-age z-scores were used to estimate long-term nutritional status and were calculated based on the WHO’s normative sample (WHO Multicentre Growth Reference Study Group, 2006). Erythrocytic AChE activity and hemoglobin concentration were measured from finger-stick blood samples using EQM Test-mate ChE Cholinesterase Test System 400 (EQM AChE Erythrocyte Cholinesterase Assay kit 470; EQM Research, Inc, Cincinnati, OH) (EQM Research Inc., 2003). Blood samples were analyzed immediately upon sample collection by study examiners, and within recommended ambient temperatures. Residential geographic coordinates were obtained using portable global positioning system receivers collected for the Local and Community Information System (SILC, Sistema de Información Local y Comunitario [Spanish]), and geospatial information of greenhouse floriculture was generated using satellite imagery. Residential distance to the nearest flower plantation perimeter was calculated using ArcGIS (Esri, Redlands, CA).

2.5. Statistical analysis

Participant characteristics.

Descriptive sample characteristics were calculated using means and standard deviations for continuous variables and frequency distributions for categorical variables, stratified by: (a) exam year and (b) categories of time after the MDH. Student’s T-Tests were used to examine statistically significant mean differences between strata of time and participant characteristics. Linear regression was used to examine trends between participant characteristics across time.

Cross-sectional associations between neurobehavior and time after the MDH.

Associations between examination days after the harvest and neurobehavior were assessed among participants examined between July and October 2016 (Nobservations [Nobs] = 523) using linear regression models. A minimally adjusted model included variables that were identified a priori as potential confounders of the main associations of analysis: age, sex, race, and height-for-age z-score. Height-for-age is an indicator for chronic nutritional status and has been found to be independently associated with neurodevelopment in Ecuadorian children (Grandjean et al., 2006). The minimally adjusted model also controlled for examination in October 2016, as this marks the beginning of a peak pesticide-use period and may result in altered neurobehavioral performance. A fully adjusted model further controlled for parental education and hemoglobin concentration because they both resulted in a 10% change in the estimate compared to the estimate from the minimally adjusted model. Parental education is a construct of socioeconomic status and has been found to be a predictor of neurobehavioral performance in children (Brooks et al., 2010). Hemoglobin concentration has also been shown to affect neurobehavioral performance (Kofman et al., 2006; Suarez-Lopez et al., 2013) via the effect of variations in hematocrit levels on the suppression of AChE activity (EQM Research Inc., 2003). To control for learning effects for subtests assessed more than once in a same year (auditory attention & response set, and inhibition were assessed in April and July–October 2016), we created an indicator variable for examination in April 2016 and July–October 2016, and included this variable in the adjusted regression models assessing the domain of Attention and Inhibitory Control. In both adjusted models, the quadratic term for time after the end of the MDH (time + time*time) was examined to evaluate a potential curvilinear relationship between time after a peak pesticide-use period and neurobehavior.

Longitudinal associations between neurobehavior and time after the MDH.

We examined the longitudinal associations between time in days after the end of the MDH and neurobehavioral outcomes (four domains: Attention and Inhibitory Control, Language, Memory and Learning, and Visuospatial Processing; Social Perception was not examined because it was not assessed in 2008) in a pooled sample of participants assessed after peak pesticide spray periods in 2008 and 2016 (Nobs~832; sample sizes vary by domain). We also examined the association between time in days after the end of the MDH and Attention and Inhibitory Control among the participants who were examined both in April and July- –October 2016 (Nobs = 606). Participants examined in April were assigned negative time values since the end of the MDH occurs in May.

Repeated measures regression (generalized linear mixed models) with a compound symmetry correlation matrix were used to test the longitudinal associations between time in days after the end of the MDH and neurobehavioral outcomes. Linear regression models were conducted using the same minimally and fully adjusted models described above. In both models, the quadratic term for time after the end of the MDH was examined to evaluate a potential curvilinear relationship between time after a peak pesticide-use period and neurobehavior. The adjusted longitudinal associations between time after the end of the MDH and neurobehavioral outcomes from the pooled sample of participants assessed after peak pesticide spray periods in 2008 and 2016 (Nobs~832) were visualized by plotting the adjusted least squares means of neurobehavioral performance across sextiles of time after the MDH.

We assessed multiplicative effect measure modification by age, sex, flower worker cohabitation, and residential distance to the nearest flower plantation in all analyses. We also assessed multiplicative effect measure modification by ESPINA examination year in the longitudinal analyses. We conducted sensitivity analyses for all regression models excluding participants with imputed values (N2008 = 20, N2016 = 10) to assess whether the findings were comparable to those of the full sample including imputed values.

All analyses were conducted using SAS Version 9.4 (SAS Institute Inc., Cary, NC).

3. Results

3.1. Participant characteristics

In the pooled sample of participants assessed after peak pesticide-use periods in 2008 and 2016 (nobs = 832), on average, participants were examined between 63 and 171 days after the end of the MDH (mean = 96 days, standard deviation [SD] = 18.2), were 11.5 years old (SD = 4.2), and had a height-for-age z-score of −1.4 (SD = 0.9). Mean neurobehavioral domain scores for Attention and Inhibitory Control, Language, Memory and Learning, Visuospatial Processing in this pooled sample were 8.4 (SD = 2.3), 6.9 (SD = 2.2), 8.5 (SD = 2.2), and 9.0 (SD = 2.6), respectively. Social Perception was only assessed in the 2016 sample, among whom the mean score was 8.2 (SD = 2.3).

Among participants assessed in 2008, children examined sooner after the MDH were younger and had lower AChE activity, hemoglobin concentrations, and Visuospatial Processing scores (p < 0.01) compared to those examined later. Mean neurobehavioral domain scores ranged from 1.0 to 17.0 (SDrange = 2.1 to 3.1). Among participants assessed in 2016, adolescents examined sooner after the MDH had lower height-for-age z-scores (p < 0.01), and higher AChE activity (p < 0.01) and hemoglobin concentrations compared to those examined later (p = 0.02). Mean neurobehavioral domain scores in 2016 ranged from 1.0 to 15.5 (SDrange = 2.0 to 2.3).

Participants examined in 2008 (n = 309) were seen, on average, 85.0 days after the end of the MDH (SD = 10.8) and had a mean age of 6.6 years, while participants examined in 2016 (n = 535) were examined, on average, 102.5 days after the end of the MDH (SD = 18.5) and had a mean age of 14.5 years (Table 1). Participants sampled in 2008 had lower height-for-age z-scores, lower hemoglobin concentrations and lower AChE activity compared to those examined in 2016 (all p-values <0.01). Mean Attention and Inhibitory Control scores did not differ significantly between participants examined in 2008 compared to those examined in 2016 (8.5 vs. 8.4, p = 0.33). However, participants examined in 2008 had lower NEPSY-II domain scores for Language (6.6 vs. 7.0, p < 0.01) but higher scores for Memory and Learning (8.8 vs. 8.3, p < 0.01) and Visuospatial Processing (9.6 vs. 8.7, p < 0.01) compared to those examined in 2016.

Table 1.

Participant characteristics by examination year and by tertiles of time after the end of the Mother’s Day harvest.

Characteristic
 Total
 Pdiff 2008 vs. 2016 Tertiles of time after the harvesta
 p trend
Range (days) 63.4–171.4 63.4–89.5 89.6–98.5 98.6–171.4
n2008(Jul-Aug) 309 176 112 21
n2016 (Jul-Oct) 523 102 168 253
npooled obs 832 278 280 274
Age (years)
 2008 6.6 (1.6) <0.01 6.4 (1.4) 6.9 (1.7) 6.9 (1.6) 0.01
 2016 14.5 (1.8) 14.3 (1.6) 14.6 (1.7) 14.4 (1.8) 0.44
Sex (male)
 2008 51% 0.51 50% 53% 52% 0.89
 2016 49% 58% 47% 46% 0.45
Race (mestizo or white)
 2008 78% 0.12 73% 81% 100% 0.94
 2016 78% 74% 63% 90% <0.01
Parental education (years)
 2008 7.4 (3.7) <0.01 7.5 (4.0) 7.2 (3.2) 7.5 (3.4) 0.81
 2016 8.1 (3.5) 8.0 (3.5) 7.3 (3.4) 8.6 (3.4) 0.01
Height-for-age z-score
 2008 −1.2 (1.0) <0.01 −1.3 (0.9) −1.3 (1.0) −0.7 (1.0) 0.99
 2016 −1.5 (0.9) −1.6 (0.9) −1.6 (0.8) −1.4 (0.9) <0.01
Hemoglobin Concentration (g/dL)
 2008 12.6 (1.2) <0.01 12.5 (1.2) 12.8 (1.1) 12.9 (1.0) <0.01
 2016 13.0 (1.2) 13.3 (1.2) 12.9 (1.2) 12.8 (1.1) 0.02
Acetylcholinesterase Activity (U/mL)
 2008 3.1 (0.5) <0.01 3.1 (0.5) 3.2 (0.5) 3.4 (0.5) <0.01
 2016 3.7 (0.5) 3.8 (0.5) 3.7 (0.5) 3.6 (0.6) <0.01
Acetylcholinesterase/Hemoglobin (U/g)
 2008 24.8 (3.1) <0.01 24.5 (3.3) 24.9 (2.7) 26.5 (2.9) 0.10
 2016 28.6 (3.4) 28.8 (3.4) 28.8 (3.2) 28.3 (3.5) <0.01
Open in a new tab

Displayed values are mean (standard deviation) or percent.

a

Tertile cut-offs are based on the pooled data from the 2008 and July–October 2016 examinations.

3.2. Cross-sectional associations between time after the MDH and neurobehavior

We found statistically significant positive associations between time after the end of the MDH and two neurobehavior domains (Nobs = 523): Attention and Inhibitory Control, and Language (Table 2). In the fully adjusted model, Attention and Inhibitory Control scores increased by 0.22 points (95% CI = 0.03 to 0.41) per 10 days after the end of the MDH (β). We observed a significant curvilinear relationship on this association (β-quadratic = 0.31, 95% CI = 0.09 to 0.53). Language also had a significant positive association in the minimally adjusted model, increasing on average by 0.19 points (95% CI = 0.00 to 0.37) per 10 days after the end of this peak pesticide-use period. The fully adjusted model indicated a borderline positive association with mean Language scores increasing by 0.16 points (95% CI = −0.03 to 0.34) per 10 days after the MDH.

Table 2.

Cross-sectional and longitudinal neurobehavioral performance differences per 10 days after the end of the Mother’s Day Harvest (β).

Neurobehavioral Domain β (95% CI)
Model 1a Model 2b
Attention and Inhibitory Controlc
 A) Summer 2016 cross-sectional analysis (nobs = 523) 0.19 (−0.00, 0.38)**d 0.22 (0.03, 0.41)**e
 B) Summer 2008 and Summer 2016 longitudinal analysis (nobs = 826) 0.20 (0.05, 0.35)** 0.19 (0.04, 0.34)**
Language
 A) Summer 2016 cross-sectional analysis (nobs = 523) 0.19 (0.00, 0.37)** 0.16 (−0.03, 0.34)
 B) Summer 2008 and Summer 2016 longitudinal analysis (nobs = 831) 0.01 (−0.14, 0.16)f −0.02 (−0.16, 0.13)g
Memory and Learning
 A) Summer 2016 cross-sectional analysis (nobs = 523) 0.10 (−0.12, 0.32) 0.05 (−0.17, 0.27)
 B) Summer 2008 and Summer 2016 longitudinal analysis (nobs = 829) 0.04 (−0.12, 0.19) 0.02 (−0.14, 0.17)
Visuospatial Processing
 A) Summer 2016 cross-sectional analysis (nobs = 523) −0.03 (−0.24, 0.19) −0.06 (−0.28, 0.16)
 B) Summer 2008 and Summer 2016 longitudinal analysis (nobs = 832) 0.13 (−0.05, 0.31)h 0.13 (−0.06, 0.31)i
Social Perception
 A) Summer 2016 cross-sectional analysis (nobs = 523) 0.20 (−0.02, 0.41)* 0.16 (−0.06, 0.38)
Open in a new tab
**

p ≤ 0.05.

*

p ≤ 0.10.

a

Adjusted for age, sex, race, height-for-age z-score, and examination in October 2016.

b

Adjusted for Model 1 + hemoglobin concentration and parental education.

c

Also adjusted for test-retest among participants assessed in April 2016 and Summer 2016.

d

Statistically significant quadratic model: β = −6.03 (95% CI: −10.46, −1.60); β-quadratic = 0.32 (95% CI: 0.09, 0.54).

e

Statistically significant quadratic model: β = −5.90 (95% CI: −10.29, −1.50); β-quadratic = 0.31 (95% CI: 0.09, 0.53).

f

Statistically significant quadratic model: β = −2.25 (95% CI: −3.63, −0.88); β-quadratic = 0.12 (95% CI: 0.05, 0.20).

g

Statistically significant quadratic model: β = −2.32 (95% CI: −3.71, −0.92); β-quadratic = 0.13 (95% CI: 0.05, 0.20).

h

Statistically significant quadratic model: β = 3.64 (95% CI: 1.86, 5.43); β-quadratic = −0.19 (95% CI: −0.29, −0.09).

i

Statistically significant quadratic model: β = 3.56 (95% CI: 1.75, 5.38); β-quadratic = −0.19 (95% CI: −0.29, −0.09).

We did not find any multiplicative effect measure modification by examination period, age, sex, flower worker cohabitation, or residential distance to the nearest flower plantation in this cross-sectional sample.

3.3. Longitudinal associations between neurobehavior and time after the MDH

Among the pooled sample of participants assessed after peak pesticide-use periods (July–August 2008 and July–October 2016 examinations, Nobs~832), we again found significant positive associations between time after the end of the MDH and Attention and Inhibitory Control in both adjustment models (Table 2) that were similar in magnitude as those observed in the cross-sectional analyses. We observed that children examined sooner after the harvest had lower scores in this domain than children examined later, and the positive associations observed were present through approximately 112 days after the harvest (Fig. 2). We observed a significant curvilinear relationship between time after the MDH and Visuospatial Processing scores in both adjusted models (fully adjusted βquadratic = −0.19, 95% CIquadratic = −0.29 to −0.09). This curvilinear association can be visualized in Fig. 2, in which the associations with Visuospatial Processing were positive only until approximately 92 days after the end of the MDH (β = 0.84 points [95% CI = 0.43 to 1.26]), but not afterwards (β = 0.06 points [95% CI = −0.40, 0.51]).

Fig. 2.

Fig. 2.

Open in a new tab

Adjusted* longitudinal associations of neurobehavior scores (sextiles using least square means) and time after the end of the Mother’s Day harvest (MDH) among ESPINA participants examined in July–August 2008 and July–October 2016. Note: to improve visualization, we did not plot 28 participants with values beyond 120 days. *Models adjusted for age, sex, race, height-for-age z-score, examination in October 2016, hemoglobin concentration, and parental education (Attention and Inhibitory Control domain also adjusted for test-retest among participants assessed in April 2016 and Summer, 2016).

Similar to the results in the cross-sectional and other longitudinal samples, we observed positive associations between time in days after the end of the MDH and Attention and Inhibitory Control among the participants who were examined both in April and July–October 2016 (Nobs = 606), although these associations were not statistically significant. In the fully adjusted model, domain scores increased by 0.07 points (95% CI = −0.10 to 0.25) per 10 days in this sample.

We did not find any multiplicative effect measure modification by examination period, age, sex, flower worker cohabitation, or residential distance to the nearest flower plantation in these longitudinal analyses. The findings from our sensitivity analyses excluding participants with imputed values were comparable to the results generated with imputed values (data not shown).

4. Discussion

Our study examined cross-sectional and longitudinal associations between time after the end of peak pesticide spray seasons and neurobehavioral outcomes and evaluated whether cyclical neurobehavior alterations previously observed in childhood continued to be observed in adolescence among participants in the ESPINA study.

In our cross-sectional analyses of 2016 participants, we found significant positive associations between time after a peak pesticide spray season and two neurobehavioral domain scores (Attention and Inhibitory Control, and Language) among adolescents living in Ecuadorian agricultural communities who did not have direct occupational exposure to insecticides. The associations with Attention and Inhibitory Control were weaker than, but consistent with, those previously reported among 4-9 year-old ESPINA participants in 2008 (2008 β per 10 days = 0.35 points vs. 2016 β per 10 days = 0.22 points) (Suarez-Lopez et al., 2017). Participants in 2016 had higher AChE activity compared to participants in 2008; this was expected, as AChE activity was previously found to be positively associated with age in children (Suarez-Lopez et al., 2012). Our findings are also consistent with results from previous ESPINA study findings that used other constructs of organophosphate exposure including AChE activity and residential distance to floricultural crops. One such study found that lower AChE activity was associated with lower Attention and Executive Functioning scores (β per AChE U/mL Decrease = −0.32, 95% CI = −1.02 to 0.38) as well as Language scores (β per AChE U/mL Decrease = −0.39, 95% CI = −1.11 to 0.32) (Suarez-Lopez et al., 2013). Another study found that children in this setting who lived closer to crops (within 50 m) had lower Attention and Inhibitory Control scores (β = −1.24, 95% CI = −2.45 to −0.04) and Language scores (β = −1.28, 95% CI = −2.50 to −0.06) compared to those living farther than 500 m from crops (Friedman et al., 2020). Although these are other proxy measures of organophosphate exposure versus time after the MDH, it is interesting that different constructs of organophosphate exposure yield consistent findings regarding transient alterations in similar neurobehavior domains.

Similar to our cross-sectional findings, we observed in our longitudinal analyses that time after a peak pesticide spray season was positively associated with Attention and Inhibitory Control and Visuospatial scores among children and adolescents in this setting. These findings suggest that these domain scores continue to improve until approximately 92–112 days after the end of the MDH. After this time, performance may have returned to basal (unexposed) levels. The lack of participants assessed beyond 112 days during low pesticide-use periods precludes us from comprehensively assessing the point at which domain scores stop increasing (presumably as a result of reaching a basal level). Our results are consistent with experimental evidence that organophosphate insecticide exposures are associated with transient neurobehavioral changes in rats and zebrafish (Levin et al., 2003; Maurissen et al., 2000; Middlemore-Risher et al., 2010). Our results are also consistent with our previous work reporting cyclical alterations in neurobehavioral performance associated with pesticide spray seasons among 4-9 year-olds in the ESPINA study (Suarez-Lopez et al., 2017), in which we observed that time after the harvest was positively associated with the domains of Attention and Inhibitory Control, Visuospatial Processing and Sensorimotor until 80–87 days after the MDH after which scores appeared to return to unexposed levels; the Sensorimotor domain was not included in the present analyses as it was not assessed in 2016. This timeframe is roughly similar with the time needed (82 days) for erythrocytic AChE activity to return to unexposed levels after irreversible inhibition due to organophosphate exposure (Mason, 2000), which perhaps provides some pathophysiological explanation, coupled with the direct associations between AChE activity and neurobehavior aforementioned, for the associations we report on this manuscript. Finally, our findings are concordant with studies conducted in Egypt among adolescent agricultural workers and nonworkers which found positive associations between time after a peak pesticide use period and self-reported neurological symptoms (Khan et al., 2014) as well as neurobehavioral impairment which remained for several months after the pesticide application period (Rohlman et al., 2016).

Our findings highlight the vulnerability of children and adolescents to secondary pesticide exposure considering that the participants in our study lived in agricultural communities but did not have direct occupational exposure. It is important to consider that these neurobehavioral alterations associated with the MDH in May are occurring during a highly influential time of the year. Since the academic school year ends in June, the children and adolescents living in this area may have poorer academic performance around the time that they are presenting for final exams or university qualifying exams due to impaired neurobehavioral performance. Decreased performance during important academic testtaking periods could subsequently affect the total grade point average, which can then affect access to jobs or higher education, and future earning potential (Beckett et al., 2007; Pollak et al., 2010; Windsor et al., 2007). Furthermore, while we have highlighted the short-term effects of pesticide spray seasons on neurobehavior, there may also be an accumulation of short-term effects throughout the year as well as chronic effects on neurodevelopment associated with pesticide exposures (Dórea, 2020) among children and adolescents who have grown up in this community.

Our study has several limitations. First, we did not assess neurobehavioral performance at multiple times after the MDH in 2008 or 2016 (e.g., repeated assessments from the same individuals both during and after peak pesticide-use periods), so we are unable to assess actual change in neurobehavioral performance within children. However, we did examine participants twice in 2016 (April and July–October exams) for the domain of Attention and Inhibitory Control. Noting that children examined in April had negative values for time after the MDH, our analyses also resulted in positive associations, albeit weaker and nonsignificant. Conducting multiple assessments over a short period of time of large population-based studies is incredibly difficult which explains the paucity of studies assessing subacute effects of pesticide exposures. If at all possible, we recommend that future studies include multiple assessments over time (before, during, and after) throughout a peak pesticide use period to have a more precise assessment of these associations. Second, some of the pooled samples in our longitudinal analyses had distinct ranges of time after the MDH. However, there was approximately a 20-day overlap between both cohorts and pooling the samples allowed us to assess a much longer length of time following a peak pesticide spray period. Additionally, the present analyses do not account for specific pesticide exposures and, as such, we are unable to discern which pesticides or pesticide classes may be driving these associations. However, the observed positive association between AChE activity and time after the MDH observed within our cohort, coupled with previous findings that AChE activity is positively associated with neurobehavioral performance (Suarez-Lopez et al., 2013) as well as anxiety and depression (Suarez-Lopez et al., 2019, 2020), provide a compelling indication that cholinesterase inhibitor pesticides such as organophosphates and carbamates may be driving the time after the MDH-neurobehavior association. Mediation by pesticide exposures of the present associations using urinary pesticide metabolite concentrations, or other pesticide constructs, is warranted.

5. Conclusions

We examined the neurobehavioral effects of a pesticide spray season in one of the largest prospective studies of children and adolescents living in agricultural settings. We observed that children examined sooner after the harvest had lower neurobehavioral performance for the domains of Attention and Inhibitory Control, Language and Visuospatial Processing compared to children examined later. This suggests that peak pesticide spray seasons may result in transient neurobehavioral performance decreases, which then increase during low pesticide-use periods back to pre-exposure levels over a period of about 92–112 days. These findings are consistent with previous studies yet should be replicated in other settings given the limited existing research on this topic. Future research should aim to parse out the acute and chronic effects of pesticide exposures and/or account for pesticide spray seasons during data collection or in analytical models.

Acknowledgements

The ESPINA study received funding from the National Institute of Occupational Safety and Health (1R36OH009402) and the National Institute of Environmental Health Sciences (R01ES030378, R01ES025792, R21ES026084). We thank Fundación Cimas del Ecuador, the Parish Governments of Pedro Moncayo County, community members of Pedro Moncayo, and the Education District of Pichincha-Cayambe-Pedro Moncayo counties for their support on this project.

Footnotes

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Credit author statement

Cristina Espinosa da Silva: Conceptualization, Methodology, Software, Formal analysis, Writing – original draft, Writing – review & editing, Visualization. Sheila Gahagan: Writing – review & editing. José Suárez-Torres: Supervision, Project administration, Data Collection, Writing – review & editing. Dolores López-Paredes: Investigation, Supervision, Project administration, Data Collection, Writing – review & editing. Harvey Checkoway: Writing – review & editing. José R. Suárez-López: Conceptualization, Study Design, Funding acquisition, Investigation, Project administration, Data Collection, Methodology, Software, Formal analysis, Writing – original draft, Writing – review & editing, Visualization.

References

  1. Abou-Donia MB, 2003. Organophosphorus ester-induced chronic neurotoxicity. Arch. Environ. Health 58, 484–497. [DOI] [PubMed] [Google Scholar]
  2. Aldridge JE, Meyer A, Seidler FJ, Slotkin TA, 2005. Alterations in central nervous system serotonergic and dopaminergic synaptic activity in adulthood after prenatal or neonatal chlorpyrifos exposure. Environ. Health Perspect 113, 1027–1031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Beckett C, Maughan B, Rutter M, Castle J, Colvert E, Groothues C, Hawkins A, Kreppner J, O’Connor TG, Stevens S, 2007. Scholastic attainment following severe early institutional deprivation: a study of children adopted from Romania. J. Abnorm. Child Psychol 35, 1063–1073. [DOI] [PubMed] [Google Scholar]
  4. Brooks BL, Sherman EMS, Iverson GL, 2010. Healthy children get low scores too: prevalence of low scores on the NEPSY-II in preschoolers, children, and adolescents. Arch. Clin. Neuropsychol 25, 182–190. [DOI] [PubMed] [Google Scholar]
  5. Crane AL, Rasoul GA, Ismail AA, Hendy O, Bonner MR, Lasarev MR, Al-Batanony M, Singleton ST, Khan K, Olson JR, 2013. Longitudinal assessment of chlorpyrifos exposure and effect biomarkers in adolescent Egyptian agricultural workers. J. Expo. Sci. Environ. Epidemiol 23, 356–362. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Delgado E, McConnell R, Miranda J, Keifer M, Lundberg I, Partanen T, Wesseling C, 2004. Central nervous system effects of acute organophosphate poisoning in a two-year follow-up. Scand. J. Work. Environ. Health 362–370. [DOI] [PubMed] [Google Scholar]
  7. Dórea JG, 2020. Exposure to environmental neurotoxic substances and neurodevelopment in children from Latin America and the Caribbean. Environ. Res 110199. [DOI] [PubMed] [Google Scholar]
  8. EQM Research Inc., 2003. Test-mate ChE Cholinesterase Test System (Model 400), Instruction Manual. [Google Scholar]
  9. Friedman E, Hazlehurst MF, Loftus C, Karr C, McDonald KN, Suarez-Lopez JR, 2020. Residential proximity to greenhouse agriculture and neurobehavioral performance in Ecuadorian children. Int. J. Hyg Environ. Health 223, 220–227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Galea KS, MacCalman L, Jones K, Cocker J, Teedon P, Cherrie JW, Van Tongeren M, 2015. Urinary biomarker concentrations of captan, chlormequat, chlorpyrifos and cypermethrin in UK adults and children living near agricultural land. J. Expo. Sci. Environ. Epidemiol 25, 623–631. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Grandjean P, Harari R, Barr DB, Debes F, 2006. Pesticide exposure and stunting as independent predictors of neurobehavioral deficits in Ecuadorian school children. Pediatrics 117, e546–e556. [DOI] [PubMed] [Google Scholar]
  12. Hernandez AF, González-Alzaga B, Lopez-Flores I, Lacasana M, 2016. Systematic reviews on neurodevelopmental and neurodegenerative disorders linked to pesticide exposure: methodological features and impact on risk assessment. Environ. Int 92, 657–679. [DOI] [PubMed] [Google Scholar]
  13. Kemp SL, Korkman M, 2010. Essentials of NEPSY-II Assessment. John Wiley & Sons. [Google Scholar]
  14. Khan K, Ismail AA, Rasoul GA, Bonner MR, Lasarev MR, Hendy O, Al- Batanony M, Crane AL, Singleton ST, Olson JR, 2014. Longitudinal assessment of chlorpyrifos exposure and self-reported neurological symptoms in adolescent pesticide applicators. BMJ Open 4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Kofman O, Berger A, Massarwa A, Friedman A, Jaffar AA, 2006. Motor inhibition and learning impairments in school-aged children following exposure to organophosphate pesticides in infancy. Pediatr. Res 60, 88–92. [DOI] [PubMed] [Google Scholar]
  16. Korkman M, Kirk U, Kemp S, 2007. NEPSY II: Clinical and Interpretive Manual. Harcourt Assessment, PsychCorp. [Google Scholar]
  17. Krenz JE, Hofmann JN, Smith TR, Cunningham RN, Fenske RA, Simpson CD, Keifer M, 2015. Determinants of butyrylcholinesterase inhibition among agricultural pesticide handlers in Washington State: an update. Ann. Occup. Hyg 59, 25–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Levin ED, Chrysanthis E, Yacisin K, Linney E, 2003. Chlorpyrifos exposure of developing zebrafish: effects on survival and long-term effects on response latency and spatial discrimination. Neurotoxicol. Teratol 25, 51–57. [DOI] [PubMed] [Google Scholar]
  19. Marks AR, Harley K, Bradman A, Kogut K, Barr DB, Johnson C, Calderon N, Eskenazi B, 2010. Organophosphate pesticide exposure and attention in young Mexican-American children: the CHAMACOS study. Environ. Health Perspect 118, 1768–1774. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Mason HJ, 2000. The recovery of plasma cholinesterase and erythrocyte acetylcholinesterase activity in workers after over-exposure to dichlorvos. Occup. Med. (Chic. Ill) 50, 343–347. [DOI] [PubMed] [Google Scholar]
  21. Maurissen JPJ, Hoberman AM, Garman RH, Hanley TR Jr., 2000. Lack of selective developmental neurotoxicity in rat pups from dams treated by gavage with chlorpyrifos. Toxicol. Sci 57, 250–263. [DOI] [PubMed] [Google Scholar]
  22. Middlemore-Risher M-L, Buccafusco JJ, Terry AV Jr., 2010. Repeated exposures to low-level chlorpyrifos results in impairments in sustained attention and increased impulsivity in rats. Neurotoxicol. Teratol 32, 415–424. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Muñoz-Quezada MT, Lucero BA, Barr DB, Steenland K, Levy K, Ryan PB, Iglesias V, Alvarado S, Concha C, Rojas E, 2013. Neurodevelopmental effects in children associated with exposure to organophosphate pesticides: a systematic review. Neurotoxicology 39, 158–168. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Peiris-John RJ, Ruberu DK, Wickremasinghe AR, van-der-Hoek W, 2005. Low-level exposure to organophosphate pesticides leads to restrictive lung dysfunction. Respir. Med 99, 1319–1324. [DOI] [PubMed] [Google Scholar]
  25. Poliak SD, Nelson CA, Schlaak MF, Roeber BJ, Wewerka SS, Wiik KL, Frenn KA, Loman MM, Gunnar MR, 2010. Neurodevelopmental effects of early deprivation in postinstitutionalized children. Child Dev. 81, 224–236. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Qiao D, Seidler FJ, Tate CA, Cousins MM, Slotkin TA, 2003. Fetal chlorpyrifos exposure: adverse effects on brain cell development and cholinergic biomarkers emerge postnatally and continue into adolescence and adulthood. Environ. Health Perspect 111, 536–544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Quandt SA, Pope CN, Chen H, Summers P, Arcury TA, 2015. Longitudinal assessment of blood cholinesterase activities over two consecutive years among Latino non-farmworkers and pesticide-exposed farmworkers in North Carolina. J. Occup. Environ. Med. Coll. Occup. Environ. Med 57, 851. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Rohlman DS, Ismail AA, Rasoul GA, Bonner MR, Hendy O, Mara K, Wang K, Olson JR, 2016. A 10-month prospective study of organophosphorus pesticide exposure and neurobehavioral performance among adolescents in Egypt. Cortex 74, 383–395. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Singleton ST, Lein PJ, Dadson OA, McGarrigle BP, Farahat FM, Farahat T, Bonner MR, Fenske RA, Galvin K, Lasarev MR, 2015. Longitudinal assessment of occupational exposures to the organophosphorous insecticides chlorpyrifos and profenofos in Egyptian cotton field workers. Int. J. Hyg Environ. Health 218, 203–211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Slotkin TA, 2004. Cholinergic systems in brain development and disruption by neurotoxicants: nicotine, environmental tobacco smoke, organophosphates. Toxicol. Appl. Pharmacol 198, 132–151. [DOI] [PubMed] [Google Scholar]
  31. Strelitz J, Engel LS, Keifer MC, 2014. Blood acetylcholinesterase and butyrylcholinesterase as biomarkers of cholinesterase depression among pesticide handlers. Occup. Environ. Med 71, 842–847. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Suarez-Lopez JR, Butcher CR, Gahagan S, Checkoway H, Alexander BH, Al- Delaimy WK, 2018. Acetylcholinesterase activity and time after a peak pesticide-use period among Ecuadorian children. Int. Arch. Occup. Environ. Health 91, 175–184. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Suarez-Lopez JR, Checkoway H, Jacobs DR Jr., Al-Delaimy WK, Gahagan S, 2017. Potential short-term neurobehavioral alterations in children associated with a peak pesticide spray season: the Mother’s Day flower harvest in Ecuador. Neurotoxicology 60, 125–133. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Suarez-Lopez JR, Himes JH, Jacobs DR, Alexander BH, Gunnar MR, 2013. Acetylcholinesterase activity and neurodevelopment in boys and girls. Pediatrics 132, e1649–e1658. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Suarez-Lopez JR, Hood N, Suárez-Torres J, Gahagan S, Gunnar MR, López-Paredes D, 2019. Associations of acetylcholinesterase activity with depression and anxiety symptoms among adolescents growing up near pesticide spray sites. Int. J. Hyg Environ. Health 222, 981–990. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Suarez-Lopez JR, Jacobs DR Jr., Himes JH, Alexander BH, Lazovich D, Gunnar M, 2012. Lower acetylcholinesterase activity among children living with flower plantation workers. Environ. Res 114, 53–59. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Suarez-Lopez JR, Nguyen A, Klas J, Gahagan S, Checkoway H, Lopez-Paredes D, Jacobs DR, Noble M, 2020. Associations of acetylcholinesterase inhibition between pesticide spray seasons with depression and anxiety symptoms in adolescents, and the role of sex and adrenal hormones on gender moderation. Expo. Heal 1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Thompson B, Griffith WC, Barr DB, Coronado GD, Vigoren EM, Faustman EM, 2014. Variability in the take-home pathway: farmworkers and non-farmworkers and their children. J. Expo. Sci. Environ. Epidemiol 24, 522–531. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. WHO Multicentre Growth Reference Study Group, 2006. WHO Child Growth Standards based on length/height, weight and age. Acta Paediatr. 450 (Suppl. 1), 76. [Google Scholar]
  40. Windsor J, Glaze LE, Koga SF, 2007. Language acquisition with limited input: Romanian institution and foster care. J. Speech Lang. Hear. Res 50, 1365–1381. [DOI] [PubMed] [Google Scholar]
  41. World Health Organization, 2008. 2008 Training Course on Child Grown Assessment 7, 25–36. [Google Scholar]

RESOURCES