Abstract
Children and adolescents may have occupational exposure to pesticides. Although previous studies examining prenatal pesticide exposure have identified neurobehavioral deficits in children, there are limited studies examining the impact of occupational exposure in children. The objectives of this study are to estimate exposures to the organophosphorus pesticide, chlorpyrifos (CPF), by measuring urinary levels of 3,5,6-trichloro-2-pyridinol (TCPy), a specific CPF metabolite, and blood cholinesterase (ChE) activities and to characterize neurobehavioral performance in adolescents working as seasonal pesticide applicators and non-applicator controls. A neurobehavioral test battery, consisting of 14 tests, was used to assess a broad range of functions. Applicators performed worse than controls on the majority of tests. Principal component analysis was used to reduce the number of outcome variables and two components, focused on reasoning-short-term memory and attention-executive functioning, showed significant deficits in applicators compared to non-applicators. Elevated metabolite levels were found in the applicators compared to the non-applicators, confirming CPF exposure in the applicators. Although this study is limited by a small sample size, it provides preliminary evidence of moderate CPF exposures, decreased blood ChE in some applicators and decreased neurobehavioral performance in an adolescent working population.
Keywords: Pesticides, Adolescents, Neurobehavioral, Biomarker, TCPy, Cholinesterase
Introduction
Many children throughout the world are engaged in agricultural work, either for pay or on family farms; these activities are often associated with risk, including exposure to pesticides. Although organophosphorus (OP) pesticides are increasingly restricted for use in the United States (US) (EPA 2002), many of the pesticides that are no longer available in the US are still being produced and used in developing countries. Human and animal studies consistently identify neurotoxicity as the primary endpoint of concern, with a primary target of acute OP exposure being inhibition of β-esterases, including acetylcholinesterase (AChE), butyrylcholinesterase (BChE) and carboxylesterase (CE) (Bushnell and Moser 2006; Costa 2006). However, the long-term effects of repeated occupational and environmental exposures to OP pesticides remain poorly understood (Rohlman et al. 2011; Colosio et al. 2009).
Vulnerability of children
Recent reports examining children have linked prenatal OP exposure with increased risk of pervasive developmental disorders, as well as delays in mental development (Bouchard et al. 2011; Rauh et al. 2011; Engel et al. 2011) and postnatal OP exposure with behavioral problems, poorer short-term memory and motor skills, and longer reaction times (Grandjean et al. 2006; Rohlman et al. 2005; Ruckart et al. 2004). Other studies have also found associations between exposure and Attention Deficit/Hyperactivity Disorder (ADHD), including prenatal exposures (Bellinger 2013; Marks et al. 2010; Rauh et al. 2006) and current exposure linking urinary metabolite levels for OPs with increased diagnosis of ADHD (Bouchard et al. 2010). Animal studies support the potential of early OP exposures to cause deficits (Dam et al. 2000; Eaton et al. 2008; Johnson et al. 1998; Moser and Padilla 1998; Slotkin 2005). However, while the majority of both human and animal research has focused on exposures during the perinatal period and early life exposures, there is limited data examining exposures during adolescence, a critical developmental period when the pattern of synaptic connections is being formed.
Adolescence is characterized as a period of rapid development. In addition to the hormonal and physiological changes associated with puberty, there are also significant developmental changes in the brain, primarily the prefrontal cortex (Spear 2010; Steinberg 2008). These changes are associated with behavioral changes including increases in novelty seeking and risk-taking behavior, emotional reactivity and changes in information processing speed and tasks of executive function (e.g., response inhibition, working memory and attention). Research is needed to determine if the changes occurring during this time of development make the adolescent brain more vulnerable to disruption by environmental toxins or more resilient (Kalia 2008; Spear 2002; Steinberg 2008). Additionally, vulnerability is impacted by the ability to metabolize toxins, which can also vary across ages (Connors et al. 2008; Eskenazi et al. 2010; Kalia 2008). Children have more years to live than adults, they have more time to develop diseases due to early exposures, and some effects may not appear until the child is older (Costa et al. 2004; Godfrey and Barker 2001; Landrigan et al. 2004; National Academy of Sciences 1993; Reuhl 1991).
Only a few studies have examined neurobehavioral effects in adolescents who are currently working in agriculture (Rohlman et al. 1999, 2007; Abdel Rasoul et al. 2008; Eckerman et al. 2006). Similar to studies with adult agricultural workers, studies with adolescents (Ismail et al. 2012; Rohlman et al. 2011) have reported neurobehavioral deficits in motor speed and coordination, information processing speed and executive functioning, attention and memory. Although these studies have identified deficits between occupationally exposed adolescents and controls, these studies are limited and do not include biomarkers characterizing exposure.
Egyptian pesticide application
Agriculture employs nearly 40 % of the work force in Egypt (Mansour 2004). The primary agricultural product in Egypt is cotton and because of its importance for the economy, the use of pesticides on that crop is highly regulated by the Egyptian Ministry of Agriculture. The national government purchases and sells the country's entire cotton production and, once farmers agree to plant cotton, applications of chemicals on those fields come under control of the Ministry of Agriculture. Thus, all pesticides, equipment and calibration procedures are standardized across the Governorate. The primary insecticide used in Egyptian cotton fields to control boll weevils is an organophosphorus pesticide (OP), Pestban™, an emulsifiable concentrate formulation that contains 48 % chlorpyrifos as the active ingredient.
Adolescents in Egypt are hired by the Ministry of Agriculture as seasonal workers to apply pesticides to the cotton crop and may work for repeated seasons. Applicators work in teams of 3–4 with adult supervisors, wear backpack sprayers and apply pesticides in a staggered line through the cotton fields without the use of personal protective equipment (PPE). Previous research in this adolescent Egyptian population found increased symptom reports, depressed cholinesterase, and extensive neurobehavioral deficits in adolescents applying chlorpyrifos (CPF) (vs. controls) (Abdel Rasoul et al. 2008; Ismail et al. 2010). While we have evidence of some of the highest reported exposures in adult Egyptian applicators (Farahat et al. 2010, 2011), quantitative exposure data in adolescents is limited (Crane et al. 2013).
The current study characterized the exposure and neurobehavioral performance in this population and examined the hypothesis that pesticide exposures damage the health of adolescents working as pesticide applicators by measuring standard neurobehavioral tests and biomarkers of exposure to chlorpyrifos.
Methods
Participants
Adolescents, 12 to 18 years of age, employed by the Egyptian Ministry of Agriculture as pesticide applicators for the cotton crop, were recruited for the study (N=21). Non-Applicator participants (N=20) from the same villages in the Nile delta with similar age and education were also recruited as controls. Pesticide applicators are seasonal workers hired from June through August. Although there are regulations in Egypt restricting adolescent work in hazardous jobs (Mosallem 2011), similar to the US, adolescents often work in agriculture and perform the same work as adults, including mixing or applying pesticides (McCauley et al. 2002). Both these seasonal workers and the non-applicators are enrolled in secondary school during the remainder of the year. The non-applicator participants were never employed as pesticide applicators for the cotton crop but may have had other opportunities for pesticide exposure. Written informed consent was obtained from all participants and their legal guardian (for those under 18). The study was approved by the Oregon Health & Science University IRB in June 2009, and by the Medical Ethics committee of the Faculty of Medicine, Menoufia University in July 2009.
Procedures
Beginning in June, the pesticide application teams begin a rigidly controlled cycle of checking for infestation and applying pesticides daily in their assigned fields during four application cycles per season: 1) ∼14 days of Bacillus thuringiensis (a natural occurring bacterium that is harmful to insects but not humans); 2) approximately 14 days of Pestban (CPF) OP application; 3) ∼14 days of pyrethrins or a less potent carbamate; and 4) approximately 14 days of Dursban (another formulation of CPF). There is the option of combining an OP and a pyrethrin for a single application on a field with a pesticide-resistant infestation. There is approximately a week of no spraying between the four application periods. This schedule of pest management has been followed rigidly for over 10 years. Data for the present study were collected during the 4th cycle in August of 2009.
Neurobehavioral test battery
Participants completed a selected battery of neurobehavioral tests based on previous research with adolescents and agricultural workers (Table 1). Initial selection was based on: a) tests demonstrating deficits in studies of Egyptian adolescent and adult pesticide workers (Abdel Rasoul et al. 2008; Farahat et al. 2003b); b) tests demonstrating deficits in other adolescent agriculture worker populations (Eckerman et al. 2006; Rohlman et al. 2001, 2007); and c) tests demonstrating associations between biomarkers and performance (Abdel Rasoul et al. 2008; Rothlein et al. 2006) (Table 1). Test-retest reliability is established and comparable for the individually-administered tests (Lezak 1995) used by (Ismail et al. 2007) and for the computerized tests (Ismail et al. 2008).
Table 1. Neurobehavioral tests that identified deficits associated with OP pesticide exposure in previous studies.
Function and test | Source |
---|---|
Memory | |
Match to sample | Rohlman 07a, Eckerman 06b |
Serial digit learning | Eckerman 06b, Rohlman 01 |
Benton visual retention | Ismail 07c, Roldan-Tapia 05, Farahat 03, Reidy 92 |
Attention/short-term memory | |
Digit span | Ismail 07c, Rohlman 07a, Eckerman 06b, Rothlein 06, Farahat 03, Kamel 03, Rohlman 01, Cole 97, Richter 92 |
Sustained attention | |
Continuous performance | Rohlman 07a, Rothlein 06d, Rohlman 01 |
Selective attention | Rothlein 06d, Rohlman 01 |
Motor speed/coordination | |
Finger tapping | Eckerman 06b, Rothlein 06d |
Santa ana pegboard | Kamel 03, London 97, Cole 97, Richter 92 |
Information processing speed | |
Reaction time | Rohlman 07, Eckerman 06b, Rohlman 01, Fiedler 97, Stephens 95, Richter 92 |
Complex visual-motor /executive function | |
Symbol digit | Rohlman 07, Rothlein 06d, Rohlman 01, Stephens 95 |
Trail making | Ismail 07c, e, Farahat 03, Cole 97, Richter 92, Korsak 77 |
Verbal abstraction | |
Similarities | Ismail 07c, Farahat 03 |
Perception | |
Block design | Ismail 07c |
Results are for workers reporting mixing/applying OP pesticides in comparison to others who had not
Results are of those rural 10–11 years old compared to their matched urban controls
Results are those of applicators 15–18 years old compared to their matched controls
Scores are significantly correlated with the summed DMTP + DMDTP urinary dialkylphosphate (DAP) metabolites, non-specific markers of OP pesticides, after controlling for age, sex, education
Latency was negatively correlated with AChE
Questionnaire
Each participant completed a questionnaire administered by a Research Assistant, ascertaining information about previous medical conditions, occupational history, socioeconomic background, education, use of personal protective equipment, and activities engaged in outside of fieldwork.
Urine sample
Spot urine samples were collected at the beginning of the work shift during the 4th application cycle on August 15, 2009. Urine samples were placed on wet ice in a cooler and transported to Menoufia University (Shebin El-Kom, Egypt), where they were stored at −20 °C until being shipped to the University at Buffalo (Buffalo, NY, USA) on dry ice for analysis. Urine samples were analyzed for 3,5,6-trichloro-2-pyridinol (TCPy), a specific metabolite of CPF, by negative-ion chemical ionization gas chromatography–mass spectrometry, using 13C 15N 3,5,6 TCPy as an internal standard, as described previously (Farahat et al. 2010). Creatinine concentrations were measured using the Jaffe reaction (Fabiny and Ertingshausen 1971), and urine TCPy concentrations are expressed as micrograms TCPy per gram creatinine. For quality assurance, duplicate samples and control samples were included with each analytical series. The within-run imprecision of this assay is very low, as shown by a <2 % coefficient of variation and an intraclass correlation coefficient of 0.997.
Blood sample analysis
Blood samples were collected by venipuncture into 10 mL lavender top (EDTA) vacutainer tubes and immediately placed on wet ice and transported to Menoufia University; where they were analyzed in duplicate for AChE and BChE activity using an EQM Test-Mate kit (EQM Research Inc., Cincinnati, OH, USA) as described previously (Farahat et al. 2011).
Statistical analysis
Continuous data were summarized in terms of means and standard deviations or medians and the Interquartile Range (IQR). Creatinine adjusted TCPy concentration in the urine was first log transformed to improve symmetry and analyzed with respect to the transformed means (via t-test). Back transformation of the (log-transformed) means results in a geometric mean, which acts as an estimate of the median response; back transformation of the difference (between log-transformed means) reflects a multiplicative fold change in these estimated medians (Aitkin et al. 1989). Performance on each neurobehavioral (NB) test was separately standardized by subtracting the group means from each observation and then dividing the difference by the standard deviation so that all NB tests would be dimensionless and have zero mean and unit variance. Seven of the outcome measures (Selective Attention Latency and Inter-Stimulus Interval, Trail Making A and B, Match-to-Sample Latency, Reaction Time Latency and Symbol-Digit Latency) had directionality of the standardized scores reversed (multiplication by −1) such that larger, more positive, standardized scores are associated with better performance for all tests.
Standardized scores were analyzed for group differences using linear regression; all models were fitted in both an unadjusted (group only) and adjusted sense (group, adjusting for age, and education). P-values were corrected for multiplicity via Holm's method. A secondary analysis sought to minimize the number of exploratory tests by using principal components to reduce the 19 individual NB outcome measures down to a smaller set of uncorrelated linear combinations of tests. The linear combinations (components) were rotated (varimax rotation) to minimize an individual test from loading on multiple components simultaneously. These components were then used as the response in the same sort of linear regression models described above to determine whether the value of these combined tests were associated with group, either by itself or after adjustment by the stated variables.
Results
Twenty-one applicators and 20 non-applicators (controls) contributed data for analysis. Ages in each group ranged from 12 to 18 years, with a mean of 15.5 years of age for both groups. The average years of education were also similar, 9.5 (SD= 1.5) years for non-applicators and 8.6 (SD=1.6) years for applicators. These characteristics are shown in Table 2. All of the non-applicator participants were enrolled in school, although four of the applicators report working another job in addition to pesticide application (2 in a metal shop, 1 as a farmer, and 1 in a grocery store). Applicators reported an average of 2.0 (SD=1.0) years of applying pesticides as part of their job, typically working 5 h a day, 4–5 days per week; non-applicators uniformly had zero years of applying pesticides.
Table 2. Demographic characteristics and CPF exposure and effect biomarkers in a sample of 21 Egyptian adolescent applicators and 20 non-applicators.
Characteristic | Range | Mean (SD) | Median (IQR) | Group comparison |
---|---|---|---|---|
Age (yrs) | t(39)=0.04, p=0.97 | |||
Non-applicator | 12–18 | 15.5 (1.5) | 16 (15–16) | |
Applicator | 12–18 | 15.5 (2.1) | 15 (14–18) | |
Education (yrs) | t(39)=1.34, p=0.19 | |||
Non-applicator | 6–12 | 9.5 (1.5) | 10 (9–10) | |
Applicator | 6–11 | 8.9 (1.6) | 9 (8–10) | |
Urine TCPy (μg/g creatinine) | t(39)=4.55, p<0.001* | |||
Non-applicator | 2.4–64.9 | 9.4 (14.3) | 4.8 (3.2–8.9) | |
Applicator | 4.9–125 | 33.6 (36.6) | 17.5 (11.1–36.8) | |
Plasma BuChE (U/ml)a | t(30)=1.05, p=0.30 | |||
Non-applicator | 1.1–3.0 | 1.7 (0.4) | 1.6 (1.5–1.8) | |
Applicator | 0.1–2.7 | 1.5 (0.8) | 1.7 (1.1–1.9) | |
RBC Ache (U/g Hgb)b | t(30)=1.54, p=0.13 | |||
Non-Applicator | 23.5–30.8 | 26.7 (1.9) | 26.6 (25.2–27.9) | |
Applicator | 17.7–29.8 | 25.4 (3.6) | 26.3 (23.1–28.5) |
IQR Interquartile range
Based on log-transformed data; GM=5.8 for non-applicators and GM=21.1 for applicators
Based on modified t-test with unequal variances
Urinary metabolites and cholinesterase activity
Biomarker data from Table 2 show that the median concentration of TCPy in the urine of applicators, adjusted for creatinine, was estimated to be 3.6 (95 % CI: 2.0–6.4) times greater than the corresponding median concentration in non-applicators. Therefore, in the case of a single applicator and non-applicator, there's an 86.6 % (95 % CI: 75.0–98.3 %) chance the applicator would have a higher TCPy concentration. Figure 1 shows that the average TCPy levels for adolescent applicators and non-applicators in the current study are similar to those found for adult pesticide applicators and non-applicators from the same region in August 2009 (adult data from, Ellison et al. 2012). The Egyptian applicators and non-applicators had much higher urinary TCPy levels than the geometric mean for adolescents (12–19 years) reported in National Health and Nutrition Examination Survey (Centers for Disease Control and Prevention 2009).
Although applicators had lower Plasma BuChE and RBC AChE activity than non-applicators, these differences were not significant. In the absence of baseline ChE data, participant's data was compared to normative values supplied by EQM Research, Inc (Fig. 2). Using the normative values for Plasma BuChE and RBC AChE activity supplied by EQM Research Inc, it was found that 33 % (n=7) of the applicators had Plasma BuChE activity lower than the EQM minimum value compared to only 5 % (n=1) of the non-applicators (Fig. 2). None of the non-applicator participants had depressed RBC AChE activity compared to 19 % of the applicators that had depressed activity based on the EQM standards.
Neurobehavioral performance
After adjusting for age and education, applicators performed worse than non-applicators on 12 of the 19 NB tests examined (Fig. 3).
The performance of the applicators was compared with the non-applicators using linear regression. Four of the tests were statistically significant, of these, two of the tests reached the level of statistical significance once corrections were made for multiple comparisons (Table 3). On each of these two tests (Match-to-Sample Score and Similarities), performance for non-applicators was above the overall group mean by at least 0.5 SD while performance for applicators was below the overall group mean to a similar degree; consequently, separation between groups was greater than 1 SD for each measure. After controlling for age and education, Match-to-Sample Score was marginally significant (Holm corrected p-value= 0.087) while the Similarities outcome measure remained highly significant (Holm corrected p-value <0.001) with each test showing approximately a 1.0 or 1.3 SD shift towards worse performance in applicators relative to controls.
Table 3. The performance of applicators and non-applicators on the neurobehavioral outcome measures. Models were adjusted for age and education.
Neurobehavioral measure | Non-applicator | Applicator | Adjusted* | ||
---|---|---|---|---|---|
|
|
||||
N | Mean (SE) | N | Mean (SE) | Diff (95 % CI) | |
Selective Att. latency | 18 | 0.043 (0.24) | 19 | −0.041 (0.23) | −0.003 (−0.741, 0.734) |
Selective Att. ISI | 18 | −0.058 (0.24) | 19 | 0.055 (0.23) | 0.239 (−0.490, 0.967 |
Trail making A | 20 | 0.232 (0.22) | 21 | −0.220 (0.22) | −0.428 (−1.145, 0.288) |
Trail making B | 20 | −0.059 (0.23) | 21 | 0.057 (0.22) | 0.111 (−0.635, 0.857) |
Match to sample latency | 20 | −0.179 (0.22) | 20 | 0.179 (0.22) | 0.451 (−0.286, 1.188) |
Match to sample score | 20 | 0.522 (0.19) | 20 | −0.522 (0.19) | −0.963 (−1.613, −.0312) a |
Reaction time latency | 18 | 0.263 (0.23) | 19 | −0.250 (0.22) | −0.346 (−1.094, 0.401) |
Symbol digit latency | 20 | 0.385 (0.21) | 21 | −0.366 (0.20) | −0.711 (−1.408, −0.014) b |
Continuous Perf. D-prime | 20 | 0.266 (0.22) | 18 | −0.251 (0.23) | −0.380 (−1.129, 0.369) |
Digit span | 19 | 0.201 (0.23) | 21 | −0.182 (0.22) | −0.277 (−1.025, 0.472) |
Serial digit learning | 17 | 0.054 (0.25) | 18 | −0.051 (0.24) | 0.178 (−0.578, 0.935) |
Finger tapping | 20 | 0.316 (0.21) | 20 | −0.316 (0.21) | −0.572 (−1.263, 0.120) |
Finger tapping alternating | 20 | 0.141 (0.22) | 19 | −0.148 (0.23) | −0.265 (−1.029, 0.499) |
Selective attention trials | 19 | 0.029 (0.23) | 19 | −0.029 (0.23) | 0.042 (−0.0728, 0.813) |
Similarities | 20 | 0.737 (0.16) | 21 | −0.702 (0.15) | −1.281 (−1.790, −0.773) c |
Benton visual retention | 20 | 0.089 (0.23) | 21 | −0.084 (0.22) | 0.179 (−0.520, 0.878) |
Block design | 20 | 0.385 (0.21) | 21 | −0.366 (0.20) | −0.595 (−1.260, 0.071) d |
Santa Ana dominate hand | 20 | 0.059 (0.23) | 21 | −0.057 (0.22) | −0.006 (-0.755, 0.744) |
Santa Ana Non-dominate | 20 | −0.071 (0.23) | 21 | 0.068 (0.22) | 0.271 (−0.460. 1.002) |
Adjusted for age and education
p=0.005; Holm corrected p=0.087
p=0.046; Holm corrected p=0.779
p<0.001; Holm corrected p<0.001
p=0.078; Holm corrected p=1.000
Principal component analysis (PCA) was used to reduce the number of neurobehavioral outcome measures. It was applied to all participants having complete data on all 19 outcome measures, which amounted to 15 applicators and 15 non-applicators. The first six components had eigen values in excess of one and accounted for 76 % of the total variance in the 19 tests. Standardized and rotated loadings are shown in Table 4.
Table 4. Principal component analysis for the neurobehavioral outcome measures.
Neurobehavioral measure | PC1 | PC2 | PC3 | PC4 | PC5 | PC6 |
---|---|---|---|---|---|---|
Selective Att Latency | 0.5578 | 0.0109 | −0.0066 | −0.0449 | 0.0045 | −0.0113 |
Selective Att ISI | 0.5506 | 0.0594 | −0.0696 | 0.0096 | −0.0108 | −0.0113 |
Selective attention trials | 0.4949 | −0.0396 | 0.1075 | 0.0210 | 0.0356 | 0.0429 |
Trailmaking A | −0.0400 | 0.5322 | 0.0432 | 0.0159 | 0.0286 | 0.0479 |
Trailmaking B | 0.0484 | 0.4748 | −0.0719 | 0.0480 | −0.1941 | 0.0022 |
Serial digit learn | 0.0548 | 0.0353 | 0.5733 | −0.0609 | −0.0915 | −0.0462 |
Symbol-Digit latency | −0.0434 | −0.0989 | 0.5650 | 0.0681 | 0.0707 | −0.0426 |
Block design | 0.1384 | −0.2276 | −0.0037 | 0.6471 | −0.0099 | 0.0665 |
Santa Ana dominate | −0.1270 | 0.2444 | −0.0358 | 0.5178 | −0.0899 | −0.0197 |
Santa Ana NonDominate | −0.0604 | 0.2107 | 0.0053 | 0.4349 | 0.2299 | −0.2880 |
Finger tap alternating | 0.0943 | −0.0352 | 0.1245 | −0.0939 | 0.5666 | −0.1454 |
Finger tapping | 0.0941 | −0.0124 | −0.2274 | 0.0874 | 0.5381 | 0.0454 |
Reaction time | −0.2099 | −0.0570 | 0.0758 | −0.0079 | 0.4235 | 0.1627 |
Match sample score | −0.1070 | 0.0548 | 0.0113 | −0.0882 | 0.1265 | 0.5946 |
Similarities | 0.0124 | −0.2932 | 0.2244 | 0.2837 | −0.1089 | 0.4461 |
Match-sample latency | −0.0649 | −0.1005 | 0.1816 | 0.0257 | 0.0474 | −0.4534 |
Continuous performance dprime | 0.0838 | 0.2772 | −0.0069 | 0.0403 | 0.2160 | 0.2636 |
Benton visual retention | 0.0899 | 0.1652 | 0.1495 | 0.0184 | −0.0960 | 0.1591 |
Digit span | 0.0165 | 0.3337 | 0.3853 | −0.0565 | 0.0993 | 0.0360 |
Bolded items indicate principal component loadings
Regression analysis involving scores from the first six principal components identified two of the six that separated according to applicator/non-applicator status. Average score for applicators on PC3 was 1.14 SD (95 % CI: 0.04–2.24 SD) below the average score for non-applicators (p=0.044). Loadings for this component were seen to be essentially a weighted average (equal weights) of Serial Digit Learning and Symbol-Digit performance. One of the constituent tests (Symbol Digit) was found to be marginally associated with poorer performance in applicators if significance was assessed without regard to multiplicity. Average scores on PC6 for applicators was 1.84 SD (95 % CI: 1–2.69 SD; p<0.001) below the average score for non-applicators. This component was seen to be a contrast involving primarily three NB tests: a weighted average of Match-to-Sample Score and Similarities performance contrasted against Match-to-Sample Latency performance. Although Match-to-Sample Latencies had standardized scores originally reversed in sign (so higher numbers indicate better performance), this test was one of the four for which applicators outperformed non-applicators, which accounts for the sign reversal in PC6. The other two constituent tests (Match-to-Sample Score and Similarities) were the two separate tests that showed the largest difference between applicators and non-applicators. Remaining components were found not to separate according to group, although they may separate according to some secondary factor (e.g., age, education, TCPy) that was not of primary concern.
Correlation between NB outcomes and biomarkers
Partial correlation (controlling for differences due to group) between NB performance and (log-transformed) TCPy level as well as cholinesterase levels was computed for those five NB tests that had been identified through PCA (MTS score, Similarity, MTS latency, Serial Digit Learning, and Symbol Digit Latency). None of the partial correlations exceeded 0.30 in absolute value and most were less than 0.20; none was significant with all unadjusted p-values >0.10. There was no significant association between (log-transformed) TCPy level and either RBC AChE (rp=−0.12, p=0.45) or plasma BuChE (rp=−0.10, p=0.52).
Discussion
This study provides evidence of moderate exposure in adolescents occupationally exposed to chlorpyrifos. Urinary levels of TCPy, a metabolite specific to chlorpyrifos and chlorpyrifos methyl, were significantly higher in pesticide applicators compared to non-applicators. A comparison to TCPy levels in US adolescents (GM=2.09 for adolescents 12–19 years) from the National Health and Nutrition Examination Survey (Centers for Disease Control and Prevention 2009), which also collected spot urine samples, demonstrated higher levels in the Egyptian adolescents (Fig. 1). Furthermore, TCPy levels in the non-applicator participants were five times greater than US levels, indicating some level of environmental exposure. Additional research in this adolescent population provides further evidence of environmental exposure; an increase in TCPy levels, above baseline, was found in both applicator and non-applicator groups during the application season, with a significantly greater increase in the applicator population (Crane et al. 2013). Although elevated levels were found in the adolescent applicators compared to the non-applicators, they are similar to the levels reported in adult pesticide workers in Egypt in August 2009 (Ellison et al. 2012), but are markedly less than observed in adult applicators in 2008 (Farahat et al. 2011). The variability in occupational exposures observed in adult applicators in part reflects the year-to-year variation in the use of pesticides on cotton fields. Applicators were asked about their use of Personal Protective Equipment (PPE) during application. The majority (90 %) report wearing long pants and long sleeves during application, but approximately half of the applicators (48 %) report going barefoot when applying. None of the applicators reported wearing earplugs, gloves, safety glasses or goggles, respirators or other PPE.
Blood cholinesterase levels, a biomarker of effect (and exposure), reveal no significant difference between applicators and controls; although a greater number of applicators showed cholinesterase depression compared to standardized norms (EQM Research Inc.) than non-applicator participants. One limitation of the current study was that blood samples were collected at a single time point at the end of the application season. Because cholinesterase activity can vary on individual basis, a comparison should include a baseline measure for each participant (Stefanidou et al. 2003). Although previous studies have reported lower cholinesterase activity in pesticide applicators compared to controls (Abdel Rasoul et al. 2008; et al. 2003a), this study did not find significant differences. Several factors may be contributing to these outcomes, including, the small sample size, the single time point, which does not account for temporal variation and the potential environmental exposure of the participants. It is acknowledged that the study is limited by only providing urine TCPy levels (exposure biomarker) at one time point near the end of the summer application period (Aug 15). It would be useful to have urine TCPy data at baseline and throughout the application period to better estimate cumulative CP-exposure. On the other hand, the observed inhibition of blood AChE and BuChE activities at this time point does reflect the impact of cumulative OP-exposure over the summer application period on this sensitive biomarker of OP exposure/effect. AChE and BuChE results in Fig. 1 indicate that several applicators had a moderate to high degree of ChE inhibition, suggesting a moderate to high cumulative OP exposure over the summer application season.
A neurobehavioral test battery, consisting of 14 tests, was used to assess a broad range of functions. The inclusion of a range of tests results in numerous outcome measures and can lead us to risk spurious correlations. To avoid this, we used the Holm's correction (Holm 1979) to adjust for multiple comparisons. Although applicators performed worse than controls on the majority of neurobehavioral tests, only two test outcomes remained significant when the p-values were adjusted for multiple comparisons (Match-to-Sample Scores and Similarities). The small sample size (N=41) and moderate CPF exposures may have contributed to the lack of significant findings. Therefore, PCA was used to reduce the number of neurobehavioral outcome measures from 19 outcome measures to six PCA factors. Linear regression models with the PCA factors revealed two factors that distinguish between applicators and non-applicators. The first factor included the Serial Digit Learning and Symbol-Digit measures, the second Match-to-Sample and Similarities. These findings replicate previous work with adolescent agricultural workers (Abdel Rasoul et al. 2008; Eckerman et al. 2006; Rohlman et al. 2007).
Urinary TCPy levels in the non-applicator participants indicate the adolescents may have environmental CPF exposures. The study was conducted in an agricultural area where both applicators and non-applicators lived near agricultural fields. Approximately half of the study participants live within 25 m of agricultural fields (62 % of the applicators and 35 % of the non-applicators). In addition to applying pesticides to the cotton crop, 25 % of the applicators report applying pesticides at home; 24 % of the non-applicators also report applying pesticides at home. Additionally four of the applicators, although they do not report applying pesticides at home, work on their family's farm. Exposures may also occur due to consumption of local produce. One can say that both applicators and non-applicators have approximately similar environmental exposures and hence, the same basic levels of pesticides' residues levels in their bodies. So, this factor can explain the non-applicators' findings in this study and might not be a confounder. These environmental factors may have contributed to the elevated exposures seen in the non-applicator participants and may have potentially confounded the neurobehavioral outcomes. Although TCPy levels in applicators are elevated compared to the non-applicators and other adult applicator groups from the US (CDC 2009), their levels are similar to or less than reported in adult pesticide applicators in Egypt (Ellison et al. 2012; Farahat et al. 2010, 2011).
No relationship between biomarkers, TCPy or cholinesterase activity, was found. This is most likely due to several factors, including the relatively small sample size, the moderate level of CPF exposure, complexities in conducting longitudinal studies and the fact that most studies in human populations have not comprehensively assessed exposure, internal dose, and factors that potentially modify bioeffective dose. Although applicators show elevated urinary metabolite levels and decreased cholinesterase activity compared to non-applicator participants, their results are not linked to neurobehavioral outcomes. This finding is similar to other studies involving adult agricultural workers (Rohlman et al. 2011). However, the lack of power due to the small sample size and the use of spot urine samples may have led to imprecise measures of exposures. Additional work is needed to more fully understand the relationship between biomarkers of exposures and neurobehavioral performance.
The majority of studies examining the impact of pesticide exposure on the developing brain have focused on prenatal exposure or studies of children whose parents work in agriculture. There is limited research on the impact of occupational exposures in children. Many children throughout the world are engaged in agricultural work, either for pay or on family farms, and even subtle changes in health outcomes among this group of adolescent workers could result in large social and economic consequences. Despite regulations protecting younger workers, agriculture is an industry where these regulations are often more lenient for young workers. In the United States, for example, children working in agriculture can work at younger ages, including work in hazardous jobs, and there are no restrictions on the number of hours children can work on farms owned or operated by their parents (Rohlman et al. 2012). A survey of adolescent farmworkers from Mexico reported over 20 % mixing or applying pesticides (McCauley et al. 2002). Interventions targeting various stakeholders, including parents and employers, implementing integrated pest management approaches, and educating young workers are various ways to reduce exposure to pesticides (Rohlman et al. 2012). Thus, there remains a critical need to investigate the relative susceptibility of children and adolescents to pesticides, since the developing brain may be uniquely sensitive to the neurotoxic effects of these agents.
Acknowledgments
We thank the Egyptian Ministry of Agriculture and the adolescents for their participation, the Research Team at Menoufia University and Barbara McGarrigle for the urinary TCPy analytical work. The work was supported by the Fogarty International Center and the National Institute of Environmental Health Sciences (R21 ES017223 and R01 ES022163).
Footnotes
Conflict of interest: OHSU and Dr. Rohlman have a significant financial interest in Northwest Education Training and Assessment, LLC, a company that may have a commercial interest in the results of this research and technology. This potential conflict of interest was reviewed and a management plan approved by the OHSU Conflict of Interest in Research Committee was implemented.
Contributor Information
Diane Schertler Rohlman, Email: diane-rohlman@uiowa.edu, Occupational and Environmental Health, College of Public Health, University of Iowa, Iowa CIty, IA, USA; Center for Research on Occupational and Environmental Toxicology, Oregon Health and Science University, Portland, OR, USA.
Ahmed A. Ismail, Community Medicine and Public Health Department, Faculty of Medicine, Menoufia University, Shebin El-Kom, Egypt
Gaafar Abdel-Rasoul, Community Medicine and Public Health Department, Faculty of Medicine, Menoufia University, Shebin El-Kom, Egypt.
Michael Lasarev, Center for Research on Occupational and Environmental Toxicology, Oregon Health and Science University, Portland, OR, USA.
Olfat Hendy, Clinical Pathology and Hematology and Immunology, Menoufia University, Shebin El-Kom, Egypt.
James R. Olson, Department of Social and Preventative Medicine, State University of New York, Buffalo, NY, USA; Department of Pharmacology and Toxicology, State University of New York, Buffalo, NY, USA
References
- Abdel Rasoul GM, Abou Salem ME, Mechael AA, Hendy OM, Rohlman DS, Ismail AA. Effects of occupational exposure on children applying pesticides. NeuroToxicology. 2008;29:833–838. doi: 10.1016/j.neuro.2008.06.009. [DOI] [PubMed] [Google Scholar]
- Aitkin M, Anderson D, Francis B, Hinde J. Statistical modelling in GLIM. Oxford University Press; New York: 1989. [Google Scholar]
- Bellinger DC. Prenatal exposures to environmental chemicals and children's neurodevelopment: an update. Saf Health Work (SH@W) 2013;4(1):1–11. doi: 10.5491/SHAW.2013.4.1.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bouchard MF, Bellinger DC, Wright RO, Weisskopf MG. Attention-deficit/hyperactivity disorder and urinary metabolites of organophosphate pesticides. Pediatrics. 2010;125:1270–1277. doi: 10.1542/peds.2009-3058. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bouchard MF, Chevrier J, Harley KG, Kogut K, Vedar M, Calderon N, Trujillo C, Johnson C, Bradman A, Barr DB, Eskenazi B. Prenatal exposure to organophosphate pesticides and IQ in 7-year-old children. Environ Health Perspect. 2011;119(8):1189–1195. doi: 10.1289/ehp.1003185. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bushnell PJ, Moser VC. Behavioral toxicity of cholinesterase inhibitors. In: Gupta RC, editor. Toxicology of organophosphate and carbamate compounds. Elsevier; San Diego: 2006. pp. 347–360. [Google Scholar]
- Centers for Disease Control and Prevention. Fourth report on human exposure to environmental chemicals, 2009. U.S. Department of Health and Human Services. Centers for Disease Control and Prevention; 2009. [Google Scholar]
- Colosio C, Tiramani M, Brambilla G, Columbi A, Moretto A. Neurobehavioral effects of pesticides with special focus on organophosphorus compounds: which is the real size of the problem? Neurotoxicology. 2009;30(6):1155–1161. doi: 10.1016/j.neuro.2009.09.001. [DOI] [PubMed] [Google Scholar]
- Connors SL, Levitt P, Matthews SG, Slotkin TA, Johnston MV, Kinney HC, Johnson WG, Dailey RM, Zimmerman AW. Fetal mechanisms in neurodevelomental disorders. Pediatr Neurol. 2008;38:163–176. doi: 10.1016/j.pediatrneurol.2007.10.009. [DOI] [PubMed] [Google Scholar]
- Costa LG. Current issues in organophosphate toxicology. Clin Chim Acta. 2006;366(1–2):1–13. doi: 10.1016/j.cca.2005.10.008. [DOI] [PubMed] [Google Scholar]
- Costa LG, Aschner M, Vitalone A, Syverson T, Soldin OP. Developmental neuropathology of environmental agents. Annu Rev Pharmacol Toxicol. 2004;44:87–110. doi: 10.1146/annurev.pharmtox.44.101802.121424. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Crane AL, Abdel Rasoul G, Ismail AA, Hendy OM, Bonner MR, Lasarev MR, Al-Batanony M, Singleton ST, Khan K, Olson JR, Rohlman DS. Longitudinal assessment of chlorpyrifos exposure and effect biomarkers in adolescent Egyptian agricultural workers. J Expo Sci Environ Epidemiol. 2013;23(4):356–362. doi: 10.1038/jes.2012.113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dam K, Seidler FJ, Slotkin TA. Chlorpyrifos exposure during a critical neonatal period elicits gender-selective deficits in the development of coordination skills and locomotor activity. Dev Brain Res. 2000;121:179–187. doi: 10.1016/s0165-3806(00)00044-4. [DOI] [PubMed] [Google Scholar]
- Eaton DL, Daroff RB, Autrup H, Bridges J, Buffler P, Costa LG, et al. Review of the toxicology of chlorpyrifos with an emphasis on human exposure and neurodevelopment. Crit Rev Toxicol. 2008;38(2):1–125. doi: 10.1080/10408440802272158. [DOI] [PubMed] [Google Scholar]
- Eckerman DA, Gimenes LS, Curi de Souza R, Galvao PL, Sarcinelli PN, Chrisman JR. Age related effects of pesticide exposure on neurobehavioral performance of adolescent farmworkers in Brazil. Neurotoxicol Teratol. 2006;29:164–175. doi: 10.1016/j.ntt.2006.09.028. [DOI] [PubMed] [Google Scholar]
- Ellison CA, Crane AL, Bonner MR, Knaak JB, Browne RW, Lein PJ, Olson JR. PON1 status does not influence cholinesterase activity in Egyptian agricultural workers exposed to chlorpyrifos. Toxicol Appl Pharmacol. 2012;265:308–315. doi: 10.1016/j.taap.2012.08.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Engel SM, Wetmur J, Chen J, Zhu C, Barr DB, Canfield RL, Wolff MS. Prenatal exposure to organophosphates, paraoxonase 1, and cognitive development in childhood. Environ Health Perspect. 2011;119(8):1182–1188. doi: 10.1289/ehp.1003183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- EPA. Interim reregistration eligibility decision for chlorpyrifos. Environmental Protection Agency 2002 [Google Scholar]
- Eskenazi B, Huen K, Marks AR, Harley KG, Bradman A, Barr DB, Holland N. PON1 and neurodevelopment in children from the CHAMACOS study exposed to organophosphate pesticides in Utero. Environ Health Perspect. 2010;118(12):1775–1781. doi: 10.1289/ehp.1002234. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fabiny DL, Ertingshausen G. Automated reaction-rate method for determination of serum creatinine with the CentrifiChem. Clin Chem. 1971;17(8):696–700. [PubMed] [Google Scholar]
- Farahat TM, Abdel Rasoul GM, Amr MM, Shebl MM, Farahat FM, Anger WK. Neurobehavioral effects among workers occupationally exposed to organophosphorus pesticides. Occup Environ Med. 2003a;60:279–286. doi: 10.1136/oem.60.4.279. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Farahat TM, Abdelrasoul GM, Amr MM, Shebl MM, Farahat FM, Anger WK. Neurobehavioural effects among workers occupationally exposed to organophosphorous pesticides. Occup Environ Med. 2003b;60(4):279–286. doi: 10.1136/oem.60.4.279. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Farahat FM, Fenske RA, Olson JR, Galvin K, Bonner MR, Rohlman DS, Farahat TM, Lein PJ, Anger WK. Chlorpyrifos exposures in Egyption cotton field workers. NeuroToxicology. 2010;31(3):297–304. doi: 10.1016/j.neuro.2010.02.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Farahat FM, Ellison CA, Bonner MR, McGarrigle BP, Crane AL, Fenske RA, Lasarev MR, Rohlman DS, Anger WK, Lein PJ, Olson JR. Biomarkers of chlorpyrifos exposure and effect in Egyptian cotton field workers. Environ Health Perspect. 2011;119(6):801–806. doi: 10.1289/ehp.1002873. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Godfrey KM, Barker DJ. Fetal programming and adult health. Public Health Nutr. 2001;4:611–624. doi: 10.1079/phn2001145. [DOI] [PubMed] [Google Scholar]
- Grandjean P, Harari R, Barr DB, Debes F. Pesticide exposure and stunting as independent predictors of neurobehavioral deficits in Ecuadorian school children. Pediatrics. 2006;117(3):e546–e556. doi: 10.1542/peds.2005-1781. [DOI] [PubMed] [Google Scholar]
- Holm S. A simple sequentially rejective Bonferroni test procedure. Scand J Statist. 1979;6:65–70. [Google Scholar]
- Ismail AA, Rohlman DS, Abou Salem ME, Mechael AA, Hendy OM, Abdel Rasoul GM. Neurobehavioral effects among adolescent pesticide applicators in Egypt. Paper presented at the 24th International Neurotoxicology Conference; San Antonio, TX. 2007. [Google Scholar]
- Ismail AA, Rohlman DS. The test-retest reliability of the behavioral assessment and research system in arabic speaking adults. 66th Annual Meeting of the Oregon Academy of Science; Portland, OR. 2008. [Google Scholar]
- Ismail AA, Rohlman DS, Abdel Rasoul GM, Abou Salem ME, Hendy OM. Clinical and biochemical parameters of children and adolescents applying pesticides. Int J Occup Environ Med. 2010;1(3):132–143. [PubMed] [Google Scholar]
- Ismail AA, Bodner TE, Rohlman DS. Neurobehavioral performance among agricultural workers and pesticide applicators: a meta-analytic study. Occup Environ Med. 2012;69(7):457–464. doi: 10.1136/oemed-2011-100204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson DE, Seidler FJ, Slotkiin TA. Early biochemical detection of delayed neurotoxicity resulting from developmental exposure to chlorpyrifos. Brain Res Bull. 1998;45(2):143–147. doi: 10.1016/s0361-9230(97)00329-8. [DOI] [PubMed] [Google Scholar]
- Kalia M. Brain development: anatomy, connectivity, adaptive plasticity, and toxicity. Metab Clin Exp. 2008;57(Suppl 2):S2–S5. doi: 10.1016/j.metabol.2008.07.009. [DOI] [PubMed] [Google Scholar]
- Landrigan PJ, Kimmel CA, Correa A, Eskenazi B. Children's health and the environment: public health issues and challenges for risk assessment. Environ Health Perspect. 2004;112(2):257–265. doi: 10.1289/ehp.6115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lezak MD. Neuropsychological assessment. 3rd. Oxford University Press; New York: 1995. [Google Scholar]
- Mansour SA. Pesticide exposure—Egyptian scene. Toxicology. 2004;198:91–115. doi: 10.1016/j.tox.2004.01.036. [DOI] [PubMed] [Google Scholar]
- Marks AR, Harley K, Bradman A, Kogut K, Barr DB, Johnson C, Calderon N, Eskenazi B. Organophosphate pesticide exposure and attention in young Mexican-American children: the CHAMACOS study. Environ Health Perspect. 2010;118:1768–1774. doi: 10.1289/ehp.1002056. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McCauley LA, Sticker D, Bryan C, Lasarev MR, Scherer JA. Pesticide knowledge and risk perception among adolescent Latino farmworkers. J Agric Saf Health. 2002;8(4):397–409. doi: 10.13031/2013.10220. [DOI] [PubMed] [Google Scholar]
- Mosallem F. Working children and their families research assessment draft 2011 [Google Scholar]
- Moser VC, Padilla S. Age- and gender-related differences in the time course of behavioral and biochemical effects produced by oral chlorpyrifos in rats. Toxicol Appl Pharmacol. 1998;149(1):107–119. doi: 10.1006/taap.1997.8354. [DOI] [PubMed] [Google Scholar]
- National Academy of Sciences. Pesticides in the diets of infants and children. National Academy Press; Washington, DC: 1993. [PubMed] [Google Scholar]
- Rauh VA, Garfinkel R, Perera FP, Andrews H, Hoepner L, Barr DB, Whitehead R, Tang D, Whyatt RW. Impact of prenatal chlorpyrifos exposure on neurodevelopment in the first 3 years of life among inner-city children. Pediatrics. 2006;118(6):e1845–e1859. doi: 10.1542/peds.2006-0338. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rauh VA, Arunajadai S, Horton M, Perera FP, Hoepner LA, Barr DB, Whyatt RM. Seven-year neurodevelopmental scores and prenatal exposure to chlorpyrifos, a common agricultural pesticide. Environ Health Perspect. 2011;119(8):1196–1201. doi: 10.1289/ehp.1003160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reuhl KR. Delayed expression of neurotoxicity: the problem of silent damage. NeuroToxicology. 1991;12:341–346. [PubMed] [Google Scholar]
- Rohlman DS, Bailey SR, Anger WK, McCauley L. Computerized tests to assess neurobehavioral function in Hispanic adolescents working in agriculture. Oregon Academy of Science; Salem: 1999. [DOI] [PubMed] [Google Scholar]
- Rohlman DS, Bailey SR, Anger WK, McCauley L. Assessment of neurobehavioral function with computerized tests in a population of Hispanic adolescents working in agriculture. Environ Res. 2001;85:14–24. doi: 10.1006/enrs.2000.4105. [DOI] [PubMed] [Google Scholar]
- Rohlman DS, Arcury TA, Quandt SA, Lasarev M, Rothlein J, Travers R, Tamulinas A, Scherer J, Early J, Marin A, Phillips J, McCauley L. Neurobehavioral performance in preschool children from agricultural and non-agricultural communities in Oregon and North Carolina. NeuroToxicology. 2005;26:589–598. doi: 10.1016/j.neuro.2004.12.002. [DOI] [PubMed] [Google Scholar]
- Rohlman DS, Anger WK, Lasarev M, Scherer J, Stupfel J, McCauley L. Neurobehavioral performance of adult and adolescent farmworkers. Neurotoxicology. 2007;28:374–380. doi: 10.1016/j.neuro.2006.10.006. [DOI] [PubMed] [Google Scholar]
- Rohlman DS, Anger WK, Lein PJ. Correlating neurobehavioral performance with biomarkers of organophosphorous pesticide exposure. NeuroToxicology. 2011;32(2):268–276. doi: 10.1016/j.neuro.2010.12.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rohlman DS, Nuwayhid I, Ismail A, Saddik B. Using epidemiology and neurotoxicology to reduce risks to young workers. Neurotoxicology. 2012;33(4):817–822. doi: 10.1016/j.neuro.2012.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rothlein J, Rohlman D, Lasarev M, Phillips J, Muniz J, McCauley L. Organophosphate pesticide exposure and neurobehavioral performance in agricultural and non-agricultural Hispanic workers. Environ Health Perspect. 2006;114:691–696. doi: 10.1289/ehp.8182. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ruckart PZ, Kakolewski K, Bove FJ, Kaye WE. Long-term neurobehavioral health effects of methyl parathion exposure in children in Mississippi and Ohio. Environ Health Perspect. 2004;112(1):46–51. doi: 10.1289/ehp.6430. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Slotkin TA. Developmental neurotoxicity of organophosphates: a case study of chlorpyrifos. In: Gupta RC, editor. Toxicity of Organophosphate and Carbamate PEsticides. Elsevier Academic Press; San Diego: 2005. pp. 293–314. [Google Scholar]
- Spear LP. Alcohol's effect on adolescents. Alcohol Res Health. 2002;26:287–291. [PMC free article] [PubMed] [Google Scholar]
- Spear LP. The behavioral neuroscience of adolesence. W.W Norton & Company; New York: 2010. [Google Scholar]
- Stefanidou M, Athanaselis S, Velonakis M, Pappas F, Koutselinis A. Occupational exposure to cholinesterase inhibiting pesticides: a Greek case. Int J Environ Health Res. 2003;13(1):23–29. doi: 10.1080/0960312021000063287. [DOI] [PubMed] [Google Scholar]
- Steinberg L. A social neuroscience perspective on adolescent risk-taking. Dev Rev. 2008;28:78–106. doi: 10.1016/j.dr.2007.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]