Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Mar 19.
Published in final edited form as: Neurotoxicol Teratol. 2011 Aug 28;34(1):168–176. doi: 10.1016/j.ntt.2011.08.014

Central nervous system function and organophosphate insecticide use among pesticide applicators in the Agricultural Health Study

Sarah E Starks 1, Fred Gerr 1, Freya Kamel 2, Charles F Lynch 3, Michael P Jones 4, Michael C Alavanja 5, Dale P Sandler 2, Jane A Hoppin 2
PMCID: PMC3601543  NIHMSID: NIHMS329241  PMID: 21907279

Abstract

Acute organophosphate (OP) pesticide exposure is associated with adverse central nervous system (CNS) outcomes, however, little is known about the neurotoxicity of chronic exposures that do not result in acute poisoning. To examine associations between long-term pesticide use and CNS function, neurobehavioral (NB) tests were administered to licensed pesticide applicators enrolled in the Agricultural Health Study (AHS) in Iowa and North Carolina. Between 2006 and 2008, 701 male participants completed nine NB tests to assess memory, motor speed and coordination, sustained attention, verbal learning and visual scanning and processing. Data on ever-use and lifetime days of use of 16 OP pesticides were obtained from AHS interviews conducted before testing between 1993 and 2007 and during the NB visit. The mean age of participants was 61 years (SD = 12). Associations between pesticide use and NB test performance were estimated with linear regression controlling for age and outcome-specific covariates. NB test performance was associated with lifetime days of use of some pesticides. Ethoprop was significantly associated with reduced performance on a test of motor speed and visual scanning. Malathion was significantly associated with poor performance on a test of visual scanning and processing. Conversely, we observed significantly better test performance for five OP pesticides. Specifically, chlorpyrifos, coumaphos, parathion, phorate, and tetrachlorvinphos were associated with better verbal learning and memory; coumaphos was associated with better performance on a test of motor speed and visual scanning; and parathion was associated with better performance on a test of sustained attention. Several associations varied by state. Overall, our results do not provide strong evidence that long-term OP pesticide use is associated with adverse CNS-associated NB test performance among this older sample of pesticide applicators. Potential reasons for these mostly null associations include a true absence of effect as well as possible selective participation by healthier applicators.

Keywords: agricultural workers, epidemiology, organophosphates, neuropsychological testing, pesticide exposure

1. Introduction

Organophosphate (OP) pesticides are widely used in the United States and internationally to protect crops from insect damage. OP pesticides accounted for approximately 35% of all insecticide pesticides used in the U.S. in 2007 with over 33 million pounds used annually [7]. Exposure is common among agricultural workers as well as the general population.

The acute nervous system toxicity of OP pesticides is well described and results from inhibition of the enzyme acetylcholinesterase (AChE) [4]. Long-term exposure to low or moderate levels of OP pesticides does not cause overt cholinergic toxicity [19,24]. However, evidence for an association between long-term low or moderate exposure to OPs and impaired neurobehavioral (NB) function or other neurological effects is inconsistent [5,8,12,13,21-23,25]. The heterogeneity of findings reported in the literature may be due to a number of methodological limitations including small sample size [5,9,28], use of poor or inaccurate exposure estimates [8,13,18,22,23], referent groups that may have differed from the exposure group on characteristics other than exposure (e.g., sheep dippers versus ceramic workers) [5,8,9,21,28], and inadequate control for potential confounding variables, such as previous pesticide poisoning [3,5,8,21].

The purpose of this investigation was to examine associations between OP pesticide use and measures of central nervous system (CNS) function in a large cohort of pesticide applicators with well characterized pesticide use histories. The primary hypothesis we tested was whether long-term OP pesticide use was associated with adverse NB outcomes.

2. Materials and Methods

2.1. Study participants

This study was conducted among private pesticide applicators enrolled in the Agricultural Health Study (AHS). The AHS is a large prospective study of private pesticide applicators, their spouses, and commercial pesticide applicators from Iowa and North Carolina. Details have been described elsewhere [2]. Briefly, between 1993 and 1997, participants completed the AHS Enrollment questionnaire at the time of pesticide licensing and recertification. Forty-percent of applicators also completed a “take-home” questionnaire within one month of enrollment. Detailed information on some OPs was collected on this second questionnaire, thus, only individuals who completed both questionnaires at enrollment were eligible. These two questionnaires comprised Phase 1 data collection. After enrollment, a five-year and a ten-year follow-up telephone interview were administered (Phase 2 and 3 data collection).

Male participants who completed all AHS interviews, resided in Iowa or North Carolina, and lived within approximately 150 miles of the testing facilities were invited to participate in the present study. Participants were not eligible if they had medical conditions that could influence central or peripheral nervous system testing results (i.e., amyotrophic lateral sclerosis, multiple sclerosis, Parkinson’s disease, retinal or macular degeneration, stroke, hypothyroidism and treatment for diabetes), reported drinking ≥42 alcoholic beverages per week, or reported a history of pesticide poisoning at the most recent interview (Phase 3). Initially, 1,807 male AHS participants were eligible to participate in the present study after these criteria were applied.

To enrich the sample for applicators with higher lifetime use of OP pesticides, we oversampled the high end of the OP lifetime use distribution based on the lifetime days of use 10 OPs assessed in detail in Phase 1. Among eligible participants, a stratified sample was selected based on equal sampling from the upper and lower portion of the OP lifetime days distribution. In Iowa, a cutpoint of ~75% was used to separate individuals; in NC the cutpoint was lower (66%) because the NC cohort is more geographically dispersed and has fewer members. Individuals were randomly selected from within the high and low groups. While the cutpoint was shifted for selection, all analyses were done based on lifetime use of pesticides which included data from all AHS phases as well as the NB appointment. Thus the sampling frame allowed us to have an enriched sample for OP use, but was not used as an analytical variable. In Iowa, testing was conducted in Iowa City and Dubuque between November 2006 and March 2007. In North Carolina, testing was conducted in Greenville and Wilmington between January and March 2008. Participants were reimbursed for time and travel expenses. Appropriate Institutional Review Boards approved the study protocol, and all participants provided written informed consent for the present study.

2.2. Exposure assessment

Information on pesticide exposure history, farm practices, medical history, and demographic factors was obtained from the questionnaires used during all three AHS phases as well as a NB study questionnaire administered at the time of testing. Copies of AHS questionnaires are available online [1].

Use of specific pesticides was quantified for each participant using information from AHS questionnaires and the NB study questionnaire. In each phase of the AHS, pesticide use was assessed in slightly different ways, and we integrated these data from all phases to create variables on ever use and lifetime days of use. In Phase 1 questionnaires, participants were queried in detail about 50 specific pesticides and asked to provide information on ever use, frequency of use and years of use. Additionally, on the “take-home” questionnaire, participants were asked to complete a checklist indicating ever use of specific chemicals, but were not asked to provide information on frequency or duration of use. In Phases 2 and 3, participants provided open-ended responses regarding pesticide use since last interview, and this information was used to create lifetime use for those time periods. The NB study questionnaire collected pesticide use information for the past 12 months including ever mix or apply and duration of use (days).

We evaluated the 16 OP pesticides used by at least 50 NB participants. Nine of these were reported in detail on the AHS enrollment questionnaire (chlorpyrifos, coumaphos, diazinon, dichlorvos, fonofos, malathion, parathion, phorate, and terbufos); six were initially queried on the take-home questionnaire checklist (acephate, dimethoate, disulfoton, ethoprop, phosmet, and tetrachlorvinphos); and a new chemical introduced in 1995 (tebupirimfos) was reported initially on the Phase 2 questionnaire. Using information from all AHS interviews and the NB study questionnaire, we created two lifetime exposure metrics: 1) ever use – based on any positive report at any interview and 2) lifetime days of use based on the sum of lifetime days of use reported at each interview. For the Phase 1 lifetime days summary measure, we multiplied the number of days used per year by the numbers of years used. As chemicals on the “take-home” checklist did not have this information, we assumed i) that the days of use per year was equal to the median number of days of insecticide use per year for that individual and ii) that the number of years used was equal to the median number of years that individual had applied insecticides using categories comparable to the Phase 1 questionnaire. For Phases 2 and 3 as well as the NB interview information, we multiplied the number of days used per year by the number of years since last interview to create the lifetime days accrued during that period. We then summed the lifetime days for Phases 1, 2, 3 and the NB interview to create the cumulative lifetime days of use each pesticide. All pesticide use occurred prior to NB testing. A summary measure of OP pesticide use (cumulative lifetime days of all OP pesticides) was also created. In addition to variables on OP use, we also created similar variables for four carbamate pesticides (aldicarb, benomyl, carbaryl and carbofuran). An overall measure of cumulative lifetime pesticide use was estimated for the 50 pesticides assessed in detail during Phase 1. This variable integrated data from all AHS interviews.

2.3. Neurobehavioral testing

NB tests were administered by trained technicians unaware of the participants’ exposure status [27]. Eight computerized tests from the Neurobehavioral Evaluation System, Version 3 (NES3), [15-17] and the manual Grooved Pegboard test (Lafayette Instruments, Lafayette, IN) [14] were administered. These tests have been used extensively in investigations of neurotoxicants and were selected to be sensitive indicators of a wide range of CNS functions. A description of the nine NB tests is presented in Table 1.

Table 1.

Neurobehavioral tests of central nervous system (CNS) function administered to pesticide applicators.

Test Function assessed Units
Continuous performancea Sustained attention m/s
Digit-symbola Visual scanning and processing seconds
Finger tapping, dominant hand Motor speed avg # of taps (4, 10-second trials)
Grooved pegboard, dominant handa Fine motor coordination seconds
Auditory verbal learning total recall Verbal learning and memory # correct (0-36)
Auditory verbal learning delayed recall Memory # correct (0-12)
Auditory verbal learning recognition Memory true positives - false positives (−12−12)
Sequences A latencya Motor speed and scanning seconds
Sequences B latencya Motor speed and scanning seconds
a

Higher score indicate poorer performance.

2.4. Assessment of potential confounders

To construct outcome-specific models, we evaluated a common set of covariates potentially associated with neurobehavioral outcomes. We selected the following questionnaire covariates as potential confounders of the NB outcomes: age, height, education, state, smoking status, alcohol consumption, head injury, current antidepressant use, caffeine consumption and exposure to other potentially neurotoxic substances such as organic solvents, soldering and welding fumes. Additionally, we administered tests of reading ability, affect, and visual acuity to control for these potential confounders as well. NES3 includes an Adult Reading Test (ART) to estimate intellectual functioning and the Positive and Negative Affect Schedule (PANAS) to measure positive and negative affect [30]. We measured visual acuity using the Optec 1000 (Stereo Optical Co, Chicago, IL).

2.5. Statistical methods

2.5.1. Linear regression analyses

We used a backward elimination procedure to create separate base models for each NB outcome measure with outcome-specific covariates. First, we examined the unadjusted association between each covariate and each outcome with linear regression. Covariates associated with a NB outcome in unadjusted linear regression models with a p-value <0.20 were selected for inclusion in an initial full multiple linear regression base model for that outcome. Covariates with p-values >0.20 were then removed sequentially from the initial full base model. The final multivariate base model for each NB outcome included only those covariates with p-values <0.20. All final multivariate base models included age (years) and ART score. In addition, the Continuous Performance Test final multivariate base model included positive affect (PA) score and caffeine consumption; the Digit-Symbol final multivariate base model included PA score, education, state, and visual acuity score; the Finger Tapping final multivariate base model included PA score and state; the Grooved Pegboard final multivariate base model included caffeine consumption, state and visual acuity score; the AVLT Total Recall final multivariate base model included negative affect (NA) score, PA score, and education; the AVLT Delayed Recall final multivariate base model included NA score, PA score, education and state; the AVLT Recognition final multivariate base model included PA score, education and state; and the Sequences A and B final multivariate base model included PA score and state.

For each NB outcome, we excluded subjects who had studentized residual values that exceeded the absolute value of 4.0. For Digit-Symbol, two subjects were excluded; for Sequences A one subject was excluded; and for Sequences B one subject was excluded.

Each pesticide was examined both as a continuous variable (cumulative lifetime days of use) and as a dichotomized variable (ever/never use). The lifetime days of pesticide use variables were log10-transformed to normalize the distribution of residuals. Adjusted associations between NB outcomes and pesticide exposures were estimated with linear regression models in which the NB outcome was regressed on the pesticide exposure variable while controlling for the covariates included in the base model. Parameter estimates for the timed tests (Continuous Performance Test, Digit-Symbol, Grooved Pegboard, Sequences A and B) were multiplied by −1 so that lower scores indicated poorer test performance for all NB outcomes. In addition, pesticide use by state was examined with the inclusion of a state by pesticide interaction term.

2.5.2. Confounding by related pesticide exposures

Pesticide applicators typically use more than one pesticide. Potential confounding of the association between NB outcomes and each pesticide by other pesticides was examined. Specifically, Spearman correlations were calculated for pesticides associated with NB outcomes with a p-value <0.10. Moderately correlated pesticide pairs (r ≥0.30) were added simultaneously to final base models and the pesticide variable parameter estimates were compared to models with only one pesticide.

2.5.3. Sensitivity analyses

For our primary analysis, we analyzed data from all participants. To assess the robustness of our results, we conducted two analyses where we excluded: 1) individuals with medical conditions that may affect NB outcomes, and 2) individuals with a history of physician-diagnosed pesticide poisoning. For the first sensitivity analysis, we excluded participants who reported use of specific medications (i.e., benzodiazepines (n=18), opiates (n=12), anticonvulsants (n=3), barbiturates (n=2), antipsychotics (n=3), and donepezil (n=1)) or medical conditions not reported on the Phase 3 interview (i.e., a history of alcoholism (n=6), brain tumor (n=5), alcohol use on day of testing (n=3), struck by lightning (n=1), renal failure (n=1), macular degeneration (n=1), and severe dementia (n=1)). Parameter estimates from models excluding these individuals were then compared to estimates from models that included these individuals. A similar strategy was employed to assess whether excluding individuals with a history of diagnosed pesticide poisoning at enrollment (n=8) influenced our results.

We used the P1RE1071201, P2RE1071202 and 07222008 releases of the AHS dataset. All analyses were performed using SAS software, version 9.2 (SAS Institute Inc., Cary, NC).

3. Results

3.1. Participation

NB testing was administered to 701 participants from the 1,807 eligible AHS participants. The overall participation rate was 39 percent. Participants were similar to non-participants with respect to age and pesticide use history (data not shown).

3.2. Characteristics of the study participants

3.2.1. Demographics

Descriptive statistics for potential confounding demographic characteristics, personal health information and chemical exposures are presented in Table 2. Among the 701 participants included in the analyses, 51% were from Iowa and 49% were from North Carolina. The mean age of the participants was 61 years (SD = 12) and approximately half reported completing at least a high school education. Over 20% of the participants reported a past head injury with or without loss of consciousness and eight (1%) reported a previous physician-diagnosed pesticide poisoning.

Table 2.

Demographic characteristics, personal health information and Chemical exposure for the 701 Agricultural Health Study male pesticide applicators.

Characteristic Mean SD No. %
Age (yrs) 61 12 -- --
Height (cm) 179 6 -- --
Adult Reading Test (0-60) 30 10 -- --
Positive affect (1-5) 3.5 0.7 -- --
Negative affect (1-5) 1.4 0.4 -- --
Testing location
 Iowa -- -- 356 51
 North Carolina -- -- 345 49
Education
 ≤ High school -- -- 355 51
 > High school -- -- 346 49
Smoking status
 Never smoked -- -- 403 57
 Current smoker -- -- 47 7
 Past smoker -- -- 251 36
Alcohol consumption (drinks/wk)
 0 drinks -- -- 401 57
 1-7 drinks -- -- 231 33
 >7 drinks -- -- 69 10
Visual acuity
 20/20 - 20/40 -- -- 592 84
 20/50 - 20/200 -- -- 109 16
Head injury
 No injury -- -- 536 76
 Injury, no loss of consciousness -- -- 71 10
 Injury, w/loss of consciousness -- -- 94 13
Antidepressants (current use) -- -- 51 7
Caffeine use (drink regularly) -- -- 525 75
Solvent exposure (ever) -- -- 288 41
Soldering exposure (ever) -- -- 36 5
Pesticide poisoning (ever) -- -- 8 1

3.2.2. Pesticide exposures

Frequencies of use of specific pesticides and means of cumulative lifetime days for the 16 OPs, four carbamates and two pesticide summary variables are presented in Table 3. Ever use of specific OPs ranged from 77% for malathion to ≤10% for dimethoate, tebupirimfos and tetrachlorvinphos. Carbaryl (63%) was the most commonly used carbamate pesticide. Most participants reported ever using any OP (97%) and all but one participant reported using at least one pesticide in their lifetimes. Lifetime days of all OP pesticides and lifetime days of all pesticides were similar between Iowa and North Carolina participants (data not shown). Carbamate pesticide use, however, was more prevalent in North Carolina (93%) than in Iowa (67%).

Table 3.

Frequencies and means of cumulative lifetime days of pesticide use for 701 male pesticide applicators.

Pesticides Na % Mean SD Min. Median Max.
Organophosphates
 Acephate 166 24 85 90 3 56 501
 Chlorpyrifos 418 60 75 103 2 39 767
 Coumaphos 94 13 74 245 1 12 1,683
 Diazinon 302 43 55 93 1 20 846
 Dichlorvos 128 18 443 1,064 1 58 8,680
 Dimethoate 66 9 46 68 2 25 457
 Disulfoton 110 16 43 42 2 25 236
 Ethoprop 121 17 45 50 3 25 316
 Fonofos 201 29 64 84 2 39 457
 Malathion 541 77 99 201 2 37 2,625
 Parathion 147 21 103 272 1 20 1,668
 Phorate 230 33 70 130 1 25 1,628
 Phosmet 101 14 61 83 3 26 600
 Tebupirimfos 69 10 51 46 4 40 250
 Terbufos 356 51 101 116 2 56 752
 Tetrachlorvinphos 69 10 65 99 3 25 582
Carbamates
 Aldicarb 131 19 86 118 2 29 742
 Benomyl 116 17 62 122 1 13 767
 Carbaryl 440 63 103 153 1 47 1,388
 Carbofuran 290 41 55 92 1 25 752
Summary variables
 All organophosphates 682 97 420 663 2 241 8,763
 All pesticides 700 100 1,604 1,617 10 1,045 11,677
a

Number of participants who reported ever use.

3.3. Associations between OP pesticide use and neurobehavioral outcomes

Descriptive statistics for the NB test results are presented in Appendix A. Because a small number of study participants were unable to complete individual tests in the allowed time or after two attempts, the total number of participants completing each test varied slightly. Specific pesticides were associated with some NB tests. Three of the nine NB outcomes we examined had at least one significant adverse association with ever-use (Table 4) or lifetime days of pesticide use (Table 5). Lifetime days of all OP use was not significantly associated in either direction with any NB test. Ever-use of ethoprop and lifetime days of malathion use were both significantly associated with poorer performance on the Digit-Symbol test. Ever-use of disulfoton, ethoprop and terbufos were significantly associated with poorer performance on the Sequences A test. Conversely, six of the nine outcomes had significantly better test performance with ever-use or lifetime days of use. Better test performance among was observed more frequently for the three Auditory Verbal Learning tests (Total Recall, Delayed Recall and Recognition). For several NB outcomes, we observed a significant state by pesticide interaction, suggesting differential effects for chlorpyrifos, coumaphos, and malathion in North Carolina and Iowa (Table 6).

Table 4.

Regression coefficients from linear regression models for neurobehavioral outcomes and ever used pesticides for the 701 male pesticide applicators.

Pesticide CPTa
(n=692)
Digit-Symbola
(n=691)
Finger
Tapping,
dominant
(n=695)
Grooved
Pegboard,
dominanta
(n=699)
AVLT Total
Recall
(n=696)
AVLT Delayed
Recall
(n=695)
AVLT
Recognition
(n=694)
Sequences Aa
(n=679)
Sequences Ba
(n=671)

β SE β SE β SE β SE β SE β SE β SE β SE β SE

Organophosphates
 Acephate 1.68 3.55 −2.12 1.83 0.22 0.97 −0.04 2.12 −0.04 0.39 0.22 0.27 −0.02 0.25 −2.07 1.25 −1.36 1.83
 Chlorpyrifos −1.57 3.12 int -- −0.64 0.70 3.47 * 1.53 0.54 0.34 0.37 0.19 0.17 0.18 −1.60 0.89 −0.55 1.30
 Coumaphos 0.92 4.39 2.30 1.87 1.24 1.00 1.82 2.19 1.25 * 0.48 0.50 0.27 0.40 0.26 0.75 1.26 2.93 1.82
 Diazinon −1.61 3.07 0.37 1.32 0.71 0.70 0.60 1.54 0.16 0.33 0.19 0.19 −0.03 0.18 −0.74 0.89 0.78 1.30
 Dichlorvos −0.46 3.90 −0.47 1.76 0.30 0.94 0.97 2.05 0.51 0.43 0.18 0.26 0.17 0.24 1.53 1.18 1.34 1.72
 Dimethoate 2.00 5.13 −1.58 2.17 −1.06 1.15 −0.71 2.53 0.84 0.56 0.21 0.32 0.55 0.30 1.98 1.45 1.42 2.11
 Disulfoton 3.73 4.18 −1.97 1.93 −1.20 1.03 −2.08 2.24 −0.18 0.46 0.37 0.28 −0.30 0.27 −2.55 * 1.31 0.72 1.93
 Ethoprop 1.51 3.99 −3.66 * 1.84 0.33 0.99 −1.33 2.15 −0.86 * 0.44 −0.23 0.27 −0.43 0.25 −3.27 ** 1.25 −1.46 1.85
 Fonofos 1.70 3.33 0.90 1.56 0.24 0.83 0.57 1.82 0.36 0.36 0.34 0.23 0.03 0.22 −0.73 1.05 −0.29 1.53
 Malathion −6.13 3.62 −2.68 1.53 0.30 0.81 int −- 0.58 0.39 0.24 0.22 −0.19 0.21 −0.91 1.04 0.20 1.51
 Parathion 3.47 3.73 0.40 1.60 −0.33 0.85 0.61 1.86 −0.26 0.40 0.41 0.23 0.06 0.22 −0.65 1.08 0.62 1.58
 Phorate −1.54 3.20 0.24 1.40 −0.21 0.75 −0.99 1.64 0.22 0.35 0.25 0.21 0.33 0.19 −0.52 0.95 0.03 1.38
 Phosmet −2.83 4.35 −2.00 1.92 0.19 1.02 0.89 2.25 0.29 0.47 −0.15 0.28 −0.21 0.27 −0.15 1.29 0.00 1.86
 Tebupirimfos −0.04 5.04 2.35 2.23 0.23 1.19 −1.29 2.62 0.36 0.55 0.19 0.33 −0.03 0.31 −0.38 1.51 1.60 2.18
 Terbufos −0.62 3.02 0.78 1.33 1.21 0.70 −1.22 1.55 0.06 0.33 0.17 0.19 0.04 0.18 −1.74 * 0.90 −0.66 1.31
 Tetrachlorvinphos 3.28 5.03 0.78 2.17 −1.28 1.16 −0.72 2.54 1.24 * 0.55 0.35 0.32 0.53 0.30 1.45 1.46 1.69 2.10
Carbamates
 Aldicarb −0.69 3.89 1.00 1.84 −0.51 0.97 2.00 2.12 0.25 0.47 0.30 0.27 0.51 * 0.25 −0.66 1.24 −0.01 1.81
 Benomyl 3.56 4.06 2.64 1.83 −0.44 0.97 −1.29 2.13 0.56 0.47 0.44 0.27 0.28 0.25 2.27 1.24 −1.78 1.81
 Carbaryl int -- 0.48 1.48 0.49 0.79 −1.76 1.74 0.83 * 0.38 0.56 ** 0.22 0.05 0.21 −0.31 1.01 1.00 1.46
 Carbofuran 4.75 3.08 1.81 1.30 1.20 0.69 2.08 1.52 0.21 0.34 0.19 0.19 −0.07 0.18 −0.75 0.88 int --
a

Regression coefficients have been multiplied by −1 so that lower scores indicate poorer performance.

**

p<0.01;

*

p<0.05.

CPT = Continuous Performance Test, AVLT = Auditory Verbal Learning Test, int = significant interaction term for state by pesticide exposure; All models were adjusted for age (years) and ART score. In addition, CPT models were adjusted for positive affect score (PA) and caffeine consumption; Digit-Symbol models were adjusted for PA, education, state, and visual acuity score; Finger Tapping models were adjusted for PA and state; Grooved Pegboard models were adjusted for caffeine consumption, state and visual acuity score; AVLT Total Recall models were adjusted for negative affect score (NA), PA score, and education; AVLT Delayed Recall models were adjusted for NA, PA, education and state; AVLT Recognition models were adjusted for PA, education and state; Sequences A and B were adjusted for PA and state.

Table 5.

Regression coefficients from linear regression models for neurobehavioral outcomes measures and cumulative lifetime days of pesticide use (log10 transformed) for 701 male pesticide applicators.

Pesticide CPTa
(n=692)
Digit-Symbola
(n=691)
Finger
Tapping,
dominant
(n=695)
Grooved
Pegboard,
dominanta
(n=699)
AVLT Total
Recall
(n=696)
AVLT Delayed
Recall
(n=695)
AVLT
Recognition
(n=694)
Sequences Aa
(n=679)
Sequences Ba
(n=671)

β SE β SE β SE β SE β SE β SE β SE β SE β SE

Organophosphates
 Acephate 0.95 1.98 −1.13 1.00 −0.08 0.53 0.02 1.16 −0.07 0.22 0.12 0.15 −0.05 0.14 −0.86 0.68 −0.33 1.00
 Chlorpyrifos −0.29 1.74 −0.99 0.73 −0.22 0.39 1.77 0.85 0.26 0.19 0.22 * 0.11 0.06 0.10 −0.85 0.50 −0.49 0.72
 Coumaphos 1.42 3.09 2.24 1.31 1.03 0.70 1.94 1.54 0.82 * 0.34 0.30 0.19 int -- 0.50 0.88 3.05 * 1.28
 Diazinon −0.67 1.96 −0.46 0.86 0.23 0.45 0.03 1.00 0.01 0.21 0.03 0.13 −0.02 0.12 −0.22 0.58 0.70 0.85
 Dichlorvos −0.63 1.82 0.13 0.82 −0.05 0.44 −0.28 0.96 0.30 0.20 0.16 0.12 0.13 0.11 0.55 0.55 0.60 0.80
 Dimethoate 1.65 3.45 −0.79 1.46 −0.14 0.78 −0.51 1.71 0.39 0.37 0.07 0.21 0.37 0.20 1.37 0.98 −1.16 1.42
 Disulfoton 3.08 2.72 −1.46 1.25 −0.76 0.66 −1.25 1.44 −0.23 0.30 0.19 0.18 −0.24 0.17 −1.52 0.85 0.28 1.25
 Ethoprop 2.17 2.58 −1.81 1.19 0.05 0.63 −0.51 1.39 −0.60 0.28 −0.18 0.17 −0.27 0.16 −1.65 * 0.81 −0.97 1.18
 Fonofos 0.78 2.02 0.41 0.94 0.08 0.50 0.85 1.09 −0.27 0.22 0.26 0.14 −0.05 0.13 −0.38 0.63 −0.53 0.91
 Malathion −0.90 1.80 −1.75 * 0.75 0.11 0.40 int -- 0.14 0.20 0.10 0.11 −0.10 0.10 −0.52 0.52 0.16 0.75
 Parathion 4.65 * 2.32 0.35 0.99 −0.60 0.53 −0.11 1.16 0.10 0.25 0.35 * 0.15 0.20 0.14 0.24 0.67 0.03 0.97
 Phorate −0.25 1.93 0.82 0.83 −0.40 0.44 −0.50 0.97 0.18 0.21 0.21 0.12 0.25 * 0.12 −0.02 0.56 0.28 0.82
 Phosmet −1.47 2.68 −1.15 1.18 0.21 0.63 0.45 1.38 0.12 0.29 −0.16 0.17 −0.12 0.16 −0.11 0.79 −0.01 1.14
 Tebupirimfos −0.44 3.12 1.51 1.38 0.08 0.73 −1.13 1.61 0.34 0.34 0.20 0.20 0.00 0.19 −0.21 0.93 1.01 1.34
 Terbufos −0.22 1.60 0.48 0.70 0.65 0.37 −0.98 0.82 0.04 0.17 0.06 0.10 0.04 0.10 −0.65 0.48 −0.47 0.69
 Tetrachlorvinphos 2.34 3.11 0.45 1.34 −0.77 0.72 −0.43 1.57 0.76 * 0.34 0.18 0.20 0.36 0.19 1.19 0.90 0.89 1.30
Carbamates
 Aldicarb 1.05 2.27 1.12 1.06 −0.26 0.56 1.88 1.23 0.30 0.27 0.27 0.15 0.30 * 0.15 −0.04 0.72 0.72 1.06
 Benomyl 2.59 2.72 2.13 1.22 −0.03 0.65 −0.01 1.42 0.60 0.31 0.41 * 0.18 0.27 0.17 2.25 ** 0.82 1.34 1.20
 Carbaryl 0.60 1.64 −0.04 0.84 −0.12 0.45 −0.79 0.98 0.21 0.22 0.25 * 0.12 0.04 0.12 −0.02 0.57 0.36 0.83
 Carbofuran 1.56 1.97 0.68 0.83 0.63 0.44 1.50 0.96 0.20 0.21 0.21 0.12 0.05 0.11 −0.45 0.56 −0.52 0.81
Summary variables
 ALL OPs −0.95 2.17 −1.22 0.93 −0.16 0.49 −0.93 1.08 0.10 0.23 0.16 0.14 −0.05 0.13 −0.68 0.63 0.18 0.91
 ALL pesticides 2.38 3.13 −1.45 1.33 0.42 0.70 −0.12 1.54 0.43 0.34 0.54 * 0.19 0.10 0.18 −1.26 0.90 −0.77 1.30
a

Regression coefficients have been multiplied by −1 so that lower scores indicate poorer performance.

**

p<0.01;

*

p<0.05.

CPT = Continuous Performance Test, AVLT = Auditory Verbal Learning Test, int = significant interaction term for state by pesticide exposure; All models were adjusted for age (years) and ART score. In addition, CPT models were adjusted for positive affect score (PA) and caffeine consumption; Digit-Symbol models were adjusted for PA, education, state, and visual acuity score; Finger Tapping models were adjusted for PA and state; Grooved Pegboard models were adjusted for caffeine consumption, state and visual acuity score; AVLT Total Recall models were adjusted for negative affect score (NA), PA score, and education; AVLT Delayed Recall models were adjusted for NA, PA, education and state; AVLT Recognition models were adjusted for PA, education and state; Sequences A and B were adjusted for PA and state.

Table 6.

Regression coefficients from linear regression models for neurobehavioral outcome measures and pesticide exposures with an interaction term for state by pesticide exposure for the 701 male pesticide applicators.

CPTa
(n=692)
Digit-Symbola
(n=691)
Grooved
Pegboarda
(n=699)
AVLT
Recognition
(n=695)
Sequences Ba
(n=671)

Iowa N.C. Iowa N.C. Iowa N.C. Iowa N.C. Iowa N.C.
Ever used
 Carbaryl 6.60 −14.52 * -- -- -- -- -- -- -- --
 Carbofuran -- -- -- -- -- -- -- -- 2.99 −3.78 *
 Chlorpyrifos -- -- −4.67 ** 1.06 -- -- -- -- -- --
 Malathion -- -- -- -- 5.85 * −3.98 -- -- -- --
Lifetime days (log10 transformed)
 Coumaphos -- -- -- -- -- -- 0.03 1.09 ** -- --
 Malathion -- -- -- -- 2.46 * −1.11 -- -- -- --
a

Regression coefficients have been multiplied by −1 so that lower scores indicate poorer performance.

**

p<0.01;

*

p<0.05.

CPT = Continuous Performance Test, AVLT = Auditory Verbal Learning Test. Results are presented for models with a significant interaction term (p<0.05); Models are adjusted for the base model covariates and an interaction term for state by pesticide exposure.

3.4. Associations between carbamate use and neurobehavioral outcomes

No statistically significant adverse associations were observed between carbamate pesticide use and adverse NB test results. Rather, each of the four carbamate pesticides was significantly associated with better performance on one or more NB tests (Tables 4-5). A significant interaction between state and ever-use of carbaryl was observed for the Continuous Performance Test (Table 6). Among North Carolina participants, significantly decreased test performance was observed with ever-use of carbaryl, whereas better, but non-statistically significant test performance was observed among Iowa participants. We also observed a significant interaction between state and ever-use of carbofuran for the Sequences B test.

3.5 Confounding by related pesticide exposures

The simultaneous inclusion of correlated pesticides in the models did not attenuate any statistically significant associations between the pesticide exposures and the NB outcome measures.

3.6. Sensitivity analyses

When we excluded the 57 participants with medical conditions or medications which may influence NB performance, we saw no difference in the results. However, when the eight individuals with a history of diagnosed pesticide poisoning were removed from the analyses, the parameter estimate of the association between ever ethoprop use and Digit-Symbol test performance was reduced from −3.66 seconds (p = 0.05) to −3.19 seconds (p = 0.09). No other meaningful changes in estimates of association were observed.

4. Discussion

This is one of the largest studies of NB function among OP pesticide-exposed workers published to date. The study included good characterization of specific pesticide use patterns and quantitative measures of NB function. Overall, among this sample of pesticide applicators, we found little evidence of an adverse association with OP pesticide use, although ever-use or lifetime days of use of at least one OP pesticide was associated with significantly poorer performance on three of nine CNS-associated NB tests and significantly better performance on six of nine tests. Given the large number of statistical tests performed and the limited number of significant associations, it is possible that some of our results may be due to chance.

The three tests for which adverse associations were observed assessed visual scanning and processing (Digit-Symbol), verbal learning and memory (AVLT Total Recall), and motor speed and scanning (Sequences A). Our findings are comparable to the results of several previous studies of agricultural workers without previous pesticide poisoning. Rohlman et al administered a battery of 10 NB tests to 119 Hispanic adults and adolescents working in agriculture and 56 Hispanic adult and adolescent referents [22]. Statistically significantly poorer test performance was observed on four NB measures, including the Continuous Performance Test, among those with any experience mixing or applying pesticides. Farahat et al studied 52 male workers occupationally exposed to OP pesticides and 50 unexposed male controls with similar demographic characteristics [8]. After adjustment for age and education, workers occupationally exposed to OP pesticides performed significantly worse than unexposed workers on six of 12 NB tests, including Digit-Symbol and Trailmaking part A and B (similar to Sequences A and B in the current study). Kamel et al conducted a cross-sectional study of NB test performance among 288 farm workers with at least one month of farm work exposure and 51 controls without farm work exposure [13]. Ever having done farm work was associated with poorer performance on four of eight NB tests including tests of verbal memory, motor speed and motor coordination.

Our predominantly null study may result from the possibility that the sample was highly selected. Although we randomly sampled from the AHS cohort, we required individuals to complete all AHS questionnaires and excluded individuals with a number of health conditions. It is possible that individuals who left farming or who were unable to complete the AHS questionnaires may have been more affected by pesticides than those who were eligible and participated in the current study. Furthermore, the average age of our population was older compared with most previous study populations. An older cohort is more likely to manifest selective survival than a younger cohort. This potential selection bias may have attenuated the observed associations between long-term pesticide use and NB outcomes. Because healthy older individuals are also likely to have used pesticides for longer, it may also have created the appearance of an association between pesticide use and improved NB test performance. Additionally our study did not capture the full range of OP exposures in the AHS cohort, as our sample sites were limited and thus, we did not include all geographic regions. For example, orchard growers are generally limited to the western part of North Carolina which was outside our sampling area. There are unique OPs and application methods used in orchards and we were limited in our ability to assess these. In addition, the overall response rate of the study was slightly less than 40% further suggesting that our study sample may not have been representative of all pesticide applicators enrolled in the AHS. However, participants were similar to the non-participants on several important characteristics including age and total lifetime days of pesticide use suggesting some comparability between them.

Several studies have reported a positive association between long-term OP pesticide exposure and neurological symptom prevalence among agricultural workers [5,11,18,20,26]. Kamel et al examined associations between long-term pesticide use and self-reported neurological symptoms among 18,782 licensed pesticide applicators enrolled in the AHS [11]. They reported associations between neurological symptoms and cumulative exposure to OP pesticides among pesticide applicators with no history of previous pesticide poisoning. While seemingly inconsistent with our findings, individuals may experience subtle differences in neurological function and report symptoms before dysfunction can be identified with objective measures [12].

We also observed several statistically significant interactions between state and specific chemical use. The explanation for these interactions is unclear. However, the difference in findings observed between Iowa and North Carolina participants may be due to uncaptured differences in pesticide use and application methods. For example, chlorpyrifos is more often used in a granular form in Iowa and more often used in a liquid form in North Carolina [29] thus, state differences may be indicative of different patterns of exposure.

One of the strengths of the AHS is the detailed pesticide exposure assessment. This detailed information allowed us to create relatively precise exposure estimates. Whereas most studies in the literature have used dichotomized exposure variables, we estimated cumulative lifetime days of use to specific OP pesticides, as well as to this class as a whole, for each study participant.

Although misclassification of pesticide use is possible, methodological studies have shown that AHS participants provide accurate and reliable pesticide use and duration of exposure information [6,10]. Using pesticide registration information, Hoppin et al showed that AHS participants provide plausible data regarding lifetime duration of use, with less than 5% reporting implausible values for specific chemicals [10]. Similar findings were reported by Blair et al who found that for repeated interviews of AHS participants the percentage agreement for specific pesticide use and application practices were high, ranging from 70% to more than 90% [6].

A major strength of the study is that it was based on a large sample of pesticide applicators selected from the AHS. The sample included pesticide applicators from two distinct geographical locations with different crops and farming practices. Therefore, the results of the present study are likely relevant to a large segment of the farming population. Unlike many prior studies, we had sufficient power to examine the associations of individual pesticides with CNS-associated NB outcomes while controlling for important covariates. However, we were not well powered for examination of interactions, so for those associations that differed between states or for those pesticides used in only one state, we had limited power.

In conclusion, we did not observe strong evidence of impaired NB function associated with OP pesticides in this large study of licensed pesticide applicators. While specific associations may be due to chance and potentially selective participation may have limited our ability to detect differences, some of the findings are consistent with previous studies.

Acknowledgements

We thank Mr. Stuart Long and Mr. Tao Zhang for programming support and the Iowa and North Carolina field station staff (Ms. Ellen Heywood and Ms. Margaret Hayslip). We are especially grateful to the late Richard Letz, PhD, for his valuable technical assistance with the design of the neurobehavioral testing procedures as well as his in-kind contribution of the NES3 software.

Funding: This work was supported by grant number R01-ES013067 from the National Institute of Environmental Health Sciences. This work was supported in part by the Intramural Research Program of the NIH, the National Institute of Environmental Health Sciences (Z01-ES049030) and the National Cancer Institute (Z01-CP010119).

Abbreviations

AHS

Agricultural Health Study

ART

Adult Reading Test

AVLT

Auditory Verbal Learning Test

BMI

body mass index

CPT

Continuous Performance Test

CNS

central nervous system

NA

negative affect

NB

neurobehavioral

OP

organophosphate

PA

positive affect

APPENDIX A. Neurobehavioral outcome measures

Descriptive summary statistics for the NB test results are presented in Table 1. We administered the Finger Tapping and Grooved Pegboard tests separately for the dominant and non-dominant hands. However, because the overall results were similar for both hands, only the results for the dominant hand are presented. Some study participants were unable to complete individual tests in the allowed time or after two attempts, therefore, the total number of participants completing each test varied.

Table 1.

Frequencies and means of central nervous system neurobehavioral outcome measures for 701 male pesticide applicators in the Agricultural Health Study.

Outcome N Mean SD Min. Median Max.
Continuous performance (ms) 693 427.9 44.9 318.6 421.2 612.3
Digit-Symbol (s) 692 117.6 23.1 73.6 112.1 213.6
Finger tapping, dominant hand (# of taps) 695 53.6 9.6 9.0 55.0 86.0
Grooved pegboard, dominant hand (s) 700 92.0 24.0 51.0 86.0 180.0
Auditory verbal learning total recall (# correct) 696 19.9 5.1 5.0 20.0 34.0
Auditory verbal learning delayed recall (# correct) 695 6.6 2.8 0 7.0 12.0
Auditory verbal learning recognition (tp-fp) 694 8.3 2.6 −3.0 9.0 12.0
Sequences A latency (s) 680 42.9 14.6 14.8 40.3 93.8
Sequences B latency (s) 672 64.6 21.2 22.8 59.5 144.4

ms = milliseconds, s = seconds, # = number, tp = true positives, fp = false positives.

Footnotes

Conflict of interest statement: We declare that we do not have any competing interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • [1].AHS Full Text of Questionnaires. 2010 Available from: http://aghealth.nci.nih.gov/questionnaires.html.
  • [2].Alavanja MC, Sandler DP, McMaster SB, Zahm SH, McDonnell CJ, Lynch CF, Pennybacker M, Rothman N, Dosemeci M, Bond AE. The Agricultural Health Study. Environ Health Perspect. 1996;104:362–9. doi: 10.1289/ehp.96104362. others. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [3].Baldi I, Filleul L, Mohammed-Brahim B, Fabrigoule C, Dartigues JF, Schwall S, Drevet JP, Salamon R, Brochard P. Neuropsychologic effects of long-term exposure to pesticides: results from the French Phytoner study. Environ Health Perspect. 2001;109:839–44. doi: 10.1289/ehp.01109839. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [4].Bardin PG, van Eeden SF, Moolman JA, Foden AP, Joubert JR. Organophosphate and carbamate poisoning. Arch Intern Med. 1994;154:1433–41. [PubMed] [Google Scholar]
  • [5].Bazylewicz-Walczak B, Majczakowa W, Szymczak M. Behavioral effects of occupational exposure to organophosphorous pesticides in female greenhouse planting workers. Neurotoxicology. 1999;20:819–26. [PubMed] [Google Scholar]
  • [6].Blair A, Tarone R, Sandler D, Lynch CF, Rowland A, Wintersteen W, Steen WC, Samanic C, Dosemeci M, Alavanja MC. Reliability of reporting on life-style and agricultural factors by a sample of participants in the Agricultural Health Study from Iowa. Epidemiology. 2002;13:94–9. doi: 10.1097/00001648-200201000-00015. [DOI] [PubMed] [Google Scholar]
  • [7].EPA . In: Pesticides Industry Sales and Usage: 2006 and 2007 Market Estimates. U.S.E.P. Agency, editor. Washington D.C.: 2011. [Google Scholar]
  • [8].Farahat TM, Abdelrasoul GM, Amr MM, Shebl MM, Farahat FM, Anger WK. Neurobehavioural effects among workers occupationally exposed to organophosphorous pesticides. Occup Environ Med. 2003;60:279–86. doi: 10.1136/oem.60.4.279. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [9].Fiedler N, Kipen H, Kelly-McNeil K, Fenske R. Long-term use of organophosphates and neuropsychological performance. Am J Ind Med. 1997;32:487–96. doi: 10.1002/(sici)1097-0274(199711)32:5<487::aid-ajim8>3.0.co;2-p. [DOI] [PubMed] [Google Scholar]
  • [10].Hoppin JA, Yucel F, Dosemeci M, Sandler DP. Accuracy of self-reported pesticide use duration information from licensed pesticide applicators in the Agricultural Health Study. J Expo Anal Environ Epidemiol. 2002;12:313–8. doi: 10.1038/sj.jea.7500232. [DOI] [PubMed] [Google Scholar]
  • [11].Kamel F, Engel LS, Gladen BC, Hoppin JA, Alavanja MC, Sandler DP. Neurologic symptoms in licensed pesticide applicators in the Agricultural Health Study. Hum Exp Toxicol. 2007;26:243–50. doi: 10.1177/0960327107070582. [DOI] [PubMed] [Google Scholar]
  • [12].Kamel F, Hoppin JA. Association of pesticide exposure with neurologic dysfunction and disease. Environ Health Perspect. 2004;112:950–8. doi: 10.1289/ehp.7135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [13].Kamel F, Rowland AS, Park LP, Anger WK, Baird DD, Gladen BC, Moreno T, Stallone L, Sandler DP. Neurobehavioral performance and work experience in Florida farmworkers. Environ Health Perspect. 2003;111:1765–72. doi: 10.1289/ehp.6341. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [14].Klove H. Clinical Neuropsychology. Med Clin North Am. 1963;47:1647–58. [PubMed] [Google Scholar]
  • [15].Letz R. NES3 User’s Manual. Neurobehavioral Systems Inc.; Atlanta, GA: 2000. [Google Scholar]
  • [16].Letz R, Gerr F, Cragle D, Green RC, Watkins J, Fidler AT. Residual neurologic deficits 30 years after occupational exposure to elemental mercury. Neurotoxicology. 2000;21:459–74. [PubMed] [Google Scholar]
  • [17].Letz R, Green RC, Woodard JL. Development of a computer-based battery designed to screen adults for neuropsychological impairment. Neurotoxicol Teratol. 1996;18:365–70. doi: 10.1016/0892-0362(96)00041-4. [DOI] [PubMed] [Google Scholar]
  • [18].London L, Nell V, Thompson ML, Myers JE. Effects of long-term organophosphate exposures on neurological symptoms, vibration sense and tremor among South African farm workers. Scand J Work Environ Health. 1998;24:18–29. doi: 10.5271/sjweh.274. [DOI] [PubMed] [Google Scholar]
  • [19].Lotti M. Low-level exposures to organophosphorus esters and peripheral nerve function. Muscle Nerve. 2002;25:492–504. doi: 10.1002/mus.10086. [DOI] [PubMed] [Google Scholar]
  • [20].Ohayo-Mitoko GJ, Kromhout H, Simwa JM, Boleij JS, Heederik D. Self reported symptoms and inhibition of acetylcholinesterase activity among Kenyan agricultural workers. Occup Environ Med. 2000;57:195–200. doi: 10.1136/oem.57.3.195. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [21].Pilkington A, Buchanan D, Jamal GA, Gillham R, Hansen S, Kidd M, Hurley JF, Soutar CA. An epidemiological study of the relations between exposure to organophosphate pesticides and indices of chronic peripheral neuropathy and neuropsychological abnormalities in sheep farmers and dippers. Occup Environ Med. 2001;58:702–10. doi: 10.1136/oem.58.11.702. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [22].Rohlman DS, Lasarev M, Anger WK, Scherer J, Stupfel J, McCauley L. Neurobehavioral performance of adult and adolescent agricultural workers. Neurotoxicology. 2007;28:374–80. doi: 10.1016/j.neuro.2006.10.006. [DOI] [PubMed] [Google Scholar]
  • [23].Roldan-Tapia L, Parron T, Sanchez-Santed F. Neuropsychological effects of long-term exposure to organophosphate pesticides. Neurotoxicol Teratol. 2005;27:259–66. doi: 10.1016/j.ntt.2004.12.002. [DOI] [PubMed] [Google Scholar]
  • [24].Romana MJ, A J, Sheridan R, Sidell FR. Health effects of low-level exposure to nerve agents. CRC Press; Boca Raton: 2001. [Google Scholar]
  • [25].Rothlein J, Rohlman D, Lasarev M, Phillips J, Muniz J, McCauley L. Organophosphate pesticide exposure and neurobehavioral performance in agricultural and non-agricultural Hispanic workers. Environ Health Perspect. 2006;114:691–6. doi: 10.1289/ehp.8182. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [26].Smit LA, van-Wendel-de-Joode BN, Heederik D, Peiris-John RJ, van der Hoek W. Neurological symptoms among Sri Lankan farmers occupationally exposed to acetylcholinesterase-inhibiting insecticides. Am J Ind Med. 2003;44:254–64. doi: 10.1002/ajim.10271. [DOI] [PubMed] [Google Scholar]
  • [27].Starks S, Gerr F, Kamel F, Lynch CF, Alavanja M, Sandler D, Hoppin J. High pesticide exposure events and central nervous system function among pesticide applicators. 2011. [DOI] [PMC free article] [PubMed]
  • [28].Stephens R, Sreenivasan B. Neuropsychological effects of long-term low-level organophosphate exposure in orchard sprayers in England. Arch Environ Health. 2004;59:566–74. doi: 10.1080/00039890409603435. [DOI] [PubMed] [Google Scholar]
  • [29].Thomas KW, Dosemeci M, Hoppin JA, Sheldon LS, Croghan CW, Gordon SM, Jones ML, Reynolds SJ, Raymer JH, Akland GG. Urinary biomarker, dermal, and air measurement results for 2,4-D and chlorpyrifos farm applicators in the Agricultural Health Study. J Expo Sci Environ Epidemiol. 2010;20:119–34. doi: 10.1038/jes.2009.6. others. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [30].Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 1988;54:1063–70. doi: 10.1037//0022-3514.54.6.1063. [DOI] [PubMed] [Google Scholar]

RESOURCES