Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Dec 1.
Published in final edited form as: Neurotoxicology. 2020 Aug 15;81:11–17. doi: 10.1016/j.neuro.2020.08.002

Associations of the Behavioral Assessment and Research System (BARS) neurobehavioral outcomes with attention problems in children living near coal ash storage sites.

Lonnie Sears a, Clara G Sears b, John V Myers c, Guy N Brock d, Kristina M Zierold e
PMCID: PMC7708521  NIHMSID: NIHMS1622784  PMID: 32810513

Abstract

Environmental exposures have been linked to childhood problems with overactivity, attention, and impulse control, and an increased risk of attention deficit hyperactivity disorder (ADHD) diagnosis. Two approaches to identify these types of exposure-related neurobehavioral problems include the use of computerized tests, such as the Behavioral Assessment and Research System (BARS), as well as the use of behavior rating scales. To assess comparability of these two types of measures, we analyzed data from 281 children aged 6 to 14 years enrolled in a 5-year research study investigating coal ash exposure and neurobehavioral health. All children lived in proximity of coal ash storage sites. We administered six computer tests from the BARS and obtained behavior measures from the parent-completed Child Behavior Checklist (CBCL) ADHD DSM oriented scale. BARS test performance was associated with age indicating that the tests could be used to evaluate neurodevelopmental changes over time or across a wide age range. Tests within the BARS including Continuous Performance (CPT) false alarm (standardized estimate 1.57, 95% confidence interval (CI) (0.67, 2.48), adjusted p=0.006), Selective Attention (SAT) wrong count (standardized estimate 2.8, 95% CI (1.17, 4.44), adjusted p=0.006), and SAT proportion correct (standardized estimate −2.45, 95% CI (−4.01, −0.88), adjusted p=0.01) were associated with attention and impulse control problems on the CBCL after adjustment for multiple comparisons. Findings support that the BARS can contribute to research on environmental exposures by assessing subclinical behaviors related to ADHD such as sustained attention, impulse control, response inhibition, associative learning, and short-term memory. Future research can examine relationships of these BARS measures with biomarkers of neurotoxic exposures related to living near coal ash storage sites to better identify the potential risk for ADHD-related behaviors among children living near coal ash storage sites.

Keywords: ADHD, Neurobehavioral tests, Children, Environmental Exposure, Neurotoxins, BARS

INTRODUCTION

Attention Deficit Hyperactivity Disorder (ADHD) is a chronic neurodevelopmental disorder characterized by attention problems, hyperactivity, and poor impulse control (APA, 2013). The prevalence of ADHD in the United States was estimated to be 7.8% in 2011 (Visser et al., 2014) while the most recent estimate from 2016 data is 9.4% (Danielson et al. 2018). The prevalence of ADHD continues to rise for unknown reasons leading to concerns that exposure to environmental pollutants may play a part in rising numbers of children diagnosed with ADHD in addition to genetic factors (NIMH, 2019). Potential environmental etiologic factors include a variety of neurotoxins such as the well-studied effects of early life lead exposure on behavior and development (Chiodo et al., 2007; Lanphear et al., 2005; Needleman et. al., 1979; Nie et al., 2011; Rodrigues et al., 2016). One understudied potential source of neurotoxins that may increase risk for ADHD-related behaviors is coal ash. Coal ash is produced throughout the world, and in some countries like the United States, stored at sites located near residential communities (EPA, 2010).

Coal ash, a waste product that is generated from burning coal for energy, is predominately comprised of silicon, aluminum, iron, oxygen, and calcium, however, particles may contain trace levels of neurotoxic heavy metals (Bednar et al., 2013; Hatori et al., 2010; Patra et al., 2012). Some of the neurotoxic heavy metals that are detected in coal ash include arsenic, cadmium, chromium, lead, and mercury (Bednar et al., 2013; Bhangare et al., 2011; Brown et al., 2011; Hatori et al., 2010; Liberda et al., 2013; Patra et al., 2012; Zierold & Odoh, 2020). Research has shown that concentrations of metals in coal ash are higher than concentrations found in the parent coal (Spencer & Drake, 1987; Bhangare et al., 2011; Verma et al., 2016), which may increase the hazard associated with exposure.

Coal ash that is not reused in products such as concrete or grout is stored in landfills and surface impoundments. These storage sites can release fugitive dust emissions increasing ambient air pollution in surrounding communities and exposing children to neurotoxic metals (EPA, 2010). However, limited research has assessed the developmental and neurobehavioral health of children exposed to coal ash (Perera et al., 2008; Sears & Zierold, 2017; Tang et al., 2008; Tang et al., 2014). From 2013–2014, Sears & Zierold conducted a cross-sectional survey in Jefferson County, Kentucky to compare the health of children living near coal ash storage sites, to a comparison population not living near coal ash storage sites in Orange County, Indiana. Parents of children in both locations were asked to fill out a questionnaire that contained multiple questions about health conditions of their children (Sears & Zierold, 2017). Responses to the questionnaire indicated that 36% of children living near the coal ash storage sites had been identified as having ADHD, compared with 16% of the non-exposed group. The parent-suspected prevalence of ADHD in the children living near the coal ash storage sites was found to be three to four times higher than the prevalence of the US.

Although ascertaining prevalence rates of ADHD from a community-based survey is a common method for determining prevalence, this approach may be imprecise due to the sensitivity of parent reported diagnoses (e.g. affected vs. not affected) and lack of a standardized protocol for ascertaining neurotoxic effects. To overcome these issues, a behavior checklist and diagnostic interview are often used in as a standardized diagnostic approach to classify individuals as affected or unaffected. However, there are concerns with this categorical approach, such as the misclassification of affected individuals and the under-identification of subclinical adverse outcomes (Sagiv et al., 2015). Another approach is to use a dimensional measure to identify adverse effects rather than the typical categorical approach. In a dimensional approach a relevant behavior can be measured and characterized in terms of degree of impairment which can capture subclinical effects of neurotoxic exposure compared to the variability within a population. The dimensional approach recognizes subclinical effects and avoids problems associated with misclassification of affected individuals (Rauh & Margolis, 2016). It provides greater statistical power and the ability to follow developmental patterns within age groups. A dimensional approach has been shown to improve studies of neurodevelopmental disorders such as autism (Constantino, 2011) and is used in mental health research such as with the Research Domain Criteria Project. (Morris & Cuthbert, 2012)

The Behavioral Assessment and Research System (BARS), was developed to identify neurobehavioral differences in adults exposed to pollutants (Anger et al., 1994) and was adapted for use in children (Rohlman et al., 2000a; Rohlman et al., 2001). The computerized tests include measures of sustained attention, processing speed, response inhibition, fine motor speed, number – symbol association, and memory. The BARS has been used in environmental exposure studies in several countries and across races/ethnicities demonstrating cross-cultural reliability and adaptability (Butler-Dawson et al., 2016; Khan et al., 2019a; Khan et al., 2019b; Rohitrattana et al., 2014; Rohlman et al., 2008; Rohlman et al., 2016). It was developed to avoid practice effects when subjects are retested over time allowing for monitoring of chronic exposures (Rohlman et al., 2000b). Many of the cognitive abilities measured by the BARS have been associated with ADHD and thought to be related to brain functioning in frontal-striatal circuits (Norman et al., 2016). Although the diagnosis of ADHD is based on a comprehensive psychological evaluation (Barkley, 2019), computerized tests similar to the BARS, such as continuous performance tests, have been shown to improve accuracy of an ADHD diagnosis (Berger et al., 2017; Hollis et al., 2018; Nikolas & Nigg, 2013) and are sensitive to the neurotoxic effects of exposures such as related to pesticides, arsenic, and chromium (Caparros-Gonzalez et al, 2019; Ismail et al., 2017a; Ismail et al., 2017b; Rodriguez-Barrano et al., 2015; Rohlman et al., 2016).

Based on the potential environmental exposures from coal ash storage sites, and our findings from a previous survey indicating a high prevalence of parent-reported suspected ADHD near coal ash storage sites (Sears & Zierold, 2017), we used the BARS to evaluate a group of children aged 6 to 14 years potentially exposed to coal ash related pollution. The sensitivity of the BARS to developmental effects was tested by comparing age with test performance. The relationship of the BARS outcome scores with to day-to-day behavior was also tested by correlating BARS test results with caregiver report of ADHD behaviors on the Child Behavior Checklist (CBCL), a validated measure of behavior problems in children (Achenbach & Rescorla, 2001; Chang et al., 2017; Ebesutani et al., 2010; Raiker et al., 2017). Furthermore, we evaluated sex differences in BARS performance considering evidence of differential sex effects of environmental exposures on BARS test performance (Caparros-Gonzalez et al., 2019; Sears et al., 2020). In general, the effects of environmental exposures on neurobehavioral outcomes may vary by sex due to differences in neurodevelopment. It was hypothesized that the BARS outcome scores would be sensitive to developmental changes related to age and sex. Furthermore, we hypothesized that poorer performance on the BARS would be associated of neurocognitive problems typical of children with ADHD making it useful for studying the neurobehavioral effect of coal ash exposure.

METHODS

Participants of the Study

Participants for this study included children aged 6 to 14 years old and one of their parents/guardians, who enrolled in a 5-year study investigating the relationship between coal ash exposure and children’s neurobehavioral health. As of December 2019, we had enrolled 282 children, who lived within proximity to coal ash storage sites in Jefferson County and Bullitt County, Kentucky. Two hundred and eighty-one children completed the BARS testing and their parents/guardians filled out the CBCL. Recruitment methods have been previously described (Odoh et al., 2019). In brief, parents/guardians were invited to enroll their child through multiple recruitment methods, including mailings to the home, flyers distributed by foot recruiting, and media outreach. The mean age of the 281 children was 10.7 years old (SD=2.5). The sample of child participants was 51.6% male and 47.3% female and 75% were white non-Hispanic and 25% other race/ethnicity.

Consenting of the parent/guardian and assenting of the child were conducted prior to any data collection. The research study and procedures were reviewed and approved by the University of Louisville Institutional Review Board (IRB #14.1069), The University of Alabama at Birmingham (IRB #300003807), and the Ohio State University (IRB #2019X0066).

BARS Test Battery

The BARS is a collection of computerized tests assessing memory, sustained attention, processing speed, visual memory, and associative learning. The BARS was developed using tests found to be sensitive to neurologic insult and based on cognitive assessment research. The background for test development has been described by Rohlman et al. (2003). Reliability and validity of the tests has been described (Farhat et al., 2003, Rohlman et al., 2003). Although the BARS battery includes ten tests, only six tests were used in this research study ensuring children could complete the testing in one session. BARS tests used for this study are indicated in Table 1 and were selected based on previous research on use of the tests to identify environmental exposures (Eckerman et al., 2007; Rodriguez-Barrano et al., 2015; Rohlman et al., 2001; Rohlman et al., 2003; Rohlman et al., 2015). (see Rohlman et al., 2003)

Table 1.

Behavioral Assessment and Research System (BARS) tests and order of administration.

Test Primary Neurocognition Measured Outcome Measure
Continuous Performance Sustained Attention D’prime, Misses
Impulsivity False Alarms
Processing Speed Hit Latency
Selective Attention Sustained Attention Omit Count
Impulsivity Wrong Count
Processing Speed Response Latency
Finger Tapping Fine Motor Speed Total Tap Both Hands
Simple Reaction Time Processing Speed Response Latency
Symbol Digit Processing Speed Response Latency
Associative Learning Total Errors
Digit Span Immediate Memory Forward Count
Working Memory Reverse Count
Open in a new tab

Testing was done in the child’s home during afternoon or evening hours. Children sat comfortably at a table in the kitchen, separated from distractions. Instructions were presented on a laptop connected to a 9-button response keypad used for response measurement for all of the tests. A psychologist administered the six tests and all the tests were given in the same order. The psychologist followed instructions in the BARS Procedure Manual (V1.0(06/17/2015)) for test administration, first completing testing with a practice group of children to ensure fidelity in test administration. Consultation with test developers from the Northwest Education and Training group (NWETA.com) occurred to resolve questions in test administration. All children completed the six BARS tests in approximately thirty minutes

ADHD DSM Oriented Scale

The Child Behavior Checklist (CBCL) was used to quantify behaviors associated with ADHD. This questionnaire is a caregiver-completed behavior checklist that provides a measure of problem behaviors including patterns suggestive of ADHD (Achenbach & Rescorla, 2001). The ADHD DSM oriented scale includes thirteen items which indicate the presence and severity of ADHD related behavior problems due to inattention, overactivity, and poor impulse control. A raw score is obtained based on the presence and severity of behaviors (0 to 26) that is converted to a t-score (Mean=50, SD=10) according to age- and gender- based norms. Higher t-scores indicate more severe problems with a t-score of 65 to 69 being borderline clinically significant while 70 and above is considered clinically significant and suggestive of ADHD. The reliability of the ADHD DSM oriented scale is high, with a Chronbach’s alpha of 0.84 (Mano et al., 2009).

Statistical Analyses

The median and interquartile range were calculated for each test measure since scores were not normally distributed for all variables. Wilcoxon rank sum tests were used to look for differences in scores separately by age group (6 to 9 years versus 10 to 14 years) and gender. Developmental effects on the BARS outcome variables were assessed with Spearman correlations using age in months and each selected BARS test outcome measure. The relationship of the BARS variables and the ADHD DSM oriented scale were assessed with linear regression using a Tobit model due to the restricted range of the CBCL ADHD DSM oriented scale with a lower cutoff t-score at 50. Linear regression was adjusted for age (in months) and gender for each BARS outcome measure since previous research indicated age (Eckerman et al, 2007; Rohlman et al., 2000b) and gender (Caparros-Gonzalez et al., 2019) differences in test performance. In addition to unadjusted p-values, adjusted p-values based on the false discovery rate (FDR) were reported to control for multiple comparisons (Benjamini & Hochberg, 1995). An FDR adjusted p-value ≤ 0.05 was considered statistically significant.

RESULTS

BARS Summary Statistics

Table 2 displays outcome measures for each BARS test indicating differences by age group (younger, older) and sex in addition to the overall summary statistics. As expected, age differences were significant for every test, based on the Wilcoxon rank-sum test, with older subjects displaying improved performance. Sex was also found to be associated with all hit latency metrics except for SDT, CPT false alarms, total finger tapping, SAT wrong count, and SAT proportion correct. In general, boys performed better on tests related to processing speed and fine motor speed, while girls had stronger performances for tests associated with impulsivity.

Table 2.

Summary statistics for BARS test measures.

BARS Test Measure Overall (n = 281) Female 6–9 yrs (n = 53) Female 10–14 yrs (n = 80) Male 6–9 yrs (n=42) Male 10–14yrs (n = 103)
Median (IQR†) [Min, Max] Median (IQR) [Min, Max] Median (IQR) [Min, Max] Median (IQR) [Min, Max] Median (IQR) [Min, Max]
Continuous Performance D’Prime + 3.69 (1.29) [0.26, 5.24] 3.43 (0.92) [1.15, 5.00] 3.92 (1.26) [1.62, 5.24] 2.86 (1.51) [0.26, 4.77] 4.03 (1.32) [1.41, 5.24]
Hit Latency (sec)*- 0.45(0.10) [0.26, 0.73] 0.52 (0.09) [0.37, 0.73] 0.42 (0.09) [0.33, 0.61] 0.50 (0.09) [0.26, 0.67] 0.42 (0.07) [0.28, 0.59]
Misses - 5 (9) [0, 54] 5 (8) [0, 52] 4.5 (10.5) [0, 38] 10.5 (12) [1, 54] 4 (9) [0, 30]
False Alarms* - 2 (4) [0, 47] 3 (3) [0, 21] 1 (3) [0, 33] 6 (11) [1, 47] 2 (4) [0, 21]
Selective Attention Response Latency (sec)*- 0.36 (0.10) [0.12, 0.70] 0.44 (0.09) [0.28, 0.70] 0.34 (0.07) [0.23, 0.52] 0.42 (0.13) [0.12, 0.65] 0.33 (0.06) [0.23, 0.49]
Wrong Count*- 2 (2) [0, 8] 2 (2) [0, 6] 1 (2) [0, 5] 3 (2) [0, 8] 2 (2) [0, 8]
Omit Count- 1 (2) [0, 15] 1 (2) [0, 10] 1 (2) [0, 6] 2 (2) [0, 15] 1 (2) [0, 7]
Proportion Correct*+ 0.90 (0.10) [0.43, 1.00] 0.90 (0.10) [0.63, 1.00] 0.90 (0.10) [0.63, 1.00] 0.83 (0.13) [0.43, 1.00] 0.90 (0.13) [0.67, 1.00]
Finger Tapping Total Tap* + 211 (48) [131, 317] 179 (33) [131, 291] 217.5 (29.5) [154, 317] 183.5 (32) [141, 233] 230 (35) [174, 311]
Simple Reaction Time Response Latency (sec)*- 0.39 (0.11) [0.26, 1.03] 0.47 (0.13) [0.35, 1.03] 0.37 (0.06) [0.28, 0.59] 0.45 (0.15) [0.32, 0.67] 0.35 (0.09) [0.26, 0.64]
Digit Span Forward Count + 5 (2) [0, 9] 4 (1) [3, 6] 5 (2) [3,9] 4 (2) [0, 6] 5 (1) [0, 8]
Reverse Count + 3 (1) [0, 8] 3 (1) [0, 6] 4 (2) [0, 7] 3 (3) [0, 4] 3 (1) [0, 8]
Symbol Digit Response Latency (sec) - 2.49 (1.17) [1.38, 11.89] 3.20 (1.28) [1,97, 8.57] 2.13 (0.53) [1.54, 3.80] 4.02 (1.92) [2.56, 11.89] 2.28 (0.67) [1.34, 3.97]
Total Errors - 0 (2) [0, 17] 0 (1) [0, 10] 0 (1) [0, 4] 1.5 (3) [0, 17] 0 (2) [0, 10]
Open in a new tab
*

Significant difference by sex (FDR adjusted p < 0.05; assessed with Wilcoxon rank-sum test due to non-normality). All BARS measures comparisons by age group were statistically significant (FDR adjusted p < 0.05; Wilcoxon rank-sum test).

†

IQR = Inter-quartile range

Note: Plus sign (+) indicates a higher score = better performance; Minus sign (−) indicates a lower score = better performance

Correlations of Age with BARS Measures Outcomes

Developmental effects were seen for all tests with age being most predictive of response latency and memory span for respective BARS tests. Figure 1 displays scatterplots with simple regression lines for significant Spearman correlations of age with BARS test outcomes. Variables with highest correlations with age for each test were; CPT average hit latency (Spearman ρ = −0.70, FDR-adjusted p<0.0001), SAT average correct latency (ρ = −0.58, FDR-adjusted p < 0.0001), FT total (ρ = 0.71, FDR-adjusted p < 0.0001), SRT average correct latency (ρ = −0.71, FDR-adjusted p < 0.0001), SDT average correct latency (ρ = − 0.79, FDR-adjusted p < 0.0001), and DST forward count (ρ = 0.42, FDR-adjusted p < 0.0001). Most tests show a fairly linear relationship with some deviations at early ages (e.g. SDT average correct latency).

Figure 1.

Figure 1.

Open in a new tab

Age-related association with BARS test performance. Scatterplots between age (in months) and BARS test performance measures. Simple linear regression lines are superimposed on each plot.

BARS and CBCL ADHD DSM Oriented Scale

Table 3 shows results for association between all BARS test measures considered in this study and the ADHD scale t-score on the CBCL. Approximately 35% (n=99) of the participants had an ADHD scale t-score of 50, with a median score of 52, a third quartile of 60, and 26 participants (9%) with a score of 70 or higher. The regression models were adjusted for age and sex. In addition to estimates per unit change in measurement, standardized estimates obtained by dividing the BARS test score by the interquartile range were calculated to allow for comparisons across measures. Significant predictive values were seen for the CPT d’prime and false alarm variables which indicated that poorer signal detection skills and increased commission errors (false alarms) were associated with higher ADHD scale t-scores. On the SAT, incorrect responses were associated with a higher ADHD scale t-score. Reduced memory span on the DST and reduced processing speed as indicated by increased response latency on the SRT test were also associated with higher ADHD scale t-scores. Likewise, errors on the SDT test were associated with higher ADHD scale t-scores. Absolute values of the standardized estimates ranged from 1.5 to 2.8, indicating that mean ADHD scale t-scores on the CBCL changed between 1.5 and 2.8 points over the middle 50% of the BARS outcome measures. Taken together, results of the BARS outcome measures indicate that reduced processing speed, poorer signal detection, increased performance errors, and poorer short-term memory were associated with higher ADHD scale t-scores on the CBCL.

Table 3.

BARS tests predictive of ADHD score on Child Behavior Checklist from Tobit regression model adjusting for age and gender.

Test BARS Measure Estimate SE CI Standardized Estimate* Standardized SE* Standardized CI* Unadjusted p-value FDR-adjusted p-value
Continuous Performan ce (CPT) D’Prime −1.77 0.74 (−3.22, −0.31) −2.28 0.96 (−4.15, −0.40) 0.02 0.05
Hit Latency (sec) 1.62 12.09 (−22.07, 25.30) 0.16 1.23 (−2.25, 2.58) 0.89 0.89
Misses 0.11 0.07 (−0.03, 0.25) 0.98 0.63 (−0.25, 2.22) 0.12 0.17
False Alarm 0.39 0.12 (0.17, 0.62) 1.57 0.46 (0.67, 2.48) <0.001 0.006
Selective Attention (SAT) Average Correct Latency (sec) 2.43 9.86 (−16.89, 21.76) 0.25 1.02 (−1.74, 2.24) 0.81 0.87
Wrong Count 1.40 0.42 (0.58, 2.22) 2.80 0.84 (1.17, 4.44) <0.001 0.006
Omit Count 0.43 0.36 (−0.27, 1.14) 0.87 0.72 (−0.54, 2.29) 0.23 0.27
Proportion Correct −24.45 8.00 (−40.13, −8.78) −2.45 0.80 (−4.01, −0.88) 0.002 0.01
Finger Tapping Total Tapping −0.04 0.03 (−0.10, 0.01) −2.05 1.38 (−4.76, 0.66) 0.14 0.18
Simple Reaction Time (SRT) Average Correct Latency (sec) 21.98 9.62 (3.13, 40.83) 2.44 1.07 (0.35, 4.53) 0.02 0.05
Digit Span (DST) Digit Span Forward Count −1.28 0.56 (−2.37, −0.19) −2.56 1.11 (−4.74, −0.39) 0.02 0.05
Digit Span Reverse Count −0.87 0.44 (−1.73, −0.002) −0.87 0.44 (−1.73, − 0.002) 0.05 0.09
Symbol Digit Test (SDT) Average Correct Latency (sec) 1.33 0.80 (−0.23, 2.90) 1.56 0.93 (−0.27, 3.39) 0.09 0.15
Total Errors 0.80 0.36 (0.10, 1.50) 1.60 0.72 (0.20, 3.01) 0.03 0.05
Open in a new tab
*

Standardized data achieved by dividing data by IQR of the BARS measure.

DISCUSSION

The purpose of this research was two-fold: (1) to investigate the sensitivity of the BARS to developmental effects and sex difference in BARS performance and (2) to assess the relationship of the BARS with day-to-day behavior, by correlating BARS test results with caregiver report of ADHD behaviors on the CBCL. Consistent with the BARS literature, age was found to be positively associated with better neurobehavioral performance, more strongly with processing speed and response latency. Furthermore, the BARS test scores demonstrated external validity based on the association of several BARS outcome scores with ADHD-related behaviors. The convergence of computerized test results with observed behaviors provides improved opportunity to identify ADHD features in children for use in studies of environmental exposures. Combining both measures can enhance identification of subtypes of ADHD related to attention, overactivity, and impulse control as well as associated problems such as mood regulation (APA, 2013), and those effects of environmental exposures in children (Rauh & Margolis, 2016). In addition, this information can be used to identify subclinical aspects of ADHD in the context of developmental changes in specific behaviors.

The relationship of age with test performance was most salient for response latency. For BARS tests assessing response speed, the decreasing response latency with age is consistent with observations from previous research with the BARS (Rohlman et al., 2000, Khan et al., 2019) as well as other computerized tests (Carlozzi et. al., 2013) indicating that the BARS is sensitive to similar developmental changes. Differences across the age span noted on the BARS can add important information for comparison of the effects of environmental exposures across age groups so that developmental differences can be compared and contrasted. In addition to processing speed, age-related differences were seen in memory span and fine motor speed also reflecting age-related performance improvements across the age group used in this study. Moving forward, BARS age-related test results can be used to identify ADHD related behaviors associated with environmental exposures, such as coal ash, from early childhood through adolescence.

Results indicated that several BARS variables were associated with ADHD behaviors as reported by caregiver-completed behavior checklist. This finding supports the external validity of the BARS based on agreement of test findings with observed behaviors used to diagnose ADHD. This observation is in agreement with previous research on the potential use of neuropsychological tests to assess ADHD and identify subtypes (Nikolas & Nigg, 2013), although this does not mean that the BARS should be used to diagnose ADHD (Barkley, 2019). The diagnosis of ADHD requires the use of objective behavioral measures of the child in multiple settings in addition to a thorough diagnostic interview. Findings support the use of the BARS in conjunction with parent/guardian report to identify children with neurobehavioral differences related to environmental exposures. Furthermore, the BARS ability to evaluate specific behaviors associated with ADHD, such as response inhibition, processing speed, memory, associative learning, and fine motor speed, allows for study of specific endophenotypes within ADHD. The evidence of validity of the BARS for identifying key aspects of ADHD supports its use for further studies seeking to use a dimensional approach to the study of environmental exposures using a measure sensitive to subclinical effects and age-related differences.

This study has several limitations including the potential bias in our sample related to participant recruitment. Parents/guardians may be more likely to enroll their child with ADHD features. However, children meeting diagnostic criteria for ADHD in this study is approximately 17% which does not statistically differ from estimates of the prevalence of ADHD in Kentucky (12%) suggesting that the study sample is representative of the population in the study area. Furthermore, recruitment material did not have the health outcomes of interest listed, so parents/guardians did not know that ADHD was a component of the study until speaking with the principal investigator. Another limitation is the lack of teacher report of behavior which is commonly used to ascertain ADHD (NIMH, 2019). Attention problems are often most apparent in the classroom and teacher input may have improved identification of children with inattentive types of ADHD. Improved identification of ADHD would enhance the use of BARS through a better ability to understand the impact of specific behaviors on daily function. Another limitation of this study is the cross-sectional nature of the overall study design. Although this manuscript is not addressing causation, cross-sectional study designs are limited by the fact that they do not prove causation. Reporting bias might have occurred from the caregiver-form of the CBCL. Parents/guardians may have been more likely or less likely to report problematic behavior in their children. Parents/guardians may have over-reported problematic behaviors if they wanted to get an opportunity for their child to get a diagnostic work-up by the psychologist of the study. However, parents/guardians may have under-reported in order to prevent “labeling” of their child. A final limitation of this study is the generalizability. Although millions of children are exposed to coal ash, there is potential that the trace concentrations of heavy metals of the coal ash that children in this study are exposed to is different than the exposure of other children. Some coal ash may have greater concentrations of heavy metals and thus may have a greater impact on neurobehavioral performance of children.

In conclusion, the BARS is a computerized neurobehavioral test that provides a developmentally sensitive measure for assessing skills commonly found to be impacted by ADHD noted in changes in performance from childhood to adolescence. Impaired BARS performance also predicted ADHD features as measured by an objective rating scales indicating that the BARS measures behaviors relevant to daily activities for the child. Based on these findings, the BARS appears to be useful for study of neurotoxins that may affect cognitive control. Future research will explore relationships of the BARS tests with environmental and biologic measures of exposure to better understand previous findings of increased ADHD, as reported by caregivers, in children living near coal ash storage sites.

HIGHLIGHTS.

  • Computerized testing used the Behavioral Assessment and Research System (BARS)

  • Children living near coal ash storage sites completed the BARS and an ADHD scale

  • The BARS was sensitive to developmental changes in children’s cognitive functioning

  • The BARS had excellent external validity based on correlations with behavior

  • The BARS provides a dimensional measure to assess neurotoxin exposure in children

ACKNOWLEDGEMENTS:

The authors would like to acknowledge Lindsay Tompkins, Chisom Odoh, Jack Pfeiffer, Carol Norton, Jillian Winn, and Paula Kingsolver for their help with the overall study. We would like to thank the community and community leaders for their participation in this study. In addition, the authors would like to acknowledge C. Hanchette (deceased, October 2017) for her contributions to the overall study.

FUNDING: This work was supported by the National Institutes of Health, National Institute of Environmental Health Sciences [grant number R01ES024757].

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Declaration of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

REFERENCES

  1. Achenbach TM & Rescorla LA. (2001). Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth and Families. [Google Scholar]
  2. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author. [Google Scholar]
  3. Anger WK, Letz R, Chrislip DW, Frumkin H, Hudnell K, Russo JM, Chappell W, & Hutchinson L (1994). Neurobehavioral test methods for environmental health studies of adults. Neurotoxicology and Teratology. 10.1016/0892-0362(94)90128-7 [DOI] [PubMed] [Google Scholar]
  4. Bednar AJ, Averett DE, Seiter JM, Lafferty B, Jones WT, Hayes CA, Chappell MA, Clarke JU, & Steevens JA (2013). Characterization of metals released from coal fly ash during dredging at the Kingston ash recovery project. Chemosphere. 10.1016/j.chemosphere.2013.04.034 [DOI] [PubMed] [Google Scholar]
  5. Barkley RA (2019). Neuropsychological testing is not useful in the diagnosis of ADHD: Stop it (or prove it)! The ADHD Report 27 (2), 1–8. [Google Scholar]
  6. Benjamini Y & Hochberg Y (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57(1): 289–300. [Google Scholar]
  7. Berger I, Slobodin O & Cassuto H (2017). Usefulness and Validity of Continuous Performance Tests in the Diagnosis of Attention-Deficit Hyperactivity Disorder Children, Archives of Clinical Neuropsychology. 10.1093/arclin/acw101 [DOI] [PubMed] [Google Scholar]
  8. Bhangare RC, Ajmal PY, Sahu SK, Pandit GG,& Puranik VD (2011). Distributions of trace elements in coal and combustion residues from five thermal power plants in India. International Journal of Coal Geology. 10.1016/j.coal.2011.03.008 [DOI] [Google Scholar]
  9. Brown P, Jones T, & BéruBé K (2011) The internal microstructure and fibrous mineralogy of fly ash from coal-burning power stations. Environmental Pollution. 10.1016/j.envpol.2011.08.041 [DOI] [PubMed] [Google Scholar]
  10. Butler-Dawson J, Galvin K, Thorne PS, & Rohlman DS (2016). Organophosphorus pesticide exposure and neurobehavioral performance in Latino children living in an orchard community. Neurotoxicology, 53, 165–172. 10.1016/j.neuro.2016.01.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Caparros-Gonzalez RA, Giménez-Asensio MJ, González-Alzaga B, Aguilar-Carduño C, Lorca-Marín JA, Alguacil J, Gómez-Becerra I, Gómez-Ariza JL, García-Barrera T, Hernandez AF, López-Flores I, Rohlman DS, Romero-Molina D, Ruiz-Pérez I, & Lacasaña M (2019). Childhood chromium exposure and neuropsychological development in children living in two polluted areas in southern Spain. Environmental Pollution. 10.1016/j.envpol.2019.06.084 [DOI] [PubMed] [Google Scholar]
  12. Carlozzi NE, Tulsky DS, Kail RV, & Beaumont JL (2013). NIH toolbox cognition battery (CB): Measuring processing speed. Monographs of the Society for Research in Child Development. 10.1111/mono.12036 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Chang LY, Wang MY, & Tsai PS (2016). Diagnostic Accuracy of Rating Scales for Attention-Deficit/Hyperactivity Disorder: A Meta-analysis. Pediatrics. 10.1542/peds.2015-2749 [DOI] [PubMed] [Google Scholar]
  14. Chiodo LM, Covington C, Sokol RJ, Hannigan JH, Jannise J, Ager J, Greenwald M, & Delaney-Black V (2007). Blood lead levels and specific attention effects in young children. Neurotoxicology and Teratology 10.1016/j.ntt.2007.04.001 [DOI] [PubMed] [Google Scholar]
  15. Constantino JN (2011). The quantitative nature of autistic social impairment. Pediatric Research. 10.1203/PDR.0b013e318212ec6e [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Danielson ML, Bitsko RH, Ghandour RM, Holbrook JR, Kogan MD, & Blumberg SJ (2018). Prevalence of parent-reported ADHD diagnosis and associated treatment among U.S. children and adolescents, 2016. Journal of Clinical Child and Adolescent Psychology. 10.1080/15374416.2017.1417860 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Ebesutani C, Bernstein A, Nakamura BJ, Chorpita BF, Higa-McMillan CK, Weisz JR, & The Research Network on Youth Mental Health (2010). Concurrent Validity of the Child Behavior Checklist DSM-Oriented Scales: Correspondence with DSM Diagnoses and Comparison to Syndrome Scales. Journal of psychopathology and behavioral assessment, 32(3), 373–384. 10.1007/s10862-009-9174-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Environmental Protection Agency (2010). Hazardous and solid waste management system; identification and listing of special wastes; disposal of coal combustion residuals from electric utilities; proposed rule (Codified at 40 CFR Parts 257, 261, 264 et al.). Federal Register, 75(118):35128–35264. [Google Scholar]
  19. Eckerman DA, Gimenes LS, Curi de Souza R, Lopes Galvao PR, Sarcinelli PN, Chrisman JR (2007). Age related effects of pesticide exposure on neurobehavioral performance of adolescent farm workers in Brazil. Neurotoxicology and Teratology. doi: 10.1016/j.ntt.2006.09.028. [DOI] [PubMed] [Google Scholar]
  20. Farahat FM, Rohlman DS, Storzbach D, Ammerman T, Anger WK (2003). Measures of short-term test-retest reliability of computerized neurobehavioral tests. Neurotoxicology. 10.1016/S0161-813X(03)00079-2 [DOI] [PubMed] [Google Scholar]
  21. Hatori Y, Matsuyama S, Ishii K, Terakawa A, Kikuchi Y, Fujiwara H, Kawamura Y, Okura S, Fujikawa M, Hamada N, Fujiki K, Inoue C, Yamazaki H, & Hashimoto Y (2010). PIXE analysis of individual particles in coal fly ash. International Journal of PIXE. 10.1142/s0129083510001975 [DOI] [Google Scholar]
  22. Hollis C, Hall CL, Guo B, James M, Boadu J, Groom MJ, Brown N, Kaylor-Hughes C, Moldavsky M, Valentine AZ, Walker GM, Daley D, Sayal K, Morriss R and (2018), The impact of a computerised test of attention and activity (QbTest) on diagnostic decision-making in children and young people with suspected attention deficit hyperactivity disorder: single-blind randomised controlled trial. The Journal of Child Psychology and Psychiatry, 59: 1298–1308. doi: 10.1111/jcpp.12921 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Ismail AA, Bonner MR, Hendy O, Rasoul GA, Wang K, Olson JR, & Rohlman DS (2017a). Comparison of neurological health outcomes between two adolescent cohorts exposed to pesticides in Egypt. PLoS ONE. 10.1371/journal.pone.0172696 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Ismail AA, Wang K, Olson JR, Bonner MR, Hendy O, Abdel Rasoul G, & Rohlman DS (2017b). The impact of repeated organophosphorus pesticide exposure on biomarkers and neurobehavioral outcomes among adolescent pesticide applicators. Journal of Toxicology and Environmental Health - Part A: Current Issues. 10.1080/15287394.2017.1362612 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Khan KM, Parvez F, Zoeller RT, Hocevar BA, Kamendulis LM, Rohlman D, Eunus M, & Graziano J (2019). Thyroid hormones and neurobehavioral functions among adolescents chronically exposed to groundwater with geogenic arsenic in Bangladesh. The Science of the total environment, 678, 278–287. 10.1016/j.scitotenv.2019.04.426 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Khan KM, Weigel MM, Yonts S, Rohlman D, & Armijos R (2019). Residential exposure to urban traffic is associated with the poorer neurobehavioral health of Ecuadorian schoolchildren. Neurotoxicology, 73, 31–39. 10.1016/j.neuro.2019.02.018 [DOI] [PubMed] [Google Scholar]
  27. Lanphear BP, Hornung R, Khoury J, Yolton K, Baghurst P, Bellinger DC, Canfield RL, Dietrich KN, Bornschein R, Greene T, Rothenberg SJ, Needleman HL, Schnaas L, Wasserman G, Graziano J, & Roberts R (2005). Low-level environmental lead exposure and children’s intellectual function: An international pooled analysis. Environmental Health Perspectives. 10.1289/ehp.7688 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Liberda EN, Chen LC (2013). An evaluation of the toxicological aspects and potential doses from the inhalation of coal combustion products. Journal of the Air & Waste Management Association. 10.1080/10962247.2013.777374 [DOI] [PubMed] [Google Scholar]
  29. Mano KJ, Davies WH, Klein-Tasman BP, Adesso VJ (2009). Measurement equivalence of the child behavior checklist among parents of African American adolescents. Journal of Family Studies. doi: 10.1007/s10826-009-9263-0. [DOI] [Google Scholar]
  30. Morris SE, & Cuthbert BN (2012). Research domain criteria: Cognitive systems, neural circuits, and dimensions of behavior. Dialogues in Clinical Neuroscience. 10.1016/s0924-977x(17)31010-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. National Institute of Mental Health (2019). Attention Deficit Hyperactivity Disorder. Retrieved March 29, 2020, from https://www.nimh.nih.gov/health/topics/attention-deficit-hyperactivity-disorder-adhd/index.shtml [Google Scholar]
  32. Needleman HL, Gunnoe C, Leviton A, Reed R, Peresie H, Maher C, & Barrett P (1979). Deficits in Psychologic and Classroom Performance of Children with Elevated Dentine Lead Levels. New England Journal of Medicine. 10.1056/NEJM197903293001301 [DOI] [PubMed] [Google Scholar]
  33. Nie LH, Wright RO, Bellinger DC, Hussain J, Amarasiriwardena C, Chettle DR, Pejović-Milić A, Woolf A, & Shannon M (2011). Blood lead levels and cumulative blood lead index (CBLI) as predictors of late neurodevelopment in lead poisoned children. Biomarkers. 10.3109/1354750X.2011.604133 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Nikolas MA, & Nigg JT (2013). Neuropsychological performance and attention-deficit hyperactivity disorder subtypes and symptom dimensions. Neuropsychology. 10.1037/a0030685 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Norman LJ, Carlisi C, Lukito S, Hart H, Mataix-Cols D, Radua J, & Rubia K (2016). Structural and functional brain abnormalities in attention-deficit/hyperactivity disorder and obsessive-compulsive disorder: A comparative meta-analysis. JAMA Psychiatry. 10.1001/jamapsychiatry.2016.0700 [DOI] [PubMed] [Google Scholar]
  36. Odoh C, Sears CG, Tompkins LK, Hagemeyer AN, Pfeiffer JA, Polivka BJ, Sears L, Brock GN, Zhang C, & Zierold KM (2019). Recruitment strategies and challenges: Lessons learned from a coal ash and children’s health study. Research in Nursing and Health. 10.1002/nur.21986 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Patra KC, Rautray TR, Tripathy BB, & Nayak P (2012). Elemental analysis of coal and coal ASH by PIXE technique. Applied Radiation and Isotopes. 10.1016/j.apradiso.2011.12.013 [DOI] [PubMed] [Google Scholar]
  38. Perera F, Li TY, Zhou ZJ, Yuan T, Chen YH, Qu L, Rauh VA, Zhang Y, Tang D (2008). Benefits of reducing prenatal exposure to coal-burning pollutants to children’s neurodevelopment in China. 10.1289/ehp.11480 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Raiker JS, Freeman AJ, Perez-Algorta G, Frazier TW, Findling RL, & Youngstrom EA (2017). Accuracy of Achenbach Scales in the Screening of Attention-Deficit/Hyperactivity Disorder in a Community Mental Health Clinic. Journal of the American Academy of Child and Adolescent Psychiatry. 10.1016/j.jaac.2017.02.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Rauh VA, & Margolis AE (2016). Research Review: Environmental exposures, neurodevelopment, and child mental health – new paradigms for the study of brain and behavioral effects. Journal of Child Psychology and Psychiatry and Allied Disciplines. 10.1111/jcpp.1253 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Rodrigues EG, Bellinger DC, Valeri L, Hasan MOSI, Quamruzzaman Q, Golam M, Kile ML, Christiani DC, Wright RO, & Mazumdar M (2016). Neurodevelopmental outcomes among 2- to 3-year-old children in Bangladesh with elevated blood lead and exposure to arsenic and manganese in drinking water. Environmental Health: A Global Access Science Source. 10.1186/s12940-016-0127-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Rodríguez-Barranco M, Gil F, Hernández AF, Alguacil J, Lorca A, Mendoza R, Gómez I, Molina-Villalba I, González-Alzaga B, Aguilar-Garduño C, Rohlman DS, & Lacasaña M (2016). Postnatal arsenic exposure and attention impairment in school children. Cortex. 10.1016/j.cortex.2014.12.018 [DOI] [PubMed] [Google Scholar]
  43. Rohitrattana J, Siriwong W, Suittiwan P, Robson MG, Strickland PO, Rohlman DS, & Fiedler N (2014). Adaptation of a neurobehavioral test battery for Thai children. Roczniki Panstwowego Zakladu Higieny, 65(3), 205–212. [PMC free article] [PubMed] [Google Scholar]
  44. Rohlman DS, Anger WK, Tamulinas A, Phillips J, Bailey SR, & McCauley L (2001). Development of a neurobehavioral battery for children exposed to neurotoxic chemicals. NeuroToxicology. 10.1016/S0161-813X(01)00049-3 [DOI] [PubMed] [Google Scholar]
  45. Rohlman DS, Gimenes LS, Ebbert C, Anger WK, Bailey SR, & McCauley L (2000a). Smiling faces and other rewards: Using the Behavioral Assessment and Research System (BARS) with unique populations. Neurotoxicology, 21(6), 973–978. [PubMed] [Google Scholar]
  46. Rohlman DS, Bailey SR, Brown M, Blanock M, Anger WK, & McCauley L (2000b). Establishing stable test performance in tests from the Behavioral Assessment and Research System (BARS). Neurotoxicology, 21(5), 715–723. [PubMed] [Google Scholar]
  47. Rohlman DS, Gimenes LS, Eckerman DA, Kang SK, Farahat FM, & Anger WK (2003). Development of the Behavioral Assessment and Research System (BARS) to detect and characterize neurotoxicity in humans. NeuroToxicology. 10.1016/S0161-813X(03)00023-8 [DOI] [PubMed] [Google Scholar]
  48. Rohlman DS, Ismail AA, Rasoul GA, Bonner MR, Hendy O, Mara K, Wang K, & Olson JR (2016). A 10-month prospective study of organophosphorus pesticide exposure and neurobehavioral performance among adolescents in Egypt. Cortex. 10.1016/j.cortex.2015.09.0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Rohlman DS, Villanueva-Uv E, Ramos EAM, Mateo PC, Bielawski DM, Chiodo LM, Delaney-Black V, McCauley L, & Ostrea EM (2008). Adaptation of the Behavioral Assessment and Research System (BARS) for evaluating neurobehavioral performance in Filipino Children. Neurotoxicology. doi: 10.1016/j.neuro.2007.10.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Sagiv SK, Kalkbrenner AE, & Bellinger DC (2015). Of decrements and disorders: Assessing impairments in neurodevelopment in prospective studies of environmental toxicant exposures. Environmental Health: A Global Access Science Source. https://doi.org/1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Sears CG, & Zierold KM (2017). Health of Children Living Near Coal Ash. Global Pediatric Health. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Sears CG, Sears L, & Zierold KM (2020). Sex differences in the association between exposure to indoor particulate matter and cognitive control among children (age 6–14 years) living near coal-fired power plants. Neurotoxicology and Teratology. 10.1016/j.ntt.2020.106855 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Spencer LLS & Drake LD (1987). Hydrogeology of an alkaline fly ash landfill in Eastern Iowa. Groundwater. 10.1111/j.1745-6584.1987.tb02881.x [DOI] [Google Scholar]
  54. Tang D, Li TY, Liu JJ, Zhou ZJ, Yuan T, Chen YH, Rauh VA, Xie J, & Perera F (2008). Effects of prenatal exposure to coal-burning pollutants on children’s development in China. Environmental Health Perspectives. doi: 10.1289/ehp.10471 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Tang D, Lee J, Muirhead L, Li TY, Qu L, Yu J, & Perera F (2014). Molecular and neurodevelopmental benefits to children of closure of a coal burning power plant in China. PLoS One. doi: 10.1371/journal.pone.0091966. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Verma C, Madan S, & Hussain A (2016). Heavy metal contamination of groundwater due to fly ash disposal of coal-fired thermal power plant, Parichha Jhansi, India. Cogent Engineering. 10.1080/23311916.2016.1179243. [DOI] [Google Scholar]
  57. Zierold KM & Odoh C (2020). A review on fly ash from coal-fired power plants: Chemical composition, regulations, and health evidence. Reviews on Environmental Health. doi: 10.1515/reveh-2019-0039. [DOI] [PubMed] [Google Scholar]

RESOURCES