ADAPTATION OF A NEUROBEHAVIORAL TEST BATTERY FOR THAI CHILDREN

Abstract

Background

Exposure to neurotoxicants is a world wide problem with significant health implications for child development. In spite of higher neurotoxicant exposures, many developing countries do not have established neuropsychological instruments.

Objective

This study evalauted the adaptation and reliability of a computer and examiner administered Behavioral Assessment and Research System (BARS) that includes tests of motor speed and dexterity, attention, memory, and visuospatial coordination for use in Thailand.

Material and methods

To assess test-retest and alternate form reliability, BARS was administered to 24 healthy, 6–8 year old urban Thai children during two testing sessions two weeks apart. A comparison group of 29 healthy, rural Thai children of similar age and sex completed the BARS as part of another study and comprised a comparison group.

Results

Test-retest reliabilities for tests without alternate forms ranged from 0.41 to 0.77, but reliabilities were lower for tests with alternate forms (0.11 to 0.83). Paired t-tests revealed few significant differences in group performance between test administrations. Performance of urban Thai participants was compared to 29 rural Thai participants of similar age and sex. Parental education was significantly greater for urban vs. rural participants, resulting in signficant differences in performance on tests of motor speed.

Conclusions

This study supports the use of BARS for epidemiologic studies of neurotoxicants in Thailand, but highlights the sensitivity of these tests to differences in parental education and the need for improved alternate test forms.

INTRODUCTION

Environmental health issues are a world health concern not only in industrialized countries but also areas of developing countries such as Thailand [8]. Adverse effects on cognitive development among children from industrialized countries have been demonstrated for exposures to a number of neurotoxicants such as lead, mercury, and pesticides [4, 6, 10, 13, 16]. Arguably, children from developing countries are being exposed to higher concentrations of neurotoxicants than in the developed world [14], but few studies have been conducted to evaluate the impact of these exposures on children’s cognitive function. The dearth of neurobehavioral research in developing countries is due, in part, to a lack of culturally relevant test batteries. The purpose of the current study is to evaluate the adaptation of a neurobehavioral battery for use in Thailand.

Few tests are available to evaluate cognitive and psychomotor function of Thai children. The Wechsler Intelligence Scale for Children (WISC) is a widely used intelligence test that has been adapted primarily for clinical assessment of Thai children [3]. The WISC is lengthy, must be administered by a psychologist, and is not suitable for field epidemiologic studies of children. A small number of studies have used the human figure drawing test, the Test of Nonverbal Intelligence version 3 or standard tests of math and verbal fluency to estimate intelligence among Thai children [9, 15, 21]. Relatively more neuropsychological tests, however, are adapted to screen for dementia among the elderly in Thailand rather than for assessment of children [11].

The Behavioral Assessment and Research System (BARS), originally developed for neurobehavioral evaluation of adults, has been adapted for use in children from 5 years old [17]. This test battery is economical, requires limited language and education abilities, and has been translated into multiple languages to include Spanish, Portuguese, Arabic, and Korean. The BARS has been used for many studies in adults, adolescents, and children in the U.S. and in developing countries and has demonstrated utility for making cross-cultural comparisons of performance [1, 5, 12, 18, 20]. Farahat et al [7] reported test-retest reliabilities for BARS administered to adults residing in the United States ranging from 0.35 to 0.85 while Rohlman et al [20] reported similar one-month, test-retest correlations for 4 to 9 year old Hispanic non-English speaking children. Neurotoxicant exposures may affect cognitive and motor skills differentially depending on the developmental stage at which exposure occurred, the frequency of exposure and a host of other variables that make it difficult to separate acute, temporary effects from persistent decrements in function. Therefore, repeated assessment of children’s cognitive and motor skills is often desirable, particularly in situations where intermittent acute exposures occur in the context of chronic background exposure such as those seen with pesticides in farming communities. Using the same tests allows direct comparisons over time and in differing exposure scenarios, but practice effects may hinder the sensitivity of the tests for detecting subtle behavior change. Therefore, the purpose of the current study is to 1) demonstrate the utility and test-retest reliability of BARS for Thai children, 2) to develop and assess the performance of alternate forms of those BARS tests vulnerable to practice effects, and 3) to compare performance of urban and rural Thai children.

MATERIAL AND METHODS

Participants

To assess the suitability, test-retest reliability, and alternate form reliability of the testing battery. Twenty-four healthy 5 years, 10 months to 8 years, 11 months Thai children from Bangkok (urban sample) volunteered to complete the test battery. The study was explained fully to parents who signed the consent form and the participating children gave verbal assent prior to participation. The study was reviewed and approved by the Institutional Review Boards of Chulalongkorn University and Rutgers-Robert Wood Johnson Medical School.

Neurobehavioral Tests

The following tests (Table 1) were presented on a computer screen equipped with a 9 BUTTON response unit: finger tapping (TAP), match-to-sample (MTS), symbol digit (SD), and the continuous performance test (CPT). In addition digit span (DST) and Object Memory (OMT) were administered by an examiner as were the following tests adapted from the Pediatric Environmental Neurobehavioral Test Battery (PENTB)[2]: Purdue pegboard (PEG), visual motor integration (VMI), and divided attention (DAT).

Table 1.

Description of neurobehavioral tests and functions for BARS

Test Description	Function	Variables
Finger tapping (TAP) Right and left hand taps for 20 seconds; 2 trials/hand	Response speed and coordination	Average number of taps each hand
Divided attention (DAT) Tap while reciting nursery rhyme (Chang song)	Divided attention	Average number of taps each hand while singing
Purdue pegboard (PEG) Number of small pegs placed in holes during two 30 second trials each hand Preferred, non-preferred, and both hand trials	Dexterity	Average number of pegs placed: preferred, non-preferred, both
Visual motor integration (VMI) Copied line drawing	Hand-Eye coordination	Total score for correct segments
Digit span (DST) Spoken presentation of number sequences Forward and reverse recall	Memory and attention	raw score maximum digits forward, backward
Object memory test (OMT) Show and name 16 objects Immediate and delayed recall Recognition of target and non-target items	Recall and recognition memory	Immediate recall; delayed recall; recognition
Symbol-Digit (SDT) Match number and symbol from key	Information processing speed	Average latency (ms) of response for correct match
Match-to-Sample (MTS) 15 stimuli shown for 3 seconds Identify target from 3 choices Delay between presentation and choice varies from 1 to 8 seconds	Visual memory	Average latency (ms) for correct choice Number correct
Continuous performance (CPT) Different shapes shown rapidly for 4 min in original version and 7 min in alternate version Press key when target (original = circle; alternate = triangle) shown	Sustained attention	Percent correct Average latency (ms) for correct response (hit) Average latency (ms) for false alarms D-Prime

Adapted in part from Rohlman et al [20].

Cultural adaptation

Some parameters in the neurobehavioral tests were adjusted or substituted for items familiar to Thai children. For example in the OMT which uses common objects, paper was substituted for envelope because envelopes are not familiar to young Thai children. In addition, hair brush was changed to a hair clip because hair brush has the same pronounciation in Thai as toothbrush; fork was also substituted for the chopstick and ruler for pen. The “Happy Birthday song” used in the DAT was changed to “Chang song”, a song about elephants that is common in Thai kindergarten. Video and audio instructions for all tests were translated into Thai.

To reduce practice effects, alternate forms were developed for the following: OMT, SD, MTS, and CPT. A second set of objects regarded as familiar to Thai children were selected for the OMT. The stimulus in CPT was a circle in the original test and a triangle in the alternate test. For CPT, the original version used 100 trials while the alternate was 200 trials to determine the effect of prolonged test time on reliability. Target samples were changed for all trials of the alternate version of MTS. For SD, the pairing of each symbol and number were changed for the alternate version. It was not possible to develop an alternate form of the motor/visuomotor tests, i.e. PEG, TAP, and VMI. There are no alternate items for VMI and therefore, the same items were administered during both test sessions.

The test administrators were doctoral and masters students from the College of Public Health Science and Faculty of Psychology, Chulalongkorn University. All examiners were trained during three separate sessions at least 3 times prior to test administration with the children. The training scheme included a brief introduction to neurobehavioral tests, purpose of each test, and demonstration of proper test administration. During the second training session, the tester practiced the test with their colleagues. They were taught how to troubleshoot the test administration, and what they should say in response to subject performance and questions during the test. For example, they were instructed not to indicate if an answer was “correct” or “wrong”, but instead to use phrases such as “go on”, “keep trying”, “try more” to encourage the child’s persistence with the test.

Procedure

Participants were tested twice, 2 weeks apart. Order of the original and alternate test form was counterbalanced and a different tester administered the two versions of the test for each child. All neurobehavioral tests were completed in a quiet classroom with one child tested at a time. In each test station, an examiner gave the child instructions. A trained examiner observed the child perform the practice tests to be certain the child understood the instructions and clarified instructions if the child did not perform the test correctly. The examiner also provided encouragement to maintain the child’s attention to the test. Each child had a 10 minute break between completion of the computerized tests and non-computerized tests. After the participant accomplished all neurobehavioral tests, the child received a small gift as a reward.

Statistical Analysis

The statistical analyses were performed using SAS for Windows, version 9.3. Mean and standard errors were calculated for each variable assessed. Pearson product-moment correlations and paired t-tests were used to assess test-retest and alternate form reliability and the effects of age and sex. The Folded F-statistic was used to assess equality of variances followed by ANCOVA with adjustment for age to compare urban and rural Thai subjects’ performance.

RESULTS

The average age of the Thai participants was 7.4 years (SD = ±0.85) with an equal number of participants at each age, grade level and sex. All children were native Thai speakers.

Test-retest results

All Thai children completed all tests. Means, standard deviations, p-values of mean differences and correlation coefficients are presented in Table 2. For tests without alternate forms, paired t-tests revealed no significant differences between tests administered at time 1 and 2 with the exception of VMI. VMI performance significantly improved from the first to the second testing session. All test-retest reliabilities were significant for tests without alternate forms ranging from r = 0.41 to 0.77.

Table 2.

Mean and standard deviation, paired t-test, and correlations of test-retest scores with no alternate BARS form (n = 24)

Test	Variables	Test (T1)		Test (T2)		Paired t-test	Correlation
		Mean	S.D.	Mean	S.D.	p-value	r	p-value
TAP	Right hand	75.40	7.30	76.90	6.30	0.166	0.71	<0.001
	Left hand	66.70	9.40	66.60	8.90	0.965	0.72	<0.001
DAT: song	Tap right average	56.70	7.30	56.90	5.70	0.853	0.67	<.001
	Tap left average	51.00	8.30	50.70	6.80	0.746	0.77	<0.001
PEG	Preferred hand	13.60	2.10	13.80	1.80	1.000	0.72	<0.001
	Non-preferred hand	12.70	2.10	12.50	1.60	0.890	0.71	<0.001
	Both hands	10.50	1.70	10.50	1.50	0.870	0.71	<0.001
VMI	Total Correct	16.00	1.60	16.60	1.30	0.022	0.64	<0.001
DST	Maximum digits forward	7.09	1.59	7.04	1.27	0.900	0.41	0.047
	Maximum digits backward	3.08	0.97	2.96	0.75	0.500	0.48	0.018

Alternate form test results

For tests with alternate forms, paired t-tests revealed no significant differences between the alternate forms with the exception of CPT hit latency and CPT latency for false alarms (i.e., incorrect responses). Speed of response was slower for the CPT alternate version in which the number of trials completed by subjects was increased (i.e., 200 trials) (Table 3). A paired t-test comparing performance on the first 100 vs. second 100 trials of the CPT alternate form revealed that hit latency was significantly slower during the second 100 trials (1^st 100 mean = 489.0 (SD = 121); 2^nd 100 mean = 540.2 (SD = 139); p = 0.005), but false alarm latency was not significantly different (p = 0.55). When hit latency for the original form (i.e., 100 trials) was compared to the latency for the first 100 trials of the alternate form, hit latency was not significantly different. Original and alternate forms of SDT, MTS, and CPT were significantly correlated for latency of correct responses and for percent hits on CPT (r = 0.63 to 0.81). Although speed of response for MTS was highly correlated between the original and alternate stimuli, the number of correct responses was not, suggesting that even though the alternate forms did not affect the overall group mean, individuals did not perform similarly on the original and alternate forms of MTS. Thus, further work will be required to insure the equivalence of the items.

Table 3.

Mean and standard deviation of original and alternate BARS scores (n=24).

Test	Variables	Original form		Alternate form		Paired t-test	Correlation
		Mean	SD	Mean	SD	p-value	r	p-value
OMTa	Immediate recall	8.29	1.81	7.63	2.16	0.175	0.32	0.129
	Delay recall	6.71	2.03	6.83	2.20	0.825	0.16	0.445
	Recognition	15.75	0.44	15.63	1.06	0.543	0.35	0.094
SDT	Latency (ms)	3947.20	1287.10	4107.60	1048.70	0.954	0.83	<0.001
MTS	Latency (ms)	3673.00	613.30	3601.20	519.60	0.486	0.63	0.001
	Correct	12.30	1.70	12.70	1.30	0.278	0.11	0.603
CPT	Percent Hits	0.84	0.19	0.84	0.13	0.897	0.78	<0.001
	HitLatency (ms) (1st 100 trials of alternate)	453.40	86.00	513.00 (488.70)	122.10 (120.80)	0.004 (p=0.10)	0.65	<0.001
	Percent False Alarms	0.11	0.09	0.09	0.08	0.142	0.77	<0.001
	FALatency (ms)	410.70	163.20	505.00	200.20	0.041	0.33	0.118
	Correct Dprime	2.70	1.20	2.70	0.90	0.990	0.81	<0.001

^aAll items included

For OMT, scores were not significantly different between the alternate forms, but correlations were low and non-sigificant, suggesting that individual subjects did not perform similarly in response to the alternate forms. Item analyses were conducted by assessing the number of children who correctly remembered each item at both immediate and delayed recall for each form of the test. The number who correctly recalled each item immediately and after a delay was used to predict the total test score for the test version in which that item was administered and the total test score for the alternate test or the test in which the item was not given. Using regression models, items that poorly predicted the alternate form total score for immediate and delayed memory were removed (i.e., negative beta weight) (i.e., comb, ball, and paper from original test; ice cream, plane, and bottle from alternate version). The correlation between immediate memory (r = 0.41; p = 0.05) and delayed memory (r = 0.44; p = 0.03) scores for these modified versions were both significant. Paired t-tests comparing mean performance between these versions were not significantly different for immediate (immediate: original mean = 6.88 (SD = 1.60); alternate mean = 6.58 (SD = 2.06)) or delayed memory (delayed: original mean = 5.17 (SD = 1.8); alternate mean = 5.96 (SD = 2.10)).

Age and sex effects

Only tests of motor speed and dexterity, and visual motor integration were significantly correlated with age (PEG right hand: r = 0.65; left hand: r = 0.73; both hands: r = 0.73; VMI: r = 0.47) while the remainder of the tests were not significantly affected by age. Females had significantly more correct responses on MTS (female mean = 12.58 (SD = 1.74); male mean = 12.38 (SD = 1.28); p = 0.03). They also were significantly faster in coding digits with symbols (SDT) (female mean = 3617.30 (SD = 768.77); male mean = 4280.66 (SD = 1043.13)).

Comparison to rural Thai children

Twenty-nine, healthy 6 to 8 years, 5 month old children from a rural shrimp farming community (rural sample) also completed the original version of the BARS battery. These subjects were recruited as a control group for a study evaluating the cognitive and motor effects of pesticide exposure. The same procedures were used to evaluate this group of children during a single testing session. Because they were of similar age, education level, and sex, comparison with the performance of urban Thai children provides another evaluation of the utility of this testing protocol for use in Thailand.

The urban and rural subjects did not differ in sex distribution, but the urban sample was significantly older (urban mean: 89.46 (SD = 11.77) months; rural mean: 82.72 (SD = 9.52) months; p <0.03) and their parents were significantly more educated (urban mean: 14.29 (SD = 2.80) years; rural mean: 8.97 (SD = 4.53) years; p <0.001) than the rural sample. Therefore, an analysis of covariance adjusting for age was used to compare performance of these groups on the original BARS battery of tests (Table 4). Urban subjects exhibited significantly greater motor speed for the right and left hand (TAP, DAT, PEG) than rural subjects, but no other significant differences were observed. Because the distibution of parental education was remarkably different between the urban and rural sample, a subset of rural children whose parents had > 9 years of education were selected (n = 14) to compare performance with the urban sample (n = 24). After adjusting for age, no significant differences in test performance were observed between the urban and a subset of the rural subjects with comparable parental education (data not shown).

Table 4.

Age-adjusted covariance analyses of BARS performance: Thai urban (n=24) vs. rural (n=29)

Variable	Group	Mean	Stanadard deviation	F value	p value
MTS_Correct	urban	12.25	1.67	2.23	0.141
	rural	10.83	2.94
MTS_Latency (ms)	urban	3676.90	625.00	0.09	0.769
	rural	3857.70	650.00
TAP_R	urban	76.10	7.54	3.61	0.063
	rural	68.35	12.20
TAP_L	urban	66.90	9.39	5.90	0.019
	rural	57.48	11.47
CPT_percent Hits	urban	0.84	0.19	0.22	0.638
	rural	0.80	0.18
CPT_percent FA	urban	0.11	0.09	0.38	0.540
	rural	0.13	0.10
CPT_Hit Latency (ms)	urban	453.40	86.03	0.90	0.348
	rural	498.70	139.20
CPT_FA Latency (ms)	urban	410.70	163.20	0.21	0.646
	rural	464.30	223.20
Correct_Dprime	urban	2.66	1.17	0.96	0.332
	rural	2.28	0.87
DAT_R	urban	56.71	7.32	5.30	0.026
	rural	48.21	12.04
DAT_L	urban	51.02	8.32	4.03	0.050
	rural	55.14	9.59
DST Maximum Digits Forward	urban	7.08	1.59	2.89	0.095
	rural	6.24	1.43
OMT_Immediate	urban	8.29	1.81	2.99	0.090
	rural	6.48	3.19
OMT_Delay Recall	urban	6.71	2.03	2.27	0.138
	rural	5.31	2.75
OMT_Recognition	urban	15.75	0.44	3.09	0.085
	rural	13.69	4.20
PEG_Pref	urban	13.58	2.09	2.55	0.117
	rural	12.14	1.90
PEG_nonPref	urban	12.71	2.09	7.68	0.008
	rural	10.88	1.47
PEG_B	urban	21.42	3.49	3.50	0.067
	rural	18.62	3.29
VMI	urban	16.00	1.56	0.11	0.738
	rural	15.69	2.14

Analyses adjusted for age

DISCUSSION

Thai children were able to complete all neurobehavioral tests with few significant differences in performance over a two week period, suggesting that BARS has utility for epidemiologic studies where repeated testing of Thai children is needed. As a group the children’s mean performance was comparable between the first and second testing sessions on all but visual motor integration and latency of response for continuous performance. Hand-eye coordination (VMI) showed significant improvement on re-testing probably because of familiarity with the figures to be copied while the increased number of trials for the alternate form of CPT contributed to differences in performance. When the 1st 100 trials for the alternate version of CPT was compared to the 2^nd 100 trials, significant slowing of response (i.e., latency of hits and false alarms) was observed, suggesting that fatigue may have been a factor. Moreover, when the first 100 trials of the alternate version were compared to the original CPT with only 100 trials, differences in latency were no longer significant. With the exception of object memory (OMT) and number correct in match to sample (MTS), test-retest reliabilities were similar to those cited previously among 4 to 9 year old Hispanic children from the U.S. [20]. Both OMT and MTS used alternate stimuli which likely contributed to lower correlations between test administrations and therefore, further work will be required to insure comparability of these alternate forms.

Repeated testing did not result in signficant changes in group performance for tests of motor speed to include tapping with and without distraction (TAP; DAT) and fine motor manipulation (PEG). However, future studies where repeated testing is planned should consider developing alternate figures for VMI to reduce practice effects. Accuracy in sustained attention (percent hits for CPT), speed and accuracy of information processing (latency for correct SDT responses), and accuracy of immediate memory (maximum digits for DST) were comparable between testing sessions with children exhibiting consistent performance (r = 0.41–0.83). Thus although children performed more slowly when given more trials for CPT, their accuracy remained consistent.

Group performance for immediate and delayed recall and recognition of objects (OMT) was also similar for alternate forms of the test, but the relative performance of each child as revealed by low test-retest reliabilities was inconsistent. The regression analysis revealed improved test-retest correlation for both immediate and delayed memory when items that were poor predictors of each version were excluded from the total. Testing with a different sample of children will be required to cross-validate this result, but further work on alternate form development would enhance the applicability of this test for sitatuions when repeated testing is required.

Within this relatively restricted age range, only motor (PEG) and visual motor tests (VMI) showed improvement with age. Females exhibited faster information processing (SDT) and more accurate visuospatial processing (MTS) than males. These findings are consistent with Rohlman’s observation that Filipino, Spanish and English speaking females performed better than males of the same age although they did not observe significant differences [19].

After adjustment for age and parental education, urban and rural Thai children performed similarly on BARS. Even when parental education was not controlled, performance of the urban and rural samples was quite similar except for speed of motor performance suggesting that although parental education was much greater in the urban sample, this made little difference in the performance of the children. Moreover, rural Thai children have much less access to extracurricular activities and their socioeconomic status is generally lower than urban children. Nevertheless, their performance was comparable further validating the suitability of BARS for children of very different cultural and socioeconomic background.

At this stage of development BARS is suitable for epidemiologic studies of children who may be exposed to neurotoxicants or for other group comparisons, but cannot be used to predict or evaluate an individual child’s performance. This study lends support to the use of BARS across diverse cultures, socioeconomic groups, and languages which can help strengthen our ability to aggregate data from disparate samples toward understanding neurodevelopment in a variety of circumstances.

CONCLUSIONS

This study demonstrates the utility of using BARS to assess cognitive and motor performance in Thai children. Overall test-retest reliability is acceptable although alternate forms need further refinement to improve their comparability. As for most neurobehavioral tests, age, sex, and parental education influence performance and must be controlled for in the design and analysis of these tests. Continuing cross cultural validation of the BARS will allow data aggregation for the purpose of assessing the effects of world wide neurotoxicant exposure on child development.