Psychometric properties of Persian version of the Sustained Auditory Attention Capacity Test in children with attention deficit-hyperactivity disorder

Abstract

Background: The purpose of the present study was to evaluate the psychometric properties (validity and reliability) of the Persian version of the Sustained Auditory Attention Capacity Test in children with attention deficit hyperactivity disorder.

Methods: The Persian version of the Sustained Auditory Attention Capacity Test was constructed to assess sustained auditory attention using the method provided by Feniman and colleagues (2007). In this test, comments were provided to assess the child’s attentional deficit by determining inattention and impulsiveness error, the total scores of the sustained auditory attention capacity test and attention span reduction index. In the present study for determining the validity and reliability of in both Rey Auditory Verbal Learning test and the Persian version of the Sustained Auditory Attention Capacity Test (SAACT), 46 normal children and 41 children with Attention Deficit Hyperactivity (ADHD), all right-handed and aged between 7 and 11 of both genders, were evaluated.

Results: In determining convergent validity, a negative significant correlation was found between the three parts of the Rey Auditory Verbal Learning test (first, fifth, and immediate recall) and all indicators of the SAACT except attention span reduction. By comparing the test scores between the normal and ADHD groups, discriminant validity analysis showed significant differences in all indicators of the test except for attention span reduction (p< 0.001).

Conclusion: The Persian version of the Sustained Auditory Attention Capacity test has good validity and reliability, that matches other reliable tests, and it can be used for the identification of children with attention deficits and if they suspected to have Attention Deficit Hyperactivity Disorder.

Introduction

Attention is an important cognitive process for successful participation in learning and training (1). This ability is a complex process that limits the amount of input information in any interval to maximize the amount of stimulus information processing and storage (2). In order to promote compatibility with internal and external needs, it seems that attention as a cognitive process is coordinated with other cognitive activities such as memory and motivation (1). It is thought that attention is not a single concept (3) and can be classified based on the type of task (2). One of the most important types of attention is sustained attention, defined as the readiness to explore and respond to certain small changes that occur randomly in the environment (4).

The clinical evaluation of sustained attention involves the use of various tools including behavioral assessment, such as continuous performance tests (CPTs). The optimal psychological features of CPTs have led to their wider use as a valuable scale for assessing sustained auditory attention capacity. Today, several different versions of these tests are available (5). The Sustained Auditory Attention Capacity Test (SAACT) is a continuous performance test that was introduced in 2007 by Feniman and colleagues to assess sustained auditory attention. A child’s auditory attention behavior can be described objectively by this test and is evaluated by determining the child’s capacity to listen to auditory stimuli over a long period of time and respond to a specific stimulus. In this test, comments are provided that describe the child’s attentional deficit by determining two types of error: inattention, or when the child raised up his/her hand in reply to the target-word; and impulsiveness error, or when the child raised up his/her hand for another word rather than the target-word. The test also provides an attention span reduction index, which represents the attentional weakness of the child during the test (6).

Attention Deficit Hyperactivity Disorder (ADHD) is one of the most common childhood disorders (7-9) and occurs in 3 to 7 percent of school-age children (10). According to surveys, the most significant characteristic of this deficit is the failure to maintain sustained attention (4). Three different types of this disorder have been proposed by the Diagnostic and Statistical Manual of Mental Disorders (DSM) as: attention deficit disorder (ADD), hyperactivity and impulsivity, and combined type (7, 11, 12). Many aspects of children’s life are affected by this disorder, including communication, parent-child relationships, academic performance, social skills, self-confidence, and future careers. Although effective treatments are available, so far no simple cause has been identified. Nevertheless, the research conducted in this field indicates that this disorder is caused by a combination of environmental and genetic factors (10).

Taking advantage of neuropsychological tools is one of the major research methods in the study of mental and cognitive function, especially in the field of measuring attention. While normalized versions of these tests have been used in many countries, few studies have been conducted in Iran in the field of psychometric properties of psychological tools such as the assessment of attention and cognition. Since during the last two decades, special attention has been paid to the attentional deficit features rather than the restless movements of children with ADHD, thus the CPTs have acquired a certain popularity for research and diagnosis for these children (8). Therefore, in the present study, the Persian version of the SAACT was administered for normal and children with ADHD, and its validity and reliability were determined. Since attention disorders are very common in children, especially in school-age, having normal values and psychometric properties for the Persian version of the SAACT in Persian children can be useful and important for the identification of anyone with attentional deficits and also for screening deficiencies in children, especially in pre-school-age children.

Methods

Our study was conducted on 46 normal children (22 girls and 24 boys) with a mean age of 8.50 and a standard deviation of 1.11 years and 41 children with ADHD (16 girls and 25 boys) with a mean age of 8.24 and standard deviation of 1.02 years, aged between 7-11 years in city of Tehran. Normal children had normal hearing thresholds (passed the pure-tone audiometry screening), a normal intelligence score (IQ≥90) according to the Persian version of the Wechsler Intelligence Scale for Children Revised (WISC-R), right hand superiority in single-handed exercises according to the Edinburgh Handedness Inventory, and were native monolinguals of Persian language that selected from student of elementary schools located in the 6^th region of Tehran province. Children with ADHD were selected among those referred from public schools in Tehran province to the three centers of identification and management of learning disabilities in Tehran. In children with ADHD, besides the inclusion criteria listed for normal children, attentional deficit was also confirmed by a skilled psychiatrist according to the (DSM-IV).

The child was excluded from the study if there was any history of neurological problems, convulsions, high fever, epilepsy, head trauma, ototoxic medications, brain surgery, or behavioral problems, according to medical records. This research was conducted with the approval of the Ethics Committee of Tehran University of Medical Sciences.

First, otoscopic inspection was performed to examine the ear canal, cerumen, and healthy eardrum and then pure tone audiometry screening performed to check each child’s hearing status at frequencies of 500, 1000, 2000 and 4000 Hz at a 20 dB HL intensity level (13). If the child failed in the screening, he/she was referred to an audiology clinic for a more accurate assessment of hearing status. After the verification of a child’s normal hearing status, the Persian version of the SAACT (14) was administered to each child.

Speech stimuli were used to construct the Persian version of the SAACT, similar to the original version constructed by Feniman and colleagues in 2007. These stimuli consisted of 100 monosyllabic words where the target word was repeated 20 times in random order. These 100 words, basically a list of 21 monosyllabic words, that were repeated and rearranged randomly (14).

This list was played six times without interruption to each child at a comfortable hearing level in a diotic fashion. Each child was asked to raise his/her hand immediately after hearing the target word. Before the test word list was given to the child for the first time, a sample list containing 50 monosyllable words (10 target words) was provided to familiarize the child with the test procedure, and to ensure that kid understood the test correctly before initiating the main experiment. This list was shorter but formed from the same original list of words and only used for ensuring familiarity with the test procedure. In order to define the results of this test, two types of errors were computed: inattentive error, or when the child raised up his/her hand in reply to the target-word; and impulsiveness, or when the child raised up his/her hand for another word rather than the target-word. The attention span reduction index was also computed that reflected the attentional weakness of the child during the test. It was calculated by subtracting the number of correct answers to the target word the first time it played from the number of correct answers after it played six times. The total number of inattentive and impulsiveness errors, namely total error score, was also interpreted as the sustained auditory attention capacity for each child (14).

In this study, the construct validity of the Persian version of the SAACT was calculated using discriminant and convergent validity. For determining discriminant validity, the test’s ability to distinguish children with ADHD from normal ones was studied. First, the normal distribution of data was assessed by the Kolmogorov-Smirnov statistical test. Since the data distribution of the data was non-normal, thus the non-parametric Mann-Whitney U-test was used. To evaluate convergent validity, the correlation between all SAACT indicators and three parts of the Persian version of Rey Auditory Verbal Learning Test (RAVLT; first, fifth, and immediate recall) (15, 16) was determined. The RAVLT is a word-registry multi-stage test more often applied in neuropsychological evaluations (17). This test consists of two lists: a list of fifteen simple monosyllabic words that are familiar and understandable for children under 6 years of age (list A) followed by an intervening word list (list B). The list B was an interposition word list of 15 new ordinary nouns that followed by free recall of these new nouns. To evaluate the effect of stimulus repetition and the learning ability of an individual, the first list was presented five times consecutively, and then the mean total score calculated by determining the mean scores of these five presentations. In this two-stage (learning and interference) test, the test material was read to the individual with a one-second interval between words and after the completion of the list, each individual asked to repeat any number of words that he/she could recall. Immediately after the interference word list, each individual was asked to repeat any number of words that he/she could recall from list A(15,16).

Statistical Analysis

Statistical analysis was conducted by SPSS.18 statistical software at significance levels of p<0.05. Since the data distribution was non-normal, the Spearman correlation coefficient and Mann-Whitney U-tests were used to examine the discriminant and convergent validity, respectively. Also, the intra-class correlation (ICC) coefficients were calculated to assess the repeatability of SAACT results. For this purpose, 23 normal children (12 girls and 11 boys) completed this test twice within a month to avoid learning effects. To interpret the acquired correlation coefficients, we considered quantities of less than 0.4 as a weak reliability, more than 0.7 as great reliability, and between 0.4 and 0.7 as tolerable to fine reliability (18).

Results

Table 1 shows the mean and standard deviation of all test indicators in the normal group and children with ADHD.

Table 1. Descriptive statistics of all SAACT indicators in the normal group and children with ADHD.

Test indicators	Normal children (number=46)		ADHD children (number=41)
	Mean	SD	Mean	SD
Inattention error	1.09	1.532	10.10	11.694
Impulsiveness error	2.07	1.971	5.98	4.602
Total error score	3.15	2.716	16.07	15.170
Attention reduction span	0.17	0.570	1.05	2.489

In determining discriminant validity, a statistically significant difference was found between the average of inattention errors (p<0.001), impulsiveness errors (p<0.001), and total error score (p<0.001), but not attention span reduction index (p=0.507), in the two groups of children.

In determining convergent validity, we calculated Spearman’s correlation coefficient between all of the SAACT indicators and three parts of the RAVLT (first, fifth, and immediate recall) in children with ADHD (Table 2 ). As shown in Table 2 , a negative significant correlation was found between all SAACT indicators (except attention span reduction index) and three parts of the RAVLT.

Table 2. Assessing the convergent validity of SAACT in children with ADHD.

Test indicators		RAVLT
		First part	Fifth part	Immediate recall
Inattention error	r	-0.494	-0.515	-0.587
	p value	0.002**	**<0.001	<0.001**
Impulsiveness error	r	-0.333	-0.457	-0.562
	p value	0.044*	0.004**	<0.001**
Total error score	r	-0.549	-0.549	-0.628
	p value	**<0.001	<0.001**	<0.001**
Attention reduction span	r	-0.079	-0.168	-0.259
	p value	0.643	0.320	0.126
N		37	37	36

**p <0.01, *p<0.05

Table 3 shows the acquired ICC quantities for evaluating the repeatability of the SAACT. These results revealed admissible reliability for all indicators of the Persian version of the SAACT except attention span reduction.

Table 3. Reliability of all indicators for the SAACT in the group of normal children (n = 23).

Test indicators		ICC	Lower bound	Upper bound	p-value
Inattention error		0.635	0.310	0.827	<0.001
Impulsiveness error		0.700	0.412	0.861	<0.001
Total error score		0.702	0.415	0.862	<0.001
Attention reduction span		0.252	-0.170	0.595	0.118

Discussion

Attention plays a major role in neuropsychological evaluation, which means that maintaining attention while doing these exercises shapes the validity of such evaluations. Since attentional deficits are common in most neuropsychological disorders, such as ADHD, there is a need for feasible tests that can be applied by healthcare specialists. However, there are few scales for the assessment of attention that are clinically appropriate and that also have confirmed validity and reliability (19). For this reason, the purpose of this study was to further extend the results of relevant studies in the field of validity and reliability of a type of attention test called the SAACT.

In determining convergent validity, a negative significant correlation was found between the third parts of RAVLT (first, fifth, and immediate recall) and all indicators of the SAACT except attention span reduction. Since attention as a cognitive process can affect and limit the amount of memory in individuals, as a child’s attention deficit increases, he/she may show poorer memory. There has been no previous study that has examined the convergent validity of the attention span reduction index. In 2001, Christensen and Joschko examined the convergent validity of a computerized CPT and Continuous Attention Test (CAT), in 47 children aged between 6 and 11 who were referred to children’s rehabilitation centers because of a variety of problems including attentional deficit, developmental delays, learning disability, and behavioral problems (19). To examine the convergent validity of CAT, these researchers have used the Trail Making Test (TMT), The Holes Steadiness Test, and The Freedom from Distractibility factor (FFD) and reported poor convergent validity of the test. They attributed observed weakness to the difference in attentional structure of the CAT in comparison with the other tests that were used to determine convergent validity. The findings showed that the CAT was an appropriate clinical tool to assess sustained attention in children. The children who tested in the study of Christensen and Joschko (2001) had other problems besides attentional problems and seven of them were left-handed (19). These factors might have had an impact on the results of their study and could be a reason for the inconsistency between Christensen and Joschko’s and our findings.

In present study, the discriminant validity of the Persian version of the SAACT was also determined and the findings showed that this test had a good ability to separate children with ADHD from normal children in all indicators except attention span reduction. Since hearing has an important role in the evolution of speech, language, learning, and hearing ability, evaluation of the effect of hearing loss on auditory attention in children is essential. To determine the discriminant validity of the SAACT, Mondelli and colleagues (2010) compared 30 children aged between 7 and 11 with normal hearing thresholds with two groups of children who had mild bilateral sensorineural and conductive hearing loss. The results showed significant differences among the three groups studied in terms of inattention, impulsiveness and the total number of errors as well as vigilance decrement. Overall, this study showed that the SAACT has a good ability to differentiate children with normal hearing from these two other groups (20). In 2010, Lemos and Feniman employed the SAACT to evaluate 25 normal children without cleft lip and palate and 30 children with cleft lip and palate aged between 7 to 7 years and 11 months with both genders. As expected, children with cleft lip and palate showed weaker performance in all indicators of the SAACT, similar to normal children with long term sensory suspension (hearing loss). This study also agreed with the findings of the other study that showed SAACT has a good ability to distinguish between children with and without cleft lip and palate (21). In 2012, Raz and colleagues examined the validity of a CPT for the separation of healthy adolescents from adolescents with ADHD. Their findings showed a high level of inattention and impulsiveness errors in adolescents with ADHD compared to healthy individuals. Overall, the findings of this study confirmed the validity of the CPT and showed that it is an effective tool for assessing attentional performance and consistent with the findings of the present study (22). Mahone and colleagues (2005) determined the discriminant validity of an Auditory Continuous Performance Test (ACPT) in preschoolers like in the present study. They carried out their study on 40 normal children and 40 with ADHD who were matched for age and gender. In their study, the performance of children suspected to have ADHD was worse than normal children in all test indicators, which is consistent with the findings of the present study. To differentiate between 23 children with mild hearing loss from 40 with normal hearing, Mahone and his colleagues reviewed the ability of an ACPT as part of their research. In this study, no significant differences were found in any indicators that were used in the ACPT between the two groups studied, including inattention error, impulsiveness error, and the average response time variability. This result likely means that the amount and type of hearing loss do not have much effect on sustained auditory attention performance, although it is still necessary to perform further studies in the area of the impact of the type and degree of hearing loss on sustained auditory attention. The discrepancies between these findings may be due to differences in type of ACPT used, as well as the number of samples (23). Discriminant validity was also reported in the study of Christensen and Joschko. Like the present study, findings of Christensen and Joschko partially confirmed the discriminant validity of this test (19). In 1996, Riccio and colleagues determined the discriminant validity of an ACPT in children with central auditory processing disorder with and without ADHD. They performed their study on 30 children with central auditory processing disorder, and also included 15 children with ADHD in addition to central auditory processing disorder. The lack of significant differences between the two groups in all test indicators suggests that this test does not have a good discriminant validity for distinguishing children with central auditory processing with and without ADHD (24).

The attentional network is known to be influenced by education and training. Electrophysiological records from the skull during attention tests indicate that training has a particular effect on the cranial distribution of Event Related Potentials (ERPs). In fact, training effect on cranial distribution has an augmented effect. Also behavioral data showed that children’s attentional performance is closer to adults or to older counterparts after practice or training (25). In psychology, this phenomenon is called the “learning effect.” Therefore to prevent this effect in the current study, we assessed the reliability of the SAACT with an interval of one month. In our study, all indicators showed a good correlation except the attention span reduction index. In 2012, Raz and colleagues evaluated the reliability of a CPT. Their findings demonstrated acceptable reliability and were in agreement with results of our study (22). Soreni and colleagues (2009) evaluated test-retest reliability of a CPT (in terms of impulsiveness error) in 12 children with ADHD. To check the reliability, they performed the test three times on these children with an interval of seven days. Like the present study, Soreni and colleagues’ research results showed that the CPT was a reliable way to evaluate children with ADHD (26).

The major limitation of this study was the unavailability of a credible Persian test that would examine auditory attention for assessing convergent validity.

For future research, we recommend that sustained auditory attention deficits be evaluated with SAACT in other clinical groups, such as; children with learning disabilities and patients with depression, anxiety and with brain injury.

Conclusions

The present study showed that the Persian version of the SAACT has a good validity and reliability and is a suitable tool to evaluate attentional performance in children with ADHD. The Persian version of the SAACT can be used to expand related studies in the field of attentional deficits, especially in the area of ​​auditory stimulation and monitoring of treatment progress in children with attentional deficits. Considering the possibility of attention defects in various disorders, this test is also useful in different fields of rehabilitation and other medical fields and psychological sciences that provide services to children. Since SAACT is a new measurement tool and little research has been done with it, the capabilities of this test in differentiating children with and without ADHD requires further study. Moreover, because an individual’s attention is inherently variable, performing a single test may not provide an accurate picture of this variability. Indeed, according to the studies, for diagnostic and clinical purposes, CPTs should not be performed alone, but as part of a more comprehensive evaluation.

Cite this article as: Soltanparast S, Jafari Z, Sameni S. J, Salehi M. Psychometric Properties of Persian Version of the Sustained Auditory Attention Capacity Test in Children with Attention Deficit-Hyperactivity Disorder. Med J Islam Repub Iran 2014 (23 Feb). Vol. 28:14.