Abstract
OBJECTIVES
Determine if early-implanted, long-term cochlear implant (CI) users display delays in verbal short-term and working memory capacity when processes related to audibility and speech production are eliminated.
DESIGN
Twenty-three long-term CI users and 23 normal-hearing controls each completed forward and backward digit span tasks under testing conditions which differed in presentation modality (auditory or visual) and response output (spoken recall or manual pointing).
RESULTS
Normal-hearing controls reproduced more lists of digits than the CI users, even when the test items were presented visually and the responses were made manually via touchscreen response.
CONCLUSIONS
Short-term and working memory delays observed in CI users are not due to greater demands from peripheral sensory processes such as audibility or from overt speech-motor planning and response output organization. Instead, CI users are less efficient at encoding and maintaining phonological representations in verbal short-term memory utilizing phonological and linguistic strategies during memory tasks.
Introduction
Despite increasing success in spoken language outcomes, long-term cochlear implant (CI) users are at risk for deficits in complex cognitive skills (Kronenberger et al. 2013; Marschark, et al. 2007), specifically, delays in verbal short-term and working memory (Kronenberger et al. 2013; Lyxell et al. 2008). Storage capacity of verbal short-term memory is commonly assessed in CI users with forward auditory digit span in which a sequence of spoken numbers is reproduced aloud in order (Harris et al. 2013; Kronenberger et al. 2013; Pisoni et al. 2011). For children, backward auditory digit span—recalling the list in reverse order—imposes an additional processing load and requires use of both short-term storage and working memory processing operations (Gathercole & Pickering 2000).
Forward and backward auditory digit spans typically involve auditory presentation and spoken responses. Because improvement in CI users’ speech perception and speech intelligibility has been observed up to five years following implantation (Calmels et al. 2004), it is likely that auditory encoding and speech production are less automatized for CI users than for NH peers of the same age. Reduced audibility and constraints on speech production in CI users may contribute to poorer performance on traditional auditory digit span tasks compared to normal-hearing controls. Thus, conventional auditory digit span tasks have a confound between short-term/working memory capacity and audibility/speech skills, which are delayed in CI users.
Group differences in forward span persist when demands of either audibility (e.g. Kronenberger et al. 2013) or spoken responses (e.g. Nittrouer et al. 2013) are eliminated. However, to rule out audibility and speech production as confounds, both must be controlled simultaneously. In the present study, forward and backward auditory digit spans were obtained along with versions of digit span which used visual presentation and touchscreen responses, eliminating the possible contribution of audibility and speech production demands.
Materials and Method
Participants
CI Sample
Twenty-three CI users (14 female; 12 bilateral) were recruited through a CI clinic and research center as well as local advertisements, with the following inclusion criteria: (1) onset of severe-to-profound hearing loss (>70 dB hearing loss in the better hearing ear) prior to age 3 years; (2) cochlear implantation prior to age 7 years; (3) minimum 7 years CI use; (4) consistent use of a multichannel CI system with updates to maps and processors as necessary; and (5) living in a home where spoken English is the primary language.
Normal-Hearing (NH) Control Sample
Twenty-three NH controls (16 female) were recruited from advertisements in the same clinics and local sites as the CI sample. All NH controls passed a hearing screening (.5–4k Hz at 20 dB bilaterally) and were matched 1:1 with the CI sample on nonverbal IQ (+/− 1 SD) and age (+/− 1.5 years at each testing session).
CI and NH participants had no other co-morbid developmental or neurocognitive delays or disabilities and their nonverbal IQ scores were within 1 standard deviation of the norm mean or higher. Demographic and hearing history measures are reported in Table 1.
Table 1.
Participant Demographic Characteristics
| CI | NH | t | p | |||
|---|---|---|---|---|---|---|
| M (SD) | Range | M (SD) | Range | |||
| Age at initial testing (yrs) | 11.8 (1.9) | 7.8–15.3 | 12.5 (2.2) | 8.2–15.3 | −1.2 | .23 |
| Age at follow-up (yrs) | 14.0 (2.4) | 10.1–17.1 | 14.0 (2.1) | 10.1–16.6 | −0.01 | .99 |
| Nonverbal IQ* | 56.0 (6.5) | 42–68 | 55.7 (7.8) | 38–69 | 0.2 | .87 |
| Receptive Language† | 89.9 (18.8) | 44–118 | 112.0 (16.5) | 83–146 | −4.1 | <0.01 |
| Time between testing (yrs) | 2.2 (1.0) | 0.6–4.0 | 1.5 (0.6) | 0.1–2.5 | 2.8 | <.01 |
| Family Income‡ | 6.0 (2.8) | 1–9 | 7.1 (2.7) | 3–11 | −1.4 | .17 |
| Speech Perception†† | 76.73 (12.4) | 40–92 | -- | -- | ||
| Onset of Deafness (mos) | .48 (1.6) | 0–6 | -- | -- | ||
| Age at Implant (mos) | 27.4 (13.0) | 8.3–57.9 | -- | -- | ||
| Years of Implant Use | 11.7 (2.4) | 7.8–15.6 | -- | -- | ||
| Best Pre-Implant PTA‡‡ | 105.5 (13.1) | 85–118.4 | -- | -- | ||
| CMRS Score§ | 5.9 (.34) | 5–6 | -- | -- | ||
| Implant Model/Processing Strategy|| (N ears)
|
||||||
| ABC Clarion—MPS | 1 | |||||
| ABC CL—HiRes | 5 | |||||
| CC Nucleus 24—ACE | 24 | |||||
| CC Nucleus System 5--ACE | 5 | |||||
| ME Combi 40+--CIS | 1 | |||||
| ME Sonata—CIS | 1 | |||||
|
| ||||||
| N | N | X2 | ||||
|
| ||||||
| Race/Ethnicity | ||||||
| -White, Nonhispanic | 19 | 17 | ||||
| -Hispanic | 1 | 0 | ||||
| -Black/AA | 0 | 4 | ||||
| -Asian | 1 | 0 | ||||
| -Mixed Race | 2 | 2 | 6.1 | .19 | ||
| Gender | ||||||
| -M | 9 | 7 | ||||
| -F | 14 | 16 | .10 | .76 | ||
Nonverbal IQ was assessed using Wechsler Abbreviated Scale of Intelligence (WASI) Matrix Reasoning T-score
Receptive Language was assessed using the standard scores of the Peabody Picture Vocabulary Test (PPVT)
Family income was reported on 1 (< $5,500) to 10 (≥ $95,000) scale (values of 3, 5, and 7 correspond to $15,000–$24,999, $35,000–$49,999 and $65,000–$79,999, respectively).
Speech perception was assessed using the Lexical Neighborhood Test (LNT); n=22.
Preimplant residual hearing (mean unaided pure-tone average [PTA] in the better-hearing ear for the frequencies 500, 1,000, and 2,000 Hz at 20 dB).
CMRS Communication Mode Rating Scale (coded on a 1 [mostly sign] to 6 [auditory verbal]; see Geers & Brenner, 2003)
ABC = Advanced Bionics Corporation; CC = Cochlear Corporation; ME = Med-El Corporation
Procedures
Participants completed computerized digit span testing during a follow-up visit that occurred 1 month to 4 years after an initial study visit at which auditory digit span, visual digit span, nonverbal IQ (Matrix Reasoning subtest of the Wechsler Abbreviated Scale of Intelligence) [Wechsler 1999], and receptive vocabulary (Peabody Picture Vocabulary Test, Fourth Edition) [PPVT; Dunn & Dunn 2007] were measured. This testing was completed in a quiet room used for speech perception and language assessments. Auditory tasks were presented face-to-face via live voice by a trained experimenter. During their initial visit, all CI users also completed a speech perception test of words presented in isolation [Lexical Neighborhood Test (LNT); Kirk, Eisenberg, & Martinez, 1999], in a sound proof booth. The CI users listened with their everyday CI map, and they were all tested by one of two certified speech-language pathologists who are experienced in testing CI users.
Memory Span Measures
In all tasks, the initial set size was 2 digits. Two lists were presented at each set size. If the participant accurately reproduced at least one list, then set size increased by one item; if the participant failed to reproduce both lists, testing was terminated. Accuracy was calculated as the total number of lists correctly reproduced.
Visual Digit Span
Sequences of digits were visually presented, simultaneously in a row, according to the instructions in the Wechsler Intelligence Scale for Children, Fourth Edition—Integrated (WISC-IV-I; Wechsler et al. 2004). Participants were instructed to orally reproduce the lists in order from left to right (VDS-F).
Auditory Digit Spans
Lists of digits were presented using live voice according to the instructions for the Digit Span Forward and Digit Span Backward subtests of the WISC-III (Wechsler 1991). Participants were instructed to orally reproduce the lists in either forward (ADS-F) or backward (ADS-B) order.
Computerized Digit Spans
Computerized digit span tasks were developed to match the WISC-III Auditory Digit Span tasks using visual presentation and manual touchscreen responses (12.1 inch, 800×600 pixel, Model Keytech L1201S). A 3×3 response grid (450×450 pixels; 1 in the upper left to 9 in the lower right) appeared 250 ms after the offset of the final list item. Participants were instructed to reproduce the list by touching the digits in either forward (CDS-F) or backward (CDS-B) order.
Results
Forward Span Measures
A 2×3 repeated-measures ANOVA of hearing status (CI vs. NH) and task (VDS-F, ADS-F, CDS-F) revealed main effects of both hearing status, F(1, 44)=23.05, p<0.001 (CI mean=6.47, SD=2.72; NH mean=8.31, SD=3.25) and task, F(2, 88)=17.44.4, p<0.001 (VDS-F mean=10.02, SD=2.78; ADS-F mean=8.25, SD=2.76; CDS-F mean=8.57, SD=2.60) but no interaction between the two. NH controls performed better than CI users on all forward span measures; scores on VDS-F were higher than scores on the other measures (Figure 1a).
Figure 1.
Means and Standard Error for (a) Forward and (b) Backward Digit Span tasks. VDS-F = Visual Digit Span –Forward; ADS-F = Auditory Digit Span – Forward; CDS-F = Computerized Digit Span – Forward; ADS-B = Auditory Digit Span – Backward; and CDS-B = Computerized Digit Span – Backward
Backward Span Measures
A 2×2 repeated-measures ANOVA of hearing status and task produced a main effect of task, F(1, 44)=6.97, p=0.01 (ADS-B mean=5.18, SD=2.14; CDS-B mean=4.35, SD=2.05) but no main effect of hearing status nor interaction of the two (Figure 1b). Both groups scored higher on ADS-B than CDS-B.
Visual inspection of Figure 1 suggested that normal-hearing participants were more impacted by requiring the lists to be reversed. This pattern was confirmed by an additional 2×2×2 repeated measures ANOVA of hearing status (CI vs. NH), direction (forward vs. backward) and task (ADS vs. CDS) which yielded a two-way interaction of hearing status and direction, F(1, 44)=13.06, p<0.001. The analysis also produced a two-way interaction of direction and task F(1, 44)=11.50, p=0.001. For both groups, reversing lists during ADS (ADS-F mean=8.25, SD=2.77; ADS-B mean=5.17, SD=2.14) affected recall less than reversing lists during CDS (CDS-F mean=8.57, SD= 2.60; CDS-B mean=4.35, SD=2.05). No other interactions were significant.
Correlations of Digit Span Tasks with Demographic/Hearing History Variables
Table 2 reports the Pearson coefficients of the first-order semi-partial correlations of each digit span task with receptive vocabulary, nonverbal IQ, speech perception, and hearing history variables after removing variation due to age from digit span scores. ADS-F for CI users was related to higher receptive vocabulary and nonverbal IQ scores as well as earlier age of onset of deafness. In contrast, VDS-F was significantly related only to receptive vocabulary, and CDS-F was significantly related to receptive vocabulary and length of CI use. Both backward span tasks were also significantly related to length of CI use. Additionally, ADS-B correlated with nonverbal IQ, and CDS-B correlated with receptive vocabulary.
Table 2.
Correlations of Digit Span Tasks with Demographic/Hearing History Variables
| Forward Digit Spans | Backward Digit Spans | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| VDS-F
|
ADS-F
|
CDS-F
|
ADS-B
|
CDS-B
|
||||||
| CI | NH | CI | NH | CI | NH | CI | NH | CI | NH | |
|
| ||||||||||
| Receptive Vocabulary | .34 | .13 | .54 | .26 | .67 | .33 | .12 | .51 | .33 | .25 |
| Nonverbal IQ | −.05 | <.01 | .33 | −.12 | −.10 | .30 | −.36 | .15 | −.15 | .12 |
| Speech Perception | .19 | -- | .18 | -- | .36 | -- | .06 | -- | .30 | -- |
| Onset of Deafness | .05 | -- | -.32 | -- | −.14 | -- | −.17 | -- | .09 | -- |
| Age at Implant | .16 | -- | −.07 | -- | −.17 | -- | .03 | -- | .02 | -- |
| Years of Implant use | .27 | -- | .05 | -- | .32 | -- | .63 | -- | .39 | -- |
Pearson product-moment correlation coefficient of semi-partial correlations between digit spans and receptive vocabulary, nonverbal IQ, Speech Perception, and hearing history variables after removing variation due to age at time of testing from digit span scores. Significant p-values (p<.05) denoted in bold.
Table 3 reports pairwise correlations of the five digit span tasks. For the CI group, all pairwise correlations were significantly related—with the exception of three correlations involving ADS-B. Of note is the strong relation between ADS-F and CDS-F. This correlation did not reach significance in the NH group.
Table 3.
Intercorrelations of Digit Span Tasks
| Forwards Span Tasks | Backward Span Tasks | ||||
|---|---|---|---|---|---|
| VDS-F | ADS-F | CDS-F | ADS-B | CDS-B | |
| Forward Span Tasks | |||||
| VDS-F | .49 | .65 | .58 | .52 | |
| ADS-F | .41 | .70 | .38 | .45 | |
| CDS-F | .74 | .40 | .34 | .67 | |
| Backward Span Tasks | |||||
| ADS-B | .30 | .60 | .29 | .28 | |
| CDS-B | .67 | .31 | .60 | .64 | |
Pearson product-moment correlation coefficients. CI users are above the diagonal in grey; normal hearing listeners are below the diagonal. Significant p-values (p<.05) denoted in bold.
Discussion
The present study utilized computer-administered forward and backward digit span tasks in which digits were presented visually and responses were entered manually in order to simultaneously eliminate demands of both audibility and speech production. Even with these demands removed, CI users reproduced fewer sequences than normal-hearing controls on CDS-F, replicating previous studies using conventional auditory presentation and spoken responses. A number of factors may have contributed to audibility during ADS-F and ADS-B. For example, in the present study only half of the CI users heard the digits bilaterally whereas all normal hearing listeners heard the digits binaurally. Because the CI users in the present study are all long-term users with at least 7 years of CI use, their speech perception and spoken language skills may have plateaued (e.g. Calmels et al. 2004); nevertheless, the CI users in the present study averaged only 76.7% (SD=12.4) on open-set spoken word recognition using the LNT. Though impressive, these isolated word recognition scores in quiet were still below the near perfect performance that would be expected from the normal hearing controls, suggesting that even at maximal performance, speech perception in long-term CI users does not mimic normal hearing. If audibility had been a contributing factor to performance on the auditory digit span tasks, then we would expect the CI users’ deficits to be reduced or eliminated on VDS-F and CDS-F because none of the stimuli were presented auditorily in these tasks. Similarly, if demands from producing speech-motor commands interfered with CI users’ performance on ADS-F and VDS-F, then they should have shown relative improvement on CDS-F. Instead, CI users’ deficit was consistent across all three digit span tasks. This pattern of results suggests that underlying cognitive mechanisms—not difficulties due to audibility or speech production—explain long-term CI users’ verbal short-term memory disturbances. Until further work is done to examine the relationship of spoken language production with verbal working memory in less experienced CI users (see Pisoni & Cleary, 2003 for review), future tests of underlying memory mechanisms should incorporate participants who have reached stable performance in speech perception and spoken language production.
Despite the delay between testing sessions, the CI users’ performance on ADS-F was strongly related to their performance on CDS-F. The strong relationship observed between auditory and computerized versions of forward digit span further supports the claim that individual differences in both tasks are driven by similar underlying cognitive processes. If audibility and speech production interfered with performance in the conventional auditory administration, performance on CDS-F and ADS-F should be minimally correlated because no such interference would be expected in the computerized administration. The delay between testing sessions is a possible limitation of the present study given that age-related improvements are expected on digit span tasks (e.g. Alloway, Gathercole, & Pickering, 2006). However, CI users had longer average delays allowing them more time for age-related increases in memory span performance. Despite having slightly more time than the NH controls for developmental improvements to occur, no relative improvement on digit span was observed.
The data presented here are consistent with contemporary models of short-term and working memory which describe basic mechanisms that may function in an atypical fashion when hearing and language are delayed1. Previous reports of poor forward span performance in prelingually deaf CI users have drawn on these models to develop specific hypotheses about which mechanisms are impaired in this population. For example, these CI users may have trouble recovering phonological structure from the speech signal leading to weaknesses in phonological storage (Nittrouer et al. 2013). Additionally, lexical processing strategies, such as visual-verbal recoding and covert verbal rehearsal, may be more effortful for CI users in contrast to normal-hearing listeners for whom these processes become increasingly more efficient and automatic with age and language experience (AuBuchon et al. 2014; Pisoni et al., 2011). Slow, resource-demanding, recoding and rehearsal processes may require CI users to engage working memory operations even during forward span tasks. By this account, observed impairments in verbal short-term memory are reflective of a shift in resource allocation that is not observed in typically-developing, normal-hearing listeners until additional task demands are imposed. Previous work has shown that adults’ performance on traditional verbal short-term memory tasks comes to approximate their performance on working memory tasks as lexical processing strategies such as rehearsal are made more difficult through task manipulations such as articulatory suppression (Unsworth & Engle, 2007).
In earlier research, CI users have lagged behind normative values in ADS-B scores (Harris et al. 2014; Pisoni et al. 2011); conversely, no differences between CI and NH samples were observed on ADS-B or CDS-B in this study. However, the effect size of the difference between groups was small-moderate (Cohen’s d=0. 41) and the present study is underpowered to detect a true group difference of this magnitude on the backward span tasks. The small effect of group for the backward tasks in the present study likely reflects the additional processing demands imposed by the instructions to reverse the list for recall. Reversing the order of the digits on the list makes the task more difficult for both groups, but is particularly detrimental to the normal hearing controls because it also interferes with their ability to initiate lexical processing strategies (Gathercole & Pickering 2000), so the backward span tasks become more equivalent for the two groups.
The present results support earlier reports that CI users have disturbances in verbal short-term and working memory that go beyond issues directly related to audibility and speech production. However, the present results do not negate the important role that early auditory and linguistic experiences play in the development of the working memory system, including the mechanisms described above. Poor audibility leads to sparsely-coded and underspecified phonological representations in long-term memory which provides less downstream support for reactivation and recovery within the short-term memory store. Further, delayed language development observed in CI users may lead to compromised or atypical lexical connectivity and lexical organization resulting in the slow lexical processing strategies described earlier. Although forward and backward digit spans are useful clinical tools to assess verbal short-term and working memory capacity in CI users, these conventional immediate memory tasks are insufficient for furthering our understanding of the foundational memory mechanisms considered here. The present study highlights the need to look to basic literature in working memory for new experimental methodologies and behavioral tasks that can better isolate the underlying deficits in phonological storage and lexical processing as well as describe CI users’ implementation of underlying lexical processing strategies.
Acknowledgments
This work was supported by NIH/NIDCD Grants R01DC00011, R01DC009581 and T32DC000012.
Footnotes
Relevant models include the Multicomponent Model of Working Memory (Baddeley & Hitch, 1974; Baddeley, 2000), the Embedded Processes Model (Cowan, 2001) and the Time-Based Resource Sharing Model (Barrouillet & Camos, 2012), although other working memory models could also account for the present data.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Alloway TP, Gathercole SE, Pickering SJ. Verbal and visuospatial short-term and working memory in children: Are they separable? Child Development. 2006;77(6):1698–1716. doi: 10.1111/j.1467-8624.2006.00968.x. [DOI] [PubMed] [Google Scholar]
- AuBuchon AM, Pisoni DB, Kronenberger WG. Verbal processing speed and executive functioning in long-term cochlear implant users. Journal of Speech, Language, and Hearing Research. 2014:1–12. doi: 10.1044/2014_JSLHR-H-13-0259. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Baddeley AD. The episodic buffer: a new component of working memory? Trends in Cognitive Science. 2000;4:417–423. doi: 10.1016/s1364-6613(00)01538-2. [DOI] [PubMed] [Google Scholar]
- Baddeley AD, Hitch G. Working memory. Psychology of Learning and Motivation. 1974;8:47–89. [Google Scholar]
- Barrouillet P, Camos V. As time goes by: Temporal constraints in working memory. Current Directions in Psychological Science. 2012;21(6):413–419. [Google Scholar]
- Calmels MN, Saliba I, Wanna G, Cochard N, Fillaux J, Deguine O, Fraysse B. Speech perception and speech intelligibility in children after cochlear implantation. International Journal of Pediatric Otorhinolaryngology. 2004;68(3):347–351. doi: 10.1016/j.ijporl.2003.11.006. [DOI] [PubMed] [Google Scholar]
- Cowan N. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences. 2001;24:87–185. doi: 10.1017/s0140525x01003922. [DOI] [PubMed] [Google Scholar]
- Dunn D, Dunn L. Peabody Picture Vocabulary Test. 4. Minneapolis, MN: Pearson Assessments; 2007. [Google Scholar]
- Gathercole SE, Pickering SJ. Assessment of working memory in six- and seven-year-old children. Journal of Educational Psychology. 2000;92(2):377. doi: 10.1348/000709900158047. [DOI] [PubMed] [Google Scholar]
- Geers A, Brenner C. Background and educational characteristics of prelingually deaf children implanted by five years of age. Ear and Hearing. 2003;24:2S–14S. doi: 10.1097/01.AUD.0000051685.19171.BD. [DOI] [PubMed] [Google Scholar]
- Harris MS, Kronenberger WG, Gao S, Hoen HM, Miyamoto RT, Pisoni DB. Verbal short-term memory development and spoken language outcomes in deaf children with cochlear implants. Ear and Hearing. 2013;34(2):179. doi: 10.1097/AUD.0b013e318269ce50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kirk KIL, Eisenberg S, Martinez A. The lexical neighborhood test: Test-retest reliability and interlist equivalency. Journal of the American Academy of Audiology. 1999;10:113–123. [Google Scholar]
- Kronenberger WG, Pisoni DB, Henning SC, Colson BG. Executive functioning skills in long-term users of cochlear implants: A case control study. Journal of Pediatric Psychology, jst034. 2013 doi: 10.1093/jpepsy/jst034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lyxell B, Sahlen B, Wass M, Ibertsson T, Larsby B, Hällgren M, Mäki-Torkko E. Cognitive development in children with cochlear implants: relations to reading and communication. International Journal of Audiology. 2008;47(S2):S47–S52. doi: 10.1080/14992020802307370. [DOI] [PubMed] [Google Scholar]
- Marschark M, Rhoten C, Fabich M. Effects of cochlear implants on children’s reading and academic achievement. Journal of Deaf Studies and Deaf Education. 2007 doi: 10.1093/deafed/enm013. [DOI] [PubMed] [Google Scholar]
- Nittrouer S, Caldwell-Tarr A, Lowenstein JH. Working memory in children with cochlear implants: Problems are in storage, not processing. International Journal of Pediatric Otorhinolaryngology. 2013;77(11):1886–1898. doi: 10.1016/j.ijporl.2013.09.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pisoni DB, Cleary M. Measures of Working Memory Span and Verbal Rehearsal Speed in Deaf Children after Cochlear Implantation. Ear and Hearing. 2003;24:106S–120S. doi: 10.1097/01.AUD.0000051692.05140.8E. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pisoni D, Kronenberger W, Roman A, Geers A. Measures of digit span and verbal rehearsal speed in deaf children following more than 10 years of cochlear implantation. Ear and Hearing. 2011;32(1):60s. doi: 10.1097/AUD.0b013e3181ffd58e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Unsworth N, Engle RW. On the division of short-term and working memory: An examination of simple and complex span and their relation to higher order abilities. Psychological Bulletin. 2007;133:1038–166. doi: 10.1037/0033-2909.133.6.1038. [DOI] [PubMed] [Google Scholar]
- Wechsler D. Wechsler intelligence scale for children (WISC) San Antonio, TX: Psychological Corporation; 1991. WISC-III. [Google Scholar]
- Wechsler D. Wechsler Abbreviated Scale of Intelligence (WASI) USA: The Psychological Corporation; 1999. [Google Scholar]
- Wechsler D, Kaplan E, Fein D, Kramer J, Morris R, Delis D, Maerlender A. Wechsler Intelligence Scale for Children – Fourth Edition – Integrated. San Antonio, TX: Harcourt Assessment; 2004. [Google Scholar]