Abstract
Purpose
The objective of the present article was to document the extent to which early expressive language skills (measured using the MacArthur–Bates Communicative Development Inventories [CDI; Fenson et al., 2006]) predict long-term neurocognitive outcomes in a sample of early-implanted prelingually deaf cochlear implant (CI) users.
Method
The CDI was used to index the early expressive language skills of 32 pediatric CI users after an average of 1.03 years (SD = 0.56, range = 0.39–2.17) of CI experience. Long-term neurocognitive outcomes were assessed after an average of 11.32 (SD = 2.54, range = 7.08–16.52) years of CI experience. Measures of long-term neurocognitive outcomes were derived from gold-standard performance-based and questionnaire-based assessments of language, executive functioning, and academic skills.
Result
Analyses revealed that early expressive language skills, collected on average 1.03 years post cochlear implantation, predicted long-term language, executive functioning, and academic skills up to 16 years later.
Conclusion
These findings suggest that early expressive language skills, as indexed by the CDI, are clinically relevant for identifying CI users who may be at high risk for long-term neurocognitive delays and disturbances.
Cochlear implantation is now recognized as the standard of care for medically treating bilateral, severe-to-profound sensorineural hearing loss. Cochlear implants (CIs) provide deaf children and adults with access to sound by stimulating the surviving spiral ganglion cells in the auditory nerve (Copeland & Pillsbury, 2004). There is now a rapidly growing body of research suggesting that a period of early auditory deprivation followed by implantation not only affects speech and language skills but also has cascading effects on brain regions responsible for the development of a set of neurocognitive skills referred to as executive functions (EFs; Castellanos et al., 2015; Conway, Pisoni, Anaya, Karpicke, & Henning, 2010; Conway, Pisoni, & Kronenberger, 2009; Pisoni, Conway, Kronenberger, Henning, & Anaya, 2010). EFs are a broad cluster of cognitive and emotional abilities, including sustained attention, shifting attention, speed of processing, working memory, novel problem solving, decision making, and inhibitory self-regulation (Barkley, 2012; Luria, 1966; McAuley & White, 2011). EF and language are robustly connected via feedback loops that include attention, working memory, and sequential processing, which are critically important for speech production and language perception (Miller & Cohen, 2001; Pisoni et al., 2010). EFs also support the regulation of externalizing behaviors and emotional states and are critically important for learning, memory, adaptive functioning, and appropriate social interactions (Barkley, 2012).
Delays and disturbances in the development of EFs have been documented in deaf children with CIs (Kronenberger, Beer, Castellanos, Pisoni, & Miyamoto, 2014) and in children with attention deficit/hyperactivity disorder (Barkley, 2012), spina bifida (O'Hara & Holmbeck, 2013), and traumatic brain injuries (Horton, Soper, & Reynolds, 2010). In deaf children with CIs, the relations between language and EFs differ from the relations observed in typically developing normal-hearing (NH) children. Research suggests that CI users may accomplish conventional speech and language tasks via different pathways, requiring additional executive functioning resources to reach similar outcomes as their NH peers (Lyxell, Andersson, Borg, & Ohlsson, 2003).
In our previous research, we identified a subset of executive functioning abilities that are important in the development of speech and language skills following cochlear implantation, which include working memory, fluency speed, and inhibition concentration. For example, Kronenberger, Colson, Henning, and Pisoni (2014) demonstrated that the speech and language skills of long-term CI users were associated more strongly with verbal working memory and fluency–speed processing skills than those of their NH peers. Importantly, delays and disturbances in EF may be present during early development following cochlear implantation. At preschool age, a subset of deaf children with CIs, compared with their NH peers, are already at an elevated risk for clinically significant delays and disturbances in core areas of neurocognitive functioning, including attention, working memory, and novel problem solving (Kronenberger, Beer, et al., 2014). Thus, early identification of deaf children who may be at high risk for poor neurocognitive outcomes after pediatric cochlear implantation, is a pressing clinical problem with vital importance for making medical decisions about appropriate habilitation options, about when to introduce interventions, and about what kinds of individualized interventions may strengthen executive functioning skills in these children.
Because of the strong relationship between EF and language, several currently available language assessments may provide a critical key to identifying deaf children who may be at risk for poor executive functioning skills. Assessments such as the Peabody Picture Vocabulary Test (PPVT; Dunn & Dunn, 2007), the Reynell Developmental Language Scales (RDLS; Reynell & Gruber, 1990), and the MacArthur–Bates Communicative Development Inventories (CDI; Fenson et al., 2006) are routinely used to evaluate the development of spoken language skills and are useful clinical tools for monitoring lexical and grammatical development and planning interventions. The PPVT and RDLS assessments are applicable to older children (PPVT older than 30 months, RDLS older than 21 months), whereas the CDI (Words and Gestures form) can be used with children as young as 8 months. The CDI is also the only parent-completed rating scale among the top five most commonly used assessments of language skills in deaf children with CIs (Perin da Silva, Comerlatto Junior, Bevilacqua, & Lopes-Herrera, 2011). Parent-completed rating scales are useful because they are easy to administer, score, and interpret. Moreover, rating scales may be completed by parents while the child is undergoing audiological testing and CI mapping, in place of more time-consuming, performance-based behavioral assessments. Parent reports also provide valuable information about real-world language development beyond conventional measures obtained in the laboratory or clinic setting.
Numerous studies have examined the short-term and long-term predictive validity of the CDI in both typically developing NH children and pediatric CI users. In typically developing NH children, Marchman and Fernald (2008) showed that expressive vocabulary size obtained at age 2 years using the CDI was predictive of working memory skills at age 8 years. In pediatric CI users, Nicholas and Geers (2008) reported that CDI scores obtained for expressive vocabulary size, mean sentence length, sentence complexity, and irregular words obtained at age 3.5 years were correlated with scores on standardized language tests, such as the Preschool Language Scale (PLS) and the PPVT, 1 year later. Similarly, high correlations have been found between the CDI and the RDLS (Stallings, Gao, & Svirsky, 2002). Aside from correlations with standardized language measures, expressive language scores obtained from the CDI are also associated with language measures derived from spontaneous language samples (Thal, DesJardin, & Eisenberg, 2007) and from mother–infant free-play sessions (Nicholas & Geers, 2006).
In this study, we investigated whether expressive language measures obtained in early toddlerhood using the CDI would be predictive of later long-term speech, language, and executive functioning skills (up to 16 years later) in prelingually deaf CI users. We have been following a sample of early-implanted prelingually deaf CI users longitudinally for more than 2 decades as they enter adolescence and young adulthood (Castellanos et al., 2014; Harris et al., 2011). These longitudinal data afford us a unique opportunity to examine the relations between early measures of speech and language outcomes following cochlear implantation and long-term neurocognitive functioning. Our goal in this study was to examine the predictability of early speech–language assessments, specifically the CDI, in identifying CI users who may be at early risk for poor long-term neurocognitive outcomes.
Method
Participants
Study participants were 32 children, adolescents, and young adults who received CIs in our center and who were evaluated as part of a larger study of long-term outcomes following cochlear implantation in childhood (see Castellanos et al., 2015, for descriptive statistics on the demographics, hearing history, speech, and language variables of the full sample). Table 1 provides descriptive statistics on the demographics and hearing history of the current sample. To be included in the long-term outcome study, participants were required to meet the following inclusionary criteria: (a) have prelingual, severe-to-profound, sensorineural hearing loss (>70 dB HL in the better hearing ear before age 3 years); (b) have received their CI before age 7 years; (c) have used their CI for 7 years or more; (d) use a currently available, state-of-the-art, multichannel CI system; (e) live in a household with spoken English as the primary language; and (f) pass a screening performed by licensed speech-language pathologists before testing, confirming no additional developmental, neurological, or cognitive conditions were present other than hearing loss. To be included in the current study, participants were also required to have the MacArthur–Bates CDI completed by one parent within 2.5 years post cochlear implantation.
Table 1.
Demographic and hearing history variables | Early post–cochlear implantation visit |
Long-term, follow-up visit |
||
---|---|---|---|---|
M (SD) | Range | M (SD) | Range | |
Onset of deafness (months) | 1.06 (3.70) | 0.00–18.00 | ||
Age at implantation (months) | 32.12 (14.85) | 9.92–67.02 | ||
Age at testing (years) | 3.71 (1.04) | 1.98–6.10 | 14.00 (3.19) | 9.10–21.55 |
Duration of CI use (years) | 1.03 (0.56) | 0.39–2.17 | 11.32 (2.54) | 7.08–16.52 |
Preimplant residual hearing PTA b | 108.58 (10.83) | 85.00–118.43 | ||
Nonverbal IQ c | — | — | 55.03 (6.58) | 42.00–65.00 |
Income Level d | — | — | 7.15 (2.09) | 3.00–10.00 |
Count (% of sample) |
||||
Age at testing (years) | ||||
≤ 3 | 9 (28.1) | |||
3.10–4.00 | 14 (43.8) | |||
4.10–6.10 | 9 (28.1) | |||
9.10–12.00 | 9 (28.1) | |||
12.10–15.00 | 12 (37.5) | |||
15.10–22.00 | 11 (34.4) | |||
Hearing device | ||||
CI and HA | 1 (3.1) | 1 (3.1) | ||
Bilateral CIs | 0 (0.0) | 10 (31.3) | ||
Unilateral CI | 31 (96.9) | 21 (65.6) | ||
Etiology of hearing loss | ||||
Meningitis | 2 (6.3) | |||
Other/unknown | 30 (93.7) | |||
Communication mode | ||||
Signed/total | 13 (40.6) | 4 (12.5) | ||
Oral/cued | 19 (59.4) | 28 (87.5) | ||
Gender | ||||
Female | 16 (50.0) | |||
Male | 16 (50.0) | |||
Race | ||||
Asian | 1 (3.1) | |||
White | 31 (96.9) | |||
Ethnicity | ||||
Hispanic | 2 (6.3) | |||
Not Hispanic | 30 (93.8) |
Note. Long-term, follow-up visits took place, on average, 10.29 (SD = 2.64, range = 5.11–15.95) years after the Communicative Development Inventories were completed. CI = cochlear implant; HA = hearing aid. Dashes indicate data on nonverbal IQ and income level were not collected during the early post–cochlear implantation visit.
N = 32.
Unaided pure-tone average (PTA) in the better ear for the frequencies 500, 1000, and 2000 Hz in dB HL.
Nonverbal IQ scores are expressed as T scores (M = 50, SD = 10) from the Wechsler Abbreviated Scale of Intelligence (WASI) Matrix Reasoning subtest for nonverbal intelligence. Our sample of deaf CI users displayed nonverbal IQ scores within the normal range.
Income level is coded on a scale from less than $5,500 (coded 1) to $95,000 and greater (coded 10), with a code of 7 = $50,000–$64,999 and 8 = $65,000–$79,999.
Demographic variables coded for each participant included chronological age, gender, family income (coded by income ranges on a 1 [under $5,500] to 10 [$95,000 and more] scale, with values of 4, 6, and 8 corresponding to income values of $15,000–$24,999, $35,000–$49,999, and $65,000–$79,999, respectively), and race/ethnicity. Additional hearing history variables included age at onset of deafness (defined as the age at which deafness was identified or age at the time of a known event causing deafness); age at time of implantation; interval of time between CDI and long-term, follow-up testing; duration of CI use at CDI testing and long-term, follow-up testing; preimplant residual hearing (mean, unaided pure-tone average [PTA] in the better hearing ear for the frequencies of 500, 1000, and 2000 Hz in dB HL); communication mode (coded 1 for total communication and 2 for oral communication) at CDI testing and long-term, follow-up testing; and etiology of deafness. Etiology of deafness included unknown (n = 23, 71.9%), familial (at least one immediate family member also had deafness of unknown etiology, n = 4, 12.5%), meningitis (n = 2, 6.3%), Mondini malformation (n = 2, 6.3%), and auditory neuropathy (n = 1, 3.1%).
On average, children in this sample were implanted with a unilateral CI before 32.12 months of age (SD = 14.85, range = 9.92–67.02). At the time of long-term, follow-up testing, 10 (31.3%) participants were fitted with bilateral CIs, and 1 (3.1%) was fitted with a CI plus a hearing aid in the opposite ear. At the time the CDI was completed, participants averaged 3.71 (SD = 1.04, range = 1.98–6.10) years old and averaged 1.03 (SD = 0.56, range = 0.39–2.17) years of CI use. At the time of long-term, follow-up testing, participants averaged 14.00 (SD = 3.19, range = 9.10–21.55) years old and averaged 11.32 (SD = 2.54, range = 7.08–16.52) years of CI use.
Procedure
All study procedures were reviewed and approved by the local Institutional Review Board, and written, informed consent was obtained for all participants or parents before initiation of any study procedures. Long-term, follow-up data were obtained from research visits to a large, university hospital–based CI clinic conducted as a part of a cross-sectional neurocognitive outcome study. The long-term outcome study provided the initial pool of potential participants, for whom CDI data were obtained from the longitudinal database. If participants received the CDI more than once during the 2 years following their cochlear implantation, the earliest data point was selected for analysis. Long-term, follow-up visits took place on average 10.29 years (SD = 2.64, range = 5.11–15.95) after the CDI was completed. Licensed speech-language pathologists evaluated all study participants and administered the language tests in the participant's mode of communication used at school or (for those not in school) in the participant's preferred mode of communication (either oral communication or total communication, see Table 1).
Measures
Early Expressive Language
Early expressive language skills were assessed on average 1.03 years (SD = 0.56, range = 0.39–2.17) post cochlear implantation with the Words and Sentences (WS) form of the MacArthur–Bates CDI (CDI:WS; Fenson et al., 2006). The CDI:WS form is a parent-completed rating scale of early language skills applicable to typically developing children between 16 and 30 months and is valid for use in chronologically older children with suspected language delays, such as children with hearing loss (Thal et al., 2007). The CDI:WS form is divided into two parts: Part 1 assesses vocabulary with a 680-word vocabulary checklist, and Part 2 assesses sentences and grammar. Part 2 of the CDI:WS is further divided into five sections: Sections A–C assess production of regular and irregular words, whereas Sections D–E assess production of multiword utterances. Parents are instructed to stop at Section C if their child has not yet begun to combine words.
In line with previous studies, CDI scores are reported here as raw scores instead of percentile scores because our sample of pediatric CI users was chronologically older 1 than the normative sample (16–30 months) for which norms were derived (Stallings et al., 2002; Thal et al., 2007). In addition, to provide consistency with published norms, CDI scores are reported here relative to the child's gender. Total number of words produced (WP), obtained from Part 1 of the vocabulary checklist, was used as the primary global measure of early expressive language skills.
Long-Term Neurocognitive Performance Outcomes
A broad range of gold-standard neurocognitive assessments with strong psychometrics, including internal consistency, test–retest reliability, and construct validity, were administered at the long-term, follow-up visit. Five composite scores were created from the individual neurocognitive assessments: (a) language, (b) verbal working memory, (c) visual–spatial working memory, (d) fluency speed, and (e) inhibition concentration. All neurocognitive assessments were administered in their standardized format using spoken instructions when appropriate.
Language. Standard scores on the PPVT–Fourth Edition (PPVT-4; Dunn & Dunn, 2007) and core language standard scores from the Clinical Evaluation of Language Fundamentals–Fourth Edition (CELF-4; Semel, Wiig, & Secord, 2003) were used to assess language skills. The PPVT-4 is a one-word, receptive vocabulary test requiring participants to choose one of four pictures matching a spoken word. The core language score of the CELF-4 is a measure of general receptive and expressive language skills, derived from several subtests depending on the participant's age, such as understanding concepts and following directions, recalling sentences, formulating sentences, and vocabulary knowledge. For children using total communication (n = 4), Signed Exact English accompanied the spoken word/phrase for these language tests.
Verbal working memory. Scaled scores on the Digit Span subtest of the Wechsler Intelligence Scale for Children–Third Edition (WISC-III; Wechsler, 1991) and scaled scores on the Visual Digit Span subtest of the WISC–Fourth Edition, Integrated (WISC-IV-I; Wechsler et al., 2004) were used to assess verbal working memory skills. The Digit Span subtest of the WISC-III requires participants to reproduce a sequence of spoken digits presented in forward (Digit Span Forward) or backward (Digit Span Backward) order, whereas the Visual Digit Span subtest of the WISC-IV-I requires participants to repeat a series of visually presented digits in forward order.
Visual–spatial working memory. Visual–spatial working memory skills were assessed using scaled scores on the Spatial Span subtest of the WISC-IV-I (Wechsler et al., 2004). In the Spatial Span subtest of the WISC-IV-I, the experimenter taps a series of blocks in a sequence, and participants are required to reproduce the sequence of blocks tapped by the experimenter in forward (Spatial Span Forward) or backward (Spatial Span Backward) order.
Fluency speed. Fluency–speed skills were assessed using standard scores on the Pair Cancellation subtest of the Woodcock–Johnson Tests of Cognitive Abilities–Third Edition (WJ-III; Woodcock, McGrew, & Mather, 2001) and scaled scores on the Coding and Coding Copy subtests of the WISC-IV-I (Wechsler et al., 2004). The Pair Cancellation subtest of the WJ-III requires participants to rapidly identify pictures within visual stimulus arrays. The Coding subtest of the WISC-IV-I requires participants to rapidly reproduce a sequence of visually unique symbols based on corresponding numerals, whereas the Coding Copy subtest requires participants to rapidly reproduce visual symbols (from the Coding subtest) without the corresponding numerals.
Inhibition Concentration. Inhibition–concentration skills were assessed using omissions, commissions, and response time variability standard scores from the Test of Variables of Attention (TOVA; Leark, Dupuy, Greenberg, Corman, & Kindschi, 1996). The TOVA requires participants to press a button when presented with a target stimulus (a square at the top of a screen) but not when presented with a distractor stimulus (a square at the bottom of a screen). Measures of omissions (failing to respond to the target), commissions (responding inaccurately to the distractor), and response time variability (variability in speed of button press in response to the target) were collected from the TOVA.
Composite Score Derivation
Composite scores for the domains of language, verbal working memory, visual–spatial working memory, fluency speed, and inhibition concentration were created by summing z-transformed scores (based on the means and standard deviations in the current study sample) of the measures within each domain described in the prior section (see Geers, Brenner, & Davidson, 2003; Kronenberger, Colson et al., 2014, for support of this technique). Empirical support for combining scores into these composite areas has been provided in two principal-components analyses (one for the 11 neurocognitive measures of verbal working memory, visual–spatial working memory, fluency speed, and inhibition concentration, and one for the two measures of language) using these measures in a larger sample of 138 CI users and NH peers (see Kronenberger, Colson et al., 2014). All correlation and regression analyses were conducted using these composite scores of language, verbal working memory, visual–spatial working memory, fluency–speed skills, and inhibition–concentration skills.
Long-Term Neurocognitive Questionnaire-Based Outcomes
The Learning, Executive, and Attention Functioning (LEAF) scale is a parent-completed rating scale of neurocognitive abilities used to assess executive functioning and related learning skills that are applicable to children and adolescents between 6 and 17 years old (Kronenberger & Pisoni, 2009). The LEAF contains 55 items, divided into eight cognitive subscales (comprehension and conceptual learning, factual memory, attention, processing speed, visual–spatial organization, sustained sequential processing, working memory, and novel problem solving) and three academic subscales (mathematics, basic reading, and written expression). The parent or teacher rates the child's behavior on a 0 (never a problem) to 3 (very often a problem) point scale, such that higher scores on the LEAF indicate greater problems in executive functioning and learning. Only parent-completed LEAF scores are reported here.
The LEAF is a reliable and valid questionnaire-based measure of executive functioning (Kronenberger & Pisoni, 2009). Internal consistency (Cronbach's α ≥ 0.69 for all LEAF subscales) and test–retest reliability (r = 0.40–0.82 for the LEAF cognitive subscales and r = 0.34–0.50 for the LEAF academic subscales) for parent-completed LEAF scales are adequate to strong. Parent–teacher interrater reliability is modest (r = 0.21–0.34 for eight of the 11 LEAF subscales). Additionally, construct validity between corresponding subscales on the LEAF and other behavior checklists is also strong (see Kronenberger, Beer et al., 2014, for comparisons of executive functioning skills in CI users and in NH children using the LEAF and another behavior checklist).
Data Analysis
To examine the relations between early expressive language skills and long-term neurocognitive outcomes, scores from the CDI:WS completed either in toddlerhood or early childhood were correlated with long-term performance outcomes using composite scores of language, verbal working memory, visual–spatial working memory, fluency speed, and inhibition concentration and with long-term, questionnaire-based, parent-reported LEAF scores. To statistically control for nonspecific effects of global intelligence on the relationship between early expressive language skills and long-term neurocognitive functioning, all correlations were run partialling out global nonverbal intelligence. The Matrix Reasoning subtest of the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999) assessed at long-term follow-up served as our index of global nonverbal intelligence. To provide more-conservative estimates of statistical significance due to multiple tests (reducing Type I error), a p value of .01 was adopted for evaluation of significance.
Next, to evaluate the independent contribution of CDI expressive vocabulary scores on long-term outcomes while also accounting for demographic and hearing-history variables, hierarchical, blockwise regression analyses were conducted with each performance outcome (composite score) as the criterion variable and blocks of variables entered sequentially as follows: Block 1 consisted of demographic and hearing history (gender; age at onset of deafness; age at implantation; best, unaided PTA preimplantation; age at CDI testing; age at long-term, follow-up testing; duration of CI use at CDI testing; duration of CI use at long-term, follow-up testing; and interval of time between the CDI and long-term, follow-up visits). To maintain an acceptable variable-to-subject ratio, only variables from Block 1 that were statistically significant at the p < .05 level were retained in the regression equation. Block 2 consisted of CDI vocabulary scores (mean number of WP), which were interpreted for significance using a p value of .01 (consistent with the analyses above). Similar, hierarchical, blockwise regression analyses were conducted with each questionnaire-based LEAF subscale score as the criterion variable.
Results
Descriptive statistics for CDI words produced (CDI:WP) for deaf children after an average of 1.03 years of CI use are shown in Table 2. Floor and ceiling effects can be observed on measures of expressive vocabulary size. Mean expressive vocabulary size (CDI:WP) ranged from 1 to 663 of 680 possible words. These findings using the CDI illustrate clearly the enormous amount of variability in lexical growth and individual differences following pediatric cochlear implantation. Descriptive statistics for the long-term neurocognitive outcome measures (language, verbal working memory, visual–spatial working memory, fluency speed, inhibition concentration, and LEAF cognitive and academic functioning) are reported in Table 3.
Table 2.
Gender | MacArthur–Bates CDI Words and Sentences: Words Produced |
|
---|---|---|
M (SD) | Range | |
Female | 274.44 (218.38) | 11.00–663.00 |
Male | 273.63 (162.27) | 1.00–499.00 |
All | 274.03 (189.25) | 1.00–663.00 |
Note. Descriptive statistics obtained on average 1.03 years post cochlear implantation. In line with the norms published on typically developing children with normal hearing, CDI scores are reported by gender.
N = 32 (16 females, 16 males).
Table 3.
Long-term neurocognitive outcome measures | N | Mean (SD) | Range |
---|---|---|---|
Performance-based composite outcomes | |||
Language | |||
PPVT-4 | 32 | 86.88 (20.38) | 48.00–123.00 |
CELF-4 core | 30 | 83.97 (25.66) | 44.00–124.00 |
Verbal working memory | |||
WISC-III Digit Span Forward | 32 | 6.31 (2.49) | 3.00–12.00 |
WISC-III Digit Span Backward | 32 | 9.59 (3.16) | 3.00–16.00 |
WISC-IV-I Visual Digit Span Forward | 32 | 8.41 (2.87) | 2.00–15.00 |
Visual–spatial working memory | |||
WISC-IV-I Spatial Span Forward | 32 | 10.22 (2.59) | 5.00–16.00 |
WISC-IV-I Spatial Span Backward | 31 | 11.06 (2.27) | 7.00–16.00 |
Fluency speed | |||
WJ-III Pair Cancellation | 32 | 100.03 (12.86) | 66.00–127.00 |
WISC-IV-I Coding | 32 | 9.31 (2.68) | 5.00–15.00 |
WISC-IV-I Coding Copy | 32 | 10.19 (2.86) | 6.00–16.00 |
Inhibition concentration | |||
TOVA omissions | 29 | 73.24 (27.83) | 40.00–113.00 |
TOVA commissions | 29 | 81.10 (23.92) | 40.00–120.00 |
TOVA response time variability | 29 | 86.59 (23.35) | 43.00–119.00 |
LEAF questionnaire-based outcomes | |||
Cognitive subscales | |||
Comprehension and conceptual learning | 31 | 4.58 (3.99) | 0.00–13.00 |
Factual memory | 31 | 3.45 (4.07) | 0.00–15.00 |
Attention | 31 | 4.06 (3.79) | 0.00–11.00 |
Processing speed | 31 | 3.81 (3.34) | 0.00–10.00 |
Visual–spatial organization | 31 | 2.68 (2.33) | 0.00–7.00 |
Sustained sequential processing | 31 | 4.33 (3.59) | 0.00–12.00 |
Working memory | 31 | 4.13 (3.85) | 0.00–15.00 |
Novel problem solving | 31 | 3.35 (3.51) | 0.00–12.00 |
Academic subscales | |||
Mathematics skills | 31 | 3.39 (3.24) | 0.00–11.00 |
Basic reading skills | 31 | 5.03 (4.70) | 0.00–15.00 |
Written expression skills | 31 | 5.00 (4.91) | 0.00–15.00 |
Note. The PPVT-4, CELF-4 core, WJ-III Pair Cancellation, TOVA omissions, TOVA commissions, and TOVA response time variability scores are expressed as standard scores (M = 100, SD = 15), whereas all other performance-based scores are expressed as scaled scores (M = 10, SD = 3). Questionnaire-based scores on the LEAF are expressed as raw scores. PPVT-4 = Peabody Picture Vocabulary Test–Fourth Edition, CELF-4 = Clinical Evaluation of Language Fundamentals; WISC-III = Wechsler Intelligence Scale for Children–Third Edition; WISC-IV-I = Wechsler Intelligence Scale for Children–Fourth Edition, Integrated; WJ-III = Woodcock–Johnson Tests of Cognitive Abilities–Third Edition; TOVA = Test of Variables of Attention; LEAF = Learning, Executive, and Attention Functioning.
Correlational Analyses
Partial correlations between CDI:WS and long-term neurocognitive performance outcomes, controlling for nonverbal intelligence, are summarized in Table 4. Early expressive vocabulary size, obtained at an average 1.03 years post cochlear implantation, was significantly correlated with three of the five long-term composite outcomes (language, verbal working memory, and fluency speed). CDI:WP was significantly correlated with long-term language (r = 0.69, p < .001), verbal working memory (r = 0.53, p = .002), and fluency speed (r = 0.58, p = .001). 2
Table 4.
Long-term performance outcomes | N | MacArthur–Bates CDI Words and Sentences: |
---|---|---|
Words Produced | ||
Language | 32 | .69*** |
Verbal working memory | 32 | .53** |
Visual–spatial working memory | 32 | .01 |
Fluency speed | 32 | .58** |
Inhibition concentration | 29 | .10 |
Note. Nonverbal IQ scores were obtained from the Wechsler Abbreviated Scale of Intelligence (WASI) Matrix Reasoning subtest. Language = Peabody Picture Vocabulary Test–4 and Clinical Evaluation of Language Fundamentals–4, core language; verbal working memory = Wechsler Intelligence Scale for Children–Third Edition (WISC-III) subtests: Digit Span Forward and Digit Span Backward. Wechsler Intelligence Scale for Children–Fourth Edition, Integrated (WISC-IV-I) subtest: Visual Digit Span; visual-spatial working memory = WISC-IV-I subtests: Spatial Span Forward and Spatial Span Backward. Fluency speed = WISC-IV-I subsets: Coding and Coding Copy. Woodcock Johnson Tests of Cognitive Abilities–Third Edition (WJ-III) subtest: Pair Cancellation; Inhibition concentration = omissions, commissions, and reaction time variability of the Test of Variables of Attention (TOVA).
p < .01.
p < .001.
Partial correlations between CDI:WS and long-term, parent-reported LEAF cognitive-functioning scores, controlling for nonverbal intelligence, are summarized in Table 5. CDI:WP was significantly correlated with two of the eight LEAF cognitive subscales (attention, r = −0.48, p = .008; and sustained sequential processing, r = −0.49, p = .006). 3 CDI:WP was also significantly correlated with long-term LEAF basic reading (r = −0.53, p = .002) and written expression skills subscales (r = −0.50, p = .005; see Table 6).
Table 5.
Long-term, follow-up outcomes | MacArthur–Bates CDI Words and Sentences: |
---|---|
Words Produced | |
LEAF scale | |
Cognitive subscales | |
Comprehension and conceptual learning | −.39* |
Factual memory | −.36* |
Attention | −.48** |
Processing speed | −.38* |
Visual–spatial organization | −.35 |
Sustained sequential processing | −.49** |
Working memory | −.40* |
Novel problem solving | −.23 |
Academic subscales | |
Mathematics skills | −.07 |
Basic reading skills | −.53** |
Written expression skills | −.50** |
N = 31.
p < .05.
p < .01.
Table 6.
Long-term, follow-up composite scores |
||||||
---|---|---|---|---|---|---|
Language |
Verbal working memory |
Fluency speed |
||||
β | SE | β | SE | β | SE | |
Block 1: Demographic and hearing history | ||||||
Early communication mode | .37* | .17 | .33* | .16 | ||
Age at implantation | .54** | .16 | ||||
Gender | .33* | .15 | ||||
R 2 | 0.13* | 0.35** | ||||
Block 2 | ||||||
CDI words produced | .65*** | .14 | .55** | .15 | .46** | .15 |
Early communication mode | .14 | .14 | .12 | .16 | ||
Age at implantation | .37* | .16 | ||||
Gender | .32* | .14 | ||||
R 2 | 0.51*** | 0.31** | 0.51*** |
Note. N = 32 for all composite scores. The standardized regression coefficient (β) and standard error (SE) of β are provided. Composite measures include language = Peabody Picture Vocabulary Test–4 and Clinical Evaluation of Language Fundamentals–4, core language; verbal working memory = Wechsler Intelligence Scale for Children–Third Edition (WISC-III) subtests: Digit Span Forward and Digit Span Backward. Wechsler Intelligence Scale for Children–Fourth Edition, Integrated (WISC-IV-I), subtest: Visual Digit Span; fluency speed = WISC-IV-I subsets: Coding and Coding Copy. Woodcock–Johnson Tests of Cognitive Abilities–Third Edition (WJ-III) subtest: Pair Cancellation. Only demographic and hearing-history variables significant at the p < .05 level are displayed in this table. CDI = MacArthur–Bates Communicative Development Inventories.
p < .05.
p < .01.
p < .001.
Regression Models Predicting Long-Term Outcomes
Table 6 displays a summary of the results from the regression analyses using early expressive vocabulary size (CDI:WP) as a predictor of long-term neurocognitive performance outcomes. CDI:WP significantly predicted long-term language, verbal working memory, and fluency–speed performance scores. Early communication mode was significantly attenuated following the entry of CDI:WP into the regression equation predicting long-term language. The overall equation with CDI:WP and early communication mode accounted for 51% of the variance in long-term language scores (p < .001). Larger early expressive vocabulary size significantly predicted better performance on verbal working memory tasks, accounting for 31% of the variance in long-term verbal working memory scores (p = .001). Larger early expressive vocabulary size also significantly predicted (p = .006) faster fluency–speed performance scores. The overall equation with CDI:WP, age at implantation, and gender (female) accounted for 51% of the variance in long-term fluency–speed scores (p < .001).
Tables 7 and 8 display summaries of the regression analyses using CDI:WP to predict long-term LEAF cognitive and academic functioning scores, with significant demographic and hearing-history variables controlled. CDI:WP significantly predicted (p = .007) only basic reading skills scores, with the equation, including early communication mode accounting for 39% of the variance in long-term basic reading skills. It is notable, however, that CDI scores were also related to several other LEAF subscales (attention, sustained sequential processing, written expression) at a p < .05 level; however, only values at the p < .01 level were interpreted for significance in order to reduce Type I errors. Additionally, early communication mode (oral) significantly predicted (p = .011) sustained sequential processing scores, with the equation, including CDI:WP accounting for 39% of the variance in long-term sustained sequential processing skills.
Table 7.
LEAF cognitive subscales |
||||
---|---|---|---|---|
Attention |
Sustained sequential processing |
|||
β | SE | β | SE | |
Block 1: Hearing history | ||||
Early communication mode | −.49** | .16 | −.54** | .16 |
R 2 | 0.24** | 0.29** | ||
Block 2 | ||||
CDI words produced | −.37* | .16 | −.35* | .16 |
Early communication mode | −.37* | .16 | −.43** | .16 |
R 2 | 0.37** | 0.39** |
Note. N = 31. The standardized regression coefficient (β) and standard error (SE) of β are provided. Only demographic and hearing-history variables significant at the p < .05 level are displayed in this table. CDI = MacArthur–Bates Communicative Development Inventories.
p < .05.
p < .01.
Table 8.
LEAF academic subscale |
||||
---|---|---|---|---|
Basic reading skills |
Written expression skills |
|||
β | SE | β | SE | |
Block 1: Hearing history | ||||
Early communication mode | −.45* | .17 | −.38* | .17 |
R 2 | 0.20* | 0.14* | ||
Block 2 | ||||
CDI words produced | −.46** | .16 | −.44* | .17 |
Early communication mode | −.30 a | .16 | −.24 | .17 |
R 2 | 0.39** | 0.31** |
Note. N = 31. The β for the standardized regression coefficient and standard error (SE) of β are provided. Only demographic and hearing-history variables significant at the p < .05 level are displayed in this table. CDI = MacArthur–Bates Communicative Development Inventories.
p < .10.
p < .05.
p < .01.
Discussion
The present investigation using the CDI is the first study, to our knowledge, to show that expressive language skills obtained in early toddlerhood are clinically meaningful and strongly predictive of long-term language and executive functioning outcomes in school-age and young adult CI users. Although CDI norms could not be used with our clinical sample because of age boundaries, the CDI still served as a practical, reliable, and highly cost-effective instrument for indexing the early expressive language skills of prelingually deaf children with CIs. Although the results of the present study are preliminary and based on a small sample size of 32 young CI users, the clinical implications for using the CDI as a screening instrument are considerable. Our findings suggest that the CDI may be used clinically as an initial tool by speech-language pathologists and audiologists to identify pediatric CI users who display low expressive language skills and thus may be at risk for neurocognitive delays and disturbances in later development after a period of long-term CI use. We believe our research approach is of important clinical significance because it can provide insights into motivating possible novel interventions that are specifically designed to target underlying mechanisms of action and to improve neurocognitive outcomes in young CI users, especially the group of deaf children who may not be achieving optimal levels of speech and language performance with their CIs, even after a long period of use (see Kronenberger, Pisoni, Henning, Colson, & Hazzard, 2011; and Nunes, Barros, Evans, & Burman, 2014, for neurocognitive training programs). In the future, in an effort to facilitate routine clinical use of the CDI, we hope to provide data normalized on a larger sample of young deaf CI users so that viable cutoffs for lexical growth can be determined.
A small proportion (12.5%) of our deaf pediatric sample made impressive lexical progress after an average of 2 years of CI use, with some deaf children having vocabulary sizes nearing the ceiling levels established by the CDI (displaying expressive vocabularies of more than 500 words). The opposite end of the spectrum was also true, as another portion (21.9%) of our deaf pediatric sample made very little lexical progress after gaining access to spoken language via a CI (displaying expressive vocabularies of less than 100 words). The individual variability observed in early lexical growth in our sample of 32 long-term CI users is quite large, although it is not anomalous or idiopathic but, rather, reflects how a period of early auditory deprivation affects the development of speech and spoken language and ultimately how early linguistic and cognitive experiences affect several foundational aspects of higher order cognition and executive functioning. Indeed, descriptive statistics on long-term neurocognitive functioning have revealed areas of executive functioning, including verbal working memory, inhibition concentration, and fluency speed, that are affected by a period of auditory and spoken language deprivation. As with previous findings, a great amount of variability can be found in verbal working memory (ranges from clinical deficits of −2 SD to above average), fluency speed (ranges from clinical deficits of −2 SD to above average), and inhibition concentration (ranges from clinical deficits of −4 SD to above average) performance. For more-stable estimates of the rate of executive functioning deficits based on a larger sample, see Kronenberger, Beer et al. (2014) for performance data and Kronenberger, Colson et al. (2014) for questionnaire-based data on relative risk.
After controlling for nonverbal intelligence, the development of early expressive language, as indexed by vocabulary size obtained from the CDI, was found to be significantly associated with long-term performance-based, and questionnaire-based measures of neurocognitive skills. Larger early vocabulary sizes (CDI:WP) were associated with better long-term performance measures of language, verbal working memory, and fluency speed and better long-term LEAF measures of attention, sustained sequential processing, basic reading, and written expression skills up to 16 years later, supporting the long-term predictability of the CDI.
In our regression analysis, early expressive vocabulary size (CDI:WP) was found to be highly predictive of long-term language, verbal working memory, and fluency–speed skills. These results replicate the Marchman and Fernald (2008) findings that demonstrated that early expressive vocabulary size was predictive of long-term language and verbal working memory in typically developing NH children. Regression analyses also revealed that early expressive vocabulary size was significantly predictive (p < .01) of only one long-term LEAF questionnaire-based measure: basic reading skills. However, regression coefficients for CDI scores fell at a p < .05 level for predicting attention, sustained sequential processing, and written expression skills (see Tables 7 and 8). Because our use of the more stringent p < .01 criterion for significance increases the chance of Type II (β) errors, these latter findings should be interpreted with caution because they may reflect insufficient power to detect a significant effect at this more stringent level. Importantly, partial correlations between CDI:WP scores and LEAF attention, sustained sequential processing, and written-expression scores (see Table 5) were statistically significant at the p < .01 level, suggesting that insufficient power in the regression equations may have influenced results for those LEAF subscales. Finally, early communication mode also predicted long-term LEAF sustained sequential processing scores. These results are consistent with previous studies indicating associations between spoken language processing and sequential learning (Conway et al., 2010). Exposure to, and experience with, spoken language (phonological sequences) aids in encoding and representing other sequential patterns in the environment (Conway et al., 2009).
Across both performance- and questionnaire-based measures, early expressive language skills using the CDI were not associated with or predictive of visual–spatial working memory and visual–spatial organization. Together with previous findings in our laboratory, the present results suggest that visual–spatial skills may be less heavily reliant on the development of early lexical organization and, therefore, may be more resilient to early auditory delays and disturbances (Kronenberger, Beer et al., 2014; Kronenberger, Colson et al., 2014).
As with other longitudinal studies, several limitations should be taken into account when interpreting the present results:
Our sample was relatively small and consisted of homogenous English speakers, limiting our ability to generalize our long-term outcome findings to more-diverse samples of CI users, such as bilingual language speakers. The CDI is currently available in multiple languages, such as Spanish, and future research should investigate the predictability of early expressive language skills on the long-term functioning of bilingual CI users because these data are sparse (see Robbins, Green, & Waltzman, 2004, for short-term language outcomes in bilingual CI users).
At the time these CDI data were collected, our sample of pediatric CI users had similar “hearing ages” (i.e., they had used their CI for an average of 1.03 years) but varied in chronological ages that ranged from 1.98 to 6.10 years old. Even after controlling for chronological age at time of CDI administration and nonverbal intelligence, early expressive vocabulary size was still found to be significantly correlated with long-term neurocognitive performance measures and questionnaire-based skills. Additionally, our regression analyses indicated that chronological age at the time the CDI was administered was not a significant predictor of long-term neurocognitive functioning.
For this research study, we sought to recruit participants from our archived database of young CI users who received the CDI within 2.5 years post cochlear implantation. Because of this recruitment strategy, our sample of CI users consisted of participants with a relatively large chronological age range (9.10–21.55 years old) at long-term, follow-up testing. We do not believe the large age range detracts from our findings because our regression analyses indicated that chronological age at long-term, follow-up testing and the interval of time between the CDI and long-term, follow-up testing visits were not significant predictors of long-term neurocognitive functioning.
Because of chronological age restrictions, we used raw CDI scores and were unable to compare the expressive language skills of our pediatric CI users to CDI percentiles based on typically developing NH controls.
We only examined early parent-reported expressive-language skills and did not present complementary data using early performance-based assessments (see Castellanos et al., 2014, for early performance data).
Although we obtained multiple data points of early expressive language skills, we only had these measures for a small subset of our long-term participants and thus, in the present study, we captured only a snapshot of the effects of early expressive language development on long-term outcomes in this clinical population.
In summary, the present study investigated the predictive value of early expressive language skills for long-term language and neurocognitive outcomes in a sample of early implanted pediatric CI users. Long-term skills were assessed with gold-standard performance-based assessments of neurocognitive function (language, verbal working memory, visual–spatial working memory, fluency speed, and inhibition concentration) and a questionnaire-based parent report of neurocognitive functioning and academic skills. Our findings suggest that early expressive vocabulary size and lexical development have long-term consequences for a broad range of neurocognitive skills in early implanted CI users, including language, verbal working memory, fluency speed, attention, sustained sequential processing, and basic reading and writing skills. These findings suggest that the CDI can be reliably used following cochlear implantation as an early index of expressive language skills and may serve as a potential early marker of long-term neurocognitive risk in this clinical population.
Acknowledgments
This research was supported by National Institute on Deafness and Other Communication Disorders Grants R01 DC009581 (awarded to David B. Pisoni and William G. Kronenberger) and T32 DC00012 (awarded to David B. Pisoni) and by National Center for Advancing Translational Sciences Grant TL1 TR001107 (awarded to Irina Castellanos).
Funding Statement
This research was supported by National Institute on Deafness and Other Communication Disorders Grants R01 DC009581 (awarded to David B. Pisoni and William G. Kronenberger) and T32 DC00012 (awarded to David B. Pisoni) and by National Center for Advancing Translational Sciences Grant TL1 TR001107 (awarded to Irina Castellanos).
Footnotes
Compared with the normative national sample, our sample of pediatric CI users had similar “hearing ages” (length of time with hearing access through the CI) despite being chronologically older.
Partial correlations controlling for chronological age at CDI administration also revealed significant associations between early expressive vocabulary size and long-term language, verbal working memory, and fluency–speed outcomes.
Partial correlations controlling for chronological age at CDI administration also revealed significant associations between early expressive-vocabulary size and two of the eight long-term LEAF cognitive outcomes and two of the three long-term LEAF academic outcomes.
References
- Barkley R. A. (2012). Executive functions: What they are, how they work, and why they evolved. New York, NY: Guilford. [Google Scholar]
- Castellanos I., Kronenberger W. G., Beer J., Colson B. G., Henning S. C., Ditmars A., & Pisoni D. B. (2015). Concept formation skills in long-term cochlear implant users. Journal of Deaf Studies and Deaf Education, 20(1), 27–40. doi:10.1093/deafed/enu039 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Castellanos I., Kronenberger W. G., Beer J., Henning S. C., Colson B. G., & Pisoni D. B. (2014). Preschool speech intelligibility and vocabulary skills predict long-term speech and language outcomes following cochlear implantation in early childhood. Cochlear Implants International, 15(4), 200–210. doi:10.1179/1754762813Y.0000000043 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Conway C. M., Pisoni D. B., Anaya E. M., Karpicke J., & Henning S. C. (2010). Implicit sequence learning in deaf children with cochlear implants. Developmental Science, 14(1), 69–82. doi:10.1111/j.1467-7687.2010.00960.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Conway C. M., Pisoni D. B., & Kronenberger W. G. (2009). The importance of sound for cognitive sequencing abilities: The auditory scaffolding hypothesis. Current Directions in Psychological Science, 18(5), 275–279. doi:10.1111/j.1467-8721.2009.01651.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Copeland B. J., & Pillsbury H. C. (2004). Cochlear implantation for the treatment of deafness. Annual Review of Medicine, 55(1), 157–167. doi:10.1146/annurev.med.55.091902.105251 [DOI] [PubMed] [Google Scholar]
- Dunn L. M., & Dunn D. M. (2007). Peabody Picture Vocabulary Test–Fourth Edition manual. Bloomington, MN: Pearson. [Google Scholar]
- Fenson L., Marchman V. A., Thal D. J., Dale P. S., Reznick J. S., & Bates E. (2006). The MacArthur–Bates Communicative Development Inventories (2nd ed.). Baltimore, MD: Brookes. [Google Scholar]
- Geers A., Brenner C., & Davidson L. (2003). Factors associated with development of speech perception skills in children implanted by age five. Ear and Hearing, 24(Suppl.), 24–35. doi:10.1097/01.AUD.0000051687.99218.0F [DOI] [PubMed] [Google Scholar]
- Harris M. S., Pisoni D. B., Kronenberger W. G., Gao S., Caffrey H. M., & Miyamoto R. T. (2011). Developmental trajectories of forward and backward digit spans in deaf children with cochlear implants. Cochlear Implants International, 12, S84–S88. doi:10.1179/146701011X13001035752534 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Horton A. M. Jr., Soper H. V., & Reynolds C. R. (2010). Executive functions in children with traumatic brain injury. Applied Neuropsychology, 17, 99–103. doi:10.1080/09084281003708944 [DOI] [PubMed] [Google Scholar]
- Kronenberger W. G., Beer J., Castellanos I., Pisoni D. B., & Miyamoto R. T. (2014). Neurocognitive risk in children with cochlear implants. Otolaryngology–Head & Neck Surgery, 140(7), 608–615. doi:10.1001/jamaoto.2014.757 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kronenberger W. G., Colson B. G., Henning S. C., & Pisoni D. B. (2014). Executive functioning and speech-language skills following long-term use of cochlear implants. Journal of Deaf Studies and Deaf Education, 4, 456–470. doi:10.1093/deafed/enu011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kronenberger W. G., & Pisoni D. B. (2009, August). Measuring learning-related executive functioning: Development of the LEAF scale. Poster presented at the American Psychological Association 117th Annual Convention. Toronto, ON, Canada. [Google Scholar]
- Kronenberger W. G., Pisoni D. B., Henning S. C., Colson B. G., Hazzard L. M. (2011). Working memory training for children with cochlear implants: A pilot study. Journal of Speech, Language, and Hearing Research, 54, 1182–1196. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leark R. A., Dupuy T. R., Greenberg L. M., Corman C. L., & Kindschi C. L. (1996). Test of Variables of Attention professional manual version 7.0. Los Alamitos, CA: Universal Attention Disorders. [Google Scholar]
- Luria A. R. (1966). Higher cortical functions in man. New York, NY: Basic Books. [Google Scholar]
- Lyxell B., Andersson U., Borg E., & Ohlsson I.-S. (2003). Working-memory capacity and phonological processing in deafened adults and individuals with a severe hearing impairment. International Journal of Audiology, 42, S86–S89. [DOI] [PubMed] [Google Scholar]
- Marchman V. A., & Fernald A. (2008). Speed of word recognition and vocabulary knowledge in infancy predict cognitive and language outcomes in later childhood. Developmental Science, 11(3), F9–F16. doi:10.1111/j.1467-7687.2008.00671.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller E. K., & Cohen J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167–202. [DOI] [PubMed] [Google Scholar]
- McAuley T., & White D. A. (2011). A latent variables examination of processing speed, response inhibition, and working memory during typical development. Journal of Experimental Child Psychology, 108(3), 453–468. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nicholas J. G., & Geers A. E. (2006). Effects of early auditory experience on the spoken language of deaf children at 3 years of age. Ear and Hearing, 27(3), 286–298. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nicholas J. G., & Geers A. E. (2008). Expected test scores for preschoolers with a cochlear implant who use spoken language. American Journal of Speech-Language Pathology, 17, 121–138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nunes T., Barros R., Evans D., & Burman D. (2014). Improving deaf children's working memory through training. International Journal of Speech & Language Pathology and Audiology, 2, 51–66. doi:10.12970/2311-1917.2014.02.02.1 [Google Scholar]
- O'Hara L. K., & Holmbeck G. N. (2013). Executive functions and parenting behaviors in association with medical adherence and autonomy among youth with spina bifida. Journal of Pediatric Psychology, 38, 675–687. doi:10.1093/jpepsy/jst007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Perin da Silva M., Comerlatto Junior A. A., Bevilacqua M. C., & Lopes-Herrera S. A. (2011). Instruments to assess the oral language of children fitted with a cochlear implant: A systematic review. Journal of Applied Oral Science, 19(6), 549–553. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pisoni D. B., Conway C. M., Kronenberger W. G., Henning S., & Anaya E. (2010). Executive function, cognitive control, and sequence learning in deaf children with cochlear implants. In Marschark M. S. & Spencer P. E. (Eds.), Oxford handbook of deaf studies, language, and education (2nd ed., Vol. 1, pp. 439–457). New York, NY: Oxford University Press. [Google Scholar]
- Reynell J., & Gruber C. P. (1990). Reynell Developmental Language Scales: Manual. Los Angeles, CA: Western Psychological Services. [Google Scholar]
- Robbins A. M., Green J. E., & Waltzman S. B. (2004). Bilingual oral language proficiency in children with cochlear implants. Archives of Otolaryngology—Head & Neck Surgery, 130(5), 644–647. [DOI] [PubMed] [Google Scholar]
- Semel E. M., Wiig E. H., & Secord W. (2003). Clinical Evaluation of Language Fundamentals (CELF-4). San Antonio, TX: The Psychological Corporation. [Google Scholar]
- Stallings L. M., Gao S., & Svirsky M. A. (2002). Assessing the language abilities of pediatric cochlear implant users across a broad range of ages and performance abilities. The Volta Review, 102(4), 215–235. [Google Scholar]
- Thal D., DesJardin J. L., & Eisenberg L. S. (2007). Validity of the MacArthur–Bates Communicative Development Inventories for measuring language abilities in children with cochlear implants. American Journal of Speech-Language Pathology, 16(1), 54–64. [DOI] [PubMed] [Google Scholar]
- Wechsler D. (1991). Wechsler Intelligence Scale for Children (3rd ed.). San Antonio, TX: The Psychological Corporation. [Google Scholar]
- Wechsler D. (1999). Wechsler Abbreviated Scale of Intelligence (WASI). San Antonio, TX: The Psychological Corporation. [Google Scholar]
- Wechsler D., Kaplan E., Fein D., Kramer J., Morris R., Delis D., & Maerlender A. (2004). Wechsler Intelligence Scale for Children: Integrated (4th ed.). San Antonio, TX: Harcourt Assessment. [Google Scholar]
- Woodcock R. W., McGrew K. S., & Mather J. (2001). Woodcock-Johnson III. Itasca, IL: Riverside. [Google Scholar]