Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Aug 1.
Published in final edited form as: J Autism Dev Disord. 2011 Aug;41(8):1065–1075. doi: 10.1007/s10803-010-1134-4

Lexical and Grammatical Skills in Toddlers on the Autism Spectrum Compared to Late Talking Toddlers

Susan Ellis Weismer 1, Morton Ann Gernsbacher 2, Sheri Stronach 3, Courtney Karasinski 4, Elizabeth R Eernisse 5, Courtney E Venker 6, Heidi Sindberg 7
PMCID: PMC3049899  NIHMSID: NIHMS251910  PMID: 21061053

Abstract

This study compared language development in 30-month-old toddlers on the autism spectrum and 25-month-old late talking toddlers without autism. Groups were matched on overall productive vocabulary (and nonverbal cognition was controlled) in order to compare language acquisition patterns related to vocabulary composition and early lexical-grammatical relationships. Findings revealed that semantic categories of words – including psychological state terms – used by toddlers on the autism spectrum were very similar to those of late talkers. Both groups were equivalent with respect to grammatical complexity and proportion of toddlers combining words, though late talkers displayed a relatively stronger association between lexical-grammatical abilities. These tentative findings are consistent with a dimensional account of early, core linguistic abilities across different populations of children with language delay.

Keywords: Autism, Language, Late Talkers, Vocabulary, Semantic Categories, Grammar, Psychological State Terms

Investigation of early language development within the autism spectrum is important from both scientific and clinical perspectives. The core linguistic abilities comprising structural language development in young children are vocabulary (lexical) acquisition and grammatical (morphological and syntactic) acquisition. The theoretical issue of interest for the current study was whether lexical and grammatical delays observed in young children on the autism spectrum constitute a unique phenomenon or whether they overlap with other types of language delays. These two characterizations of structural language delays across different populations have been referred to as the distinct category account versus the dimensional account (see Gernsbacher, Geye, and Ellis Weismer, 2005). The dimensional account has also been described in terms of a ‘language endowment spectrum’ with respect to typical talkers, late talking toddlers, and young children with specific language impairment (Ellis Weismer, 2007; Rescorla, 2009; Rescorla & Roberts, 2002). Research focused on contrasting the distinct category versus dimensional accounts by employing cross-population comparisons has implications for identifying phenotypic markers of autism used in genetic investigations (Lindgren, Folstein, Tomblin, & Tager-Flusberg, 2009; Ruser et al., 2007; Whitehouse, Barry, & Bishop, 2007), as well as implications for planning interventions for children with autism. Previous investigators have emphasized the benefits of cross-population comparisons of language development and suggested that more research of this type is needed (Beeghly, 2006; Rice and Warren, 2004).

Earlier research has investigated overlap between children with specific language impairment (SLI), pragmatic language impairment (PLI), and autism (Bishop, 2000; Bishop & Norbury, 2002). However, we are interested in core linguistic abilities or structural language, which refers to phonological, semantic, and grammatical components of language rather than aspects of pragmatics or social communication. While children on the autism spectrum routinely have pragmatic language difficulties associated with their atypical social interaction, there is a wide range of variability in structural language ability. In typical development, all of these components of language interact and are critical for successful communicative competence. In the current study we focus on the semantic (lexical) and grammatical features of structural language. The particular aspects of language that were targeted represent key areas of early language acquisition for all children– namely, the use of different categories of words, combination of words into multi-word phrases, use of morphological markers (such as plural “s” and past tense “ed”), and early appearing syntactic structures to form sentences. Psychological state terms are of interest as a special semantic category of words because they have been linked with performance on theory of mind tasks and both children with SLI and children on the autism spectrum have been reported to have theory of mind difficulties.

Several studies have reported differences in certain aspects of vocabulary development in children on the autism spectrum compared to various other groups (Bruckner, Yoder, Stone, & Saylor, 2007; Tager-Flusberg, 1992; Williams, 1993; Ziatas, Durkin, & Pratt, 1998). Two of these studies focused exclusively on psychological (or mental) state terms, stemming from an interest in ‘theory of mind’ development. However, other investigations have found a number of similarities in early vocabulary acquisition between young children on the autism spectrum and comparison groups (Charman, Drew, Baird & Baird, 2003; Luyster, Lopez, & Lord, 2007; Swensen, Kelley, Fein, & Naigles, 2007). It is difficult to resolve these conflicting findings with existing data because the studies have varied considerably in terms of methodological features (comparison groups, type of task), developmental level (infancy through 8 years), and diagnostic characteristics of the sample (autism versus autism spectrum).

An early case report by Williams (1993) examined vocabulary development in an autistic boy from age 3 years, 3 months to 4 years. Findings revealed a predominance of nominals, characteristic of a referential (rather than expressive) pattern of language acquisition. This child used only 5 of 36 frequent words reported in several other studies of typically developing children, and his vocabulary consisted of more modifiers and fewer action words than vocabularies of typically developing children. While descriptive, longitudinal data of this type can be suggestive of possible developmental patterns, findings from subsequent, larger-scale studies have offered better evidence regarding the nature of early vocabulary acquisition in children on the autism spectrum. Bruckner et al. (2007) compared the mental lexicon (i.e., vocabulary knowledge) of a large group of toddlers on the autism spectrum (n=209) to a group of younger typically developing infants (n=272). Groups were matched on total size of receptive vocabulary as measured by the MacArthur Communicative Development Inventory, CDI-Infant Form (Fenson et al., 1991). A logistic regression analysis revealed differential item functioning for 25 words on this 396-item vocabulary checklist, indicating that these items functioned differently across the two groups and were not representative of the mental lexicons of both groups. That is, some of the words were more likely to be understood by typically developing infants than by toddlers on the autism spectrum, whereas other words were more likely to be understood by toddlers on the autism spectrum than by typically developing infants. Bruckner et al. suggested that differences in the mental lexicons of the two groups (or parents’ interpretation of the content of their children’s mental lexicon) could be due to age differences or atypical orienting, social communication, or object use on the part of the toddlers on the autism spectrum.

Studies examining psychological state terms, such as happy, hurt, and mad, have reported differences between children with autism and comparison groups. Tager-Flusberg (1992) analyzed the use of terms expressing desire, perception, emotion, and cognition in spontaneous language samples of children with autism and children with Down syndrome who were matched on age and expressive language level. While the children with autism were comparable to the Down syndrome controls in their use of desire, perception, and emotion terms, they used significantly fewer terms to refer to cognitive mental states; this finding was interpreted as reflecting a deficit in theory of mind acquisition. Ziatas and colleagues (1998) directly examined the relationship between comprehension and production of belief terms think, know, and guess and performance on a false belief task. Children with autism and Asperger syndrome were compared to children with specific language impairment (SLI) and typically developing children, with all groups matched on receptive vocabulary level. Findings revealed an association between knowledge of false belief terms and performance on the false belief task across groups. The autistic group performed significantly poorer than the other groups on the false belief task as well as on comprehension and production of false belief terms.

In contrast, other investigations have found more similarities than differences in the patterns of early vocabulary development in children on the autism spectrum relative to comparison groups. Charman and colleagues (2003) investigated language development in 134 preschool children on the autism spectrum using the CDI-Infant Form, comparing their performance to the CDI’s normative sample. While children on the autism spectrum displayed an atypical pattern in which word comprehension was delayed relative to word production, they demonstrated a number of similarities with typically developing infants. Similarities included wide variability in vocabulary acquisition (though the group as a whole was significantly delayed), word comprehension preceded word production in absolute terms, and the pattern of acquisition across word categories was generally consistent with typical development. The findings of a similar study by Luyster et al. (2007) were generally consistent with those of Charman et al. (2003). Although children on the autism spectrum were delayed in their lexical development, they demonstrated a profile of expressive and receptive vocabulary that was similar to a typically developing control group and a group with developmental delay.

Rather than focusing on vocabulary profiles per se, a few recent studies have examined language acquisition processes presumed to support word learning, such as fast mapping and the noun bias (McDuffie, Yoder, & Stone, 2006; Swensen et al., 2007). For instance, a study by Swensen et al. (2007) used an intermodal preferential looking paradigm to compare language acquisition processes in toddlers on the autism spectrum to younger typically developing toddlers who were matched for expressive vocabulary on the CDI. Swensen et al.’s findings indicated that toddlers on the autism spectrum, like their typically developing language-matched peers, demonstrated a noun bias by attaching a novel word to objects rather than actions.

One group who offers an interesting comparison to toddlers on the autism spectrum is ‘late talkers.’ Late talkers are toddlers who exhibit late onset of language development but are developing typically in other domains, such as cognitive, motor, social-emotional areas (see Ellis Weismer, 2007). Vocabulary development in late talkers – including use of psychological state terms – has been investigated by Rescorla and colleagues (Lee & Rescorla, 2002, 2008; Rescorla, Alley, & Christine, 2001). Rescorla et al. investigated word frequencies in the lexicons of late talking toddlers compared to a large community sample of typically developing toddlers using the Language Development Survey (Rescorla, 1989), a parent report checklist. Frequency was examined by computing a percentage value for each word based on the number of toddlers reported to produce the word. Frequency correlation statistics revealed that the words used most often by typically developing toddlers were also among the words used most often by the late talkers. This finding suggests that the overall pattern of vocabulary development is similar for toddlers with and without language delay with respect to the specific words used. Although some differences in word frequencies were noted between 3-year-old late talkers and 2-year-old typically developing toddlers whose lexicons were similar in size, these differences were attributed to differences in interests and abilities of 2- versus 3-year-olds. For example, the words fork, knife, spoon, and ABC were higher frequency words for the late talking 3-year-olds, whereas the words peekaboo, crib, and diaper were higher frequency for the younger, typically developing 2-year-olds.

Lee and Rescorla (2002, 2008) investigated late talkers’ use of psychological state terms at ages 3, 4, and 5 years in spontaneous language samples collected during mother-child play sessions. Four types of psychological state words were examined: physiological, emotional, desire, and cognitive terms. When children who had been identified as late talkers at 24–31 months were compared to age-mates with typical language development, several differences were noted. Late talkers used significantly more physiological state words than the comparison group at ages 3 and 4 but not at age 5. Late talkers used significantly fewer cognitive terms than controls at each age level. When 5-year-old late talkers were matched to 3-year-olds from the comparison group on overall language level (as indexed by mean length of utterance), there was no significant difference between the groups’ use of any of the categories of psychological state words, including the later developing cognitive terms.

Prior research has also examined grammatical abilities in children with autism (Eigsti, Bennetto, & Dadlani, 2007; Kjelgaard & Tager-Flusberg, 2001; Roberts, Rice, & Tager-Flusberg, 2004; Scarborough, Rescorla, Tager-Flusberg, Fowler, et al., 1991) and in late talkers (Moyle, Ellis Weismer, Evans, & Lindstrom, 2007; Rescorla & Roberts, 2002); however, few studies have investigated the association between early lexical and grammatical development. Bates and colleagues have suggested that advances in grammar occur only after vocabulary reaches a ‘critical mass’ (Bates & Goodman, 2001; Marchman & Bates, 1994). According to this view, young children must acquire a minimum threshold of vocabulary knowledge upon which grammatical knowledge can begin to be constructed. Research has documented links between lexical and grammatical skills during the earliest stages of typical development across various languages (Dionne, Dale, Boivin, & Plomin, 2003; Maitel, Dromi, Sagi, & Bornstein, 2000; Thordardottir, Ellis Weismer, & Evans, 2002). Moyle et al. (2007) examined longitudinal relationships between lexical and grammatical development (from 24 to 66 mo) in typically developing children and late talkers. Their findings indicated a strong lexical-grammatical association for both groups (with the timing of the association delayed for the late talkers), but suggested that syntactic growth may be less facilitative of lexical development in late talkers than typical talkers. Although previous research has not investigated early stages of grammatical development in very young children on the autism spectrum, there is some indication that somewhat older children have more difficulties in syntactic abilities than would be expected given their vocabulary level. For example, Eigsti et al. (2007) reported that five-year-old children with autism displayed significant syntactic delays compared to comparison groups with developmental delays or typical development who were matched on lexical level and nonverbal mental age.

Purpose of the Current Study

The present study explored whether early language patterns displayed by young children on the autism spectrum are unique or whether they overlap with patterns of acquisition observed in other types of language delays. Prior research has investigated this general issue for older, school-aged children on the autism spectrum compared to children with SLI or other neurodevelopmental disorders (Eigsti et al., 2007; Roberts et al., 2004). However, this is the first study, to our knowledge, to compare language acquisition patterns in toddlers on the autism spectrum to toddlers without autism who have late onset of language and are at risk for later language impairment. This particular comparison allows speculation about early differences in core linguistic abilities that might be attributable to autistic characteristics versus ‘pure’ language delay that is not associated with deficits in other areas of development. By matching the groups on vocabulary level and controlling for nonverbal cognition, we attempted to compare more fine-grained aspects of language acquisition patterns involving the types of words comprising the children’s vocabulary repertoire and the relation between early vocabulary and grammatical development. Specifically, we examined whether toddlers on the autism spectrum (a subgroup who produced spoken words) and late talkers, matched on overall productive vocabulary, display differences in their use of a) semantic categories; b) psychological state terms; and c) word combinations and grammatical complexity, taking into account differences in nonverbal cognitive level. The distinct category account predicts that the two groups will display qualitatively different types of word use (semantic categories) and that the pattern of lexical-grammatical associations will be distinct across the groups. In contrast, the dimensional account predicts that while the groups may exhibit quantitative differences (i.e., differences in severity of delay), their patterns of word use and lexical-grammatical associations will be qualitatively similar.

Methods

Participants

Two groups of toddlers (22–37 months) participated in this study: 40 toddlers on the autism spectrum (35 boys, 5 girls) and 40 late talkers without autism (27 boys, 13 girls). Both groups of participants were part of larger longitudinal investigations of language and communication development. Participants for the current study were drawn from these larger samples by selecting 40 toddlers for both groups who produced at least 15 different words based on a parental report measure. Participants within the Autism Spectrum group and the Late Talker group were individually matched on parent report of words produced (± 9 words) on the MacArthur-Bates Communicative Development Inventories: Words & Sentences (CDI) such that the groups were equivalent in terms of overall vocabulary, t(78)= −.13, p=.89 (two-tailed). The two groups did not differ significantly with respect to SES as indexed by years of maternal education, t(76) = 1.18, p=.24 (two-tailed). As seen in Table 1, vocabulary matching resulted in a significant age difference between the groups, t(78)= −.6.52, p=.00 (two-tailed), with the mean age of the Autism Spectrum group exceeding that of the Late Talkers by approximately 5 months. The groups also differed significantly in terms of cognitive level as measured by nonverbal cognitive items from the Bayley Scales of Infant Development – Second Edition (Bayley-II) (Bayley, 1993), t(75)=8.68, p=.00 (two-tailed). Prior research has used nonverbal items from the Bayley-II as the basis for matching groups for nonverbal cognitive level (Moyle et al., 2007; Rescorla, 2009). Bayley-II scores were available for 39 of 40 toddlers in the Autism Spectrum group and 38 of 40 toddlers in the Late Talker group (sample size for each measure is noted in Table 1).

Table 1.

Summary of group characteristics.

Autism Spectrum Group Late Talker Group
CDI Words Produceda (n=40) (n=40)
 Mean 108.15 105.90
 Standard Deviation 76.11 74.93
 Range 17–298 16–302
Chronological Age (months) (n=40) (n=40)
 Mean 30.33 25.63
 Standard Deviation 3.57 2.84
 Range 23–37 22–30
Bayley -II Nonverbal Cognitionb (n=39) (n=38)
 Mean 3.46 7.37
 Standard Deviation 2.01 1.94
 Range 1–9 4–11
Bayley- III Cognitive Compositec (n=40)
 Mean 85.52 NA
 Standard Deviation 10.83 NA
 Range 57–105 NA
Maternal Education (years) (n=40) (n=38)
 Mean 14.90 15.47
 Standard Deviation 2.23 2.06
 Range 12–19 12–19
a

CDI = MacArthur-Bates Communicative Development Inventories: Words & Sentences.

b

Raw scores based on a total of 11 nonverbal items from the Bayley- II.

c

Standard scores based on a mean of 100 and standard deviation of 15; Bayley-III was only administered to the Autism Spectrum group.

Autism spectrum diagnoses were determined using comprehensive diagnostic evaluations by an experienced clinician that included the Autism Diagnostic Interview-Revised (Lord, Rutter, & Le Couteur, 1994) and the Autism Diagnostic Observation Schedule (ADOS) (Lord, Rutter, DiLavore, & Risi, 1999) or Autism Diagnostic Observation Schedule – Toddler Module (Luyster et al., 2009). The Autism Spectrum group displayed considerable variability in cognitive abilities as indexed by performance on the Bayley- II and Bayley Scales of Infant and Toddler

Development – Third Edition (Bayley-III) (Bayley, 2005), as seen in Table 1. Late talkers were initially identified at 24 months on the basis of their delayed expressive language on the CDI. For the purposes of the current study, Late Talkers were defined as those toddlers who scored at or below the 10th percentile on words produced or below the 20th percentile on words produced and were not yet combining words. The mean CDI percentile rank for words produced by the Late Talker group was 5.55, SD=6.10 (CDI percentile ranks were not computed for the Autism Spectrum group because some children in that group exceeded the upper age level for the normative sample). Late Talkers exhibited no other areas of delay (besides language) based on a detailed background information form completed by the parents and administration of the Denver Developmental Screening Test II (Frankenburg & Dobbs, 1990). It is particularly important to note that toddlers in the Late Talker group performed within normal range on the social/personal section of the Denver and that parents reported no concerns regarding social development. For those toddlers who were enrolled in early intervention for language delays, Birth-to-Three providers reported no concerns about possible autism spectrum diagnoses. Additionally, this sample of children was followed longitudinally until kindergarten (5 years of age). At that point children in the Late Talker group did not meet criteria on the Social Communication Questionnaire (Rutter, Bailey, Berument, Lord, & Pickles, 2006) and none of the children had received an autism or autism spectrum diagnosis in their communities. All participants were monolingual speakers of American English. This study was approved by the Social and Behavioral Sciences Institutional Review Board at the University of Wisconsin-Madison and parents provided written consent prior to their child’s participation.

Procedure

Semantic categories

Vocabulary usage of late talkers and toddlers on the autism spectrum was compared in terms of the breakdown across semantic categories on the CDI. There are 22 different categories of vocabulary items on the CDI: Words & Sentences form. Half of the categories consist of nouns and half are comprised of other word types. The number of vocabulary items within a given category on the CDI ranges from 6 (for Connecting Words) to 103 (for Action Words), with a mean of 31 words per category.

Psychological state terms

The analysis of psychological state terms from the CDI was based on a combination of categories employed in prior research (Johnston, Miller, & Tallal, 2001; Lee & Rescorla, 2002). Table 2 provides a summary of the five categories examined – emotion, desire, cognitive, physiological, evaluation – and specific words within each category that are assessed by the CDI. A total of 33 psychological state terms are included on the CDI, 10 words within the Action Word category and 23 words within the Descriptive Words category. The CDI contains a relatively large number of physiologic al terms that are acquired earlier and very few of the later developing cognitive terms.

Table 2.

Categories of psychological state terms and vocabulary items on the MacArthur-Bates Communicative Development Inventories: Words & Sentences from each category.

Emotion Desire Cognitive Physiological Evaluation
cry hate pretend sleep, asleep bad
happy like think taste better
mad love wake, awake cute
sad wish broken gentle
scared cold good
hot naughty
hungry nice
hurt pretty
sleepy yucky
thirsty
tired

Early grammatical abilities

Two indices of early grammatical abilities from the CDI were examined: Combining Words and Complexity. On the CDI parents are asked whether their child has begun to combine words yet and are provided examples. A response of “Not Yet” receives a score of 0, “Sometimes” receives a 1, and “Often” receives a 2. If the child is not combining words, parents are instructed not to complete the Complexity section of the CDI. In the Complexity section, pairs of utterances are presented that vary in morphosyntactic sophistication and parents select which one sounds more like their child (e.g., two shoe versus two shoes). A score of 1 point is given for each pair in which the more grammatically complex form is selected, with possible scores ranging from 0 to 37.

Results

Semantic Categories of Vocabulary Words

The first research question examined whether there were group differences in the types of vocabulary words used by late talking toddlers and toddlers on the autism spectrum based on the semantic categories of the CDI. Prior to conducting statistical analysis of the data, we examined the proportion of children in both groups who produced at least one word within each of the CDI categories. Based on this examination of the individual data, we omitted four of the 22 categories from the analyses because less than half of the children in both groups had contributed data to those categories and those children who did contribute data had produced very few instances of words in each of the four categories. The categories that were eliminated included Question Words, Helping Verbs, Connecting Words, and Time Words. For the remaining 18 categories, the proportion of children producing words in a given category ranged from all children in both groups (Sounds Effects, People, Games & Routines) to approximately two-thirds of the children (Places, Furniture & Rooms, Quantifiers & Articles). A summary of the descriptive data is provided in Table 3, including the data for the four categories omitted from the statistical analysis.

Table 3.

Group means, standard deviations (in parentheses), and ranges for number of words and percent of vocabulary for each semantic category on the MacArthur-Bates Communicative Development Inventories: Words & Sentences.

CDI Semantic Category Autism Spectrum Group Late Talker Group
# of Words % Vocabulary # of Words % Vocabulary
Mean (SD)
Range
Mean (SD)
Range
Mean (SD)
Range
Mean (SD)
Range
Sound Effects (n=12) 7.50 (2.89)
1–12
10.08 (7.66)
2–36
7.62 (2.87)
1–12
10.18 (6.43)
2–26
Animals (n=43) 11.25 (8.75)
1–30
9.95 (3.89)
3–18
9.45 (8.43)
0–30
8.34 (5.12)
0–18
Vehicles (n=14) 4.10 (3.72)
0–14
3.81 (2.99)
0–11
4.08 (3.41)
0–11
3.67 (2.72)
0–10
Toys (n=18) 4.33 (2.60)
1–10
4.59 (1.93)
2–10
4.35 (3.24)
0–12
4.05 (1.96)
0–9
Food & Drink (n=68) 12.40 (11.34)
0–37
9.98 (4.91)
0–21
13.08 (13.02)
0–50
11.39 (6.15)
0–25
Clothing (n=28) 3.92 (3.33)
0–12
3.55 (2.32)
0–12
4.85 (4.40)
0–15
4.34 (2.88)
0–13
Body Parts (n=27) 6.98 (6.07)
0–19
6.14 (3.44)
0–16
6.80 (5.68)
0–21
6.14 (3.17)
0–15
Household Items (n=50) 6.83 (7.05)
0–22
5.16 (4.30)
0–20
6.65 (7.50)
0–25
4.88 (3.96)
0–14
Furniture & Rooms (n=33) 3.45 (4.29)
0–13
2.35 (2.60)
0–12
2.75 (3.87)
0–16
1.91 (1.83)
0–6
Outside Things (n=31) 5.70 (5.78)
0–19
4.51 (2.97)
0–10
5.10 (5.11)
0–20
4.13 (2.98)
0–9
Places to Go (n=22) 2.00 (2.55)
0–11
1.32 (1.27)
0–4
2.28 (2.63)
0–10
1.86 (1.82)
0–8
People (n=29) 5.48 (3.09)
1–13
6.64 (3.96)
1–18
5.48 (2.47)
2–13
7.35 (5.03)
2–27
Games & Routines (n=25) 8.40 (4.02)
2–20
9.71 (4.19)
5–19
8.48 (3.46)
4–16
11.27 (6.71)
3–31
Action Words (n=103) 11.08 (10.85)
0–46
8.95 (4.72)
0–20
7.35 (9.42)
0–40
5.59 (4.42)
0–16
Descriptive Words (n=63) 6.98 (6.77)
0–22
5.42 (3.40)
0–13
6.18 (6.26)
0–27
5.78 (3.88)
0–20
Time (n=12) 0.40 (0.81)
0–4
0.29 (0.66)
0–3
0.40 (0.90)
0–4
0.33 (0.72)
0–4
Pronouns (n=25) 1.63 (1.61)
0–6
1.76 (1.99)
0–8
1.83 (1.57)
0–6
2.16 (2.03)
0–9
Question Words (n=7) 0.85 (1.05)
0–4
0.83 (1.25)
0–5
0.40 (0.71)
0–3
0.44 (0.88)
0–4
Prepositions & Locations (n=26) 2.67 (2.30)
0–8
2.58 (2.51)
0–10
2.38 (2.68)
0–13
2.53 (3.15)
0–19
Quantifiers & Articles (n=17) 1.38 (1.41)
0–7
1.47 (1.59)
0–6
0.83 (1.11)
0–5
0.93 (1.09)
0–4
Helping Verbs (n=21) 0.85 (0.98)
0–4
0.92 (1.28)
0–6
0.58 (1.11)
0–5
0.58 (1.11)
0–5
Connecting Words (n=6) 0.15 (0.53)
0–3
0.11 (0.44)
0–3
0.00 (0.00)
0–0
0.00 (0.00)
0–0

To address the first research question 18 analyses of covariance (ANCOVAs) were conducted in which nonverbal cognitive score from the Bayley-II was used as a covariate. Given the large number of comparisons tested, a Bonferroni correction was used to establish the alpha level of p<.0027 (p<.05 divided by 18 comparisons) to minimize Type I errors; however, findings based on the unadjusted alpha level of p<.05 are also reported in consideration of Type II errors. Results of the ANCOVAs revealed no significant group differences in the number of words produced for each of 18 semantic categories on the CDI, Fs(1,74)=2.10-.00, ps=.15–.99 using either the adjusted or unadjusted alpha level. Next, the proportion of words per semantic category was assessed. This measure was derived by computing each child’s proportion per category and then averaging across the group. Descriptive data are provided in Table 3. The two groups displayed equivalent proportional use across all semantic categories using the adjusted alpha, Fs(1,74)=7.41-.00, ps=.01–.97. The groups differed on one category using the unadjusted alpha level such that the Late Talker group used a significantly higher proportion of clothing words than the Autism Spectrum group, F(1,74)=7.41, p=.01, ηp2=.09.

Use of Psychological State Terms

The second research question examined possible group differences in the use of psychological state terms. Given concerns regarding floor effects we decided only to analyze group differences in total psychological state terms rather than the individual subcategories of psychological state terms. An examination of individual data indicated that at least three-fourths of the toddlers in both groups contributed data to the analysis of total psychological state terms (i.e., were reported to produce some words within this broad category) and there was similar variability in use across groups (Late Talker group: 0–12 words; Autism Spectrum group: 0–14 words). In addition to reporting both groups’ data for total psychological state terms, we have provided group means and standard deviations for the individual subcategories of psychological state terms in Table 4 for descriptive purposes. An ANCOVA controlling for nonverbal cognition revealed no significant group difference for the total number of psychological state terms used by the two groups, F(1,74)=.15, p=.70.

Table 4.

Group means and standard deviations (in parentheses) for total psychological state terms produced and each category of psychological state terms as reported on the MacArthur-Bates Communicative Development Inventory: Words & Sentences.

Psychological State Terms Autism Spectrum Group Late Talker Group
Mean (SD)
Range
Mean (SD)
Range
Emotion 0.40 (0.71)
0–3
0.32 (0.69)
0–3
Desire 0.23 (0.42)
0–1
0.20 (0.46)
0–2
Cognitive 0.02 (0.16)
0–1
0 (0)
NA
Physiological 1.53 (1.74)
0–7
1.85 (1.59)
0–7
Evaluation 0.87 (1.30)
0–5
0.90 (1.39)
0–6
TOTAL 3.05 (3.44)
0–14
3.28 (3.15)
0–12

Early Grammatical Abilities

The third research question pertained to group differences in early grammatical abilities. Both groups displayed identical patterns of onset of word combination. In both groups, 22 toddlers were “not yet” combining words (received a score of 0), 16 toddlers were “sometimes” combining words (score of 1), and 2 toddlers were “often” combining words (score of 2). Given the identical data there was no need to statistically test for group differences in word combinations. We then examined nonverbal cognitive scores from the Bayley-II for toddlers who were not yet combining words compared to those who were sometimes combining words. For the Late Talker group, nonparametric analyses revealed that the mean rank of 17.40 for toddlers not combining words was not significantly different from the mean rank of 20.03 for toddlers who were sometimes combining words, Mann-Whitney U=134.50, p=.45 (two-tailed). Similarly, the mean rank of 17.26 for toddlers in the Autism Spectrum group who were combining words was not significantly different from the mean rank of 21.28 for toddlers who sometimes combined words, Mann-Whitney U=131.50, p=.25 (two-tailed).

Only those toddlers who were combining words at least some of the time (i.e., who received scores of 1 or 2) could be evaluated with respect to grammatical complexity (n=18 per group). Complexity scores on the CDI ranged from 0 to 9 for the Late Talker group and from 0 to 16 for the Autism Spectrum group. Results of a Mann-Whitney test indicated a mean rank of 17.97 for the Late Talker group and a mean rank of 19.03 for the Autism Spectrum group. Results of this nonparametric analysis indicated that there was no significant difference between these vocabulary-matched groups in CDI complexity scores (Mann-Whitney U=152.500, p=.74, two-tailed). Because the groups differed significantly in terms of nonverbal cognitive scores, we assessed whether CDI complexity scores were associated with cognitive level. Spearman rank correlation coefficients revealed that complexity scores were not significantly correlated with Bayley-II nonverbal scores for the Late Talker group (n=17), ρ=−.16, p=.53 (two-tailed) or the Autism Spectrum group (n=18), ρ=−.04, p=.87 (two-tailed). On the other hand, Spearman’s correlation indicated that there was a significant association between CDI words produced and CDI grammatical complexity scores for the Late Talker group (n=18), ρ=.76, p=.00, whereas this association approached, but did not reach statistical significance for the Autism Spectrum group (n=18), ρ=.44, p=.07 (however, with 20 participants ρ>.447 would be significant at the p<.05 level, two-tailed).

Discussion

Semantic Categories

The Autism Spectrum group in this study had more severe language delay than the Late Talker group as evidenced by the fact that they were significantly older when they produced the same number of words. This is consistent with results from Bruckner et al. (2007) who reported that matching on total size of receptive vocabulary on the CDI Infant checklist resulted in a younger group of typically developing infants (under 18 months of age) compared to the infants on the autism spectrum (above 18 months of age). However, findings for the current study related to the first research question revealed that the groups were quite comparable in terms of the number of words used within each semantic category on the CDI and the percentage of total vocabulary that each semantic category represented. For example, both groups used more people words than vehicle labels. The only way in which the groups differed was on the proportion of Clothing Words comprising their vocabulary (using an unadjusted alpha level). Thus, it appears that the categories of words used by toddlers on the autism spectrum are very similar to those used by late talking toddlers.

Psychological State Terms

Examination of psychological state terms revealed that, on average, both groups used a small number of these words; however, there was large variation in usage (such that the standard deviations were larger than the groups mean in most cases). Although the index of psychological state terms was derived from a parent-report checklist in the present study, it is interesting to note that the percentage of word types is consistent with other studies assessing word tokens via spontaneous language samples which have reported that psychological state terms account for approximately 2–3% of total words produced by children (Bartsch & Wellman, 1995; Bretherton & Beeghly, 1982; Lee & Rescorla, 2008). In the current study psychological state terms comprised an average of 2.8% (3/108) of the words produced by the Autism Spectrum group and 3.1% (3/106) of the words produced by the Late Talker group. At this early point of language acquisition, neither group was reported to use any cognitive terms, so the current study cannot speak to prior findings that older autistic children (but not those with Asperger’s syndrome) produce fewer cognitive/belief terms than matched controls with developmental disabilities or specific language impairment (Tager-Flusberg, 1992; Ziatas et al., 1998). Like the present study, Tager-Flusberg (1992) found no overall deficit in psychological state terms. The language-matched groups in that study were comparable on desire, perception, and emotion terms, although Tager-Flusberg suggested it was possible that very young children might have initial impairments in these types of mental state terms. The current data do not appear to support that conjecture.

Grammatical Abilities

A significant association between vocabulary size and grammatical complexity was found for the Late Talker group, supporting prior findings regarding early links between these linguistic domains for typically developing toddlers and late talkers (Moyle et al., 2007). The relation between number of words produced and grammatical complexity only approached significance for the Autism Spectrum group (p=.068). However, it is important to keep in mind that the correlation analyses could be conducted only on a relatively small (n=18) subset of children within each group who were combining words. Additional research is needed to examine this relationship in a larger sample of children on the autism spectrum who are at the early stages of grammatical development. The other two measures of grammatical abilities indicated that the two groups displayed equivalent profiles. That is, with similar levels of productive vocabulary the groups exhibited identical patterns of word combinations (in terms of the number of children who were ‘not yet’, ‘sometimes’ or ‘often’ combining words) and similar grammatical complexity scores. These findings suggest that, at least at early stages of development, toddlers on the autism spectrum are following a pattern that is much like that of toddlers with specific language delay with respect to associations between lexical and grammatical abilities.

Conclusion

Verbal toddlers on the autism spectrum displayed a more severe delay in productive vocabulary development than non-spectrum late talkers, controlling for nonverbal cognition. However, the findings indicated that toddlers on the autism spectrum used very similar semantic categories of words compared to the categories of words used by the late talking toddlers. Toddlers on the autism spectrum also did not differ from late talking toddlers in their use of psychological state terms. When matched on vocabulary level, both groups were equivalent in terms of grammatical complexity and word combining, although the late talkers displayed a significant lexical-grammatical association while the toddlers on the autism spectrum did not. Nonverbal cognition did not appear to be related to early grammatical development in that there were no significant differences in cognitive level for toddlers in either group who were or were not combining words and nonverbal cognition was not significantly correlated with grammatical complexity scores.

In summary, when matched on overall vocabulary level and taking nonverbal cognition into account, 30-month-old toddlers on the autism spectrum exhibited word use patterns and early grammatical abilities which were qualitatively very similar to 25-month-old late talking toddlers without autism. Findings from the current study provide no evidence to support the distinct category account; instead, these initial results are consistent with the dimensional view. While quantitative differences were observed that suggested more severe language delay in the toddlers on the autism spectrum, the groups did not differ significantly in their patterns of early language development. The lack of significant group differences must be interpreted cautiously. Alternative explanations for the similarities across groups, discussed below, include the influence of the type of structured parent report measure that was used and lack of power to detect real differences that exist across the groups.

Limitations and Future Directions

As is the case for most investigations, this study has certain limitations. The first limitation pertains to the nature of the measure that was used. The CDI provides an index of different types of words that the child produces but does not assess tokens (i.e., number of times a child uses a particular word). It may be the case that toddlers on the autism spectrum and late talking toddlers differ in the frequency of word use within various semantic categories, but it was not possible to examine this issue using the CDI. Further, it could be argued that because the CDI consists of a closed set of vocabulary words that it would be difficult to discover atypical patterns of vocabulary use (even though it is certainly possible that different patterns could emerge across the 22 categories of words). In the current study parents were encouraged to list all additional words that their child used in the section provided on the CDI; however, there were few reports of words not included on the form for either group. Finally, vocabulary checklists such as the CDI do not provide information about the contexts in which the words were used, their particular meanings, or their pragmatic functions. Thus, it is possible that these groups differ along some of these dimensions.

A second limitation of this investigation is that the actual number of instances of uses of psychological state terms was quite limited for both groups, as were the uses of some of the semantic categories of words represented on the CDI (words about time, question words, helping verbs, and connecting words). Additional investigation is needed with a substantially larger sample of children in order to ensure adequate power to confirm the lack of significant differences for each of the less frequently used categories of words.

A third issue that may be viewed as a limitation of the current study is that groups were not matched in terms of sex ratios. It is the case that the higher proportion of girls in the Late Talker group than the Autism Spectrum group may have contributed to the age level difference for the groups when matched on overall vocabulary (since girls are typically slightly ahead of boys and the Late Talker group consisted of more girls). However, it is not clear that prior research would lead one to predict different types of word class usage or differences in patterns of early grammatical abilities based on sex. Had we found significant group differences, then one might question whether those effects could have been attributed to differences in sex ratios; however, since differences were not found it is difficult to see how this feature of our study could have confounded the results. Nevertheless, this is a potential limitation of the current study that might be addressed in future investigations.

A final limitation of the study relates to the relatively narrow period of development that was investigated. The objective of the present study was to determine whether vocabulary usage and the vocabulary-grammar association differed across these groups at the earliest stage of development. These findings may not generalize to older age ranges when children on the autism spectrum have been reported to develop more highly specialized vocabularies related to their specific areas of interest and children without autism routinely begin to include a larger variety of psychological state terms, including cognitive terms, in their conversations. Some research has reported differences in language abilities of older autistic children and children with SLI. The current findings are not necessarily inconsistent with those findings. Language delay and language acquisition are dynamic processes. Some of the toddlers in the Late Talker group will develop SLI whereas others will not. Similarly, some children on the autism spectrum will develop normal range structural language abilities while others will continue to exhibit language disorders. The lack of group differences at an early point in development, does not rule out the possibility of later differences. It is also important to keep in mind that these findings pertain to a subset of toddlers on the autism spectrum who produced at least 15 different words. It is unclear the extent to which the results would generalize to autistic children with more severely delayed early language development. Further research is needed to investigate these issues more thoroughly.

Acknowledgments

These data were collected with support from the National Institutes of Health, NIDCD R01 DC007223 and NIDCD T32 DC005359. We express our sincere thanks to the children and parents who participated in this research.

Contributor Information

Susan Ellis Weismer, Department of Communicative Disorders and Waisman Center, University of Wisconsin-Madison, Madison, WI, USA.

Morton Ann Gernsbacher, Department of Psychology, University of Wisconsin-Madison, Madison, WI, USA.

Sheri Stronach, Waisman Center, University of Wisconsin-Madison, Madison, WI, USA during this study but is currently enrolled in doctoral studies at Florida State University, Tallahassee, FL, USA.

Courtney Karasinski, Department of Communicative Disorders and Waisman Center, University of Wisconsin-Madison, Madison, WI, USA.

Elizabeth R. Eernisse, Department of Communicative Disorders and Waisman Center, University of Wisconsin-Madison, Madison, WI, USA

Courtney E. Venker, Department of Communicative Disorders and Waisman Center, University of Wisconsin-Madison, Madison, WI, USA

Heidi Sindberg, Waisman Center, University of Wisconsin-Madison, Madison, WI, USA.

References

  1. Bartsch K, Wellman HM. Children’s talk about the mind. New York: Psychological Corporation; 1995. [Google Scholar]
  2. Bates E, Goodman J. On the inseparability of grammar and the lexicon: Evidence from acquisition. In: Tomasello M, Bates E, editors. Language development: The essential readings. Malden, MA: Blackwell; 2001. [Google Scholar]
  3. Bayley N. Bayley Scales of Infant and Toddlers Development. 3. San Antonio, TX: Pearson; 2005. [Google Scholar]
  4. Bayley N. Bayley Scales of Infant Development. 2. New York: Cambridge University Press; 1993. [Google Scholar]
  5. Beeghly M. Translational research on early language development: Current challenges and future directions. Development and Psychopathology. 2006;18:737–757. doi: 10.1017/s0954579406060366. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bishop DVM. Pragmatic language impairment: A correlate of SLI, a distinct subgroup, or part of the autistic continuum? In: Bishop DVM, Leonard LB, editors. Speech and language impairments in children: Causes, charactertistics, intervention and outcome. Hove, UK: Psychology Press; 2000. pp. 99–113. [Google Scholar]
  7. Bishop DVM, Norbury CF. Exploring the borderlands of autistic disorder and specific language impairment: A study using standardized diagnostic instruments. Journal of Child Psychology and Psychiatry. 2002;43:917–929. doi: 10.1111/1469-7610.00114. [DOI] [PubMed] [Google Scholar]
  8. Bretherton I, Beeghly M. Talking about internal states: The acquisition of an explicit theory of mind. Developmental Psychology. 1982;18:906–921. [Google Scholar]
  9. Bruckner C, Yoder P, Stone W, Saylor M. Construct validity of the MCDI-I receptive vocabulary scale can be improved: Differential item functioning between toddlers with autism spectrum disorders and typically developing infants. Journal of Speech, Language, and Hearing Research. 2007;50:1631–1638. doi: 10.1044/1092-4388(2007/110). [DOI] [PubMed] [Google Scholar]
  10. Dionne G, Dale P, Boivin M, Plomin R. Genetic evidence for bidirectional effects of early lexical and grammatical development. Child Development. 2003;74:394–412. doi: 10.1111/1467-8624.7402005. [DOI] [PubMed] [Google Scholar]
  11. Charman T, Drew A, Baird C, Baird G. Measuring early language development in preschool children with autism spectrum disorder using the MacArthur Communicative Development Inventory (Infant Form) Journal of Child Language. 2003;30:213–236. doi: 10.1017/s0305000902005482. [DOI] [PubMed] [Google Scholar]
  12. Eigsti IM, Bennetto L, Dadlani M. Beyond pragmatics: Morphosyntactic development in autism. Journal of Autism and Developmental Disorders. 2007;37:1007–1023. doi: 10.1007/s10803-006-0239-2. [DOI] [PubMed] [Google Scholar]
  13. Ellis Weismer S. Typical talkers, late talkers, and children with specific language impairment: A language endowment spectrum? In: Paul R, editor. Language disorders and development from a developmental perspective. Mahwah, NJ: Lawrence Erlbaum Assoc; 2007. pp. 83–101. [Google Scholar]
  14. Fenson L, Dale PS, Reznick JS, Thal D, Bates E, Hartung JP, et al. Technical manual for the MacArthur Communicative Development Inventories. San Diego, CA: San Diego State University Press; 1991. [Google Scholar]
  15. Frankenburg WK, Dobbs JB. Denver Developmental Screening Test II. Denver, CO: Denver Developmental Materials, Inc; 1990. [Google Scholar]
  16. Gernsbacher MA, Geye HM, Ellis Weismer S. The role of language and communication impairments within autism. In: Fletcher P, Miller JF, editors. Language disorders and developmental theory. Philadelphia: John Benjamins; 2005. pp. 73–93. [Google Scholar]
  17. Jarrold C, Boucher J, Russell J. Language profiles in children with autism. Autism. 1997;1:57–76. [Google Scholar]
  18. Johnston JR, Miller JF, Tallal P. Use of cognitive state predicate by language-impaired children. International Journal of Language & Communication Disorders. 2001;36:349–370. doi: 10.1080/13682820110045838. [DOI] [PubMed] [Google Scholar]
  19. Kjelgaard M, Tager-Flusberg H. An investigation of language impairment in autism: Implications for genetic sub-groups. Language and Cognitive Processes. 2001;16:287–308. doi: 10.1080/01690960042000058. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Lee EC, Rescorla L. The use of psychological state terms by late talkers at age 3. Applied Psycholinguistics. 2002;23:623–641. [Google Scholar]
  21. Lee EC, Rescorla L. The use of psychological state terms by late talkers at ages 3, 4, and 5 years. Applied Psycholinguistics. 2008;29:21–39. [Google Scholar]
  22. Lindgren K, Folstein S, Tomblin JB, Tager-Flusberg H. Language and reading abilities of children with autism spectrum disorders and specific language impairment and their first-degree relatives. Autism Research. 2009;2:22–38. doi: 10.1002/aur.63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lord C, Rutter M, DiLavore P, Risi S. Autism Diagnostic Observation Schedule. Los Angeles, CA: Western Psychological Services; 1999. [Google Scholar]
  24. Lord C, Rutter M, Le Couteur A. Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders. 1994;24:659–685. doi: 10.1007/BF02172145. [DOI] [PubMed] [Google Scholar]
  25. Luyster R, Gotham K, Guthrie W, Coffing M, Petrak R, DiLavore P, Pierce K, Bishop S, Esler A, Hus V, Richler J, Risi S, Lord C. The Autism Diagnostic Observation Schedule – Toddler Module: A new module of a standardized diagnostic measure for autism spectrum disorders. Journal of Autism and Developmental Disorders. 2009;39:1305–1320. doi: 10.1007/s10803-009-0746-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Luyster R, Lopez K, Lord C. Characterizing communicative development in children referred for Autism Spectrum Disorders using the MacArthur-Bates Communicative Development Inventory (CDI) Journal of Child Language. 2007;34:623–654. doi: 10.1017/s0305000907008094. [DOI] [PubMed] [Google Scholar]
  27. Maitel SL, Dromi E, Sagi A, Bornstein M. The Hebrew Communicative Development Inventory: Language specific properties and cross-linguistic generalizations. Journal of Child Language. 2000;27:43–67. doi: 10.1017/s0305000999004006. [DOI] [PubMed] [Google Scholar]
  28. Marchman V, Bates E. Continuity in lexical and morphological development: A test of the critical mass hypothesis. Journal of Child Language. 1994;21:339–366. doi: 10.1017/s0305000900009302. [DOI] [PubMed] [Google Scholar]
  29. McDuffie A, Yoder P, Stone W. Fast-mapping in young children with autism spectrum disorders. First Language. 2006;26:421–438. [Google Scholar]
  30. Moyle MJ, Ellis Weismer S, Evans JL, Lindstrom MJ. Longitudinal relationships between lexical and grammatical development in typical and late-talking children. Journal of Speech, Language, and Hearing Research. 2007;50:508–528. doi: 10.1044/1092-4388(2007/035). [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Rescorla L. The Language Development Survey: A screening tool for delayed language in toddlers. Journal of Speech and Hearing Disorders. 1989;54:587–599. doi: 10.1044/jshd.5404.587. [DOI] [PubMed] [Google Scholar]
  32. Rescorla L. Age 17 language and reading outcomes in late-talking toddlers: Support for a dimensional perspective on language delay. Journal of Speech, Language, and Hearing Research. 2009;52:16–30. doi: 10.1044/1092-4388(2008/07-0171). [DOI] [PubMed] [Google Scholar]
  33. Rescorla L, Alley A, Christine JB. Word frequencies in toddlers’ lexicons. Journal of Speech, Language, and Hearing Research. 2001;44:598–609. doi: 10.1044/1092-4388(2001/049). [DOI] [PubMed] [Google Scholar]
  34. Rescorla L, Roberts J. Nominal versus verbal morpheme use in late talkers at ages 3 and 4. Journal of Speech, Language, and Hearing Research. 2002;45:1219–1231. doi: 10.1044/1092-4388(2002/098). [DOI] [PubMed] [Google Scholar]
  35. Rice M, Warren S. Developmental language disorders: From phenotypes to etiologies. Mahwah, NJ: Erlbaum; 2004. [Google Scholar]
  36. Roberts JA, Rice ML, Tager-Flusberg H. Tense marking in children with autism. Applied Psycholinguistics. 2004;25:429–448. [Google Scholar]
  37. Ruser T, Dowd M, Putnam S, Winklosky B, Rosen-Sheidley B, Piven J, Tomblin JB, Tager-Flusberg H, Folstein S. Communicative competence in parents of children with autism and parents of children with specific language impairment. Journal of Autism and Developmental Disorders. 2007;37:1323–1336. doi: 10.1007/s10803-006-0274-z. [DOI] [PubMed] [Google Scholar]
  38. Rutter M, Bailey A, Berument SK, Lord C, Pickles A. The Social Communication Questionnaire. Los Angeles, CA: Western Psychological Services; 2006. [Google Scholar]
  39. Scarborough HS, Tager-Flusberg H, Fowler AE, et al. The relation of utterance length to grammatical complexity in normal and language-disordered groups. Applied Psycholinguistics. 1991;12:23–45. [Google Scholar]
  40. Swensen L, Kelley E, Fein D, Naigles L. Processes of language acquisition in children with autism: Evidence from preferential looking. Child Development. 2007;78:542–557. doi: 10.1111/j.1467-8624.2007.01022.x. [DOI] [PubMed] [Google Scholar]
  41. Tager-Flusberg H. Autistic children’s talk about psychological states: Deficits in the early acquisition of a theory of mind. Child Development. 1992;63:161–172. [PubMed] [Google Scholar]
  42. Thordardottir ET, Ellis Weismer S, Evans JL. Continuity in lexical and morphological development in Icelandic and English-speaking 2-year-olds. First Language. 2002;22:3–28. [Google Scholar]
  43. Whitehouse A, Barry JA, Bishop DVM. The broader language phenotype of autism: A comparison with specific language impairment. Journal of Child Psychology and Psychiatry. 2007;48:822–830. doi: 10.1111/j.1469-7610.2007.01765.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Williams T. Brief report: Vocabulary development in an autistic boy. Journal of Autism and Developmental Disorders. 1993;23:185–191. doi: 10.1007/BF01066427. [DOI] [PubMed] [Google Scholar]
  45. Ziatas K, Durkin K, Pratt C. Belief term development in children with autism, Asperger syndrome, specific language impairment, and normal development: Links to theory of mind development. Journal of Child Psychology and Psychiatry. 1998;39:755–763. [PubMed] [Google Scholar]

RESOURCES