Parent Certainty and Consistency When Completing Vocabulary Checklists in Young Autistic Children

Abstract

Purpose:

Using a novel parent report measure, this study investigated whether asking parents to rate their certainty when reporting on child vocabulary skills provided additional insight into parent report and emerging language abilities in young autistic children. Specifically, we investigated whether parent certainty varied based on whether the child was reported to understand, understand and say, or neither understand nor say the word and whether standardized measures of expressive and receptive language abilities and/or autistic traits predicted parent certainty. Lastly, we investigated whether certainty was associated with inconsistency in parent report of child word knowledge.

Method:

Twenty-one parents and their autistic children ages 2–5 years participated. One parent per child completed the MacArthur–Bates Communicative Development Inventories (MCDI) Words and Gestures form and a custom vocabulary checklist including 24 object nouns from the MCDI. Within the custom form, parents indicated whether their child understood, understood and said, or neither understood nor said 24 target nouns and reported how certain they were about their responses using a 5-point scale. Expressive language, receptive language, and autistic traits were measured via direct assessment using standardized measures.

Results:

Parent certainty varied widely and was higher for words the parents reported the children understood and said compared to that for words children either understood or neither understood nor said. Certainty ratings were higher when a child had higher standardized receptive and expressive language scores. Lastly, parent certainty was associated with reporting consistency, clarifying previous findings of inconsistencies in parent report of child vocabulary.

Conclusion:

Findings from this study indicate that measuring parent certainty provides critical information when assessing early vocabulary skills in autistic children.

Supplemental Material:

https://doi.org/10.23641/asha.23671497

Assessing emerging language in early communicators can pose a significant challenge (Kasari et al., 2013; Muller & Brady, 2016; Venker et al., 2013). One common approach for measuring early language includes using vocabulary checklists where parents report what words they believe their child understands and/or says (e.g., MacArthur–Bates Communicative Development Inventories [MCDI]; Fenson et al., 2006). While parent report measures have several strengths, current parent report measures also have a number of limitations. Specifically, they require parents to make inferences regarding their child's vocabulary abilities and require a dichotomous yes/no response to a gradual process (i.e., emerging word knowledge).

The challenges of language assessment are particularly relevant when examining language in young autistic children (DiStefano & Kasari, 2016; Kasari et al., 2013; Muller & Brady, 2016; Venker et al., 2013; Volden et al., 2011). Autism spectrum disorder (ASD) is characterized by challenges in social communication and social interaction as well as restricted and repetitive patterns of behaviors, activities, and interests (American Psychiatric Association, 2013). While vocabulary deficits are not a defining feature of ASD, learning language can be particularly difficult for autistic children (Ellis Weismer & Kover, 2015; Hart & Curtin, 2021; Luyster, Lopez, & Lord, 2007; Volden et al., 2011). Many researchers study language abilities in autistic individuals to characterize language delays and trajectories, identify mechanisms underlying language impairments, and determine how to best support early language development (Allen et al., 2015; Dimitrova et al., 2016; Ellis Weismer & Kover, 2015; Kasari et al., 2014; Oller et al., 2010; Venker et al., 2015, 2021). Informative research hinges on the ability to accurately assess language abilities in ASD.

To address current limitations in parent report measures, this study investigated a novel measure of child vocabulary that included parent certainty ratings when reporting on their child's vocabulary for children on the autism spectrum. As a first step in examining the utility of parent certainty, this study investigated whether certainty varied based on whether the child was reported to understand, understand and say, or neither understand nor say the word. This study also examined whether standardized measures of child expressive and receptive language abilities and/or autistic traits predicted parent certainty. Lastly, we investigated whether certainty was associated with inconsistency in parent report of child word knowledge.

Vocabulary Skills in Autistic Children

Language delays often appear early in autistic children (Eigsti et al., 2007, 2011; Ellis Weismer & Kover, 2015; Tager-Flusberg et al., 2009; Torras-Mañá et al., 2016). In fact, expressive language delays are one of the first recognized signs of ASD (Becerra-Culqui et al., 2018; Tager-Flusberg et al., 2009). In addition to vocabulary delays, the profile of vocabulary development may differ in ASD. Recent evidence suggests that the specific types of nouns autistic children produce (e.g., words for animals vs. words for foods) may vary from those of neurotypical children (Haebig et al., 2021); conversely, other research found that the words produced by autistic children and late talkers did not differ based on semantic categories (Ellis Weismer et al., 2011). Some autistic individuals also exhibit better expressive language abilities than would be expected based on receptive language abilities, although not all research has found this pattern (Charman et al., 2003; Ellis Weismer et al., 2010; Haebig & Sterling, 2017; Kwok et al., 2015; Luyster, Lopez, & Lord, 2007; Luyster et al., 2008; Woynaroski et al., 2016). Beyond characterizing current vocabulary skills in young autistic children, measuring vocabulary is also important as it is a predictor of later language abilities in this population (Luyster, Qiu, et al., 2007; Woynaroski et al., 2016). Identifying the specific words a child understands and says is critical for developing interventions that utilize scaffolding to teach new skills. Despite the importance of assessing receptive and expressive lexical skills accurately among emerging communicators, measuring these skills continues to present challenges.

Challenges in Language Assessment in Young Children

Language skills can be measured in multiple ways, such as through language samples, standardized assessments administered by speech-language pathologists (SLPs), or parent report. Each approach has benefits and drawbacks. Standardized assessments utilize consistent administration procedures, which are helpful for comparing across children and tracking change over time. However, the demands often associated with standardized, direct assessments administered by clinicians can pose a significant challenge for many young children, including in ASD (DiStefano & Kasari, 2016; Kasari et al., 2013; Muller & Brady, 2016; Venker et al., 2013; Volden et al., 2011). Direct assessments often require the child to attend to the task, listen to the examiner, and follow complex directions to indicate receptive vocabulary knowledge of a single word (e.g., “Look at all these pictures. Point to the ball.”). This type of assessment context requires attention, compliance, and social interaction with a potentially less familiar individual, which may be particularly difficult for autistic children (Kasari et al., 2013; Muller & Brady, 2016). Due to developmental skills and environmental demands, many autistic children perform at floor on standardized assessments (Charman et al., 2003; Muller & Brady, 2016; Venker et al., 2013; Volden et al., 2011).

Language samples across different contexts can provide a comprehensive picture of a child's current communication abilities. In contrast to standardized assessments, language sampling provides information on a child's expressive language in a naturalistic context but obtaining information regarding receptive language abilities is much more difficult (Muller & Brady, 2016). Moreover, relatively brief language samples in contrived contexts may not capture a representative picture of a child's language abilities and may underestimate expressive language skills, particularly for young autistic children with social communication challenges (Kover et al., 2014). It is possible to sample language across contexts and with multiple communication partners, but transcribing and analyzing multiple language samples is a time-intensive and costly process.

In part because of the aforementioned barriers, parent report measures in the form of vocabulary checklists are commonly used in research when assessing language in young children (Charman et al., 2003; Haebig et al., 2021; Luyster, Lopez, & Lord, 2007; Luyster, Qiu, et al., 2007; McDaniel et al., 2018; Styles & Plunkett, 2009). In contrast to direct assessments, parent report measures require no real-time behavioral responses from children and allow parents to draw on their observations of their child's communication across daily settings. Moreover, parent report measures may be particularly valuable for measuring language in autistic children with limited spoken language and greater behavioral challenges, who are often excluded from research studies (Tager-Flusberg & Kasari, 2013). However, researchers have expressed concerns regarding the validity and reliability of parent report measures (Brady et al., 2014; Feldman et al., 2000; Muller & Brady, 2016; Yoder et al., 1997). The validity of parent report measures relies on parents to accurately report on their child's language abilities. To complete such measures, caregivers must make inferences about what their child understands as well as recall words their child has spontaneously produced. Most parents are not trained language experts and are provided minimal instruction when completing these measures, which may lead to variability in the validity and reliability of results (Feldman et al., 2000). Nevertheless, vocabulary checklists are appealing because they are relatively inexpensive and efficient tools for researchers and clinicians to use as part of their assessment process.

Evaluating Emerging Word Knowledge

The MCDIs (Fenson et al., 2006) are parent report checklists used to index early language abilities in ASD as well as other populations (Charman et al., 2003; Luyster, Lopez, & Lord, 2007; Luyster, Qiu, et al., 2007; Nordahl-Hansen et al., 2014). Currently available parent report measures, such as the MCDI, ask parents to report whether specific words are present or absent in their child's vocabulary (i.e., a forced-choice format). Reporting on what words a child knows may seem like an easy task at first glance. However, determining whether or not a child knows a particular word is much more complex, although this is not currently reflected in the dichotomous nature of parent report measures. In reality, learning a single word is a gradual process that develops over time as children get better at recognizing and processing words in different contexts (Fernald & Marchman, 2012; Fernald et al., 1998, 2006). In the process of learning a word, children must generalize their word knowledge across representations (e.g., children must learn that the word “shoe” is the label assigned for all types of shoes varying in shape, size, and color, and not just their own shoes). Children can demonstrate knowledge of words in some but not all contexts, as research suggests that familiarity may contribute to noun comprehension in young neurotypical children (Garrison et al., 2020). Therefore, what may initially seem like an easy task for parents may actually be much more difficult.

Given the challenges of judging whether or not a child knows a word, it may not be surprising that previous work found that parents of both autistic children and neurotypical children underestimate their children's receptive vocabulary skills (Houston-Price et al., 2007; Venker et al., 2016). Specifically, in a study including 22 autistic children with a mean age of 30 months, Venker et al. (2016) used a looking-while-listening eye gaze paradigm to test word recognition of words that parents reported the children not to know. Evidence from the eye gaze task indicated that autistic children showed recognition of words parents reported as unknown, suggesting that parents may underestimate child receptive vocabulary knowledge. Based on this finding, it is possible that parents are reticent to endorse words they do not perceive their child to have mastered, although no research currently exists to support this hypothesis. By asking parents to rate how certain they are in their children's word knowledge, we may be able to tap into the gradual process of word learning that is not currently captured in parent report measures of child vocabulary. This study aimed to characterize parent certainty of children's word knowledge when completing vocabulary checklists as a first step in investigating whether certainty may be an informative clinical tool for tapping into emerging vocabulary knowledge in autistic children. Parent certainty ratings also have the potential to provide SLPs with richer information about child vocabulary skills compared to many currently used parent report measures. Thus, investigating parent certainty could lead to new insights in refining parent report vocabulary measures to maximize their accuracy and informativeness.

Parental certainty may differ based on whether a parent is reporting on expressive or receptive vocabulary. While Fenson et al. (1994) reported overall high test–retest reliability for the MCDI, Pearson correlations between the first and second administrations were descriptively higher for words understood and said (.95) compared to those for words understood only (.87). Studies on concurrent validity also found descriptively higher correlations for expressive over receptive language measures (Luyster et al., 2008; Nordahl-Hansen et al., 2014). Thus, it may be more difficult to report on words a child understands compared to says, although it is unclear if parents would rate themselves as being less certain when reporting on child receptive versus expressive vocabulary knowledge.

Exploring Child Factors and Parent Certainty

Parent certainty may also vary based on child factors such as language abilities or autistic traits. It may be particularly difficult to ascertain the extent to which an autistic child understands or even spontaneously says a word for several reasons. Regarding understanding a word, autistic children often do not display the same social cues and evidence decreased levels of joint attention and engagement commonly observed in neurotypical children and children with other developmental disabilities (Adamson et al., 2009, 2019; Franchini et al., 2019). Such skills may cue parents into a child's word comprehension. Thus, parents of autistic children who are less likely to produce such behaviors may be more uncertain about their child's comprehension. Even production may be more difficult to gauge, as autistic children demonstrate more variability in the trajectory of their language development, including instances of language regression (Ellis Weismer et al., 2010; Ellis Weismer & Kover, 2015; Goldberg et al., 2008; Hilvert et al., 2020; Woynaroski et al., 2016). Autistic children also show discrepant language performance across contexts and their expressive language may include echolalia (American Psychiatric Association, 2013; Gladfelter & VanZuiden, 2020; Hilvert et al., 2020; Kover et al., 2014). All of these factors may contribute to lower parent certainty when completing vocabulary checklists, although it is unclear if expressive and receptive language abilities, and/or ASD traits contribute to parent certainty ratings of child vocabulary.

Consistency and Inconsistency in Parent Report

Previous work has used the MCDI as a predictor of later language and as an outcome measure and to track changes in early communication (Luyster, Qiu, et al., 2007; McDaniel et al., 2018; McDuffie & Yoder, 2010). As with any measure, it is crucial that the MCDI have strong psychometric properties. The authors of the MCDI found excellent test–retest reliability, with Pearson correlations generally above .80 (Fenson et al., 1994). However, it is notable that for the subsample of children with scores below the median on words produced, the correlation was considerably lower, at .43. This finding suggests that the reliability of the MCDI may be lower for children with few spoken words. Although this study did not report on the presence or absence of ASD in the normative sample, the lower reliability in the sample with limited spoken language is pertinent information in the study of ASD, where many children show considerably delayed expressive language abilities (Haebig et al., 2021; Hart & Curtin, 2021; McDuffie et al., 2005).

Subsequent studies have also examined concurrent validity, agreement, parent rating consistency, and item-by-item stability of the MCDI in autistic children, neurotypical children, and children with developmental delays (Arunachalam et al., 2022; Luyster et al., 2008; Miniscalco et al., 2012; Nordahl-Hansen et al., 2014; Styles & Plunkett, 2009; Yoder et al., 1997). Luyster et al. (2008) compared the MCDI and a direct assessment, the Mullen Scales of Early Learning (MSEL; Mullen, 1995), in a sample of autistic children ages 18–33 months. They found that although there were significant partial correlations (controlling for age) between MCDI and MSEL for both expressive and receptive language skills, correlations were descriptively higher for expressive (.82) over receptive (.52) scales. In another study of autistic children ages 2–4 years, Nordahl-Hansen et al. (2014) found high correlations between two direct assessment measures and both MCDI expressive vocabulary (≥ .95) and receptive vocabulary (≥ .81) scores. Taken together, these findings suggest moderate-to-high concurrent validity between the MCDI and direct measures of language in autistic children, although absolute values are generally higher for expressive over receptive measures.

While overall test–retest reliability and concurrent validity for the MCDI are generally found to be in the acceptable range (Feldman et al., 2000; Luyster et al., 2008; Nordahl-Hansen et al., 2014; Styles & Plunkett, 2009), previous literature has identified inconsistencies in parent report at the item level (Arunachalam et al., 2022; Yoder et al., 1997). Item-by-item stability, or consistency, refers to the extent to which parents consistently report a child's vocabulary comprehension and/or production on individual vocabulary items across multiple parent report vocabulary measures. A study by Yoder et al. (1997) included a sample of young children with developmental delays and found that the overall level of stability for the number of words understood was high (i.e., G-coefficients ≥ .80), but item-by-item stability was comparably much lower, with a mean κ value of .47. Notably, item-by-item stability was significantly higher for the nouns understood compared to the item-level stability of all types of words understood. Arunachalam et al. (2022) conducted a study examining parent consistency when reporting on expressive vocabulary skills, including autistic children and neurotypical children ages 14–43 months. They found that approximately 10% of nouns and 15% of verbs were reported inconsistently by parents of autistic children. In the neurotypical group, 9% of nouns and 12% of verbs were reported inconsistently, indicating no significant difference in the percent of inconsistency for parents in the two groups.

Researchers have posited several explanations for the inconsistent findings at the item level, including child age, vocabulary size, socioeconomic status, and parent uncertainty (Arunachalam et al., 2022; Yoder et al., 1997). Previous work speculated that if parents are less certain about their responses, they may also be more inconsistent in those responses (Arunachalam et al., 2022; Yoder et al., 1997). Inconsistencies may also be due in part to the nature of the assessment tools. As previously mentioned, the MCDI asks parent to report whether a child understands or produces a given word, using a forced-choice format. Thus, parents must decide even when uncertain. Yoder et al. (1997) hypothesized that the relatively low item-by-item stability was in part a reflection of a parent's confidence in a child's word knowledge. Similarly, Arunachalam et al. (2022) suggested that inconsistencies in parent report of vocabulary may reflect their level of certainty regarding their child's word knowledge. These proposals/hypotheses suggest that it may be informative to ask parents to report not only whether their child knows and/or says a given word but also how certain they feel about their response. To our knowledge, no studies have examined parent certainty in this context or examined whether parent certainty is associated with reporting consistency.

Purpose of This Study

Parent report measures utilize forced-choice options to index early child vocabulary skills without measuring parent certainty, which might be a valuable source of information regarding emerging word knowledge and potential vocabulary targets for further assessment and intervention. This study sought to examine parent certainty when reporting whether a child understands, understands and says, or neither understands nor says different object nouns. We were also interested in whether certainty was associated with standardized language measures and/or autistic traits. Given that previous studies indicated some inconsistencies in parent reports of vocabulary skills in autistic children, we also examined whether consistency in parent report of child vocabulary knowledge was associated with parent certainty. Specifically, this study asked the following.

1. Does parent certainty differ based on response type (“Understands,” “Understands and Says,” “Neither”)? We predicted higher parent certainty for words the children understood and said compared to that for words the parents reported the child only understood or neither understood nor said.

2. Do standardized scores of child expressive and receptive language or autistic traits predict parent certainty? We predicted that parents of children with higher expressive and receptive language scores and fewer autistic traits would be more certain of their child's vocabulary skills.

3. Does parent certainty predict consistency in parent ratings of child vocabulary? We hypothesized that greater parent certainty would predict higher consistency when reporting on their child's vocabulary knowledge.

Method

Participants

Participants in this study were part of a larger study on language processing and word learning in autistic children (Venker et al., 2022). Twenty-one parents (20 mothers, one father) and their autistic children (five girls, 16 boys) ages 2–5 years participated (M = 42.81 months, SD = 11.88). In terms of race, parents reported that 19 children were White and two children were Black or African American. All parents reported that their children were not Hispanic or Latino. To be eligible for the larger study, participants needed to be monolingual English speakers, have a community diagnosis of ASD prior to study enrollment, and have no other known genetic disorders per parent report. Participants were recruited within the lower peninsula of Michigan through social media, parent listservs, offices and clinics, flyers posted in the community, and word of mouth. Total years of education for the parent completing the demographic form ranged from 12 to 24 years (M = 15.53 years, SD = 2.91). See Table 1 for child descriptive statistics.

Table 1.

Child participant descriptives.

Variable	M	SD	Range
Chronological age (years)	42.81	11.88	26–71
MSEL Visual Reception age equivalent	27.43	13.03	14–66
MSEL Visual Reception T-score	25.86	8.27	20–46
Autistic traits	8.52	1.40	6–10
PLS-5 Expressive Communication age equivalent	23.33	11.17	9–59
PLS-5 Expressive Communication growth scale value	361.19	58.67	304–498
PLS-5 Expressive Communication standard score	69.95	11.06	50–93
PLS-5 Auditory Comprehension age equivalent	23.19	12.70	13–60
PLS-5 Auditory Comprehension growth scale value	353.67	53.13	273–497
PLS-5 Auditory Comprehension score	64.38	15.20	50–98
MCDI-WG Words Understood	187.38	115.44	11–343
MCDI-WG Words Produced	116.19	118.20	0–322
a Certainty	4.15	0.89	3.13–5.00
b Percent consistent	76.98	15.79	47.83–100.00

Note. Autistic traits measured from the Autism Diagnostic Observation Schedule–Second Edition severity scores (Gotham et al., 2009; Lord, Luyster, et al., 2012; Lord, Rutter, et al., 2012). MSEL = Mullen Scales of Early Learning (Mullen, 1995); PLS-5 = Preschool Language Scales–Fifth Edition (Zimmerman et al., 2011); MCDI-WG = MacArthur–Bates Communicative Development Inventory Words and Gestures (Fenson et al., 2006).

^a Mean certainty ratings based on a 5-point rating scale from Very Uncertain (1) to Very Certain (5).

^b Calculated based on the number of items that parents reported consistently across the two parent-report vocabulary checklists, divided by the total number of items, and multiplied by 100.

Procedure

Participants visited the research lab on two different days. Legal guardians provided informed consent prior to participation. Children completed several standardized, experimental, and play-based tasks to measure language, cognitive abilities, and autistic traits. Parents also completed demographic paperwork and reported on child language abilities. All procedures were prospectively approved by the university's institutional review board.

Study Measures

ASD traits. ASD diagnosis was confirmed, and autistic traits were measured using the Autism Diagnostic Observation Schedule–Second Edition (ADOS-2; Lord, Luyster, et al., 2012; Lord, Rutter, et al., 2012). The ADOS-2 is a semistructured assessment composed of several activities with social presses designed to elicit and evaluate child behavioral responses. All ADOS-2 administrations were conducted by a research reliable examiner, and in some instances, an examiner training for reliability observed the administration. Autistic traits were measured via the ADOS-2 severity scores (Gotham et al., 2009). Scores range from 1 to 10, with higher scores reflecting more prevalent ASD traits.

Nonverbal skills. A trained examiner administered the Visual Reception subscale of the Mullen Scales of Early Learning, which is a play-based developmental assessment for children ages birth to 68 months (MSEL; Mullen, 1995). The Visual Reception subscale provided an index of nonverbal problem-solving skills. MSEL Visual Reception T-scores have a mean of 50 and an SD of 10.

Expressive and receptive language measures. Child receptive and expressive language skills were measured by a trained examiner using the Auditory Comprehension and Expressive Communication scales of the Preschool Language Scales–Fifth Edition (PLS-5; Zimmerman et al., 2011). The PLS-5 is a standardized measure of child language abilities across domains. Standard scores have a mean of 100 and an SD of 15. The PLS-5 also provides growth scales values from the Auditory Comprehension and Expressive Communication scales. Growth scale values have a mean of 500 and an SD of 100. Whereas standard scores reflect a child's ability relative to age-matched peers, growth scale values are transformed raw scores that result in an absolute measure of a child's language on an equal-interval scale. Standard scores are relative to a child's chronological age; thus, a younger child with a score of 90 could have lower language abilities than an older child with a score of 85. In contrast, growth scales values are not relative to a child's age but provide a measure of language ability independent of age; therefore, a higher growth scale value score is always indicative of higher language abilities. Previous research including a large sample of autistic children ages 18–48 months found that growth scale values were the most sensitive type of score for detecting change on the PLS-5 compared to raw scores, standard scores, and age equivalent scores (Kwok et al., 2022).

Parents also completed the MCDI Words and Gestures (MCDI-WG) form (Fenson et al., 2006). This version of the MCDI asks parents to report on several child skills, such as first signs of understanding (e.g., response to name); early communicative actions and gestures; understanding of phrases; and, of interest to this study, comprehension and production of 396 vocabulary words covering a range of categories. Parents are asked to fill in whether the child “Understands” or “Understands and Says” each of the 396 items. Leaving an item blank indicates that a child neither understands nor says the given word. The Words and Gestures form of the MCDI was originally developed for children ages 8–18 months, but given the presence of language delays in ASD, studies have used this assessment for autistic children beyond this age range (Luyster, Qiu, et al., 2007; McDaniel et al., 2018; McDuffie & Yoder, 2010). The MCDI-WG was deemed an appropriate measure for this study's sample as none of the children obtained the maximum for the number of words understood or spoken on this MCDI measure (see Table 1).

In addition to the MCDI-WG, parents were asked to complete a custom vocabulary checklist including 24 object nouns selected from the MCDI-WG (e.g., ball, shoe, truck). The specific items selected from the MCDI-WG are included in Supplemental Material S1 and were chosen for two primary reasons. First, the nouns selected are among the earliest acquired words in a child's vocabulary (Fenson et al., 2006; Frank et al., 2017). In addition, when choosing among the earliest acquired words, we specifically selected highly imageable nouns; this was due to the larger study's aims and subsequent experimental tasks that utilized visual stimuli representing the same list of nouns in this study when measuring language comprehension directly. Notably, previous research suggests that imageability is a significant predictor of early vocabulary skills in autistic children (Lin et al., 2022).

Similar to the MCDI-WG, parents were asked to indicate whether their child understood, understood and said, or neither understood nor said each of the 24 nouns. Unlike the MCDI-WG, parents were also asked to report how certain they were about their response on each item. Certainty ratings ranged from Very Uncertain (1) to Very Certain (5) based on a 5-point rating scale (see Supplemental Material S1 for a copy of the custom measure). Similar to the study by Arunachalam et al. (2022), the order in which the two parent report vocabulary measures were completed was not explicitly controlled in this study. However, parents received a packet of paperwork that included the custom vocabulary checklist first, followed by the MCDI-WG. In addition, the examiner consistently explained the custom vocabulary checklist before the MCDI-WG. Based on informal observation, parents almost overwhelmingly completed the custom form prior to completing the MCDI.

Data Analysis Plan

Prior to conducting analyses to address the research questions, we examined parent responses for the 24 individual nouns on the custom vocabulary checklist. We elected to remove one item, “bib,” as preliminary descriptive statistics suggested that this single item behaved differently from the rest of the items in the custom vocabulary checklist. Specifically, 14/21 parents reported that their child neither understood nor said “bib” (i.e., only 7/21 parents reported their child either understood or understood and said “bib”). For the remaining items, at least 15/21 parents reported that their child either understood or understood and said the target noun. In addition, for 4/21 participants (i.e., 19% of the sample), parents reported that their child either understood or understood and said all items except “bib.” Therefore, all analyses were completed based on the 23 remaining nouns on the custom vocabulary checklist. After examining the key variables for statistical assumptions (i.e., normality, linearity, homoscedasticity, outliers), we determined that the data were acceptable for parametric analyses.

For the first research question, we examined whether parent certainty varied by response type on the custom vocabulary checklist (i.e., whether the parent reported that the child understood, understood and said, or neither understood nor said the object noun). Parent certainty was indexed by the mean rating of parent certainty across items and was calculated separately for words parents reported children understood, understood and said, or neither understood nor said. We conducted a repeated-measures analysis of variance where response served as the independent variable with three levels (“Understands,” “Understands and Says,” and “Neither”) and mean parent certainty served as the dependent variable. To directly compare certainty on the three response types, we followed significant results with three post hoc tests with Holm–Bonferroni corrections and reported corrected p values (Holm, 1979).

We estimated three separate linear regression models with Holm–Bonferroni corrections for Research Question 2, which examined whether expressive language, receptive language, or autistic traits (i.e., the independent variables) separately predicted mean parent certainty (i.e., the dependent variable in each of the three models; Holm, 1979). Once again, we reported corrected p values. Expressive language and receptive language were indexed by the PLS-5 Expressive Communication and Auditory Comprehension growth scale values, respectively. We elected to use growth scale values rather than standard scores because we were interested in the skills the child demonstrated, rather than the extent of delay relative to same-age peers. Growth scale values are a measure of absolute ability on an equal-interval scale and thus appropriate for parametric analyses. Growth scale values are also less vulnerable to floor effects compared to standard scores. In this study, six of the 21 participants (i.e., 28.6%) scored at floor based on standard scores on the PLS-5 for the auditory comprehension scale. Moreover, none of the children in the current sample reached the mean standard score of 100 on the PLS-5 Auditory Comprehension or Expressive Communication scales. In addition, using standard scores in samples with disabilities can result in restricted ranges of scores, a problem less common with growth scale values (see Table 1 for ranges in the current sample). In contrast, growth scale values, which are an example of person ability scores, are better able to differentiate between individuals with relatively small differences in abilities and provide a more accurate measure of between-individual differences for individuals with fewer abilities (Farmer et al., 2020). Autistic traits were indexed by the ADOS-2 calibrated severity scores (Esler et al., 2015; Gotham et al., 2009). We elected to collapse certainty across response types when estimating the linear regression models in Research Questions 2 and 3. Therefore, parent certainty was indexed by the mean rating of parent certainty across the 23 nouns on the custom vocabulary checklist. This decision was based on the exploratory nature of the study. This approach also minimized the risk for Type I error.

To answer the third research question regarding whether parent certainty predicted consistency in parent report, we estimated a linear regression with parent certainty as the independent variable and consistency as the dependent variable. Consistency was indexed as the number of items that parents reported consistently across the MCDI-WG and custom checklist divided by 23, or the total number of items, multiplied by 100 (i.e., the percent consistent). Of note, one participant had missing data on one item, and therefore, the denominator for their percent consistent calculation was 22. For example, an item was considered consistent if the parent reported the same response on both measures (e.g., understands), whereas the item was considered inconsistent if the parent reported a different response on the custom vocabulary checklist and the MCDI-WG (e.g., if the parent reported the child understood the item on the custom vocabulary checklist but both understood and said the item on the MCDI-WG).

Results

Research Question 1: Differences in Certainty by Response Type

Although responses ranged from Very Uncertain (1) to Very Certain (5), overall certainty ratings were high (M = 4.15, SD = 0.89). Parents most frequently rated their certainty as “certain” or “very certain.” We examined whether parent certainty varied by response type (understands vs. understands and says vs. neither understands nor says). There was a significant difference in parent certainty by response type, F(2, 14) = 4.96, p = .023, η² _G = .20. Post hoc comparisons with Holm–Bonferroni corrections revealed that parent certainty was significantly higher for the “Understands and Says” responses compared to that for the “Understands” responses, p = .004, and “Neither Understands nor Says” responses, p = .028. Parent certainty did not significantly differ between “Neither Understands nor Says” responses and “Understands” responses, p = .494. See Table 2 and Figure 1.

Table 2.

Parent certainty measure items and ratings by response type.

Measure	Neither M (SD), Range	Understands M (SD), Range	Understands and says M (SD), Range
a Number of items	3.19 (4.12), 0–13	9.43 (7.03), 0–21	10.33 (8.95), 0–23
Certainty	3.43 (0.93), 2–5	3.82 (0.64), 2–5	4.49 (0.52), 3–5

^a Out of 23 total items.

Figure 1.

Mean parent certainty broken down by response type. Response was the parents' report of whether children understood, understood and said, or neither understood nor said each word on the custom vocabulary checklist. Certainty was each participant's mean rating of parent certainty calculated separately for each response type. Parent certainty was significantly higher for words the parents reported the children understood and said compared to that for words the children either understood or neither understood nor said.

Research Question 2: Examining Potential Predictors of Certainty

Next, we investigated whether standardized expressive and/or receptive language scores or autistic traits predicted parent certainty ratings. Using linear regression models with Holm–Bonferroni corrections, we found that expressive language, F(1, 19) = 6.07, p = .047, η² = .24, and receptive language, F(1, 19) = 7.99, p = .032, η² = .30, but not autistic traits, F(1, 19) = 0.07, p = .790, η² < .01, predicted parent certainty (see Table 3 and Figure 2). Higher standardized expressive and receptive language skills predicted higher parent certainty.

Table 3.

Statistics for the three linear regression models predicting parent certainty from child characteristics.

Models/variables	B	SE	t	p	p a	ηp 2
Model 1: Expressive language
Intercept	2.23	0.79	2.83	.011
Expressive language	0.01	< 0.01	2.46	.024	.047	.24
Model 2: Receptive language
Intercept	2.19	0.70	3.11	.006
Receptive language	0.01	< 0.01	2.83	.011	.032	.30
Model 3: Autistic traits
Intercept	3.93	0.83	4.75	< .001
Autistic traits	0.03	0.10	0.27	.790	.790	< .01

Note. Expressive language and receptive language measured from the Preschool Language Scales–Fifth Edition Expressive Communication and Auditory Comprehension growth scale values, respectively (Zimmerman et al., 2011). Autistic traits measured from the Autism Diagnostic Observation Schedule–Second Edition severity scores (Gotham et al., 2009; Lord, Luyster, et al., 2012; Lord, Rutter, et al., 2012). SE = standard error.

^a Reported p values are after Holm–Bonferroni corrections.

Figure 2.

Associations between mean parent certainty and (A) receptive language, (B) expressive language, and (C) autistic traits. Parent certainty was indexed by the mean rating of parent certainty by participant across the 23 items on the custom vocabulary checklist. Expressive language and receptive language were indexed by the Preschool Language Scales–Fifth Edition Expressive Communication and Auditory Comprehension growth scale values, respectively (Zimmerman et al., 2011). Autistic traits were indexed by the Autism Diagnostic Observation Schedule–Second Edition calibrated severity scores (Gotham et al., 2009; Lord, Luyster, et al., 2012; Lord, Rutter, et al., 2012). Parent certainty was significantly associated with receptive and expressive language but not with autistic traits.

Research Question 3: Parent Certainty as a Predictor of Consistency

The percent consistency across the two parent report measures averaged 76.98% (SD = 15.79), although the percentage ranged from 47.83% to 100%. Parent certainty predicted 38.98% of the variance in the percent consistent, B = 16.80, F(1, 19) = 12.14, p = .002, η² = .39. Thus, higher parent certainty significantly predicted a higher percent consistent (see Figure 3).

Figure 3.

The significant association between mean parent certainty and percent consistent. Parent certainty was the mean rating of parent certainty by participant across the 23 nouns on the custom vocabulary checklist. Percent consistent represented the number of items that parents reported consistently across the MacArthur–Bates Communicative Development Inventory Words and Gestures form (Fenson et al., 2006) and the custom checklist, divided the total number of items, and multiplied by 100.

Discussion

Parent report is a commonly used tool when assessing a child's emerging language skills. However, prior evidence suggests that parents may underestimate child vocabulary skills (Tardif et al., 1999; Venker et al., 2016) and parent report measures show some inconsistencies (Arunachalam et al., 2022; Yoder et al., 1997). To our knowledge, this is the first study to examine parent certainty ratings of child lexical knowledge in autistic children. Mean levels of parent certainty ranged widely across the sample. Overall, parents were significantly less certain when reporting on words their children understood or neither understood nor said compared to when reporting on words their children both understood and said. We also found that parents were more certain of their child's vocabulary abilities when their child had higher expressive and receptive language abilities. Parent certainty predicted the consistency with which parents reported their children's expressive and receptive vocabulary knowledge. This study builds on previous work in autistic children by testing a new approach to measuring parent report of child vocabulary, linking certainty with direct measures of child language, and providing insight into previous reports of parent rating inconsistency.

Measuring Parent Certainty

Mean certainty ratings for individual parents in this study ranged from 3.13 to 5, indicating that on average, most parents were neutral to very certain of their children's vocabulary skills. However, ratings from individual items ranged from 1 (very uncertain) to 5 (very certain), reflecting the variability in parent confidence when rating specific nouns. In line with our hypothesis, parent certainty was significantly higher for words the parents reported the children understood and said compared to that for words the children either understood or neither understood nor said. When parents reported that their children either understood or neither understood nor said the word, mean ratings were 3.82 and 3.43, respectively, indicating that parents were, on average, either “neutral” or “certain” of their children's lexical knowledge. In contrast, parent certainty averaged 4.49 and parents were most commonly “certain” or “very certain” when they reported their children understood and said the words. This finding is in line with previous work, which found that test–retest reliability is higher for expressive compared to that for receptive vocabulary (Fenson et al., 1994), and concurrent validity is higher for expressive versus receptive language measures (Luyster et al., 2008; Nordahl-Hansen et al., 2014).

Receptive language can be difficult to assess in young autistic children (Muller & Brady, 2016). In contrast to expressive vocabulary skills, which are often easily observed through spoken language, parents must infer their child's receptive word knowledge based on potentially subtle behaviors across environmental contexts. Moreover, acquisition of vocabulary knowledge is a gradual process, and parent certainty may reflect how well the child knows the word. That is, if a child has shown a parent that they understand a word in one context but not another, the parent may subsequently report that the child understands the word but rate their certainty as “neutral” or “certain” rather than “very certain” since the child does not exhibit word knowledge across contexts. In contrast, if a child has demonstrated that they know a word across several contexts and in multiple situations, a parent may report that they are “very certain” that their child understands the target word. In a broad sense, the associations between expressive and receptive language abilities and parent certainty in this study provided preliminary support for the hypothesis that parent certainty may provide a window into emerging language abilities in young autistic children. It is also critical to note that parent certainty may also reflect, at least in part, the extent to which a parent is attuned to their child's language. However, whether parent certainty reflects a child's emerging language abilities, a parent's familiarity with their child's language, or both goes beyond the scope of this study, and future research should continue to examine parent certainty and its potential utility for measuring emerging word knowledge.

Asking parents to report on their child's vocabulary knowledge is inherently linked with a deeper question of what it actually means to know a word. The question of what it means to know a word has led to studies employing eye tracking methodology in neurotypical children, autistic children, and late talkers that examine the accuracy and efficiency with which infants and toddlers process words (Fernald & Marchman, 2012; Fernald et al., 1998, 2006; Garrison et al., 2020; Venker et al., 2016). Findings in neurotypical children, late talkers, and autistic children suggest that word knowledge comes online over time as evidenced through increasing efficiency and accuracy during word recognition tasks in longitudinal studies and for older compared to younger children (Fernald & Marchman, 2012; Fernald et al., 1998). In addition, previous evidence using eye tracking indicates that parents may underestimate their child's word knowledge (Houston-Price et al., 2007; Venker et al., 2016). These studies provide foundational information on the development and measurement of word knowledge in young children, but eye tracking is not a readily available clinical tool, and it remains important to accurately assess emerging word knowledge in clinical populations. In comparison, parent report measures of child vocabulary are readily available, but current versions do not tap into emerging word knowledge. Pairing certainty ratings with looking-while-listening eye tracking paradigms may be a worthwhile endeavor to understand whether parent certainty can be used as a metric of emerging receptive language knowledge.

Muller and Brady (2016) argued that many language assessments, particularly those that measure receptive language, fail to measure emerging skills and result in floor effects. Unlike traditional parent report measures, eye tracking and parent certainty are measured across continuums (e.g., proportion of looks to target and certainty on a rating scale). These more nuanced measures may be more sensitive to emerging language compared to measures with dichotomous responses. Developing novel ways to measure early and emerging language abilities among autistic children is critical for developing interventions that build on current skills. If parent certainty ratings align with child performance on looking-while-listening tasks, then certainty ratings may be a useful clinical tool for identifying emerging word recognition and potential target words within interventions.

Understanding Consistency

Similar to certainty, the range of percent consistency in this study was large, with the lowest percent consistent for an individual parent being 47.83% and the highest being 100%. The mean percent consistent in this study (76.89%) was considerably lower than in the study by Arunachalam et al. (2022), where parents of autistic children averaged 89% consistency overall. The difference in overall percent consistency between the two studies may have multiple explanations. Because the previous study utilized the MCDI Words and Sentences form, they only examined consistency of words produced and not words understood. Similar to parent certainty, consistency may be higher when exclusively examining words produced compared to this study, which examined expressive and receptive lexical knowledge. In addition, Arunachalam et al. (2022) compared parent report on two different MCDI forms (i.e., the MCDI-WG and the MCDI Words and Sentences), whereas in this study, we included the MCDI-WG and a custom form.

Arunachalam et al. (2022) speculated that the inconsistency of parent report in their study reflected parent uncertainty regarding their child's lexical knowledge. We took an important next step in empirically measuring whether this was the case. Our findings confirm the interpretation of Arunachalam et al. (2022); parents who were less certain about their child's vocabulary knowledge were also less consistent in their reporting. This finding sheds light on previous studies showing a range of consistency in parent report (Arunachalam et al., 2022; Yoder et al., 1997) and provides an explanation for why parents may not always be consistent reporters. In the future, it will also be important to determine whether consistency, similar to certainty, varies based on words the parents report the children to understand, understand and say, or neither understand nor say.

Associations Between Certainty, Expressive Language, Receptive Language, and Autistic Traits

Previous work in children with developmental delays found associations between consistency and factors such as parent educational attainment, occupational status, family access to resources, and the extent to which parents attribute nonverbal behaviors as child communication (Yoder et al., 1997). This study extended this work by examining how individual differences in child language and/or autistic traits contributed to parent certainty. Consistent with our hypothesis, we found that parent certainty was significantly higher when children scored higher on standardized expressive and receptive language measures. These findings are in line with Feldman et al. (2000), who posited that parent judgments of child vocabulary for younger children (or children with lower language) require more interpretation than judgments for older children (or children with stronger language skills). We also hypothesized that autistic traits would predict parent certainty, but this was not the case. However, the lack of association between parent certainty and autistic traits is consistent with previous research that did not find differences in parent consistency when comparing ratings from parents of autistic children to ratings from parents of neurotypical children (Arunachalam et al., 2022). These findings may indicate that parent certainty about children's vocabulary knowledge more closely taps into a child's language abilities rather than autistic traits. However, additional research is needed to understand how or if specific child behaviors (e.g., joint attention, repetitive behaviors) or family factors (e.g., socioeconomic status, number of children in the family) affect parent certainty of child lexical skills.

Study Limitations and Future Directions

This study introduced a novel measure of parent certainty to the field and contributed critical information regarding how parent certainty may play a role in the consistency of parent report measures. However, this study had multiple limitations. Because investigating parent certainty was not the primary aim of the broader study, we did not counterbalance or consistently track the order in which the MCDI and custom vocabulary checklist were completed (also see Arunachalam et al., 2022). Nevertheless, the format of the two parent report assessments varied, so caregivers were perhaps less likely to remember their responses across the two forms. The custom vocabulary checklist contained a reduced number of items, and this study exclusively examined object nouns. Previous research suggests that parents are less consistent in their reports of verbs compared to nouns (Arunachalam et al., 2022). Thus, a follow-up to this study should include parent certainty ratings of other parts of speech and should also expand on the number of words included. This study did not validate the custom vocabulary checklist including the parent certainty measure. Based on the findings of this line of research, it may be appropriate to pursue larger-scale validation studies of a parent certainty measure. Validation would be a necessary step prior to broader clinical use.

It is unclear what strategies parents utilized to report certainty, and it is possible that parents employed different strategies. Nevertheless, it is important to note that this limitation is also present in traditional parent report measures such as the MCDI, where it is unclear what information parents use when selecting whether or not their child understands a word. While a strength of this study was the range of language abilities represented in the sample, the child participants lacked racial and ethnic diversity. In addition, because the larger study focused specifically on autistic children, we obtained and thus reported on limited parent demographic information.

Clinical Implications

Parent certainty and consistency ranged widely across participants in this study. Notably, the parent with the lowest certainty (3.13) had a mean percentage consistency of 73.91% and the child had among the lowest standardized expressive and receptive language scores in the sample. In contrast, one of the parents with the highest mean certainty (5.00) was 100% consistent and the child had among the highest expressive and receptive language scores. These findings suggest that parent certainty when rating vocabulary abilities is associated with their consistency; in addition, it may be more difficult for parents to assess child language abilities when their child is an early communicator. Parent report measures are a frequently utilized clinical tool for emerging communicators, when direct assessments by SLPs are difficult to administer (Kasari et al., 2013; Muller & Brady, 2016). At a minimum, it may be beneficial to discuss with parents in greater depth (a) how they assess their child's comprehension and (b) how certain they feel about their responses. This study's findings also suggest that parent report measures should be used in combination with direct assessment measures, especially for early communicators when parents are particularly uncertain. Parent certainty measures of child vocabulary may help SLPs determine specific target words to directly assess, such as by selecting words where parents indicate low levels of certainty.

Parent certainty may also be a useful tool for guiding goal development and selecting specific target words in intervention. Taking a zone of proximal development approach (Vygotsky, 1978), it may be advantageous to target words that the parent reports the child knows, but with the least certainty, to efficiently teach receptive language and solidify emerging word knowledge. In contrast, it may be beneficial to target words that parents are certain their children know when supporting expressive language development through spoken or augmented modalities. Measuring parent certainty can also provide an opportunity for clinicians and caregivers to discuss how children demonstrate word knowledge and what adults can do to support generalization.

Conclusions

This study investigated a novel measure of parent certainty of child vocabulary skills among young autistic children. Parent certainty was higher for words the parents reported the children understood and said compared to that for words children either understood or neither understood nor said. Certainty ratings were also higher when a child had higher standardized receptive and expressive language scores. Lastly, this study found that parent certainty was associated with reporting consistency, which clarified previous findings of inconsistencies in parent report of child vocabulary skills. This study provided preliminary evidence that parent certainty on ratings of child vocabulary may enrich current parent report measures by offering insight into emerging child vocabulary knowledge. Overall, this study provided foundational information on the utility of parent certainty for comprehensively assessing early vocabulary skills in autistic children.

Data Availability Statement

Data are available from the corresponding author upon reasonable request.

Funding Statement

Funding for this study was supported by the National Institutes of Health under Awards R21 DC016102 (Courtney E. Venker, principal investigator [PI]) and R01 DC020165 (Courtney E. Venker, PI).