Abstract
Purpose:
Given the time demands of language sample analysis, this study aimed to identify efficient, flexible sampling procedures that maintain strong reliability. We examine how language sample length in minutes, timing (i.e., beginning vs. later in the elicitation task following a warm-up period), and elicitation context (i.e., play or conversation) influence the reliability of four common language measures in young deaf and hard of hearing (DHH) children.
Method:
Participants included 21 DHH children (3–5 years old) who completed 10-min play and conversation samples in English. Samples were transcribed and analyzed for mean length of utterance in words, mean length of utterance in morphemes, total number of words per minute, and number of different words per minute. We evaluated the reliability of these measures by comparing shorter segments (i.e., 1, 3, 5, and 7 min) to the full sample using Cronbach's alpha, with additional comparisons based on the sample's timing and elicitation context.
Results:
Language samples with durations of 3, 5, and 7 min yielded strong reliability across all measures. Timing and elicitation context had minimal impact overall, particularly when sample length was sufficient.
Conclusions:
The findings support the use of samples as short as 3 min as a reliable and time-efficient approach to language sample analysis in DHH preschoolers. Results also challenge the assumption that a “warm-up” period is necessary before sampling, suggesting that reliable measurement can begin as soon as the elicitation task begins.
Supplemental Material:
When a clinician or educator sits down to collect a language sample, they face a series of practical questions: How long should the sample be? Should I start recording right away or wait until the child has warmed up? Should I elicit language through play or conversation? These decisions shape how professionals working across populations approach language sampling, influencing the time required for analysis and the utility of the results. For deaf and hard of hearing (DHH) children who use spoken English, these decisions carry added weight. Some DHH children score within the average range on omnibus norm-referenced language measures yet still show weaknesses in spontaneous connected speech, including shorter utterances and lower lexical diversity (Werfel & Douglas, 2017). Language sample analysis (LSA) complements standardized assessment by examining language use in context and offering a flexible way to support comprehensive assessment and progress monitoring.
Yet, despite its clinical relevance, LSA remains underutilized. Clinicians and teachers of the deaf widely recognize the value of language sampling, but they continue to rely heavily on norm-referenced assessments when making decisions about service eligibility and intervention planning (Blaiser & Shannahan, 2018; Hayes et al., 2009; Nicholas & Geers, 2008). This disconnect between perceived utility and practical application stems primarily from logistical constraints. Limited time for eliciting, transcribing, analyzing, and interpreting language samples has been consistently identified as the most significant barrier to routine clinical use (Heilmann, 2010; Kemp & Klee, 1997; Pavelko et al., 2016; Wilder & Redmond, 2024). As demands on clinician time continue to grow, streamlining the process has become a priority. If shorter samples can yield reliable data, clinicians may be able to reduce unnecessary time demands for each step.
Several studies have examined the stability of language metrics for shorter versus longer samples in children with typical development. Findings suggest that shorter samples can yield reliable results comparable to those from longer samples (Guo & Eisenberg, 2015; Heilmann et al., 2010; Pavelko et al., 2020; Thordardottir, 2016). For example, Heilmann et al. (2010) found that 3-min samples closely approximated those from 7-min samples for several measures, including mean length of utterance (MLU) in morphemes (MLUm), total number of utterances, and number of different words (NDW). Guo and Eisenberg (2015) similarly reported consistent estimates for total number of words (TNW) and MLUm across sample lengths but noted that NDW was inflated in shorter samples.
Few studies have extended this work beyond typically developing children to clinical populations, and no known studies have examined whether brief language samples yield reliable estimates of common LSA measures in DHH children. Measures frequently examined in this literature, including MLU, TNW, and NDW, are also routinely used by clinicians working with DHH children as part of their LSA practices (Blaiser & Shannahan, 2018). DHH children often produce shorter utterances and show vulnerabilities in grammatical morphology relative to their hearing peers (Koehlinger et al., 2013), raising questions about whether reliability findings from hearing children generalize to DHH samples.
In addition to total length, the timing of the sample within the elicitation task (i.e., whether the sample is collected at the beginning or later in the task) may also influence the reliability of the derived language measures. A common clinical assumption is that the initial minutes of an elicitation task may be less representative than later portions of the same task. In practice, clinicians often delay the analyzed portion of a sample until later in the task rather than analyzing language from the start. Additionally, some research protocols exclude the opening minutes of the sampled task from analysis (Nittrouer et al., 2014; Salmani et al., 2022). However, empirical support for this practice is limited. Casby (2011) found no significant differences in MLUm based on whether samples were drawn from the beginning, middle, or end of a sample in children with language impairment. McDaniel and Brady (2022) compared samples drawn from the beginning and midpoint of communication samples in minimally verbal autistic children. Samples from the beginning of the elicitation task yielded stronger agreement with data from the full task than samples from the midpoint. If these findings replicate for DHH children, LSA procedures could be further streamlined.
Beyond considerations of how much and when to sample, the context in which language is elicited may also influence the reliability of language sample data. Prior research has shown that structured interview contexts yield more complex and consistent language than play-based interactions in school-age children with language impairment (Evans & Craig, 1992). Similarly, Heilmann et al. (2010) found that short samples from conversation and narrative tasks yielded stable measures in typically developing children, with no interaction between context and sample length. Although play is widely used in clinical settings with young children, it often involves less examiner control and greater variability in materials used and topics discussed, which can introduce variability into elicited language (Evans & Craig, 1992). Because clinicians often collect language samples in play and conversation, it is important to understand how elicitation context shapes language use and whether those differences affect the reliability of commonly used language sample measures in DHH children.
Purpose and Research Questions
This study examines how sample length, timing, and elicitation context influence the reliability of common language sample measures in preschool-aged DHH children. Our goal is to provide practical, evidence-based guidance for optimizing LSA in time-constrained clinical, educational, and research settings.
We address three research questions (RQs). RQ1: Do shorter samples yield reliable estimates of (a) MLU in words (MLUw), (b) MLUm, (c) TNW per minute (TNW/min), and (d) NDW per minute (NDW/min) in preschool-aged DHH children? We predicted that reliability would improve with sample length, and that samples as short as 3–5 min may be sufficient for some or all language measures. Because prior studies have reported mixed patterns across measures (e.g., Guo & Eisenberg, 2015; Heilmann et al., 2010), we considered two alternatives regarding measure-specific reliability. One possibility was that all four measures would reach acceptable reliability at similar sample lengths (Heilmann et al., 2010). A second possibility was that the productivity measure (TNW/min) would stabilize more quickly than the lexical diversity measure (NDW/min) and the utterance length measures (MLUw, MLUm; Guo & Eisenberg, 2015).
RQ2: Does the timing within the elicitation task affect the reliability of these measures? This comparison reflects the clinical decision to analyze language from the start of a play or conversation sample versus later portions of that same task. Accordingly, we compare language samples elicited at the beginning of each sampled task to language elicited later in the same sample. We expected that samples collected at the beginning of the elicitation task would yield reliability comparable to those collected later, consistent with findings from other clinical populations (Casby, 2011; McDaniel & Brady, 2022).
RQ3: Does the elicitation context (i.e., play vs. conversation) influence the reliability of these measures? We expected conversation samples to yield stronger reliability than play samples due to the more structured nature of the interaction (Evans & Craig, 1992).
Method
Participants
Participants included a subset of 21 DHH children aged 3 to 5 years old (M = 4.31 years; SD = 0.41 years; 14 girls and seven boys) enrolled in the Kansas Language Assessment Program – Deaf/Hard of Hearing (LAP–DHH; Language Assessment Program, 2018). Twenty participants reported English as their primary language. One participant was described as a balanced bilingual student who primarily defaulted to English. Five participants used American Sign Language (ASL) to some degree at the time that the samples were collected. Eight participants had unilateral hearing loss, classified as slight (n = 1), mild sloping to severe (n = 1), moderate-to-severe (n = 1), severe (n = 1), and profound (n = 4). The remaining 13 participants had bilateral hearing loss, classified as mild (n = 2), mild to severe (n = 1), mild to profound (n = 1), moderate (n = 1), moderate to severe (n = 2), moderate to profound (n = 1), severe to profound (n = 1), and profound (n = 4). Five children used bilateral cochlear implants, six used bilateral hearing aids, three used unilateral cochlear implants, four used unilateral hearing aids, and three did not use hearing technology. Age of exposure to spoken English was reported for 12 participants. Of those, 10 were reported to have exposure since birth, one at 9 months, and one at 11 months of age (M = 1.67 months; SD = 3.92 months). Of the 11 participants for whom age of amplification was reported, amplification ranged from 2 weeks of age to 12 months (M = 5.64 months; SD = 3.61 months). Caregivers identified 17 participants as White, two as Black or African American, one as Asian, and one as two or more races.
Caregivers provided written informed consent for participants. The University of Kansas (#145175) and Vanderbilt University Medical Center (#221188) institutional review boards approved all study procedures. Study records and data-management materials were maintained using REDCap electronic data capture tools hosted at Vanderbilt University Medical Center (Harris et al., 2009, 2019).
Procedure
Language Sample Elicitation
As part of the LAP–DHH, comprehensive evaluations are conducted at least annually to assess receptive, expressive, and social communication skills in spoken English and/or ASL, depending on each child's language profile. English and ASL are assessed independently by a separate team of administrators. For the present analysis, we examined the spoken English language samples. Licensed teachers of the deaf, speech-language pathologists, and early childhood special educators administer these assessments, incorporating LSA as a key component. All language sampling sessions were video-recorded using the built-in video-recording application on employer-provided laptops with caregiver permission. Examiners placed the laptop to capture the child's face and relevant materials.
Language samples were collected during semistructured interactions in two contexts: play and conversation. To ensure consistency, examiners adhered to standardized guidelines emphasizing child-led interactions, prompt responsiveness, and minimal directives or questioning to encourage spontaneous language (Abbeduto et al., 2020; Brady et al., 2014). In the play context, children played freely with a set of predetermined toys, while examiners followed the child's lead and responded naturally without directing or structuring the interaction. In the conversation context, examiners followed a structured sequence of open-ended prompts on age-appropriate topics (e.g., school, family, pets) to elicit extended discourse while ensuring uniformity across participants (Abbeduto et al., 2020). The order of play and conversation samples was not counterbalanced. Procedural fidelity was evaluated using a rubric with context-specific items. Example items for the play context include following the child's lead, giving the child time to formulate responses, providing topic-continuing comments, showing appropriate nonverbal communication, and refraining from talking too often or too fast for the play context. Examples for the conversation context include adherence to the scripted introduction and conclusion, topic sequence and transitions, use of open-ended questions and contingent follow-up prompts, minimization of examiner interruptions, and supportive verbal and nonverbal behaviors to encourage elaboration. Procedural fidelity was assessed for 24% of the analyzed language samples, with mean scores of 97% accuracy for play (SD = 3%, range: 93%–100%) and 95% for conversation (SD = 3%, range: 90%–98%).
To be included in the analysis, participants were required to have both play and conversation samples exceeding 10 min in length, collected within 30 days of each other. For most participants, play and conversation samples were collected on the same day; when collected on different days (n = 3), the interval ranged from 1 to 8 days. Although the protocol specified a target duration of 12 min, actual sample lengths varied due to procedural deviations and scheduling constraints (M = 12.97, SD = 2.03, range: 10.18–21.52 min). The language sample procedure and coding manual is available upon request from the last author. The samples were collected between February 2021 and May 2024.
Transcription
The last author and trained research assistants (RAs) transcribed the language samples following a detailed coding manual. Utterances were segmented at the Communication Unit level, defined as “an independent clause with its modifiers” (Loban, 1976) and coded for Brown's 14 grammatical morphemes (Brown, 1973) following Systematic Analysis of Language Transcripts (SALT; Miller & Chapman, 2020) conventions. All transcribers completed training by (a) reviewing the coding manual and receiving direct instruction and (b) achieving a minimum of 80% small/large agreement on analyzed variables for three expert-coded nonstudy samples. Following Abbeduto et al. (2020), the primary transcriber generated an initial transcript for each video-recorded language sample. A secondary transcriber reviewed the transcript, proposed modifications, and collaborated with the primary transcriber to reach a consensus on the final version. To assess interobserver agreement, 24% of the language samples were independently transcribed by another team of transcribers. Two-way random-effects single-measure intraclass correlation coefficients (ICCs) with absolute agreement were used to assess interrater reliability between teams of transcribers. ICC values indicated strong interrater reliability across all variables, exceeding the commonly accepted benchmark of .70 for very good interrater reliability (Mitchell, 1979). The ICCs were .98 for MLUw, .98 for MLUm, .93 for TNW, and .89 for NDW.
Derivation of Dependent Variables
To evaluate the effects of sample length and timing, we analyzed a 10-min language sample for each participant and segmented it at predetermined intervals. For samples longer than 10 min, only the first 10 min were analyzed for comparability across participants. Aligning with prior work (Guo & Eisenberg, 2015; Heilmann et al., 2010; McDaniel & Brady, 2022), we analyzed the full 10-min sample and derived segments of 1, 3, 5, and 7 min in duration. We also examined whether the timing of a sample within each play or conversation task affected reliability. For each sample length, we extracted two matched segments: one beginning sample drawn from the start of the task and one later sample drawn from the end. For example, a 3-min beginning sample included the first 3 min of the task (0:00–3:00), while the corresponding later sample included the final 3 min (7:00–10:00). This mirrored design allowed for direct comparison of reliability across matched-length samples collected at different points in the same task, isolating any potential warm-up effects. All timestamp placements were independently verified by a second RA for accuracy.
We used the SALT software (Miller & Chapman, 2020) to derive values for four language measures (i.e., MLUw, MLUm, TNW/min, and NDW/min) from the full transcript and segmented samples. Only complete and intelligible verbal utterances were included. Using SALT's transcript cut function, we analyzed the full 10-min sample and eight segment-level samples per participant in each context (i.e., play and conversation). Two RAs independently derived variable values and verified accuracy across all samples.
To assess the reliability of each shorter segment relative to the full 10-min transcript, we calculated Cronbach's alpha (Cronbach, 1951) for each language measure. Cronbach's alpha was chosen for its ability to quantify internal consistency, reflecting the stability of each language measure when derived from shorter segments of the same transcript (Heilmann et al., 2010; McDaniel & Brady, 2022). All statistical analyses were conducted in SPSS (Version 29.0.2). Values of α ≥ .80 were interpreted as evidence of “very good” reliability and served as the threshold for sufficient reliability (DeVellis & Thorpe, 2021).
We conducted complementary post hoc analyses using two-way random-effects single-measure ICCs with absolute agreement to characterize agreement between each shorter segment and the 10-min reference sample. Following established guidelines, ICC values ≥ .75 were interpreted as evidence of strong agreement (Cicchetti, 1994; Koo & Li, 2016).
Results
We examined the minimum sample length required to achieve reliable measurement of MLUw, MLUm, TNW/min, and NDW/min (RQ1). Table 1 displays descriptive statistics for the full language samples, illustrating the participants' performance on the language measures examined in this study. Descriptive statistics for each shorter sample length are provided in Supplemental Material S1 (play) and S2 (conversation). Reliability was assessed using Cronbach's alpha, calculated for 1-, 3-, 5-, and 7-min samples. As shown in Tables 2 and 3, a consistent pattern emerged: Longer samples produced stronger reliability. All four language measures reached the threshold for sufficient reliability in 3-, 5-, and 7-min samples. One-minute samples derived from the beginning did not reach the reliability threshold for any measure. Although select measures reached the threshold in later 1-min segments, these effects were not consistent across measures or contexts.
Table 1.
Descriptive statistics for language measures in full play and conversation samples.
| Measure | Play |
Conversation |
||||
|---|---|---|---|---|---|---|
| M | SD | Range | M | SD | Range | |
| C&I utterances | 72.95 | 27.85 | 33–125 | 58.95 | 10.58 | 33–79 |
| MLUw | 3.69 | 0.73 | 2.56–5.29 | 4.21 | 1.80 | 1.98–9.03 |
| MLUm | 4.00 | 0.83 | 2.66–5.79 | 4.55 | 2.03 | 2.00–9.82 |
| TNW | 273.10 | 131.39 | 93–656 | 253.86 | 133.27 | 92–619 |
| NDW | 97.33 | 29.22 | 48–168 | 105.43 | 37.73 | 48–193 |
Note. C&I = complete and intelligible; MLUw = mean length of utterance in words; MLUm = mean length of utterance in morphemes; TNW = total number of words; NDW = number of different words.
Table 2.
Cronbach's alpha for language sample measures across sample lengths and timing for play samples.
| Measure | Segments from beginning of sample |
Segments from later in sample |
||||||
|---|---|---|---|---|---|---|---|---|
| 7 min | 5 min | 3 min | 1 min | 7 min | 5 min | 3 min | 1 min | |
| MLUw | .98 | .95 | .89 | .45 | .97 | .97 | .94 | .78 |
| MLUm | .99 | .96 | .89 | .48 | .97 | .97 | .95 | .76 |
| TNW/min | .99 | .98 | .94 | .69 | .99 | .97 | .95 | .89 |
| NDW/min | .98 | .97 | .82 | .44 | .96 | .91 | .83 | .62 |
Note. Bold values meet or exceed .80. MLUw = mean length of utterance in words; MLUm = mean length of utterance in morphemes; TNW/min = total number of words per minute; NDW/min = number of different words per minute.
Table 3.
Cronbach's alpha for language sample measures across sample lengths and timing for conversation samples.
| Measure | Segments from beginning of sample |
Segments from later in sample |
||||||
|---|---|---|---|---|---|---|---|---|
| 7 min | 5 min | 3 min | 1 min | 7 min | 5 min | 3 min | 1 min | |
| MLUw | .99 | .99 | .98 | .75 | .99 | .99 | .96 | .87 |
| MLUm | .99 | .99 | .98 | .78 | .99 | .99 | .96 | .89 |
| TNW/min | .99 | .97 | .88 | .45 | .97 | .95 | .91 | .87 |
| NDW/min | .98 | .94 | .85 | .47 | .98 | .91 | .82 | .60 |
Note. Bold values meet or exceed .80. MLUw = mean length of utterance in words; MLUm = mean length of utterance in morphemes; TNW/min = total number of words per minute; NDW/min = number of different words per minute.
Post hoc ICC analyses revealed patterns that closely paralleled those observed for Cronbach's alpha. Across measures, agreement between shorter segments and the full 10-min sample increased as sample length increased. Measures that exhibited stronger internal consistency in the Cronbach's alpha analyses likewise demonstrated higher agreement coefficients in the ICC analyses. ICC values for all measures, sample lengths, and timing conditions are reported in Supplemental Material S3 (play) and S4 (conversation).
We evaluated whether the timing of the sample impacted reliability (RQ2). All language measures demonstrated sufficient reliability for 3-, 5-, and 7-min samples (see Tables 2 and 3). For 1-min samples, reliability depended on timing and measure. Beginning-of-sample segments did not reach the reliability threshold for any tested measures in either context. In later segments, TNW/min reached threshold in play and conversation, and MLU measures did so in conversation only. NDW/min did not reach threshold in any 1-min condition.
To evaluate whether elicitation context influenced measurement reliability (RQ3), analyses were based on samples taken from the beginning of each elicitation task to isolate the effects of context apart from timing. Play and conversation samples yielded similar patterns across all lengths (see Tables 2 and 3). When sample length was sufficient (i.e., 3 min), reliability was consistently strong regardless of context. These findings support that elicitation context had little practical impact on reliability.
Discussion
Reliable measurement is foundational to effective LSA. Yet, the reliability of LSA measures can vary depending on how, when, and how much language is collected. We evaluated the effects of sample length, timing, and elicitation context on the reliability of four common LSA measures in preschool-aged DHH children using spoken English, with secondary agreement analyses included to contextualize the reliability findings.
Minimum Length Required for Sufficient Reliability Across Measures
Sample length emerged as the most influential factor in measurement reliability, aligning with our hypothesis that longer samples would yield more stable estimates. Segments of at least 3 min were sufficient to achieve strong reliability for each language measure. All four measures met the reliability threshold (α ≥ .80) in 3-, 5-, and 7-minute samples. ICC analyses indicated that agreement met criterion (ICC ≥ .75) for all four measures in 5- and 7-min samples across contexts and all measures except NDW/min in 3-min samples. One-minute samples derived from the beginning did not reach acceptable reliability for any measure across contexts. Although some measures reached threshold in later 1-min segments, these effects were not consistent across measures or contexts and did not alter conclusions regarding minimum sample length.
These findings align with those of Heilmann et al. (2010), who used Cronbach's alpha and reported strong reliability for productivity, lexical diversity, and utterance length measures in 3-min language samples from typically developing children. Studies in other populations likewise suggest that several commonly used language measures can achieve stable estimates with relatively short samples, though the specific measures and analytic frameworks differ (Guo & Eisenberg, 2015; Pavelko et al., 2020; Thordardottir, 2016). Although some studies have applied α ≥ .70 as adequate (e.g., Heilmann et al., 2010), we applied a more conservative α ≥ .80 threshold to guide clinical interpretation. Under this conservative standard, 1-min samples were insufficiently reliable. We provide the Cronbach's alpha values in Tables 2 and 3 so that readers can evaluate the results using thresholds other than α ≥ .80 that better align with their purposes.
The present findings should be interpreted with attention to the distinction between internal consistency and agreement. Cronbach's alpha was used to evaluate the stability of language measures derived from shorter segments of the same transcript, whereas complementary ICC analyses quantified agreement between shorter segments and the full reference sample. Together, these findings support the conclusion that 3-, 5-, and 7-min segments can yield stable and interpretable estimates of key language measures, without implying strict interchangeability at the individual-sample level. Clinically, this suggests that 3- to 5-min samples may be useful for efficiently estimating common language sample measures. Further work is needed to determine whether brief samples are valid for identifying language difficulty or tracking change in specific vulnerable linguistic areas.
The descriptive data in Table 1 illustrate the level of linguistic specificity captured by the LSA measures examined in the present study. Although these measures can provide efficient estimates of broad language production and point to areas for further analysis, summary-level LSA measures do not directly characterize specific linguistic vulnerabilities, including grammatical morphology differences in DHH children (Koehlinger et al., 2013). In the present sample, MLUw and MLUm were closely aligned across contexts and sample lengths. This pattern may be consistent with low morphological density in the elicited language. However, a relatively small difference between word-based and morpheme-based MLU is not necessarily unusual in preschool language samples (Parker & Brorson, 2005; Rice et al., 2010), so the interpretation should remain cautious. Direct analysis of specific grammatical morphemes was beyond the scope of the present study.
Limited Influence of a “Warm-Up” Period
Sample timing showed minimal impact on reliability, supporting our prediction that a warm-up period is not required within play and conversation tasks to obtain sufficiently reliable language samples for DHH children. Reliability estimates were consistent across timing conditions, with no systematic advantage for either. Although minor fluctuations occurred, they were small and unlikely to be clinically meaningful. A small but notable difference emerged in 1-min MLU measures from conversation samples; later samples exceeded the reliability threshold, but beginning samples did not. This pattern was confined to the 1-min condition and did not persist in 3-min or longer samples. These findings reinforce that reliable measurement can be obtained at any point in the elicitation task when sufficient sample length is collected. Similar results have been reported by Casby (2011) and McDaniel and Brady (2022), who found that a warm-up period was not required to achieve adequate reliability of language and vocalization measures in children with language impairment and autism, respectively. Of note, the majority of our sample was assessed by unfamiliar adults who did not work at the participants' school. Only five participants were assessed by their school's speech-language pathologist, who may or may not have provided services to them. Together, these findings challenge the assumption that children require a warm-up period before yielding stable estimates on common language sample measures. Clinicians may therefore begin sampling at the start of a play or conversation task without compromising measurement quality.
Consistent Results Across Play and Conversation Contexts
Elicitation context had little influence on reliability. Our prediction of greater reliability for conversation samples relative to play samples was partially supported. Conversation samples produced marginally higher reliability estimates for MLU measures at the 1-min length, but these differences were small and did not persist in longer samples. By 3 min, all measures reached strong reliability in both contexts. These findings align with prior work suggesting that structured tasks may elicit more consistent language (Evans & Craig, 1992), but they also indicate that elicitation context becomes less important when sample length is sufficient. Clinically, this finding suggests that either play or conversation can be used effectively for language sampling, provided that the sample is at least 3–5 min long.
Limitations and Future Directions
This study offers valuable insights into the reliability of brief language samples. Nonetheless, several limitations should be considered. First, the sample reflects variation in hearing levels, hearing technology use, and language experience, but the sample size (N = 21) limited our ability to analyze potential differences across subgroups. The range of characteristics represented supports generalizability of findings for a broad population of DHH children. Future studies with larger samples are needed to evaluate whether reliability varies systematically by child-level factors (e.g., communication modality, hearing level).
Second, although reliability is a necessary foundation for interpreting language sample data, it does not guarantee validity. The present analyses were not designed to determine whether brief samples are sufficient for evaluating specific linguistic targets or monitoring change in those targets over time. Future research should examine whether brief samples yield representative and clinically useful estimates of key linguistic features, including grammatical morphemes and complex syntax.
Finally, we evaluated internal consistency within a single transcript. Incorporating other reliability metrics (e.g., test–retest reliability and generalizability theory) to examine the stability of brief samples across sessions and elicitation conditions would provide additional support for the findings and guidance for application in clinical and research settings.
Conclusions
This study examined how sample length, timing, and elicitation context influence the reliability of language sample measures in preschool-aged DHH children. Sample length was the primary driver of measurement reliability and agreement. The results challenge the assumption that a “warm-up” period is necessary and support the use of brief, flexible sampling procedures that better align with real-world clinical demands. This work contributes to a growing body of evidence supporting efficient and individualized approaches to language sampling that are realistic for clinical and educational environments. As the field continues to balance practicality and rigor, this study offers a framework for optimizing both.
Data Availability Statement
The data sets generated and/or analyzed in this study are available from the last author upon reasonable request at jena.mcdaniel@vumc.org.
Supplementary Material
Acknowledgments
This research was supported by Project REACH (H325D220072, awarded to Vanderbilt University; U.S. Department of Education) and the National Institute on Deafness and Other Communication Disorders (R01DC021188; Principal Investigator: Jena McDaniel). However, contents do not necessarily represent the policy of the funding agencies, and endorsement by the federal government should not be assumed. This article is published Open Access under a read and publish agreement between Vanderbilt University and the American Speech-Language-Hearing Association. The authors thank the children and families who participated in this research, as well as the Kansas State School for the Deaf and the Language Assessment Program – Deaf/Hard of Hearing for their collaboration to make this work possible.
Funding Statement
This research was supported by Project REACH (H325D220072, awarded to Vanderbilt University; U.S. Department of Education) and the National Institute on Deafness and Other Communication Disorders (R01DC021188; Principal Investigator: Jena McDaniel). However, contents do not necessarily represent the policy of the funding agencies, and endorsement by the federal government should not be assumed. This article is published Open Access under a read and publish agreement between Vanderbilt University and the American Speech-Language-Hearing Association.
References
- Abbeduto, L., Berry-Kravis, E., Sterling, A., Sherman, S., Edgin, J. O., McDuffie, A., Hoffman, A., Hamilton, D., Nelson, M., Aschkenasy, J., & Thurman, A. J. (2020). Expressive language sampling as a source of outcome measures for treatment studies in fragile X syndrome: Feasibility, practice effects, test–retest reliability, and construct validity. Journal of Neurodevelopmental Disorders, 12(1), Article 10. 10.1186/s11689-020-09313-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Blaiser, K. M., & Shannahan, M. A. (2018). Language sample practices with children who are deaf and hard of hearing. Language, Speech, and Hearing Services in Schools, 49(4), 950–964. 10.1044/2018_LSHSS-17-0130 [DOI] [PubMed] [Google Scholar]
- Brady, N., Warren, S. F., Fleming, K., Keller, J., & Sterling, A. (2014). Effect of sustained maternal responsivity on later vocabulary development in children with fragile X syndrome. Journal of Speech, Language, and Hearing Research, 57(1), 212–226. 10.1044/1092-4388(2013/12-0341) [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brown, R. (1973). A first language: The early stages. Harvard University Press. 10.4159/harvard.9780674732469 [DOI] [Google Scholar]
- Casby, M. W. (2011). An examination of the relationship of sample size and mean length of utterance for children with developmental language impairment. Child Language Teaching and Therapy, 27(3), 286–293. 10.1177/0265659010394387 [DOI] [Google Scholar]
- Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. 10.1037/1040-3590.6.4.284 [DOI] [Google Scholar]
- Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. 10.1007/BF02310555 [DOI] [Google Scholar]
- DeVellis, R. F., & Thorpe, C. T. (2021). Scale development: Theory and applications (4th ed.). Sage Publications. [Google Scholar]
- Evans, J. L., & Craig, H. K. (1992). Language sample collection and analysis: Interview compared to freeplay assessment contexts. Journal of Speech, Language, and Hearing Research, 35(2), 343–353. 10.1044/jshr.3502.343 [DOI] [PubMed] [Google Scholar]
- Guo, L.-Y., & Eisenberg, S. (2015). Sample length affects the reliability of language sample measures in 3-year-olds: Evidence from parent-elicited conversational samples. Language, Speech, and Hearing Services in Schools, 46(2), 141–153. 10.1044/2015_LSHSS-14-0052 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harris, P. A., Taylor, R., Minor, B. L., Elliott, V., Fernandez, M., O'Neal, L., McLeod, L., Delacqua, G., Delacqua, F., Kirby, J., Duda, S. N., & REDCap Consortium . (2019). The REDCap consortium: Building an international community of software partners. Journal of Biomedical Informatics, 95, Article 103208. 10.1016/j.jbi.2019.103208 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009). Research electronic data capture (REDCap): A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381. 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hayes, H., Geers, A. E., Treiman, R., & Moog, J. S. (2009). Receptive vocabulary development in deaf children with cochlear implants: Achievement in an intensive auditory-oral educational setting. Ear and Hearing, 30(1), 128–135. 10.1097/AUD.0b013e3181926524 [DOI] [PubMed] [Google Scholar]
- Heilmann, J. J. (2010). Myths and realities of language sample analysis. Perspectives on Language Learning and Education, 17(1), 4–8. 10.1044/lle17.1.4 [DOI] [Google Scholar]
- Heilmann, J. J., Nockerts, A., & Miller, J. F. (2010). Language sampling: Does the length of the transcript matter? Language, Speech, and Hearing Services in Schools, 41(4), 393–404. 10.1044/0161-1461(2009/09-0023) [DOI] [PubMed] [Google Scholar]
- Kemp, K., & Klee, T. (1997). Clinical language sampling practices: Results of a survey of speech-language pathologists in the United States. Child Language Teaching and Therapy, 13(2), 161–176. 10.1177/026565909701300204 [DOI] [Google Scholar]
- Koehlinger, K. M., Owen Van Horne, A. J., & Moeller, M. P. (2013). Grammatical outcomes of 3- and 6-year-old children who are hard of hearing. Journal of Speech, Language, and Hearing Research, 56(5), 1701–1714. 10.1044/1092-4388(2013/12-0188) [DOI] [PMC free article] [PubMed] [Google Scholar]
- Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. 10.1016/j.jcm.2016.02.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Language Assessment Program, Kansas Pub. L. No. 75-5397e. (2018).
- Loban, W. (1976). Language development: Kindergarten through grade twelve. NCTE Committee on Research Report No. 18. National Council of Teachers of English. https://eric.ed.gov/?id=ED128818
- McDaniel, J., & Brady, N. C. (2022). The influence of communication sample length on reliability and convergent validity of vocal measures derived from the Communication Complexity Scale. Journal of Speech, Language, and Hearing Research, 65(10), 3881–3889. 10.1044/2022_JSLHR-22-00010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller, J., & Chapman, R. (2020). Systematic Analysis of Language Transcripts (Version 24) [Computer software]. SALT Software, LLC. https://www.saltsoftware.com/
- Mitchell, S. K. (1979). Interobserver agreement, reliability, and generalizability of data collected in observational studies. Psychological Bulletin, 86(2), 376–390. 10.1037/0033-2909.86.2.376 [DOI] [Google Scholar]
- Nicholas, J. G., & Geers, A. E. (2008). Expected test scores for preschoolers with a cochlear implant who use spoken language. American Journal of Speech-Language Pathology, 17(2), 121–138. 10.1044/1058-0360(2008/013) [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nittrouer, S., Sansom, E., Low, K., Rice, C., & Caldwell-Tarr, A. (2014). Language structures used by kindergartners with cochlear implants: Relationship to phonological awareness, lexical knowledge and hearing loss. Ear and Hearing, 35(5), 506–518. 10.1097/AUD.0000000000000051 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Parker, M. D., & Brorson, K. (2005). A comparative study between mean length of utterance in morphemes (MLUm) and mean length of utterance in words (MLUw). First Language, 25(3), 365–376. 10.1177/0142723705059114 [DOI] [Google Scholar]
- Pavelko, S. L., Owens, R. E., Jr., Ireland, M., & Hahs-Vaughn, D. L. (2016). Use of language sample analysis by school-based SLPs: Results of a nationwide survey. Language, Speech, and Hearing Services in Schools, 47(3), 246–258. 10.1044/2016_LSHSS-15-0044 [DOI] [PubMed] [Google Scholar]
- Pavelko, S. L., Price, L. R., & Owens, R. E., Jr. (2020). Revisiting reliability: Using sampling utterances and grammatical analysis revised (SUGAR) to compare 25- and 50-utterance language samples. Language, Speech, and Hearing Services in Schools, 51(3), 778–794. 10.1044/2020_LSHSS-19-00026 [DOI] [PubMed] [Google Scholar]
- Rice, M. L., Smolik, F., Perpich, D., Thompson, T., Rytting, N., & Blossom, M. (2010). Mean length of utterance levels in 6-month intervals for children 3 to 9 years with and without language impairments. Journal of Speech, Language, and Hearing Research, 53(2), 333–349. 10.1044/1092-4388(2009/08-0183) [DOI] [PMC free article] [PubMed] [Google Scholar]
- Salmani, M., Seyed, S., Moradi, S., Shirkavand, Z., Sadati, S., & Tabatabaei, M. S. (2022). Production of Persian morphosyntactic structures based on P-LARSP: Comparing children with/without hearing loss. Journal of Rehabilitation, 22(4), 482–505. 10.32598/RJ.22.4.3307.1 [DOI] [Google Scholar]
- Thordardottir, E. (2016). Long versus short language samples: A clinical procedure for French language assessment. Canadian Journal of Speech-Language Pathology and Audiology, 40(3), 176–197. [Google Scholar]
- Werfel, K. L., & Douglas, M. (2017). Are we slipping them through the cracks? The insufficiency of norm-referenced assessments for identifying language weaknesses in children with hearing loss. Perspectives of the ASHA Special Interest Groups, 2(9), 43–53. 10.1044/persp2.sig9.43 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wilder, A., & Redmond, S. M. (2024). Updates on clinical language sampling practices: A survey of speech-language pathologists practicing in the United States. Language, Speech, and Hearing Services in Schools, 55(4), 1151–1166. 10.1044/2024_LSHSS-24-00035 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data sets generated and/or analyzed in this study are available from the last author upon reasonable request at jena.mcdaniel@vumc.org.

This work is licensed under a