The Language ENvironment Analysis (LENA) System in Toddlers With Early Indicators of Autism: Test–Retest Reliability and Convergent Validity With Clinical Language Assessments

ABSTRACT

Clinical language assessments often influence the types of services that autistic children are eligible to receive. However, these assessments often take place outside of the child's natural language environment. In this study, we assess the potential of using naturalistic language processing technology, the Language ENvironment Analysis (LENA) system, in clinical research. Within a sample of caregivers and autistic toddlers aged 16–33 months (N = 100), the current study examined associations between all LENA‐generated variables and two clinical assessments of language: the Vineland Adaptive Behavior Scales, Third Edition: Communication Domain and the MacArthur Bates Communicative Development Inventories: Vocabulary Checklist. We also evaluated LENA test–retest reliability in a subsample of participants (n = 81). Some LENA‐generated variables—specifically, the Conversational Turn Count, Vocal Productivity, and Automated Vocalization Assessment—exhibited small‐to‐moderate significant positive correlations with clinical language assessment variables. Additionally, all LENA‐generated variables demonstrated moderate‐to‐good test–retest reliability within a 2‐week period. To our knowledge, this is the first study that examines the psychometric properties of all LENA‐generated variables in a single large sample. Findings show promising evidence of LENA's utility as a source of naturalistic language data for research with autistic toddlers.

Trial Registration: ClinicalTrials.gov identifier: NCT05114538 (“Improving the Part C Early Intervention Service Delivery System for Children with ASD”)

Summary.

• Language evaluations for children with early signs of autism often take place during short clinical visits outside of the home.

• The Language ENvironment Analysis (LENA) software records many hours of children's language in their homes, which may give researchers more information about their language abilities.

• This study examines how consistently LENA is able to measure children's language environment and vocal ability over time. We also examine associations between LENA data and clinical language measures.

• Our findings show that LENA measures are relatively consistent and have moderate to small associations with clinical language measures.

1. Introduction

Language assessments are important for identifying services that autistic children are eligible to receive and are an important marker of intervention success; thus, capturing individualized profiles of children's language development is important (Guthrie et al. 2023; Wetherby et al. 2014). Correlates of spoken language, such as vocal development and language environment, can provide researchers with deeper understanding of autistic children's language trajectories and suggest potential targets for intervention (Plumb and Wetherby 2013; Gilkerson et al. 2018).

A working group of experts convened by the National Institute on Deafness and Other Communication Disorders recommended naturalistic language sampling to capture autistic children's communicative abilities (Tager‐Flusberg et al. 2009). Because naturalistic language data is captured in‐home with the child's caregivers, it has higher ecological validity than clinic‐based language samples. However, naturalistic observations often require extensive human and financial resources due to the need for hours of transcribing and coding linguistic stimuli (McDaniel et al. 2020a, 2020b).

1.1. Understanding LENA

LENA (Language ENvironment Analysis) technology provides novel opportunities to examine large amounts of naturalistic language data without requiring extensive resources for transcription and coding (McDaniel et al. 2020a, 2020b). LENA employs a wearable recording device that captures up to 16 h of audio of children's linguistic environments and segments recordings into ~20,000–50,000 audio segments (Xu et al. 2009a, 2009b, 2009c; Ganek and Eriks‐Brophy 2018). LENA's analysis software aggregates specific types of audio segments to calculate a series of variables.

Here, we describe two classes of variables the LENA software calculates. The first class consists of count variables that are considered indicative of the language environment the adult and child produce together. The LENA system tracks both adult and child vocalizations, as the quantity and timing of child‐directed adult vocalizations scaffold child language development (Warlaumont et al. 2014; Yoder and Warren 1999). LENA software produces three key count variables: Adult Word Count (AWC), quantifying the number of adult vocalizations during the recording session; Child Vocalization Count (CVC), quantifying the number of key child vocalizations during the recording session; and Conversational Turn Count (CTC), capturing the number of times a key child and adult vocalized sequentially (Warren et al. 2010). The LENA system separates the CTC into both Adult‐Initiated Conversational Turns (AITC) and Child‐Initiated Conversational Turns (CITC).

The second class of LENA variables, which are relatively less‐studied, reflect the degree to which child vocalizations have word‐like phonology. The first is LENA's Vocal Productivity variable (VP), defined as the length of children's vocal output in canonical syllables per utterance. According to LENA's developers, canonical syllables are defined as “well‐formed consonant vowel pairs” (Du et al. 2017). The second is LENA's Automatic Vocalization Assessment (AVA), which is a standard score comparing multiple word‐like aspects of the key child's vocalizations to those of age‐matched peers. The AVA, claimed to be a quantifiable estimate of vocal development, measures the degree to which child speech resembles adult speech by computing the average difference between adult and individual child speech units standardized by chronological age (Richards et al. 2009). Here, we propose that VP and AVA might reflect children's vocal developmental level. Word‐like phonology of vocalizations become proportionally higher and more frequent as children approach formal language use (Fitch and Giedd 1999; Richards et al. 2009). Thus, VP and AVA may be particularly useful as correlates of formal language use.

Initial studies of LENA variables examined correlations between automated and human transcription of units that provide the basis for the variables. A systematic review of 33 studies comparing LENA‐derived count variables to equivalents based on human‐coded transcripts yielded strong positive correlations, with weighted mean Pearson R values of 0.79 for adult speech and 0.77 for child vocalizations (Cristia et al. 2020). LENA counts all utterances regardless of language spoken, and similar correlations supporting LENA's validity have been reported in many languages, including Spanish, French, Mandarin, Korean, and Vietnamese (Ganek and Eriks‐Brophy 2018). However, associations have not been high between LENA's CTC variable and human coders' indices of conversational turns. In the review described above, LENA's CTC did not achieve high correlations with human coders' conversational turns, with a mean Pearson R value of 0.34 (Cristia et al. 2020). This lower association was replicated in a separate meta‐analysis, reporting a relation between CTC and human coders of r = 0.36 (Wang et al. 2020).

Discrepancies between human coders and the CTC may be due to LENA's definition of turn‐taking. LENA's CTC sums instances of sequential key child and adult vocalizations with no more than 5 s of intermittent silence, without regard for the content of vocalizations. Therefore, many CTC events may be chance instances of a child vocalization following an adult vocalization. Because the child likely vocalizes less than the adult during the recording, the CVC is likely highly correlated with the CTC. If so, one can question whether CTC provides important information above and beyond CVC. Further examination of the CTC is necessary to determine whether it is clinically useful despite these limitations.

1.2. LENA as a Clinical Research Tool

LENA's count variables (i.e., CVC, CTC, and AWC) have been found to have significant relations with standard clinical assessments of language (Dykstra et al. 2013; Gilkerson et al. 2017a, 2017b; Warren et al. 2010). A meta‐analysis of 17 studies assessing language skills in autistic and non‐autistic children with multiple standardized measures (i.e., Preschool Language Scale, Bayley Scales of Infant and Toddler Development, the MacArthur‐Bates Communicative Development Inventory [CDI], the Receptive Expressive Emergent Language Test) reported aggregated relations with small‐to‐medium effect sizes, ranging from 0.21 and 0.32 between standardized language assessments and LENA's CVC, CTC, and AWC (Wang et al. 2020).

When examining effect sizes of correlations of count variables with language measures in autistic children, the results are quite varied. An early LENA study of 26 autistic children between 16‐ and 48‐months of age, not included in the above meta‐analysis, found much stronger positive correlations between the CTC and AWC variables with the CDI (r = 0.80 and r = 0.55) and Communication and Symbolic Behavior Scales (r = 0.76 and r = 0.63) (Warren et al. 2010). In contrast, a recent study of 99 autistic children aged 14–47 months examined associations between the Communication subscale of the Vineland Adaptive Behavior Scale and the CVC variable (r = 0.22). Thus, the size of correlations between LENA's count variables and clinical language measures for autistic children is unclear.

The sparse research examining associations between the less‐studied VP and AVA variables with clinical language measures in autistic children has been promising but also inconsistent. For example, significant positive relations were found between VP scores and the Vineland Communication subscale (r = 0.22) as well as receptive (r = 0.41) and expressive (r = 0.32) language scales of the Mullen Scales of Early Learning in a sample of 99 speaking and minimally speaking autistic children aged 14–47 months (Sulek et al. 2022). Additionally, in a sample of 360 autistic children aged 2–48 months, the AVA was highly correlated with expressive language composite scores derived from the PLS‐4 and REFL‐3 (r = 0.75); however, these findings incorporated the same normative participant sample used to compute AVA standard scores (Richards et al. 2009; Xu et al. 2009a, 2009b, 2009c). In a small independent sample of autistic toddlers (n = 20), no significant association was found between AVA scores and future spoken vocabulary as measured by the CDI (Woynaroski et al. 2017).

Observed differences in correlations between LENA variables and clinical language assessments among studies may be due to the settings and populations in which data was collected. Specifically, many of the above studies were conducted in settings and/or with study populations outside of LENA's original intended use (i.e., daylong, in‐home recordings with children 2–48 months old), thus contributing to greater variability among findings.

1.3. Gaps in the Literature

Many studies showing associations between LENA variables and language measures did not examine all LENA variables. Examining the associations of interest in the same study sample makes comparisons among LENA variables straightforward. Notably, few studies included LENA's VP and AVA variables in analyses. In a 2017 review of the use of LENA in research, 38 studies investigated CTC, 35 studies investigated AWC, 26 studies investigated CVC, one study investigated AVA, and no studies investigated VP (Ganek and Eriks‐Brophy 2018). Similarly, in a 2020 meta‐analysis examining LENA's associations with language development, no papers included AVA or VP (Wang et al. 2020). Additionally, no papers presented the CTC subcomponents: AITC and CITC. This study addressed this gap by examining all LENA variables within a relatively large sample of autistic toddlers within LENA's recommended age range.

Similarly, no studies to date have examined test–retest reliability of all LENA‐generated variables within one participant sample. Further, the results of previous studies examining test–retest reliability of LENA variables, albeit few, have been uneven. In a sub‐study of 52 non‐autistic children, CVC test–retest reliability was reported as r = 0.67, CTC as r = 0.56, and AWC as r = 0.66 for recordings taken within the same week (Gilkerson et al. 2017a, 2017b). A sample of 360 children yielded test–retest reliability for AVA as r = 0.76 over 1 month and r = 0.65 over 2 months (Richards et al. 2009). Stability of the AVA has been reported as g = 0.85 with one daylong recording in a sample of 30 autistic preschool children and as g = 0.88 with two recordings in a sample of 20 autistic infants and 20 non‐autistic siblings (Woynaroski et al. 2017; Markfeld et al. 2023).

When estimates of convergent validity are based on small sample sizes, the confidence intervals around the estimates are wide. The current study examines a large sample of multiple daylong LENA recordings of 100 autistic children who were within the LENA system's intended age range of 2–48 months. This relatively large sample size is useful for estimating psychometrics of LENA variables more precisely than in smaller samples.

1.4. Aims and Hypotheses

The first aim of the study was to examine test–retest reliability (n = 81) for all automatically‐generated LENA variables—AWC, CVC, CTC, AITC, CITC, VP, and AVA. We hypothesized that all LENA variables would approach or exceed the threshold of 0.75 for good test–retest reliability (Koo and Li 2016). Our second aim was to evaluate associations between these LENA variables and two standard measures of language, the Communication subscale of the Vineland Adaptive Behavior Scales, Third Edition and the vocabulary checklist portion of the MacArthur‐Bates Communicative Development Inventory (CDI). We hypothesized that all LENA variables would have small to moderate significant positive correlations with all clinical language variables and would not be associated with the Vineland Motor domain.

2. Methods

2.1. Participants

Participants (N = 100) were autistic children aged 16–33 months receiving Part C Early Intervention (EI) services and participating in an ongoing multisite community‐based Hybrid Type I implementation/effectiveness trial of Caregiver‐Implemented Reciprocal Imitation Teaching administered by EI providers and their caregivers (NCT05114538). This study was approved by the Michigan State University (MSU) institutional review board (IRB), which served as the cross‐site IRB of record. All participants in the sample had received an autism diagnosis in the community and/or received an autism diagnosis upon completing the study from a licensed clinical psychologist. Table 1 provides demographic characteristics of the sample.

TABLE 1.

Demographics of full sample (N = 100).

Demographic variables	M	SD
Child age (in months) at VABS‐3 administration	25.82	3.729

	n	%
Child race
Indigenous or Alaska Native	1	1.0
Asian	4	4.0
Black or African American	21	21.0
White	54	54.0
Other (not listed)	3	3.0
More than one race	12	12.0
Prefer not to answer	5	5.0
Child ethnicity
Hispanic or Latinx	31	31.0
Not Hispanic or Latinx	67	67.0
Prefer not to answer	2	2.0
Child sex assigned at birth
Male	67	67.0
Female	33	33.0
Total household income
Less than $10,000	6	6.0
$10,000–$24,999	9	9.0
$25,000–$49,999	25	25.0
$50,000–$74,999	15	15.0
$75,000–$99,999	7	7.0
$100,000–$124,999	8	8.0
$125,000–$149,999	8	8.0
$150,000–$174,999	3	3.0
$175,000–$199,999	1	1.0
$200,000 or above	6	6.0
Prefer not to answer	12	12.0
Child language
English	95	95.0
Spanish	5	5.0

2.2. Procedures

Families provided between one and 3 day‐long LENA recordings (mean length of 14.3 h per recording) for analysis, with 184 recordings provided across 100 participants. Data from participants who provided more than one LENA recording (n = 81) were examined for test–retest reliability based on the two longest recordings provided within a 2‐week period.

Families were provided with LENA devices and vests to wear over the child's clothing. Vests included a front‐facing pocket that was insulated on the side adjacent to the child's chest to minimize discomfort. LENA devices were hand‐delivered or mailed to family homes and returned in pre‐paid addressed envelopes. Caregivers were given instructions to record any day that the child was at home, with preference for days that were typical in terms of the child's routines. During periods when the child was asleep or bathing, caregivers were instructed to take the vest off the child and place it near them so that the device could still record vocalizations. There were no restrictions on which days of the week caregivers could record, but many working caregivers recorded on weekends, when they were at home with their children. Families were instructed to use any languages spoken at home.

In addition to providing LENA language recordings, caregivers completed questionnaires and were interviewed about their child's adaptive behavior with the Vineland Adaptive Behavior Scales, Third Edition in English or Spanish.

2.3. Measures

2.3.1. Demographics

Information about the child and family was collected through a 30‐item Family Demographic Information Form, which was completed by the caregiver enrolled in the larger research study.

2.3.2. Adaptive Behavior

The Vineland Adaptive Behavior Scales, Third Edition (Vineland; Sparrow et al. 2016) is a semi‐structured interview that assesses a child's adaptive functioning in four domains: Communication, Daily Living Skills, Socialization, and Motor Skills. The Vineland yields raw scores and growth scale values (GSVs) for subdomains, domain‐level standard scores, and a summary cross‐domain standard score, the Adaptive Behavior Composite. GSVs are derived from the Rasch model of item response theory, in which raw scores undergo a linear transformation to an equal‐interval scale that can be applied uniformly across ages (Sparrow et al. 2016). This report focuses on the Communication and Socialization domains, which are most closely related to what the LENA system purports to measure. For expressive and receptive subdomains, GSVs were used instead of raw scores because they minimize the impact of floor effects, providing continuous measurement across different levels of functioning. Motor domain raw scores were also included as a potential measure of divergent validity. The Vineland has strong psychometric properties, with reliability coefficients ranging from 0.83 to 0.99 (Sparrow et al. 2016).

2.3.3. Expressive Language

The MacArthur‐Bates Communicative Development Inventories (CDI) comprise a series of caregiver questionnaires about a child's current receptive and expressive vocabulary, and use of gestures and phrases (Fenson et al. 2006). For this project, the Vocabulary Checklist, which asks caregivers to identify the words on a 100‐word vocabulary list their child says, was used as an indicator of expressive language. This form is reported to have excellent reliability (α = 0.99) and concurrent validity with CDI long form (r = 0.93). For children exposed to both Spanish and English, caregivers completed forms in both languages.

2.3.4. LENA Variables

Five LENA variables were included in analyses that provide information about the child's language environment: Adult Word Count (AWC), Child Vocalization Count (CVC), Conversational Turn Count (CTC), Adult‐Initiated Turn Count (AITC), and Child‐Initiated Turn Count (CITC). These variables quantify the total number of specific vocal events that occur over the course of the entire recording session. The Adult Word Count (AWC) indicates the number of times a near adult vocalizes during the session. The Child Vocalization Count (CVC) indicates the number of times a key child vocalizes during the session. The Conversational Turn Count (CTC) indicates how many times a key child and near adult engage in a conversational turn by engaging in child‐adult or adult‐child communicative vocalizations within a five‐second interval with no other interrupting vocal event. The Adult‐Initiated Turn Count (AITC) is a subcomponent of the CTC and indicates how many times an adult initiates a conversational turn. The Child‐Initiated Turn Count (CITC) is also a subcomponent of the CTC and indicates how many times a key child initiates a conversational turn. More detailed information about the LENA variables can be found in LENA Technical Report (LTR) 02‐2 and 05‐2 (Gilkerson and Richards 2008; Xu et al. 2009a, 2009b, 2009c).

Two additional LENA variables included in the analyses are standard scores, derived from age‐based norms, which serve as LENA's proposed indicators of vocal development: the Automated Vocalization Assessment (AVA) and the Vocal Productivity (VP) measure. Each compares certain word‐like characteristics of the key child's vocalizations to those of age‐matched peers. More information about the VP and AVA can be found in LTR 11‐1 and 08‐1, respectively (Du et al. 2017; Richards et al. 2009).

2.4. Data Analysis

2.4.1. Data Preparation

Participants were excluded from analysis if they did not meet the criteria for ASD or if they did not provide any LENA recordings of sufficient length for analysis (≥ 5 h). Individual recordings shorter than 5 h were also excluded from analysis.

2.4.2. Assessing Undue Influence, Distributions, and Data Transformations

Cook's distance (D _i ) measures were computed for all datapoints in each bivariate correlation model to identify outliers that exerted undue influence on regression models (D _i > 1.0). Two participants were removed from analyses because they had multiple datapoints that yielded values of D _i > 1.0, indicating undue influence.

Descriptive statistics indicated that the following two variables had significant positive skewness and kurtosis: CDI vocabulary checklist and weighted mean values for AVA standard scores. A log transformation was performed on the CDI vocabulary checklist values (log10(x + c)) and a reciprocal transformation (−1/x) was performed on the AVA standard score values. Transformed variables had acceptable values for skewness and kurtosis. Because the CDI vocabulary checklist contained many zero values, supplemental non‐parametric correlations were conducted (Kendall's tau and Spearman's rho) between LENA variables of interest and the original, non‐transformed CDI vocabulary checklist scores. Descriptive information about participants' language measure scores is presented in Table 2.

TABLE 2.

Descriptive information about participants' performance on language assessments.

Variable	Mean	Standard deviation	Range	Minimum	Maximum
VABS‐3 receptive growth scale value	67.14	16.61	84	21	105
VABS‐3 expressive growth scale value	85.14	16.36	92	36	128
VABS‐3 communication standard score	46.27	18.73	67	20	87
VABS‐3 socialization standard score	66.02	11.56	67	36	103
CDI vocabulary checklist raw score	11.11	17.61	98	0	98

2.4.3. Analyzing Test–Retest Reliability

For the sub‐sample of 81 participants who contributed two or more LENA recordings longer than 5 h within a 2‐week period, the two longest recordings were used for test–retest analysis. Duration between LENA recordings ranged from 1 to 13 days (mean = 2.68, SD = 2.75). Because participants' recordings often varied in duration, LENA count variables were transformed into weighted variables to control for the duration of each recorded LENA session, which were calculated by dividing each of the variables by their corresponding recording duration and then multiplying the quotient by the duration of a full‐day LENA recording in seconds.

Test–retest reliability was computed by calculating intraclass correlation coefficients between LENA values across the first and second recordings. A two‐way mixed model was used. Additionally, to determine whether systematic mean differences between variables existed across recordings, repeated‐measures ANOVA was conducted for all LENA variables across recordings.

2.4.4. Analyzing Associations Between Clinical Assessments and LENA Variables

Associations were assessed by computing Pearson correlation coefficients (r) between weighted means of LENA‐generated variables and clinical assessments (Vineland Communication domain raw score, Vineland Socialization domain raw score, Vineland expressive subdomain GSV, Vineland receptive subdomain GSV, Vineland Motor domain raw score, and CDI vocabulary checklist total raw score).

Partial correlations were also computed to examine the relationship between CTC/AITC/CITC and clinical language assessments while controlling for CVC to acknowledge conversational turn variables' likely shared variance with child vocalizations. Higher rates of child vocalizations increase the probability of chance turn‐taking events (i.e., sequential child‐adult or adult‐child vocalizations). Partial rather than bivariate correlations were computed for the AITC and CITC due to high intercorrelations among CTC, AITC, and CITC.

To assess LENA's proposed indicators of vocal development (AVA and VP) scores' utility above and beyond child vocalization quantity, partial correlations were also conducted between these variables and clinical assessments, controlling for CVC.

3. Results

3.1. Test–Retest Reliability

Intraclass correlation coefficient values for each LENA variable across two recordings are presented in Table 3. All LENA variables demonstrated moderate‐to‐good test–retest reliability, ranging from 0.652 to 0.861 (Koo and Li 2016). Descriptive statistics for LENA variables at the first and second test–retest timepoints are presented in Table 4. There were no mean differences across recordings for the AWC, CTC, CVC, CITC, AVA, and VP variables. However, significant differences were found for AITC (F = 5.140, p = 0.026, η _p ² = 0.60), with values significantly greater at the first relative to the second recording.

TABLE 3.

Test–retest reliability.

Variable	Intraclass correlation coefficient	95% Confidence interval		n
		Upper bound	Lower bound
AWC	0.669	0.529	0.774	81
CVC	0.747	0.633	0.830	81
CTC	0.652	0.507	0.762	81
AITC	0.664	0.522	0.770	81
CITC	0.670	0.530	0.775	81
AVA	0.861	0.784	0.911	69
VP	0.734	0.602	0.807	68

Note: Intraclass correlation coefficient (ICC) values such that 0.75 > ICC > 0.5 indicate moderate reliability and values such that ICC > 0.75 indicate good reliability.

TABLE 4.

Descriptive statistics for LENA variables analyzed for test–retest reliability at both time points.

	N	Mean	Standard deviation	Range	Minimum	Maximum
Time 1 AWC	81	9650.93	5458.00	25,537	691	26,228
Time 2 AWC	81	8868.27	5051.31	25,668	821	26,489
Time 1 CTC	81	325.02	210.09	983	14	997
Time 2 CTC	81	284.32	173.42	672	21	693
Time 1 CVC	81	1585.57	953.72	4265	182	4447
Time 2 CVC	81	1406.09	864.55	4182	30	4212
Time 1 AITC	81	175.47	114.27	546	9	555
Time 2 AITC	81	146.43	92.12	364	2	366
Time 1 CITC	81	162.17	108.05	514	5	519
Time 2 CITC	81	148.53	97.72	374	6	380
Time 1 VP	75	82.85	20.98	93.56	41.62	135.18
Time 2 VP	69	82.93	22.07	100.44	40.67	141.12
Time 1 AVA	74	79.56	12.79	48.52	64.90	113.42
Time 2 AVA	72	79.00	11.49	43.30	64.90	108.20

3.2. Examining Bivariate Correlations

Pearson correlations among LENA‐generated variables are presented in Table 5. Pearson correlations between LENA‐generated variables and clinical language variables are presented in Table 6. LENA variables exhibited many significant positive correlations with clinical language variables. For the CDI vocabulary checklist, non‐parametric analyses replicated Pearson correlations described below.

TABLE 5.

Bivariate correlations among LENA variables.

	AWC	CTC	AITC	CITC	CVC	AVA	VP
AWC
Pearson correlation	—	0.703**	0.794**	0.564**	0.287**	0.140	−0.184
Sig. (two‐tailed)	—	< 0.001	< 0.001	< 0.001	0.004	0.187	0.084
CTC
Pearson correlation	—	—	0.961**	0.965**	0.754**	0.410**	0.171
Sig. (two‐tailed)	—	—	< 0.001	< 0.001	< 0.001	< 0.001	0.109
AITC
Pearson correlation	—	—	—	0.888**	0.638**	0.338**	0.052
Sig. (two‐tailed)	—	—	—	< 0.001	< 0.001	< 0.001	0.627
CITC
Pearson correlation	—	—	—	—	0.818**	0.398**	0.226*
Sig. (two‐tailed)	—	—	—	—	< 0.001	< 0.001	0.033
CVC
Pearson correlation	—	—	—	—	—	0.392**	0.437**
Sig. (two‐tailed)	—	—	—	—	—	< 0.001	< 0.001
AVA
Pearson correlation	—	—	—	—	—	—	0.428**
Sig. (two‐tailed)	—	—	—	—	—	—	< 0.001

Note: Bolded values followed by ** are significant at p < 0.01 and bolded values followed by * are significant at p < 0.05.

TABLE 6.

Bivariate correlations between LENA variables and clinical assessment variables.

	AWC weighted mean	CVC weighted mean	CTC weighted mean	VP standard score	AVA standard score
VABS‐3 receptive growth scale value
Pearson correlation	0.174	0.190	0.338**	0.138	0.121
Sig. (two‐tailed)	0.085	0.059	0.002	0.200	0.255
VABS‐3 expressive growth scale value
Pearson correlation	−0.010	0.176	0.266**	0.317**	0.236*
Sig. (two‐tailed)	0.925	0.082	0.008	0.003	0.025
VABS‐3 communication raw
Pearson correlation	0.085	0.146	0.293**	0.258*	0.246*
Sig. (two‐tailed)	0.402	0.150	0.003	0.015	0.019
VABS‐3 socialization raw
Pearson correlation	0.085	0.136	0.201*	0.230*	0.018
Sig. (two‐tailed)	0.401	0.180	0.046	0.031	0.866
VABS‐3 motor raw
Pearson correlation	005	0.135	0.156	0.057	−0.064
Sig. (two‐tailed)	0.958	0.185	0.125	0.597	0.553
CDI vocabulary checklist raw score
Pearson correlation	−0.11	0.182	0.197*	0.388**	0.421**
Sig. (two‐tailed)	0.958	0.070	0.050	< 0.001	< 0.001

Note: Bolded values followed by ** are significant at p < 0.01 and bolded values followed by * are significant at p < 0.05.

Adult word count (AWC) weighted mean values and child vocalization count weighted mean values (CVC) demonstrated no significant correlations with any of the clinical assessments.

Conversational turn count (CTC) weighted mean values demonstrated significant correlations with all five clinical language variables. CTC weighted mean values had moderate positive correlations with Vineland receptive GSV (r = 0.312, p = 0.002). Small positive correlations were found with Vineland expressive GSV (r = 0.266, p = 0.008), Vineland Communication total raw score (r = 0.293, p = 0.003), Vineland Socialization total raw score (r = 0.201, p = 0.046), and CDI vocabulary checklist (r = 0.197, p = 0.050). No significant correlations were found with the Vineland Motor total raw score.

Vocal Productivity (VP) standard scores significantly correlated with four of five clinical language variables. VP values exhibited moderate positive correlations with the Vineland expressive GSV (r = 0.317, p = 0.003) and CDI vocabulary checklist (r = 0.388 p < 0.001) and small positive correlations with Vineland Communication total raw score (r = 0.258, p = 0.15) and Vineland Socialization total raw score (r = 0.230, p = 0.031). VP was not significantly correlated with the Vineland Receptive GSV or the Vineland Motor total raw score.

Finally, Automated Vocalization Assessment (AVA) standard scores were significantly correlated with three of the five clinical language variables. AVA scores demonstrated moderate positive correlations with the CDI vocabulary checklist (r = 0.421, p < 0.001) and small positive correlations with Vineland expressive GSV (r = 0.236, p = 0.025) and Vineland Communication total raw score (r = 0.246, p = 0.019). AVA scores did not significantly correlate with Vineland receptive GSV, Vineland Socialization total raw score, or Vineland Motor total raw score.

3.3. Examining Partial Correlations

Partial correlations between LENA's proposed indicators of vocal development (VP and AVA) and clinical language variables are presented in Table 7. After controlling for CVC, the VP standard score demonstrated a moderate positive correlation with the CDI vocabulary checklist (r = 0.328, p = 0.002) and small positive correlations with the Vineland expressive GSV (r = 0.271, p = 0.011) and Vineland Communication total raw score (r = 213, p = 0.048). After controlling for CVC, the AVA standard score demonstrated a moderate positive correlation with the CDI vocabulary checklist (r = 353, p < 0.001).

TABLE 7.

Partial correlations between LENA‐generated standard scores and clinical assessments controlling for child vocalizations.

Variable	VP weighted mean	AVA weighted mean
VABS‐3 receptive growth scale value
Correlation	0.088	0.070
Sig. (two‐tailed)	0.416	0.512
VABS‐3 expressive growth scale value
Pearson correlation	0.271*	0.166
Sig. (two‐tailed)	0.011	0.120
VABS‐3 communication raw
Pearson correlation	0.213*	0.191
Sig. (two‐tailed)	0.048	0.073
VABS‐3 socialization raw
Pearson correlation	0.183	−0.043
Sig. (two‐tailed)	0.089	0.692
CDI vocabulary checklist raw score
Pearson correlation	0.328**	0.353**
Sig. (two‐tailed)	0.002	< 0.001

Note: Bolded values followed by ** are significant at p < 0.01 and bolded values followed by * are significant at p < 0.05.

Partial correlations between LENA‐generated measures of conversational turn‐taking (CTC, AITC, and CITC) and clinical language variables are presented in Table 8. The CTC retained significance with three of five clinical language variables when controlling for CVC, including Vineland receptive GSV (r = 0.261, p = 0.010), Vineland expressive GSV (r = 0.207, p = 0.041), and Vineland Communication total raw score (r = 0.282, p = 0.005). The CITC also demonstrated small positive correlations with the Vineland receptive GSV (r = 0.233, p = 0.021), Vineland expressive GSV (r = 0.268, p = 0.008), and Vineland Communication total raw score (r = 0.294, p = 0.003). When controlling for CVC, the AITC demonstrated a significant small positive correlation with the Vineland expressive GSV (r = 0.224, p = 0.027) only.

TABLE 8.

Partial correlations between turn‐taking variables and clinical assessments controlling for child vocalizations.

Variable	CTC weighted mean	AITC weighted mean	CITC weighted mean
VABS‐3 receptive growth scale value
Correlation	0.261**	0.224*	0.233*
Sig. (two‐tailed)	0.010	0.027	0.021
VABS‐3 expressive growth scale value
Correlation	0.207*	0.091	0.268**
Sig. (two‐tailed)	0.041	0.370	0.008
VABS‐3 communication raw
Correlation	0.282**	0.195	0.294**
Sig. (two‐tailed)	0.005	0.055	0.003
VABS‐3 socialization raw
Correlation	0.151	0.114	0.172
Sig. (two‐tailed)	0.137	0.264	0.091
CDI vocabulary checklist raw score
Correlation	0.092	−0.002	0.149
Sig. (two‐tailed)	0.364	0.986	0.141

Note: Bolded values followed by ** are significant at p < 0.01 and bolded values followed by * are significant at p < 0.05.

4. Discussion

This study examined the utility of LENA technology as a clinical research tool in a relatively large sample of autistic toddlers and their caregivers by evaluating test–retest reliability and associations between LENA‐generated variables and clinical language assessments frequently used in autism research. To our knowledge, this is the first study to examine the psychometric properties of all primary LENA variables in a large sample of autistic toddlers, including LENA's purported indices of vocal development—the AVA and VP.

4.1. Test–Retest Reliability

This study is the first to examine the test–retest reliability of all LENA‐generated language variables within a single sample, providing estimates of test–retest stability that can be compared. Additionally, this is the first study to our knowledge that documents the test–retest reliability of the VP, AITC, and CITC variables.

Four of seven LENA variables, the AWC, CTC, CITC, and AITC, demonstrated moderate test–retest reliability. Two of seven LENA variables, the CVC and VP, approached good test–retest reliability. One of seven LENA variables, the AVA, demonstrated good test–retest reliability within a 2‐week period.

In line with our hypotheses, the AVA, VP, and CVC variables all exceeded or approached the threshold of good test–retest reliability of 0.75 (Koo and Li 2016). Contrary to our hypotheses, the AWC, CTC, AITC, and CITC variables exhibited moderate test–retest reliability. However, confidence interval upper bounds for all variables exceeded the threshold for good test–retest reliability. While this numerical threshold is a helpful benchmark, the specific value of 0.75 itself is arbitrary. Additionally, our findings provide similar values to test–retest analyses conducted in prior studies (Woynaroski et al. 2017; Markfeld et al. 2023; Gilkerson et al. 2017a, 2017b; Richards et al. 2009). Therefore, these findings are promising in that LENA‐generated variables seem to be relatively stable within a 2‐week period.

Notably, conversational turn variables (CTC, CITC, and AITC) yielded relatively poorer test–retest reliability across LENA variables (0.652, 0.670, and 0.664, respectively). This finding may be related to the construction of these variables, in that each consists of two vocal agents, a key child and a near adult, as opposed to other variables consisting of only one vocal agent, such as a key child or near adult. Therefore, smaller test–retest reliability values for the CTC, CITC, and AITC may be due to compounded error associated with combining two vocal agents' vocalizations.

4.2. Interpreting Mean‐Level Differences Across LENA Recordings

We also examined mean differences for LENA scores across test–retest recordings. If each occasion primarily measures true variation on the same construct, there should not be significant changes over such a short interval. Surprisingly, values for AITC were significantly higher for the first versus the second LENA recordings provided for analysis. These significant differences may be due to a reactivity effect associated with putting the LENA device on the first time, in which heightened device awareness leads caregivers to work harder to engage their children than they typically would.

4.3. Bivariate Correlations Between LENA Variables and Clinical Assessments

In agreement with our hypotheses, the CTC exhibited moderate positive correlations with the Vineland receptive GSV and small positive correlations with the Vineland expressive GSV, Vineland Communication total raw score, Vineland Socialization total raw score, and CDI vocabulary checklist. Additionally, the VP exhibited moderate positive correlations with the CDI vocabulary checklist and Vineland expressive GSV and small positive correlations with the Vineland Socialization and Vineland Communication total raw scores. Lastly, the AVA exhibited moderate positive correlations with the CDI vocabulary checklist and small significant correlations with Vineland expressive GSV and Vineland Communication total raw score.

These associations were largely maintained even when controlling for child vocalization quantity. This is noteworthy because if associations with language became nonsignificant when controlling for CVC, then the rationale for the more complex LENA variables would be weakened. Additionally, as expected, no LENA variables were significantly correlated with the Vineland Motor domain raw score, which was included as an indicator of divergent validity.

The CTC exhibited small‐to‐moderate positive associations with all clinical measures of language and communication. Specifically, the CTC significantly correlated with clinical indices of expressive language, receptive language, and social communication with values similar to other comparable studies (Dykstra et al. 2013; Wang et al. 2020). The CTC, which captures sequential parent and child vocalizations, is a more complex indicator of children's language environments than LENA variables that merely count child or adult vocalizations. This complexity may account for greater convergent validity with clinical language assessments than the AWC and CVC.

The VP and AVA variables demonstrated moderate positive associations with clinical assessments of expressive language and small positive associations with clinical assessments of overall communication. The VP and AVA did not demonstrate significant associations with the Vineland's receptive GSV, thus suggesting that they demonstrate convergent validity with clinical indicators of expressive and not receptive language. There have been few studies exploring VP and AVA variables. Thus, the current findings warrant deeper investigation to support the use of VP and AVA in clinical research.

Contrary to our hypotheses, LENA's CVC and AWC did not demonstrate any significant positive associations with clinical language assessments. These findings contrast with some previous studies described above that do report significant associations between these variables and clinical language assessments. However, nonsignificant correlations found in our analyses are similar in magnitude to those presented by others (Sulek et al. 2022; Wang et al. 2020). When studies find different patterns of associations for the same variables, sampling error, population differences, and data collection differences are possible explanations. Such variance speaks to the need for meta‐analyses on the convergent validity of LENA variables. Future meta‐analyses should focus on more complex LENA variables such as CTC, which indicates bidirectional conversational exchanges, as well as on AVA and VP, which are normed indices of vocal development that focus on consonant‐vowel pairs and phone‐like utterances.

4.4. Study Limitations

Despite this study's large sample of autistic toddlers, this study has notable limitations. First, many of the toddlers in the sample had very limited spoken language. This may be in part because LENA recordings were conducted at the baseline timepoint of a larger study, when toddlers were quite young (mean age = 25.87 months). The relatively young age of participants, compounded with language delays, led to many children scoring near or at the floor of clinical language assessments. Therefore, less variability across language assessment performance was observed, which may have attenuated validity estimates. Thus, findings could be considered a lower limit of validity of the LENA variables.

Additionally, LENA is limited to the analysis of segmented language data and provides no semantic or grammatical analysis. Without considerations of content, it is likely that some “conversational turns” consist of adult and child vocalizations that occur sequentially by chance without contributing to a meaningful conversational exchange. We attempted to address this limitation by controlling for CVC in our analysis of CTC. Validity estimates continued to be significant after such control. However, future research is needed to improve automatic measurement of conversational turns.

4.5. Future Research

This study also highlights opportunities for potential future directions. One important future direction will be to extend the current study longitudinally. A longitudinal design would allow for tests of whether LENA‐generated variables predict gains in future language status. A later assessment would also likely improve the variability of language measures. Additionally, LENA may be useful in large‐scale data collection projects in which direct assessments may not be feasible. Secondly, future studies could attempt to reduce chance‐based errors inherent to the CTC by considering three‐point conversational turns. For example, one type of three‐event exchanges is a child vocalization followed by an adult vocalization, which is then followed by a second child vocalization. Three‐event turns are, theoretically, more indicative of a bidirectional conversational exchange, as it is required for the initiator to vocalize again when the desired conversational partner responds to their initial vocal bid (Harbison et al. 2018). Third, it will be useful to determine whether LENA is sensitive to intervention effects that are observed in clinical language measures, as this would be strong evidence for utility in clinical research. Lastly, the results of this study provide evidence for the increased use of VP and AVA variables in future clinical research studies of autistic toddlers.

5. Conclusions

This study examined test–retest validity among LENA variables and convergent validity between LENA variables and clinical language assessments in a sample of autistic children. These findings show promise for using LENA's CTC variable, along with previously unexplored AVA and VP variables, as clinical research tools to provide more comprehensive information about autistic children's language environments and vocal development.

Conflicts of Interest

The authors declare no conflicts of interest.

Nadwodny, N. , Yoder P. J., Ingersoll B. R., et al. 2025. “The Language ENvironment Analysis (LENA) System in Toddlers With Early Indicators of Autism: Test–Retest Reliability and Convergent Validity With Clinical Language Assessments.” Autism Research 18, no. 8: 1568–1579. 10.1002/aur.70062.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.