Mechanisms by which Early Eye Gaze to the Mouth During Multisensory Speech Influences Expressive Communication Development in Infant Siblings of Children with and without Autism

Abstract

Looking to the mouth of a talker early in life predicts expressive communication. We hypothesized that looking at a talker’s mouth may signal that infants are ready for increased supported joint engagement and that it subsequently facilitates prelinguistic vocal development and translates to broader gains in expressive communication. We tested this hypothesis in 50 infants aged 6–18 months with heightened and general population-level likelihood of autism diagnosis (Sibs-autism and Sibs-NA; respectively). We measured infants’ gaze to a speaker’s face using an eye tracking task, supported joint engagement during parent-child free play sessions, vocal complexity during a communication sample, and broader expressive communication. Looking at the mouth was indirectly associated with expressive communication via increased higher-order supported joint engagement and vocal complexity. This indirect effect did not vary according to sibling status. This study provides preliminary insights into the mechanisms by which looking at the mouth may influence expressive communication development.

Lay Abstract

At birth, infants tend to look predominantly at the eyes of their communication partners. Midway through the first year of life, however, infants begin to look increasingly at the mouth of a talker producing audiovisual speech. This developmental change has been linked with the development of communication in both theory and research. The present study provides novel insights into precisely how looking at the mouth may scaffold communication development in infant siblings of children with and without autism.

Autism is a neurodevelopmental condition characterized by social communication differences in addition to restricted and repetitive patterns of behavior, interests, and activities (American Psychiatric Association, 2013). Many children with autism¹ also present with broader language and communication impairments (Tager-Flusberg et al., 2005). In fact, difficulty acquiring expressive communication during the first few years of life is the single-most replicated predictor of long-term outcomes in this population (e.g., Billstedt et al., 2007; Eisenberg, 1956; Venter et al., 1992). These findings have spawned a large and ever-growing body of research seeking to identify factors that predict expressive communication in young children on the spectrum (see Yoder et al., 2015 for a review).

Looking at the Face as a Replicated Predictor of Expressive Communication

One replicated predictor of expressive communication is eye gaze. Early eye gaze patterns follow a known developmental trajectory. Specifically, at birth, infants tend to look predominantly at the eyes of their communication partners (e.g., Haith et al., 1977; Maurer & Salapatek, 1976). Midway through the first year of life, however, infants begin to shift their gaze towards the mouth of their communication partners, in particular when such partners are producing audiovisual speech or “talking” (e.g., Lewkowicz & Hansen-Tift, 2012; Tenenbaum et al., 2013). Looking at a talker’s mouth is especially prominent during periods marked by qualitative shifts in expressive communication development (i.e., around 8–18 months; Hillairet de Boisferon et al., 2018; Lewkowicz & Hansen-Tift, 2012; during this time, infants begin to imitate adult speech, communicate with purpose, and acquire their first words). The temporal coincidence of increased looking at the mouth with early communication milestones has led researchers to hypothesize that looking at audiovisual speech, in particular early in life, facilitates expressive communication development, perhaps by boosting speech processing efficiency and/or perceptual accuracy (e.g., Calvert et al., 1998; Middelweerd & Plomp, 1987; Sumby & Pollack, 1954; van Wassenhove et al., 2005).

There is increasing empirical support for the hypothesis that looking at the mouth during audiovisual speech bootstraps expressive communication development in individuals with autism. Several studies, for example, have demonstrated an association between looking at the mouth of a talker and aspects of language development and/or broader communicative competence in children and adults with autism both concurrently (e.g., Chawarska et al., 2012; Habayeb et al., 2020; Klin et al., 2002; Norbury et al., 2009) and prospectively (e.g., Campbell et al., 2014). The mechanisms by which looking to the mouth may scaffold expressive communication development in autism, however, remain unclear.

Relations between looking to the eyes of a speaker during audiovisual speech and expressive communication have been observed less frequently. However, in some instances looking at the both the mouth and the eyes of a talker have been shown to account for variance in communication skills (e.g., individual differences in word learning in Tenenbaum et al., 2014). Additionally, an emerging line of evidence suggests that looking at the eyes during audiovisual speech at 12 months is correlated with concurrent social skills (Pons et al., 2019) and that looking to the eyes at 6 months is correlated with later social skills (Wagner et al., 2018) in non-autistic infants. These findings suggest that social skills may benefit from attention to the social information inherent in a talker’s eyes more than linguistic information inherent in a talker’s mouth. Thus, given the core impairment in social communication skills associated with autism, it is important to also examine links between looking at the eyes during audiovisual speech and communication skills.

Putative Mechanisms By Which Looking at Audiovisual Speech May Influence the Development of Expressive Communication in Autism

We hypothesize that early looking to audiovisual speech may indirectly influence the development of expressive communication via parent-child engagement and children’s prelinguistic vocal complexity. Specifically, the shift in looking to a talker’s mouth in the first year of life is followed by a developmental period marked by increased joint engagement between parent-child dyads. This increase in joint engagement is evident first in what is termed supported joint engagement and, later in the course of development, via coordinated attention between a child and caregiver. We focus on supported joint engagement as the putative mechanism by which early looking to the mouth may support the development of expressive communication here because this type of joint engagement is most proximal, from a developmental perspective, to the predictor of interest.

Supported joint engagement.

Supported joint engagement has been frequently evaluated as a potential predictor of later communication in children with autism. Early in life, caregivers support children’s engagement in joint play routines in a manner that scaffolds development across a number of domains (Bakeman & Adamson, 1984; Bottema-Beutel et al., 2014). There is at least some evidence to suggest that caregiver supported joint engagement is linked with communication in children with or at increased likelihood for autism (Adamson et al., 2009; Gulsrud et al., 2016; Kasari et al., 2008).

Recent work has differentiated between lower-order supported joint engagement (LSJE; wherein a caregiver appears to influence a child’s play, but the child does not actively acknowledge the caregiver with gaze) and higher-order supported joint engagement (HSJE; wherein a caregiver influences a child’s play, and the child acknowledges the caregiver by engaging in reciprocal play; Bottema-Beutel et al., 2014). Several studies have found HSJE may be superior to LSJE for predicting later expressive communication in children with autism (Bottema-Beutel et al., 2014; Bottema-Beutel, Woynaroski, et al., 2019; Crandall et al., 2019).

We suspect that a child’s generalized tendency to look to the mouth during face-to-face interactions with an adult producing audiovisual speech may support the acquisition of communication skills by signaling to caregivers that children are increasingly “tuned in” to the parent’s communication and developmentally ready for supported joint engagement, in particular HSJE, wherein the child actively acknowledges the communication partner. This may lead the caregiver to make increased efforts to engage the child in scaffolded interactions marked by reciprocal engagement around objects or events, a context in which child communication is routinely made relevant and which, thus, is especially likely to facilitate prelinguistic vocal development and subsequently translate to broader gains in expressive communication.

Prelinguistic vocal complexity.

We hypothesize that the influence of looking to the mouth on expressive communication development through increased supported joint engagement will be detectable first in increased prelinguistic vocal complexity. Prelinguistic vocal complexity is commonly operationalized as the number of consonants and/or the proportion of canonical syllables (i.e., consonant-vowel combinations produced with adult-like speech timing) that a child produces in prelinguistic communication acts (i.e., consonant inventory and canonical syllabic communication; e.g., McDaniel et al., 2019; Saul & Norbury, 2020; Wetherby et al., 2007; Woynaroski et al., 2014, 2016, 2017). Recent research has shown that these indices of vocal complexity are useful for predicting language growth and outcomes in children with autism, even after controlling for a number of other factors previously linked with language development (e.g., parent linguistic input, responding to joint attention, intentional communication; Wetherby et al., 2007; Woynaroski et al., 2017; Yoder et al., 2015).

Thus, the links for both supported joint engagement, in particular HSJE, and prelinguistic vocal complexity with expressive communication development are well-established in autism. However, to our knowledge, no prior work has evaluated the hypothetical link between early looking to the mouth and expressive communication development through HSJE and prelinguistic vocal complexity. We aimed to do so by statistically testing whether HSJE and prelinguistic vocal complexity partially or completely accounted for (i.e., mediated; Hayes, 2017) associations between looking to the mouth during audiovisual speech and concurrent expressive communication, particularly early in life.

Rationale for a Focus on Infants at Heightened Likelihood for Autism

A challenge to testing this hypothesized indirect path through which early looking to audiovisual speech may influence expressive communication development is that autism cannot always be reliably diagnosed in the earliest stages of development (Ozonoff et al., 2015; Turner & Stone, 2007), in particular during the timeframe in which looking to audiovisual speech has been posited to predict vocal development (i.e., beginning at approximately 8 months of age; for discussion, see Lewkowicz & Hansen-Tift, 2012). A potential solution to this problem is to prospectively study infants who are known to be at increased likelihood for a future diagnosis of autism based on their status as younger siblings of children who are already diagnosed with autism (Sibs-autism; Costanzo et al., 2015). Approximately one-third of infant siblings of children with autism will go on to receive a diagnosis of autism and/or language impairment; many more will show sub-clinical characteristics associated with the disorder (Charman et al., 2017; Messinger et al., 2013; Ozonoff et al., 2011). These Sibs-autism are often contrasted with a group of infants at relatively lower, general population-level likelihood for autism (Sibs-NA; i.e., infant siblings of non-autistic, otherwise typically developing children). Prior studies taking this approach to date have found positive associations between looking to a talker’s mouth and later expressive communication and negative associations between looking to a talker’s eyes and later expressive communication in infants at increased likelihood of being diagnosed with autism (Elsabbagh et al., 2014; Wagner et al., 2018; Young et al., 2009). Though concurrent links have between looking to the mouth and concurrent expressive communication have been reported in young children with autism (e.g., Chawarska et al., 2012; Tenenbaum et al., 2014), no study to our knowledge has evaluated these links concurrently in Sibs-autism. In the present study, we employ this approach in an attempt to test our hypotheses regarding how looking to audiovisual speech influences concurrent expressive communication development in infants with an elevated likelihood of autism diagnosis.

Research Questions

The research questions for the present study were as follows:

1. Do Sibs-autism differ from Sibs-NA in their early eye gaze patterns to audiovisual speech (i.e., looking to the mouth and eyes of a talker)?

2. Do individual differences in looking to audiovisual speech indirectly influence concurrent expressive communication through parent-child engagement and prelinguistic vocal complexity? Does this indirect effect vary according to sibling status?

Methods

Design Overview

Recruitment and study procedures were carried out in accordance with the approval of the Vanderbilt University Medical Center Institutional Review Board. Parents provided written informed consent prior to their child’s participation in the study. All families were compensated for their participation.

Participants

Participants were 50 infants (28 Sibs-autism, 22 Sibs-NA; see Table 1 for sample characteristics) drawn from an ongoing longitudinal study (e.g., Bottema-Beutel, Kim, et al., 2019; Feldman et al., 2021). Participants made one to two visits to Vanderbilt University Medical Center over the course of a two-week period during which all study measures were collected.

Table 1.

Participant Demographics By Sibling Group

	Sibs-autism (n = 28) M (SD) Min-Max	Sibs-NA (n = 22) M (SD) Min-Max
Age in Months	11.86 (3.78) 6–18	12.73 (3.51) 6–18
Percent Male	53.6%	50.0%
MSEL ELC**	92.64 (12.19) 71–115	102.45 (9.85) 84–121
Race	n	n
White	28	20
Black/African-American	0	1
Multiple	0	1
Ethnicity	n	n
Hispanic/Latino	1	1
Not Hispanic/Latino	27	21
Primary Caregiver’s Highest Level of Education	n	n
High School Diploma or GED	2	0
College/Technical (1–2 Yrs)	10	3
College/Technical (3–4 Yrs)	8	7
Graduate/Professional School (1–2 Yrs)	5	5
Graduate/Professional School (3+ Yrs)	3	7

Note. MSEL ELC = Mullen Scales of Early Learning (Mullen, 1995) Early Learning Composite.

^**Groups differed at p < .01

Inclusion criteria were: (a) chronological age 6–18 months, (b) predominantly English-speaking household, and (c) an older sibling (30 months-18 years) who was either diagnosed with autism (Sibs-autism) or who had no autism diagnosis (Sibs-NA). Groups did not differ at the group level on chronological age or biological sex (p values > 0.5). The diagnostic status of older siblings with autism was confirmed by record review (in the cases wherein the older sibling had been diagnosed at Vanderbilt) or by an independent assessment that included detailed history, parent interview, and research-reliable administration of the Autism Diagnostic Observation Schedule, second edition (Lord et al., 2012). Diagnostic status of non-autistic older siblings was confirmed by a score below the screening threshold for autism concern on the Social Communication Questionnaire (Rutter et al., 2003) and no history of developmental concerns per parent report. Exclusion criteria for all participants included: (a) adverse neurological history, (b) known genetic condition, and (c) pre-term birth (gestation < 37 weeks). Communication assessments for the protocol were administered in English and, thus, families who did not speak English at home were excluded. The Mullen Scales of Early Learning (MSEL; Mullen, 1995) was administered to further characterize the sample.

Procedures

Looking to Audiovisual Speech

Eye gaze to a talker’s mouth, eyes, and broader face regions was measured while infants were exposed to a 50 s video of a monologue produced by a female actor speaking in an infant-directed manner in the infants’ native language (English). The infant-directed speech was characterized by a slowed rate, as well as exaggerated speech prosody and pitch/frequency contours. This video has been utilized in prior research (Dunham et al., 2020; Lewkowicz & Hansen-Tift, 2012). Participants were seated 60 – 80 cm in front of a 22 inch computer monitor connected to an SMI REDn SensoMotoric Instruments (SMI, Teltow, Germany) remote eye tracker running on a laptop computer at a sampling rate of 60 Hz. The eye tracker’s camera was connected to the bottom of the external computer monitor. We used SMI’s iViewRed software to control the eye tracker camera and to process the eye gaze data and SMI’s Experiment Center software to control stimulus presentation and data acquisition. We presented the video on the external monitor and the auditory stimuli through an external speaker placed behind the monitor and tracked the participants’ eye gaze. Prior to showing the video, we ran a 5-point calibration routine by showing a small star in the middle of the screen as well as in each corner of the screen.

Analysis of eye gaze data was completed using SMI’s BeGaze program. To determine how much time each infant spent gazing to the eyes and mouth regions of the stimulus, ovular areas of interest (AOIs) were imposed onto the talker’s mouth, eyes, and face in the video clip (see Figure 1). The AOI around the eyes was defined by a region that was demarcated on the top by a curved line slightly above the eyebrows and on the bottom by a curved line at the bridge of the nose. Left and right edges of the AOI were demarcated by curved lines at the edge of the hairline on either side of the speaker’s face. The AOI around the mouth was defined by a region that was delineated on the top by a curved line at the bottom of the nose and at the bottom by a curved line at the middle of the chin. Both the left and right edges of the mouth AOI were delineated by curved lines at the middle of the cheek on either side of the face. Then, an AOI was created encompassing the talker’s entire face. These AOIs were static (i.e., the AOIs did not change on a frame by frame basis).

Figure 1.

Image of audiovisual speech stimulus video with eyes, mouth, and face areas of interest (AOIs) depicted.

Using these AOIs, we computed the percent of total looking time to the talker’s eyes and mouth out of the total looking time to the talker’s face. We chose to use these ratio metrics because (a) they are very stable in children (Dunham et al., 2020), (b) they are consistent with ratio metrics used in many prior studies (e.g., Hillairet de Boisferon et al., 2018; Lewkowicz & Hansen-Tift, 2012; Pons et al., 2019; Wagner et al., 2013, 2018), and (c) they allow us to compare time spent looking to the eyes and mouth regions. Please note, however, that some previous studies have utilized other variables, such as an eye-mouth index (i.e., time spent looking at the eyes divided by total time looking at eyes and mouth combined; e.g., Elsabbagh et al., 2014; Young et al., 2009), to quantify gaze to eye versus mouth regions of a talker. Therefore, we also present analyses based on this index in Supplemental Material.

Parent-Child Engagement

Two types of supported joint engagement – LSJE and HSJE – were coded from two 15 minute semi-structured parent-child free play (PCFP) sessions. Procedures were identical to those used previously by Bottema-Beutel et al. (2014; Bottema-Beutel, Kim, et al., 2019). During the PCFP sampling sessions, participants and their parents played with a standard set of toys (e.g., a doll, a toy barn and animals, beaded necklaces); parents were instructed to play as they normally do at home using any of the available toys, while taking care not to play with their backs to the camera.

Following the recommendations from Bottema-Beutel, Kim, et al. (2019), the PCFP was collected twice for each participant to increase stability of derived metrics of supported joint engagement. Each sample was coded by a trained off-site coder naïve to sibling status and study hypotheses. Scores for HSJE and LSJE were averaged across the two coded samples for each participant. Reliability coding was completed on 20 (20%) of the PCFP samples. Intraclass correlation coefficients (ICCs) for HSJE and LSJE were .96 and .76, indicating excellent and good agreement, respectively.

Prelinguistic Vocal Complexity

Infants’ prelinguistic vocal complexity was measured using the Communication and Symbolic Behavior Scales Developmental Profile Behavior Sample (CSBS; Wetherby & Prizant, 2002). This standardized, play-based assessment was approximately 30–35 minutes in duration and was administered by a member of the research team that was trained to fidelity. The CSBS was used as a behavior sampling context and was coded for intentional communication acts using a 5-second partial interval coding system, as in Woynaroski et al. (2017). Intentional communication acts were defined as: (a) non-word vocal or unconventional gestural acts (e.g., moving an object toward an adult and giving an object to an adult) in combination with coordinated attention to object and person; (b) conventional gestures (e.g., distal pointing and head nodding) with attention to adult; and (c) symbolic forms (e.g., nonimitative words and sign language). The intervals coded for intentional communication acts were subsequently coded for the presence/absence of vocalizations including canonical syllables and for the number of different consonants produced by the child. Canonical syllables were defined as a vocalization that consisted of three components: (a) consonantal sound (e.g., /m/, /b/, /t/); (b) fully resonant vowel sound; and (c) transition from consonant to vowel that is rapid and uninterrupted, as judged by a human observer (e.g., “ba” or “ug”). The number of different consonants used within communication acts was coded according to Wetherby’s True Consonant Inventory List, which includes selected supraglottal consonants that emerge early and are relatively frequent and easy to code reliably (Wetherby & Prizant, 2002). The two metrics of vocal complexity that were derived from this communication sample were canonical syllabic communication (i.e., the proportion of intentional communication acts that included a canonical syllable) and consonant inventory (i.e., the total number of different consonants used in communication acts from Wetherby’s True Consonant Inventory List).

One of every five CSBS samples (10 total; 20%) were randomly selected and coded for interrater reliability. ICCs quantifying interrater reliability for canonical syllabic communication and consonant inventory were .96 and .76, indicating excellent and good reliability, respectively.

Expressive Communication

Expressive communication was assessed using three measures: the MSEL, a standardized test that assesses development in several domains, including expressive language, for children birth-68 months; the Vineland Adaptive Behavior Scales, second edition (VABS; Sparrow et al., 2005), a parent report measure that assesses adaptive function in several domains, including expressive communication for children and adults from birth; and the MacArthur-Bates Communicative Development Inventories: Words and Gestures (MCDI; Fenson et al., 2007), a parent report that assesses early-emerging vocabulary and broader language ability in infants. Expressive language age equivalency scores from the MSEL, expressive communication age equivalency scores from the VABS, and the expressive vocabulary raw scores from the MCDI were derived as indices of expressive communication.

Analytic Plan

First, all variables used in analyses were examined for skewness > |1| and kurtosis > |3| and transformed as necessary to ensure assumptions were met for parametric statistics. MCDI expressive vocabulary and HSJE were square root transformed to correct for positive skew. In keeping with current recommendations (see Enders, 2010), we then imputed missing data (ranging from 0 – 10% missingness across variables used in analyses) using the missForest package (Stekhoven & Bühlmann, 2012) in R (R Core Team, 2020).

When indices from measures purported to tap the same construct were sufficiently correlated, aggregate variables were generated to increase their stability and potential predictive validity (Rushton et al., 1983). Thus, aggregates of prelinguistic vocal complexity (from consonant inventory and canonical syllabic communication; r = 0.70) and expressive communication (from all three measures of expressive communication; r values ≥ 0.75) were generated.

To answer our first research question, a mixed model ANOVA with group (Sibs-autism vs Sibs-NA) as the between-subjects factor and AOI (mouth vs eyes) as the within-subjects factor was carried out to evaluate whether eye gaze patterns to audiovisual speech varied by group. To answer our second and third research questions, mediation analyses were carried out using PROCESS (Hayes, 2017). These models statistically tested whether relations between looking to the mouth and eyes and expressive communication was mediated by supported joint engagement (i.e., HSJE or LSJE, tested in separate models) and prelinguistic vocal complexity (for simple mediation models testing indirect effects of each putative mediator in isolation, see Supplementary Material). Cook’s D was utilized to monitor for the presence of undue influence (defined as Cook’s D ≥ 1).

Results

Differences in Eye Gaze Patterns

Results of the 2 (group) x 2 (AOI) ANOVA revealed a main effect of AOI, such that infants looked significantly more to the mouth region compared to the eye region across groups, F(1, 48) = 4.16, p = .047, ${\eta }_p^2$ = .08 (see Figure 2). There was not a significant effect of group, F(1, 48) = 0.31, p = .58, ${\eta }_p^2$ = .01, nor was there a significant interaction between group and AOI, F(1, 48) = 0.54, p = .47, ${\eta }_p^2$ = .01. There was, however, a high degree of heterogeneity in looking to audiovisual speech across groups. Variance was not significantly different between groups (p value for Levene’s test = .34 and .62 for looking to the mouth and eyes, respectively).

Figure 2.

Box plot for time spent looking to the mouth by sibling group. Across groups, infants looked more to the mouth than the eyes, F(1, 48) = 4.16, p = .047, ${\eta }_p^2$ = .08. The main effect of group and the interaction between group and area of interest were not significant. Sibs-autism = infant siblings of children diagnosed with autism. Sibs-NA = infant siblings of non-autistic, otherwise typically developing children.

Direct Associations Between Looking to the Mouth, Expressive Communication, and Putative Mediators

For correlations between all variables of interest across groups, see Table 2 (see Table S1 for correlations between all variables within groups). Although the correlation between looking to the mouth and expressive communication only trended towards significance (r = .27, p = .059), the putative mediators are significantly associated with both looking to the mouth (zero-order correlations = .30 and .44 for HSJE and prelinguistic communication, respectively; ps = .033 and .001) and expressive communication (zero-order correlations = .65 and .76, respectively; ps < .001). The lack of a direct association between looking to the mouth and expressive communication does not preclude us from assessing an indirect association (see Hayes, 2009).

Table 2.

Zero-Order Correlations Between Variables Used in Analyses Across Groups

Variable	1	2	3	4	5	6	7
1. Looking to the Mouth	—
2. Looking to the Eyes	−.81***	—
3. HSJE	.30*	−.19	—
4. LSJE	.24	−.33*	.26	—
5. Prelinguistic Vocal Complexity	.44**	−.22	.60***	−.01	—
6. Expressive Communication	.27	−.20	.65***	.13	.76***	—
7. MSEL ELC	.05	−.02	−.16	.25	−.15	−.07	—

Note. HSJE = Higher-order supported joint engagement, LSJE = Lower-order supported joint engagement, Prelinguistic Vocal Complexity = aggregate of canonical syllabic communication and consonant inventory from the Communication and Symbolic Behavior Scales Developmental Profile Behavior Sample (CSBS; Wetherby & Prizant, 2002), Expressive communication = aggregate of Mullen Scales of Early Learning (MSEL; Mullen, 1995) expressive language age equivalency score, Vineland Adaptive Behavior Scales, second edition (VABS; Sparrow et al., 2005) expressive communication age equivalency score, and the MacArthur-Bates Communicative Development Inventory: Words and Gestures (MCDI; Fenson et al., 2007) number of words the child says, ELC = Early Learning Composite.

^*p < .05

^**p < .01

^***p < .001

Indirect Associations Between Looking to the Mouth and Expressive Communication

The mediation model testing the indirect effect of looking at the mouth on expressive communication through HSJE and prelinguistic vocal complexity indicated that the relation between looking to the mouth during audiovisual speech and expressive communication was mediated by HSJE and prelinguistic vocal complexity (95% CI = [0.001, 0.009]), consistent with our hypothesis. All paths comprising this indirect effect were statistically significant, with moderate to large effect sizes (see Figure 3).

Figure 3.

Indirect effect of looking to the mouth on expressive communication through higher-order supported joint engagement (HSJE) and prelinguistic vocal complexity. a₁ = the relation between looking to the mouth and HSJE, not controlling for any other factors. a₂ = the relation between looking to the mouth and prelinguistic vocal complexity, not controlling for any other factors. b₁ = the relation between HSJE and expressive communication, controlling for looking to the mouth. b₂ = the relation prelinguistic vocal complexity and expressive communication, controlling for looking to the mouth. d₂₁ = the relation between HSJE and prelinguistic vocal complexity, controlling for looking to the mouth. c’ = the direct effect of looking to the mouth on expressive communication (i.e., the c’ path), controlling for HSJE and prelinguistic vocal complexity. Note that c’ is non-significant, meaning that the association between looking to the mouth and expressive communication is completely mediated by HSJE and prelinguistic vocal complexity. This indirect effect is not moderated by sibling group. All values are standardized coefficients. *p < .05, **p < .01, ***p < .001, ns = non-significant result.

The relation between looking to the mouth and expressive communication was not mediated by LSJE and prelinguistic vocal complexity (95% CI = [–0.004, 0.001]). Additionally, the relation between looking to the eyes and expressive communication was not mediated by HSJE and vocal complexity (95% CI = [–0.009, 0.000]) or LSJE and prelinguistic vocal complexity (95% CI = [-0.002, 0.005]). Thus, the mediated relation appears to be specific to the association between looking to the mouth and expressive communication as mediated by HSJE and prelinguistic vocal complexity.

None of the aforementioned models were moderated by sibling group. Additionally, the results of all mediation models (a) were consistent when controlling for the Mullen Early Learning Composite (IQ) and (b) did not vary according to chronological age.

Discussion

This study sought to examine whether eye gaze patterns, specifically looking to the mouth and eyes during audiovisual speech (a) differed between younger siblings of children with and without autism at 6–18 months of age and (b) indirectly influenced expressive communication via parent-child engagement and child prelinguistic vocal complexity. Additionally, we tested whether the aforementioned indirect effects varied according to sibling group. Results suggest that looking behavior in infancy, on average, does not differentiate infants at high versus general population-level likelihood for autism. Looking to the mouth of a talker, however, may be useful for explaining individual differences in expressive communication, through its influence on one aspect of parent-child engagement – HSJE – and child prelinguistic vocal complexity.

Failure to Find Between-Group Differences in Early Eye Gaze Patterns

Our null result for between-group differences in looking patterns during audiovisual speech was somewhat surprising. It is notable, however, that several studies which have focused on infants to date have failed to specifically find between-group differences in gaze to mouth and eye regions during audiovisual speech for infant siblings of children with autism in comparison to infant siblings of children without autism, particularly when using proportion metrics similar to those utilized in this study (i.e., proportion of time spent looking to the mouth to time spent looking to the face; e.g., Elsabbagh et al., 2014; Shic et al., 2014; Young et al., 2009). The past studies that have found differences in looking at the mouth versus the eyes during audiovisual speech have mostly focused on children with autism and non-autistic comparison groups (e.g., Chawarska et al., 2014; Hosozawa et al., 2012; Johnels et al., 2014; Nakano et al., 2010), though a small number of studies have found differences in early eye gaze patterns between infant sibling groups (e.g., differences in looking to social scenes, people, and faces, as well as differences in scanning inner versus outer features of faces during speech; Chawarska et al., 2012, 2013; Shic et al., 2014). Thus, although the finding of non-significant differences between groups did not support our a priori hypothesis, it is generally consistent with findings from previous studies of at-risk infants.

Looking to the Mouth, Not Eyes, Is Positively Associated with Putative Mediators

Using a brief, passive remote eye tracking task, we found moderately-sized positive associations between looking to the mouth during audiovisual speech and our two putative mediators. Specifically, looking to the mouth was positively associated with HSJE. This specific form of supported joint engagement, wherein parents support the child’s engagement and the child acknowledges the caregiver in the course of play, has been previously linked with language and communication development in young children with autism (Bottema-Beutel et al., 2014; Bottema-Beutel, Woynaroski, et al., 2019; Crandall et al., 2019). The present study provides increased support for parsing out this particular form of supported joint engagement from LSJE, for which associations in our study did not hold.

Additionally, looking to the mouth was moderately associated with the index of prelinguistic vocal complexity (comprised of canonical syllabic communication and consonant inventory) across the infants in our study sample. These results extend past work that has linked looking at the mouth to language and communication skill in infants at increased likelihood for autism, children with autism, and autistic adults (e.g., Chawarska et al., 2012; Klin et al., 2002; Norbury et al., 2009; Tenenbaum et al., 2014; Young et al., 2009) and provide empirical support for the view that increased looking to audiovisual speech cues is linked with qualitative changes in prelinguistic development that are observed early in life (Lewkowicz & Hansen-Tift, 2012).

Looking to the eyes was not positively associated with HSJE, prelinguistic vocal complexity, or expressive communication. Previous work has found evidence for concurrent associations in infants between looking to the eyes and communication, specifically positive correlations with concurrent and later social communication and negative correlations with later expressive communication in non-autistic infants (Pons et al., 2019) and in Sibs-autism and Sibs-NA (Wagner et al., 2018),though these studies collected their eye tracking data at a narrower window of time (i.e., within a 1 month age range; this study utilized a 12 month age range). Thus, additional work is needed to determine how looking to the eyes relates to various aspects of communication across the first and second year of life.

Looking to the Mouth Influences Expressive Communication Via Parent and Child Variables

Mediation analyses indicated that looking to the mouth influences expressive communication indirectly through its associations with supported joint engagement, specifically HSJE, and child prelinguistic vocal complexity. Although a direct link between looking to the mouth on concurrent expressive communication has been observed in several prior studies of children with autism (e.g., Chawarska et al., 2012; Habayeb et al., 2020; Klin et al., 2002; Norbury et al., 2009) and longitudinally in infants at increased likelihood for autism (e.g., Chawarska et al., 2012, 2013; Shic et al., 2014), it is notable that this indirect effect was “complete” in our study, meaning that the direct link between these variables was non-significant when controlling for HSJE and prelinguistic vocal complexity. Additionally, this indirect effect did not vary according to sibling group. These findings provide empirical support for the notion that parent-child engagement and prelinguistic vocal behavior are mechanisms by which early eye gaze may influence expressive communication development in infants, regardless of sibling status. The present study provides some support for our hypothesis that attention to a speaker’s face, specifically the speaker’s mouth, may signal developmental readiness for more HSJE, which subsequently scaffolds increased prelinguistic vocal complexity in the child and translates to gains in his/her broader expressive communication. To our knowledge, these findings are the first to statistically test and find evidence in support of this theoretical possibility.

Notably, the indirect relation does not hold in models that include (a) LSJE as a mediator or (b) looking to the eyes as the predictor. Additionally, the mediation models are still significant when a different looking metric is utilized (i.e., eye-mouth index; see Supplemental Materials) and when cognitive ability (i.e., Mullen Early Learning Composite) is included as a covariate. Thus, our hypothesis is strengthened by the specificity and robustness of these results.

Limitations and Future Directions

Our study is limited in several ways. First, the present investigation is limited by of our use of a concurrent correlational design. Though we were able to provide preliminary support for mediated associations between early eye gaze patterns and expressive communication, further research is needed to determine whether an early-emerging tendency to look to the mouth during audiovisual speech temporally precedes, or is causally related to, qualitative changes in expressive communication, through a cascade of changes in HSJE and child vocal complexity. Causality could be rigorously tested in the context of a clinical trial which attempts to intervene upon or directly target early eye gaze and systematically measures the effects of treatment on looking behavior and HSJE, as well as more distal prelinguistic and linguistic outcomes in infant siblings of children with autism.

Second, we cannot presently determine how the present findings apply to infant siblings of children with autism who do or do not go on to be diagnosed with autism. Nonetheless, our findings provide novel insights into the possible mechanisms underlying the replicated relation between looking to a talker’s mouth and communication. Our innovative approach, focused on the elucidation of predictors of skills within discrete developmental domains (e.g., continuously quantified communication skill), has the potential to facilitate early identification and preventative intervention for infants known to be at increased likelihood of autism and other communication delays and disorders.

Third, some recent evidence suggests that gaze patterns obtained during live interactions may not correlate with looking during prerecorded videos, at least in adolescents with autism (Grossman et al., 2019). Given that our stimuli were specifically designed to limit head movements to accommodate static AOIs, it is possible that our looking task may not represent looking behaviors during live interactions. Some technologies do exist that may provide increased ecological validity to eye gaze paradigms in infants at increased likelihood of autism (Edmunds et al., 2017). These methods should be leveraged to explore the convergent validity between virtual and in vivo measures of eye gaze in infants at high and low likelihood for autism in future work.

Finally, additional research is needed to ascertain the neural substrates underlying the replicated relations between eye gaze patterns and linguistic development in children at increased likelihood for, or diagnosed with, autism. Looking to a talker’s mouth provides infants with access to concurrent visual and acoustic articulatory cues whose dynamic properties are not only highly synchronized over time but also correspond in terms of variations in their intensity and prosody (Chandrasekaran et al., 2009). Access to such redundant audiovisual speech cues has been shown to increase the efficiency of speech processing as indexed by event-related potentials in non-autistic adults and children, as well as in children with autism (Knowland et al., 2014; van Wassenhove et al., 2005; Woynaroski et al., 2019). Ongoing work in our laboratory is exploring whether such brain-based measures of audiovisual speech processing may explain the associations that we have observed between looking behavior and expressive communication development in infancy.

Conclusion

This study provides preliminary support for the hypothesis that infants’ gaze patterns influence expressive communication via increased supported joint engagement which, in turn, scaffolds prelinguistic development (i.e., canonical syllables and for the number of different consonants produced by the child). This indirect effect is specific to the link between looking to the mouth during audiovisual speech and expressive communication as mediated by HSJE and prelinguistic communication. Additional research is needed to further characterize and increase confidence in the mediated relations observed, particularly longitudinally and with consideration to later diagnosis of autism, and to advance us towards application of these findings in clinical practice.