Abstract
Typically-developing human infants preferentially attend to biological motion within the first days of life1. This ability is highly conserved across species2,3 and is believed to be critical for filial attachment and for detection of predators4. The neural underpinnings of biological motion perception are overlapping with brain regions involved in perception of basic social signals such as facial expression and gaze direction5, and preferential attention to biological motion is seen as a precursor to the capacity for attributing intentions to others6. However, in a serendipitous observation7, we recently found that an infant with autism failed to recognize point-light displays of biological motion but was instead highly sensitive to the presence of a non-social, physical contingency that occurred within the stimuli by chance. This observation raised the hypothesis that perception of biological motion may be altered in children with autism from a very early age, with cascading consequences for both social development and for the lifelong impairments in social interaction that are a hallmark of autism spectrum disorders8. Here we show that two-year-olds with autism fail to orient towards point-light displays of biological motion, and that their viewing behavior when watching these point-light displays can be explained instead as a response to non-social, physical contingencies physical contingencies that are disregarded by control children. This observation has far-reaching implications for understanding the altered neurodevelopmental trajectory of brain specialization in autism9.
Preferential attention to biological motion is a fundamental mechanism facilitating adaptive interaction with other living beings. It is present throughout a wide range of species, from humans10,11 to monkeys12 to birds13. Developmentally, it can be found in newly-hatched chicks14 and in human infants as young as 2 days of age1. Recognition of biological motion remains intact in a variety of forms, from degraded presentations, through varying states of occlusion, and in cases when information-bearing components are reduced to their most minimal15,16. In addition, perception of biological motion can be preserved even when other types of motion perception are impaired, as in individuals with Williams syndrome17 (a condition noted for visuo-spatial deficits) and in patients suffering from circumscribed brain lesions18. Lastly, biological motion perceived through other sensory modalities as when listening to sounds of human motion19 evokes activity in the same areas of the brain that are typically responsive to visual presentations.
Collectively, these findings describe a mechanism that is evolutionarily well-conserved; developmentally early-emerging; highly robust in signal-detection (withstanding degradation on signalling and receiving sides); and redundantly represented via multiple sensory modalities. Each of these aspects suggests ready benefits for adaptive interaction with other living beings: following the movements of a conspecific; looking at others to entreat or avoid interaction; learning by imitation; or directing preferential attention to cues that build on biological motion (such as facial expression and gaze direction5).
Strikingly, many of those same behaviors have also been noted as deficits in children with autism: deficits in social interaction, diminished eye contact and reduced looking at others, problems with imitation, deficits in recognizing facial expressions, and difficulties following another’s gaze20. Autism is a lifelong, highly prevalent, and strongly genetic disorder defined by impairments in social and communicative functioning and by pronounced behavioral rigidities21. While the preponderance of evidence points to prenatal factors instantiated in infancy, knowledge of the first two years of life in autism remains largely limited to retrospective data and indirect observations20: because autism is rarely diagnosed before 18 months, relatively little is known about autism during the first 2 years of development.
In later life, much more is known about the consequences cognitive, social, and behavioural of having autism. Altered visual scanning, of both faces and social scenes22,23, as well as altered neural processing of social information, have been documented24,25. In school-age children with autism, perception of biological motion is impaired26, but the manner in which very young children with autism relate to biological motion in early life, during periods critical for brain development and before compensatory coping strategies are established, has not been previously studied.
In the current study, we sought to address two questions: Is preferential attention to biological motion altered in children with autism by the age of 2 years? And, if children with autism fail to orient towards biological motion, what other factors might guide their visual attention?
To answer these questions, we created 5 sets of point-light animations, counterbalanced for 10 total. The animations consisted of children’s games, such as playing “peek-a-boo” or “pat-a-cake”, and were created with live actors and motion capture technology (please see Supplementary Materials: Motion Capture Stimuli). The motion capture sessions included simultaneous audio recording. The experimental task was a preferential looking paradigm (Figure 1A and Supplementary Movie 1): a point-light animation of biological motion was presented on one half of a computer screen together with the audio soundtrack of the actor’s vocalizations. On the other half of the screen, the same animation was presented, but that point-light figure was inverted in orientation (shown upside-down) and played in reverse order (the frames of animated action played from the end of the sequence until its beginning). Only the one (forward) audio soundtrack was presented.
Inverted presentation disrupts perception of biological motion in young children27 and is processed by different neural circuits in infants as young as 8 months of age28. Also, by playing the inverted animation backwards, its relative levels of motion complexity, speed, and gestalt coherence were preserved, but its motion was not an exact mirror of the upright. Each animation lasted an average of 30.5 seconds. Order of presentation was randomized, and presentation of the upright figure was counterbalanced to appear on the left side of the screen as often as on the right.
Evidence for recognition and preferential attention to biological motion was measured by the child’s viewing patterns: increased looking towards the upright figure indicated preferential attention to biological motion1 and the perceptual matching of human voice with a mental template of human action8. Visual scanning was measured with eye-tracking equipment, with data collected at 60 Hz (Figure 1B–1D). (Please see full Methods in Supplementary Materials.)
With the written, informed consent of their parents or legal guardians, 76 children with mean chronological age of 2.05 (.62) participated. These children comprised 3 groups (please see Supplementary Table 1): 21 toddlers with autism spectrum disorders (ASD), 39 typically-developing toddlers (TD), and 16 developmentally-delayed but non-autistic toddlers (DD). Toddlers with autism were matched to the typically-developing toddlers on nonverbal mental age and chronological age, and matched to the developmentally-delayed, non-autistic toddlers on verbal mental age and chronological age. (Please see Supplementary Materials: Participant Characterization.)
While the typically-developing toddlers provide normative data, the developmentally-delayed but non-autistic children serve as controls against developmental confounds, assuring that findings are specific to autism rather than attributable to delays in cognitive development or language function.
Results are plotted in Figure 1E–F. When viewing point-light displays of human biological motion, two-year-olds with autism spectrum disorders are random in their looking patterns: 50.7% on the upright figure vs. 49.3% on the inverted (Figure 1E). In contrast, both control groups demonstrated significant preferential attention to the upright animations: 62.7% upright for the TD group, and 58.9% upright for the DD group (Figures 1F, 1G). Comparison across groups was significantly different (by one-way ANOVA [analysis of variance], F2,73 = 7.95, p < .001). In pairwise comparisons, looking by the ASD group differed significantly from that of each control group (p < .001 in comparison with the TD group, and p = .0185 relative to the DD group). The two control groups did not differ significantly from one another (p = .27). All data were normally distributed (all p > .4, k < .15, Lilliefors).
Results in Figure 1 are for 4 of the 5 types of animations presented. In earlier research7, a serendipitous observation led us to recognize that one of the animations contained a confound. While 4 animations presented only moving point lights with accompanying human voice, 1 animation included a different sound. The actor in that animation plays pat-a-cake (see Supplementary Movie 2), and the sound of clapping is heard at the same time that two point-lights the actor’s hands collide. The collision of point-lights and the resulting clapping sound create a causal physical contingency: rather than merely co-occurring (as with the speech sounds and movements in the other animations), the movements of the point-light hands in this case actually cause a noise to occur. In the earlier research, we found that a 15-month-old with autism was very sensitive to the occurrence of this clapping: her preferential looking went from random during other animations to 93.1% upright during the pat-a-cake animation7.
During the clapping, the causal physical contingency only exists on the upright side: the single audio track plays normally (forward), matching the upright movements, but the action of the inverted figure, playing in reverse, does not move in time to the clapping sounds.
When analyzed independently (Figure 2), the toddlers with ASD showed a significant preference for the upright clapping figure: 65.9% upright during the pat-a-cake animation. Looking towards the upright figure during this animation was significantly increased relative to other animations: 65.9% upright during pat-a-cake vs. only 50.7% in the other 4 animations, t20 = 2.43, p = .02. Behavior of the TD and DD groups was unchanged: they continued to give preferential attention to the upright figure: 58.6% upright during pat-a-cake vs. 62.7% in the other 4 animations for TD [t38 = .79, p = .44]; and 54.4% vs. 58.9% for DD [t15 = .66, p = .51]). Overall, on this animation, results for the 3 groups did not differ significantly (F2,73 = .67, p =.52). All data were normally distributed (all p > .36, k < .15, Lilliefors).
Following this observation, we questioned whether the presence of more subtle synchronies might have played an unanticipated role in the viewing of all animations: that is, whether visual scanning that had appeared random by the toddlers with ASD might actually be related to audiovisual synchronies less obvious than clapping.
To test this, we quantified levels of audiovisual synchrony (AVS) in all animations (Figure 3). In the pat-a-cake animation, when the point-light hands collide and a clapping sound occurs, an abrupt change in motion coincides with a large change in sound amplitude. We measured audiovisual synchrony in our stimuli to match this case: the synchronous occurrence of change in motion and change in sound29.
We measured change in motion by first measuring each point-light’s trajectory over time (Figure 3A). From each point-light’s trajectory, we calculated its velocity and then the magnitude of its change in velocity, |ΔV| (Figures 3B, 3C). This served as our measure of change in motion. To measure change in sound, we measured the audio amplitude of the soundtrack (its short term amplitude envelope) and then calculated its rate of change, |ΔA| (magnitude of ΔA) (Figure 3D). Level of audiovisual synchrony (AVS) of each point-light was then calculated as the product of change in velocity and change in sound amplitude (Figures 3E,3F). This measure of AVS was computed for all point-lights on both the upright and inverted sides. (More information is provided in Supplementary Movie 3 and in Supplementary Materials: Quantification of Audiovisual Synchrony).
By then summing the AVS signals of all point-lights over time, we generated cumulative maps of audiovisual synchrony for each animation (Figure 3H). From the cumulative maps, we calculated the difference between maximum audiovisual synchrony on the upright side and maximum synchrony on the inverted side (as a percentage difference to normalize across animations).
Across different animations, this measure of upright versus inverted synchrony then served as a prediction of which side of the animation would be preferentially attended if the viewing patterns of children were related to attention to audiovisual synchrony. The relationship between synchrony and preferential viewing was tested by regression (Figure 4). For the ASD group, preferential looking was significantly and strongly correlated with level of audiovisual synchrony, R2 = 0.90 and p = .01 (Figure 4A). In the TD and the DD groups, there was no significant correlation between viewing and audiovisual synchrony: R2 = 0.29 and R2 = 0.17, respectively (Figures 4B,4C). Correlation coefficients for the three groups were significantly different from one another (χ2 = 7.24, p <.05)30, with the r value of the ASD group differing from that of the TD group (z = 2.41, p <.05) as well as that of the DD group (z = −2.25, p <.05). The two control groups did not differ significantly. The pat-a-cake animation had greatest upright audiovisual synchrony. When we withheld that animation and reanalyzed, the correlation between preferential viewing and audiovisual synchrony remained significant for the ASD group (R2 = 0.95 and p = .018), but was still not significant for the other groups (R2 = 0.04 for TD and R2 = 0.001 for DD).
The results from this post-hoc quantification of audiovisual synchrony and preferential viewing indicated that the viewing patterns of toddlers with autism random relative to social content showed instead a marked reliance on audiovisual synchrony. This one measure accounted for 90% of the autism group’s variance in preferential viewing. In contrast, the looking patterns of typically-developing and of developmentally-delayed, non-autistic children showed no relationship with levels of audiovisual synchrony. The control children gave preferential attention to biological motion, disregarding audiovisual synchrony in favor of more socially relevant signals.
To test whether audiovisual synchrony could predict looking behavior in new animations, we designed a follow-up experiment (Figure 4D). We created two new types of animations with increased levels of audiovisual synchrony, filling the gap in synchrony signal strength of our original stimuli. We recruited 10 additional toddlers with ASD, characterized in the same manner and matched to the original ASD cohort (please see Supplementary Materials). We used our original results to build a predictive model for expected behavior, creating weighted binomial prediction intervals around the original regression line31, with specific predictions for each animation. The probability of both results falling within their respective prediction intervals is equal to the probability of obtaining a value in one interval multiplied by the probability of obtaining a value within the other (p = [.1674 – .0002] × [.0024 0)]).
Preferential viewing by this second cohort of toddlers with autism, watching new animations, fit the predictions based on audiovisual synchrony: their viewing on each animation followed the model, a result with chance likelihood of p = .0004.
Overall, these results indicate that a skill present in two-day-old, typically-developing infants1, as well as in chronologically-, nonverbally-, and verbally-matched control children (the TD and DD groups herein), is not functioning properly in children with autism at the age of two.
What are the implications of a disruption to such a basic and highly-conserved mechanism? One immediate implication of this finding concerns our understanding of another very basic behavior: how infants with autism look at the faces of other people. We recently found that in comparison with control children, 2-year-olds with autism look less at the eyes of others and attend instead to their mouths24. The present results suggest an explanation: Where on the face is there greatest audiovisual synchrony? These children’s sensitivity to synchrony in the present biological motion stimuli is consistent with fixating on the ongoing synchronies between lip motion and speech sound; and the lack of preferential attention towards biological motion is consistent with diminished attention to the eyes and diminished expertise in social action and interaction found in later life.
Developmentally, these results mark an important, early point along an alternate path of neural and behavioral specialization. While individual and species-specific genetics begin the development of mind and brain, that development over time is shaped critically by experience. For infants with autism, this would imply that genetic predispositions are likely to be exacerbated by experiences that are increasingly atypical. By two years of age, the data in this report show that these children are on a substantially different developmental course, having learned already from a world in which the physical contingencies of coincident light and sound are quantifiably more salient than the rich social information imparted by biological motion. Future investigations will benefit from studies, starting still earlier in life, of the developmental unfolding of such selective learning profiles. Exactly which signals are spontaneously attended to and which are missed, and the consequences thereof for structural and functional brain development, may shed light on the neurobiological anomalies that predispose these altered avenues of learning.
Methods Summary
Children were recruited through a federally-funded STAART Center (Studies to Advance Autism Research and Treatment, NIMH U54-MH66494) based in the Autism Program of the Yale Child Study Center, New Haven, CT. The research protocol was approved by the Human Investigations Committee of the Yale University School of Medicine, and families were free to withdraw from the study at any time. The children were shown counterbalanced presentations of each of 5 point-light biological motion animations (for a total of 10 presentations in the original experiment), and 2 additional animations (4 presentations in the follow-up experiment). [See Figure 1A and Supplementary Movies.] Preferential viewing in our design was a binary choice, upright vs. inverted. Visual scanning was measured with eye-tracking equipment (ISCAN, Inc., Woburn, MA). The equipment uses a dark pupil/corneal reflection technique with data collected at the rate of 60 Hz. Analysis of eye movements and coding of preferential fixation data were performed with software written in MATLAB.
Supplementary Material
Footnotes
Supplementary Information is linked to the online version of the paper at www.nature.com/nature. It includes full description of methods.
Author Contributions: A.K. and W.J. developed the initial idea and design of the study, interpreted data, wrote the final manuscript, and take full responsibility for the integrity of data and the accuracy of data analysis. A.K. supervised participant characterization. W.J. supervised all technical aspects of experimental procedure, data acquisition, and analysis. P.G. contributed to initial development of AVS methods and data analysis. D.L., with W.J. and G.R., developed the final AVS methods. G.R., with W.J. and A.K., helped develop new animations for the 2nd experiment. W.J. created the figures. A.K. and W.J. performed final revision of the manuscript for intellectual content.
Competing Interests Statement The authors declare that they have no competing financial interests.
References
- 1.Simion F, Regolin L, Bulf H. A predisposition for biological motion in the newborn baby. Proc Natl Acad Sci USA. 2008;105(2):809–13. doi: 10.1073/pnas.0707021105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Regolin L, Tommasi L, Vallortigara G. Visual perception of biological motion in newly hatched chicks as revealed by an imprinting procedure. Anim Cognit. 2000;3:53–60. doi: 10.1007/s10071-008-0198-4. [DOI] [PubMed] [Google Scholar]
- 3.Blake R. Cats perceive biological motion. Psychol Sci. 1993;4:54–57. [Google Scholar]
- 4.Johnson MH. Biological motion: a perceptual life detector? . Curr Biol. 2006;16(10):R376–7. doi: 10.1016/j.cub.2006.04.008. [DOI] [PubMed] [Google Scholar]
- 5.Pelphrey KA, Morris JP, Michelich CR, Allison T, McCarthy G. Functional anatomy of biological motion perception in posterior temporal cortex: an fMRI study of eye, mouth and hand movements. Cerebr Cortex. 2005;15 (12):1866–76. doi: 10.1093/cercor/bhi064. [DOI] [PubMed] [Google Scholar]
- 6.Frith CD, Frith U. Interacting minds: a biological basis. Science. 1999;286(5445):1692–1695. doi: 10.1126/science.286.5445.1692. [DOI] [PubMed] [Google Scholar]
- 7.Klin A, Jones W. Altered face scanning and impaired recognition of biological motion in a 15-month-old infant with autism . Dev Sci. 2008;11(1):40–6. doi: 10.1111/j.1467-7687.2007.00608.x. [DOI] [PubMed] [Google Scholar]
- 8.Klin A, Jones W, Schultz RT, Volkmar F. The enactive mind from actions to cognition: lessons from autism. Philos Transact Bio Sci. 2003;358:345–360. doi: 10.1098/rstb.2002.1202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Johnson M. Functional brain development in humans. Nat Rev Neurosci. 2001;2:475–483. doi: 10.1038/35081509. [DOI] [PubMed] [Google Scholar]
- 10.Johansson G. Visual perception of biological motion and a model for its analysis. Percept Psychophys. 1973;14:201–211. [Google Scholar]
- 11.Fox R, McDaniel C. The perception of biological motion by human infants. Science. 1982;218:486–487. doi: 10.1126/science.7123249. [DOI] [PubMed] [Google Scholar]
- 12.Oram MW, Perrett DI. Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. J Neurophysiol. 1996;76(1):109–29. doi: 10.1152/jn.1996.76.1.109. [DOI] [PubMed] [Google Scholar]
- 13.Omori E, Watanabe S. Discrimination of Johansson’s stimuli in pigeons. Int J Comp Psychol. 1996;9:92. [Google Scholar]
- 14.Vallortigara G, Regolin L, Marconato F. Visually inexperienced chicks exhibit spontaneous preference for biological motion patterns. Plos Biology. 2005;3 (7):e208. doi: 10.1371/journal.pbio.0030208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Thompson JC, Clarke M, Stewart T, Puce A. Configural processing of biological motion in human superior temporal sulcus. J Neurosci. 2005;25(39):9059–66. doi: 10.1523/JNEUROSCI.2129-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Neri P, Morrone MC, Burr DC. Seeing biological motion. Nature. 1998;395(6705):894–6. doi: 10.1038/27661. [DOI] [PubMed] [Google Scholar]
- 17.Jordan H, Reiss JE, Hoffman JE, Landau B. Intact perception of biological motion in the face of profound spatial deficits: Williams syndrome. Psychol Sci. 2002;13:162–167. doi: 10.1111/1467-9280.00429. [DOI] [PubMed] [Google Scholar]
- 18.Jokisch D, Troje NF, Koch B, Schwarz M, Daum I. Differential involvement of the cerebellum in biological and coherent motion perception. Eur J Neurosci. 2005;21(12):3439–46. doi: 10.1111/j.1460-9568.2005.04145.x. [DOI] [PubMed] [Google Scholar]
- 19.Bidet-Caulet A, Voisin J, Bertrand O, Folumpt P. Listening to a walking human activates the temporal biological motion area. NeuroImage. 2005;28(1):132–9. doi: 10.1016/j.neuroimage.2005.06.018. [DOI] [PubMed] [Google Scholar]
- 20.Chawarska K, Klin A, Volkmar FR, editors. Autism spectrum disorders in infants and toddlers: diagnosis, assessment and treatment. Guilford Press; New York, NY: 2008. [Google Scholar]
- 21.Volkmar FR, Lord C, Bailey A, Schultz RT, Klin A. Autism and pervasive developmental disorders. J Child Psychol Psychiatry. 2004;45:1–36. doi: 10.1046/j.0021-9630.2003.00317.x. [DOI] [PubMed] [Google Scholar]
- 22.Klin A, Jones W, Schultz R, Volkmar F, Cohen D. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Arch Gen Psychiatry. 2002;59(9):809–16. doi: 10.1001/archpsyc.59.9.809. [DOI] [PubMed] [Google Scholar]
- 23.Jones W, Carr K, Klin A. Absence of preferential looking to the eyes of approaching adults predicts level of social disability in 2-year-olds with autism spectrum disorder. Arch Gen Psych. 2008 doi: 10.1001/archpsyc.65.8.946. [DOI] [PubMed] [Google Scholar]
- 24.Schultz RT. Developmental deficits in social perception in autism: the role of the amygdala and fusiform face area. Int J Dev Neurosci. 2005;23(2–3):125–41. doi: 10.1016/j.ijdevneu.2004.12.012. [DOI] [PubMed] [Google Scholar]
- 25.Dalton KM, Nacewicz BM, Johnstone T, Schaefer HS, Gernsbacher MA, et al. Gaze fixation and the neural circuitry of face processing in autism. Nat Neurosci. 2005;8(4):519–26. doi: 10.1038/nn1421. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Blake R, Turner LM, Smoski MJ, Pozdol SL, Stone WL. Visual recognition of biological motion is impaired in children with autism. Psychol Sci. 2003;14(2):151–157. doi: 10.1111/1467-9280.01434. [DOI] [PubMed] [Google Scholar]
- 27.Pavlova M, Sokolov A. Orientation specificity in biological motion perception. Percept Psychophys. 2000;62:889–899. doi: 10.3758/bf03212075. [DOI] [PubMed] [Google Scholar]
- 28.Reid VM, Hoehl S, Striano T. The perception of biological motion by infants”: an event-related potential study. Neurosci Lett. 2006;395:211–214. doi: 10.1016/j.neulet.2005.10.080. [DOI] [PubMed] [Google Scholar]
- 29.Driver J, Spence C. Multisensory perception: beyond modularity and convergence. Curr Biol. 2000 Oct 19;10(20):R731–5. doi: 10.1016/s0960-9822(00)00740-5. [DOI] [PubMed] [Google Scholar]
- 30.Chen PY, Popovich PM. Correlation: Parametric and Nonparametric Measures. Thousand Oaks: Sage Publications; 2002. [Google Scholar]
- 31.Von Collani E, Drager K. Binomial Distribution Handbook for Scientists and Engineers. Boston: Birkhauser; 2001. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.