Life Detection From Biological Motion

Abstract

Life motion, the active movements of people and other animals, contains a wealth of information that is potentially accessible to the visual system of an observer. Biological-motion point-light displays have been widely used to study both the information contained in life motion stimuli and the visual mechanisms that make use of it. Biological motion conveys motion-mediated dynamic shape, which in turn can be used for identification and recognition of the agent, but it also contains local visual invariants that humans and other animals use as a general detection system that signals the presence of other agents in the visual environment. Here, we review recent research on behavioral, neurophysiological, and genetic aspects of this life-detection system and discuss its functional significance in the light of earlier hypotheses.

In 2006, one of us authored an article titled “The Inversion Effect in Biological Motion Perception: Evidence for a ‘Life Detector’?” (Troje & Westhoff, 2006). This article reported an experiment in which scrambled biological-motion point-light displays elicited an unexpected perceptual response when observers looked at them right side up—a response that disappeared, also unexpectedly, when they were shown upside down.

Biological-motion point-light displays are visual stimuli that consist of an array of moving dots, typically between 10 and 20, that describe trajectories as if attached to the body of a moving person. They were introduced by Gunnar Johansson (1973) and fascinated researchers studying visual perception for their ability to generate the inescapable percept of a moving person in action. The phenomenon demonstrated the power and automaticity of perceptual organization. Particularly when the dots were placed on joint locations, such as shoulders, elbows, or wrists, the observer seemed to fill in the lines connecting the dots, perceiving the whole display more like an articulated stick figure rather than a cloud of individual dots (see Figs. 1a and 1b).

Fig. 1.

Stimuli and results from Troje and Westhoff (2006). The intact biological-motion stimulus used in this experiment (a) consisted of 11 dots moving as if attached to the major joints of a stationary walker as seen from the side. The dotted lines shown were not part of the stimulus. They are used here to illustrate the perception of the articulated dynamic shape that participants experience. The middle row illustrates the trajectories of the individual dots in (b) the upright stimulus with coherent biological motion, (c) a scrambled stimulus (in which the local motion of the dots was unchanged, but the locations of the trajectories were randomly displaced within the display area), (d) the inverted stimulus with coherent biological motion, and (e) an inverted scrambled stimulus (in which the trajectories of the dots were flipped upside down and the locations of the trajectories were randomly chosen). On each trial, participants were asked to report whether the stimulus shown was facing to the right or the left. The graph (f) shows the rate of correct responses for the four types of stimuli, illustrating that participants can discriminate facing direction even from scrambled biological-motion displays, but only if they are presented upright. Error bars indicate standard errors of the mean.

In scrambled biological motion, the local motion of the individual dots remains intact, but the locations of the individual trajectories are randomly displaced within the display area (e.g., Fig. 1c). This manipulation disrupts the percept of an articulated figure. Scrambled biological motion was (and still is) a popular control stimulus in biological-motion research because it provides the same local motion as coherent biological motion without encoding global shape in any way.

Inversion, that is, turning the display upside down by flipping the intact, nonscrambled point-light display about a horizontal axis (Fig. 1d), also results in impaired recognition of biological motion (Sumi, 1984). The phenomenon is reminiscent of the face-inversion effect, which has been associated with processing of the configuration of facial features rather than the characterization of the features themselves (Diamond & Carey, 1986; Maurer et al., 2002).

The fact that the inversion effect is also observed with biological-motion perception seemed to further confirm the association between configural processing and stimulus orientation. However, the results obtained in the 2006 experiment did not fit that picture (Troje & Westhoff, 2006). In that experiment, the point-light displays consisted of dots that followed the main joints of a walking human body seen from the side. Overall translation had been subtracted such that the person remained stationary as if walking on a treadmill. Participants were asked to indicate whether the person was facing left or right. Performance was at a high level even though the displays were masked with an array of 100 additional flickering dots, but it dropped significantly when the stimuli were inverted. Similar behavior had been shown before (Bertenthal & Pinto, 1994) and was thus not surprising.

The first new observation was that participants still performed reasonably well if the intact point-light walker was replaced with a scrambled version. Participants no longer perceived the point-light cloud as originating from a human walker, but they were nevertheless able to assign a facing direction to it. The second new observation was that when this scrambled display was flipped upside down (Fig. 1e), participants were no longer able to discern a facing direction (Fig. 1f). The pattern of results was the same when the human point-light walkers were replaced with biological motion obtained from cats and pigeons.

Random scrambling of point-light displays entirely disrupts configural information. Therefore, the inversion effect observed with scrambled biological motion cannot be attributed to the orientation specificity of configural processing observed in face recognition. What else could possibly drive the inversion effect? And which stimulus feature, if not motion-mediated articulated shape, still informed observers about the facing direction of the upright, scrambled walker?

A second experiment (still Troje & Westhoff, 2006) revealed that the critical information about direction was carried by the dots representing the feet of the walker. Provided that the foot trajectories were present and displayed in their original orientation, observers could still indicate the direction in which the unscrambled walker faced. If these foot trajectories were displayed upside down, participants responded randomly. The critical feature that provided facing direction was demonstrated to be the “rolling wheel” cue (Chang & Troje, 2009a): If the dots representing the feet moved clockwise along their trajectory, observers reported rightward locomotion, and if they moved counterclockwise, observers reported leftward motion. However, this rolling-wheel cue was employed only if the trajectory also contained vertical motion that accelerated and decelerated in ways compatible with the orientation of gravitational forces (Chang & Troje, 2009a). Only downward acceleration, but not upward acceleration, as occurs after stimulus inversion, validates the rolling-wheel cue. Subsequently, we showed that the location of the foot trajectories within the scrambled display provides additional validation: The rolling-wheel cue is more salient when these dots are in the lower part of the display, which means that the information contained in the local motion of the feet is not entirely independent of the global configuration of the display (Hirai, Chang, et al., 2011). However, it remains helpful to distinguish between information contained in the local motion of individual features, mainly the feet, from information contained in the relative motion within the array of dots (i.e., information about global configuration).

Evidence for an Early Life-Detection System?

Why are people sensitive to the facing direction of scrambled biological motion? Why did such a curious ability evolve? Our initial answer to this question (Chang & Troje, 2009b; Troje, 2008) was motivated by earlier work on face recognition in infants and imprinting behavior in chickens. Morton and Johnson (1991) suggested that early development of face recognition in humans is guided by an innate system that consists of a coarse visual filter that responds to yet unspecified human faces and triggers the tendency to orient and attend to such stimuli immediately after birth. They termed this mechanism CONSPEC and contrasted it with a second mechanism, CONLEARN, that provides the prerequisites to learn and then fine-tune more sophisticated processing by which the mature human visual system can extract information from human faces. CONLEARN requires functional CONSPEC, which guarantees a constant supply of the required training data. The role of CONSPEC might change later in development. For instance, Senju and Johnson (2009) suggested that it is responsible for the immediate, visceral response to eye contact in humans.

Is it possible that the sensitivity of the human visual system to the footfall of terrestrially locomoting animals serves as a quick visual alert system that provides notification about the presence of another living being? Such a “life detector” may guide attention during visual development toward a stimulus class that requires individual learning and eventually lead to the maturation of a critically important tool for social perception later in life. The same system may continue to work in the mature visual system as a general alert system signaling the presence of potentially interesting or dangerous conspecifics and other large animals in the visual environment.

Several findings following the 2006 experiment seemed to point in the direction of such a life detector. We mention them here only briefly (for a more extensive review, see Troje & Chang, 2013). We showed that the ability to perceive direction and its dependence on foot motion also correlated with perceived animacy (Chang & Troje, 2008). Other researchers demonstrated that sensitivity to the local motion of a footfall is present already in newborn infants (Simion et al., 2008) and is likely evolutionarily old, as it is also evident in other vertebrates (Vallortigara & Regolin, 2006). Also, human peripheral vision responds sensitively to footfalls (B. Thompson et al., 2007), which efficiently guide overt attention (Hirai, Saunders, & Troje, 2011). Finally, in contrast to processing of motion-mediated shape, processing of the motion of a footfall is not susceptible to learning (Chang & Troje, 2009b).

Since the publication of our review (Troje & Chang, 2013), several additional lines of research have contributed to consolidate understanding of the distinction between local-motion processing and more global integration processes in the perception of biological motion. As we discuss later, the final picture is not entirely consistent, opens new questions, and indicates the need for more research.

Biological Motion in the Brain

In the past two decades, researchers have learned a great deal about how the primate brain responds to biological motion. Early neurophysiology work revealed that neurons in the macaque superior temporal cortex respond selectively to body movements. These neurons, located in a polysensory, anterior segment of the superior temporal sulcus, were posited to arrive at their biological-motion responses by integrating form-sensitive responses in the inferotemporal cortex and motion data in the middle temporal cortex. Subsequent human neuroimaging work (e.g., Grossman & Blake, 1999) has brought an understanding that biological-motion perception is governed by a rather broad collection of areas, including regions in the posterior lateral-occipital cortex, the posterior superior temporal sulcus, the posterior inferotemporal sulcus, extrastriate and fusiform body areas, and portions of the frontal and parietal cortex. Responses of the superior temporal sulcus, in particular, have received emphasis because (a) its anatomical positioning fits well with theoretical models (Giese & Poggio, 2003), (b) the topographical manner in which the region responds depends on the type of motion shown (e.g., hand vs. face motion), and (c) it has been implicated in the perception of social, intentional actions (Adolphs, 2009; Yokoyama et al., 2021).

In our recent imaging work, we sought to unravel responses to motion-mediated shape versus local biological kinematics, carefully isolating the two by presenting stimuli that preserved solely global configuration or local kinematics or contained neither shape nor local kinematics (Chang et al., 2018, 2021). Curiously, we found that both shape and kinematics elicited responses in a relatively old part of the forebrain, a thalamic area called the ventral lateral nucleus, in addition to responses in many of the same cortical areas reported previously to be responsive to biological motion (J. C. Thompson & Baccus, 2012). The ventral lateral nucleus is considered a major relay for motor planning and coordination. Implication of this thalamic motor area in processing of biological motion fits well with previous work implicating another motor region, the cerebellum (Ferrari et al., 2022; Sokolov et al., 2012). The cerebellum has also been increasingly implicated in social cognition (Cattaneo et al., 2022), and cerebellar dysfunction has been associated with deficits in social and affective cognition (Schmahmann & Sherman, 1998). The posterior cerebellum has strong functional connections with cortical regions conventionally associated with the social brain (Adolphs, 2009), including regions previously associated with action mirroring (posterior superior temporal sulcus, inferior frontal gyrus, pre- and postcentral gyri) and person or event mentalization (medial prefrontal cortex, temporal parietal junction, posterior cingulate; Van Overwalle et al., 2015). Before the local-motion aspects of biological motion became apparent and a subject of research, Jokisch and colleagues (2005), using stimuli that addressed structural aspects of biological motion, found that lesions in the cerebellum did not affect performance on a biological-motion task that was sensitive only to motion-mediated shape.

The exact roles of the cerebellum, motor thalamus, and central motor system more broadly in the perception of social stimuli are still unclear, though current hypotheses posit that the cerebellum predicts current and future states of the body using stored models of motion patterns that typically occur in specific contexts—a necessary tool given the delay with which proprioceptive data regarding the body’s state return to the central nervous system. This line of thinking extends the well-established conclusion that the cerebellum plays a role during action execution.

The emerging brain data appear to make a lot of sense considering our earlier work regarding scenarios under which the rolling-wheel cue is validated (Hirai, Chang, et al., 2011). Action sequences ought to be contextualized not only by the immediate environment (e.g., the presence of other objects or agents) but also by the anatomical and biomechanical constraints they convey (e.g., in the case of body shape) and by Newton’s laws of motion (in the case of kinematics). The role of the motor circuitry in action perception, then, could be to weigh shape and motion information in accordance with its conformity with anatomy and physics. These weighted signals could then be served to polysensory integration centers to achieve a representation of the perceived action. According to classic social-brain hierarchies (Adolphs, 2009), these polysensory representations would proceed to inform emotional and motivational responses (e.g., in the amygdala, orbitofrontal cortex, and even neuroendocrine systems), which go on to modulate responses in additional memory-, attention-, and reasoning-relevant areas (e.g., hippocampus, cingulate, prefrontal cortex). It seems now clear that the social-brain hierarchy needs an update, however. As has been shown in other contexts (e.g., Rizzolatti & Craighero, 2004), the motor system is not merely an end point in the production of effector responses following the complete assessment of a social stimulus, but also plays a role much earlier in weighing the relevance of the stimulus features (i.e., form and motion data) en route to the formation of the perceptual representation.

Nature Versus Nurture

The idea that quick detection of locomoting terrestrial organisms large enough to become relevant as prey, predators, or social partners, on the one hand, and more careful analysis of the particular nature of detected creatures, on the other hand, are served by two distinct processes is supported by a study that investigated the heritability of the abilities to process local biological motion and motion-mediated articulated shape (Wang et al., 2018). Both abilities vary across individuals. Wang and colleagues (2018) compared correlations between these abilities among monozygotic (identical) and dizygotic (nonidentical) twins, which allowed them to quantify the relative contribution of environmental and genetic factors to these abilities. Results showed that whereas almost 50% of the interindividual variance in local-motion processing is genetically determined, variance for global shape processing is affected to a large degree by environmental factors, and other nonspecified factors, but not by genetic predispositions. This finding supports the results of earlier research on newborn infants that also pointed to an innate ability to process local biological motion (Simion et al., 2008).

However, this picture is not unchallenged. The perceptual salience of biological motion in healthy adults and the question about the innateness of underlying processing mechanisms makes biological motion an attractive tool for studying visual recovery in patients who were born with cataracts that rendered them blind for part of their lives but then underwent restorative surgery to remove the cataracts and implant artificial lenses. Photoreceptors and neural connectivity in the retina are intact in these patients. Nevertheless, at least in patients who receive restorative surgery only after infancy, visual functions such as the detection of coherent motion, stereovision, and face recognition remain weak even several years after sight restoration. Apparently, the lack of vision during sensitive periods in early visual development has lasting effects. If biological-motion processing is dependent on an early sensitive phase during which the local life-detection mechanism ensures the visual input required for subsequent learning, biological-motion perception should also remain affected after extended congenital visual deprivation.

However, that seems not to be the case. Biological-motion perception seems to recover fully in patients following surgery for congenital cataracts (Bottari et al., 2015; Rajendran et al., 2020). The patients in these studies were tested many years after they recovered their sight, so it is conceivable that, during the time between surgery and testing, they went through the process that, according to our initial hypothesis, occurs in infancy: That is, the local-motion detector in these patients could have guided attention and secured visual input for subsequent learning processes. However, the results could also mean that orientation responses driven by local biological motion are not mandatory at all for the development of functional processing of life motion. The only available case study in which biological motion was presented to a congenital-cataract patient right after sight-restoration surgery showed that at least some aspects of recognition of point-light displays are already functional immediately after surgery (Ben-Ami et al., 2022).

Conclusion

Since first reported in 2006, the observation that human observers perceive direction and animacy from upright scrambled biological motion has been replicated multiple times. There is ample evidence that this pattern reflects a visual invariant that is orientation dependent (because it assumes gravitational acceleration) and that it directs attention toward conspecifics and other mid-sized locomoting terrestrial animals. The visual mechanism that detects it seems to be innate in humans, and given that it has also been found in birds, it is likely to be shared by other vertebrates, too. The fact that sensitivity to this invariant is heritable and that older parts of the brain are involved in processing and conveying the signal also points to a mechanism that is highly conserved and evolutionarily old.

Whether this local-motion detector is required to develop the more sophisticated forms of life-motion recognition that characterize the visually competent adult brain is not as clear. Most of the results discussed in this review do not clearly contradict the view that detection of local life motion is a necessary prerequisite for the development of a more general global-motion-mediated recognition system—a proposition made by Morton and Johnson (1991) for human face recognition. However, the results we have discussed also do not provide unambiguous evidence for that proposal. The two mechanisms may instead independently fulfill complementary functions. The local-motion mechanism might be used to guide attention to visual stimuli that are then scrutinized to derive further information from motion-mediated global dynamic shape. But that in itself does not mean that the former system is necessary to develop the latter one, although we suggested that was the case in earlier work (e.g., Chang & Troje, 2009b; Troje, 2008).

The idea that the relation between the two mechanisms is similar to the relation between CONSPEC and CONLEARN in face recognition remains attractive, but still must be considered hypothetical and calls for more explicit verification.

Recommended Reading

Rajendran, S. S., Bottari, D., Shareef, I., Pitchaimuthu, K., Sourav, S., Troje, N. F., Kekunnaya, R., & Röder, B. (2020). (See References). Provides evidence that the development of general motion perception requires visual input during an early sensitive period, as least some aspects of biological motion (BM) can develop without that early visual experience.

Troje, N. F., & Chang, D. H. F. (2013). (See References). Reviews the role of local-motion signals in biological-motion patterns and elaborates on early findings, which are only briefly touched upon in the current review.

Troje, N. F., & Westhoff, C. (2006). (See References). Reports the first study providing evidence that the inversion effect in biological-motion perception is also present when biological motion is scrambled, challenging the idea that biological-motion inversion effect is due to configural processing.

Wang, Y., Wang, L., Xu, Q., Liu, D., Chen, L., Troje, N. F., Sheng, H., & Jian, Y. (2018). (See References). Reports an elegantly designed twin study showing that life detection through local motion is a heritable ability and that genetics explains interindividual differences in shape processing only to a much lesser extent.

Transparency

Action Editor: Marlene Behrmann

Editor: Robert L. Goldstone