Identifying critical kinematic features of animate motion and contribution to animacy perception

Summary

Humans can distinguish flying birds from drones based solely on motion features when no image information is available. However, it remains unclear which motion features of animate motion induce our animacy perception. To address this, we first analyzed the differences in centroid motion between birds and drones, and discovered that birds exhibit greater acceleration, angular speed, and trajectory fluctuations. We further determined the order of their importance in evoking animacy perception was trajectory fluctuations, acceleration, and speed. More interestingly, people judge whether a moving object is alive using a feature-matching strategy, implying that animacy perception is induced in a key feature-triggered way rather than relying on the accumulation of evidence. Our findings not only shed light on the critical motion features that induce animacy perception and their relative contributions but also have important implications for developing target classification algorithms based on motion features.

Graphical abstract

Highlights

• •Inherent differences exist in centroid motion of birds (living objects) and drones
• •Birds have greater acceleration, angular speed, and trajectory fluctuations
• •The bird’s wingbeat frequency is reflected in the fluctuations of its flight path
• •Human animacy perception depends on the instant triggering of key motion features

Neuroscience; Cognitive neuroscience; Psychology

Introduction

Over millions of years of evolution, animals have developed specific motion patterns under the combined effects of gravitational and biomechanical forces. These patterns are difficult for non-living objects to mimic, at least with current technology. At the same time, humans and non-human animals have developed the ability to identify whether a moving object is animate or not, based on motion cues. This ability, known as animacy perception,^1,2,3 is crucial for survival (e.g., foraging and avoiding danger) and fundamental for social interactions (e.g., identifying social partners). It is assumed that animacy perception evolved under natural selection pressures and is determined by the statistical regularity of animal movement characteristics.⁴

The motion of any object can be characterized by two aspects: relative motion in the object-based reference system and common motion in the global reference system.⁵ Accordingly, Troje⁶ classified the motion of animals into two types: biological motion and animate motion. Biological motion usually refers to relative motion and intrinsic motion, which is the motion of body parts in the object-based reference system. In contrast, animate motion usually refers to common motion and extrinsic motion, that is the motion of the entire object across the whole space. Studies on biological motion usually used the actual motion stimuli generated by the point-light display technique proposed by Johansson,⁷ while studies on animate motion usually utilized synthetic paths instead of actual paths.⁸ Therefore, it remains unclear which are the features and statistical regularity of actual animate motion.

Using synthetic paths, previous studies have identified several important motion cues that are important in triggering animacy perception. These cues can be divided into two categories. One category is the interaction between moving objects, such as chasing,⁹ and helping and hindering.¹⁰ The other is the motion characteristics of a single object, such as self-propulsion, that is the ability to move from the rest without external force;^3,11 active resistance, that is actively reducing speed after being pushed;¹² and suddenly changing the direction of speed to avoid possible collisions.¹³ In addition, adult humans perceive an object moving downward as less animate than an object moving upward.⁴ Similarly, newly hatched chicks also prefer objects moving against gravity.¹⁴ Stewart (1982) thus proposed the “energy violation” hypothesis, which suggests that the moving object is perceived as animate when its motion would require the object to have a hidden energy resource.¹⁵ This resource enables objects to change the magnitude and direction of speed, and make movements that defy “Newtonian laws”. This hypothesis seems reasonable since most non-living objects (such as leaves) throughout the evolution lack internal energy source and cannot move on their own without external forces. However, with the advances in technology, some non-living objects (e.g., unmanned aerial vehicles and planes) have now gained internal energy source and can display similar motion features (including self-propulsion, active resistance, and suddenly changing the direction of speed) as living objects. Given these developments, it is crucial to explore whether people can still differentiate between animate and inanimate motion when both animate and inanimate objects possess energy sources. Tremoulet and Feldman¹¹ have refuted the “energy violation” hypothesis by showing that local orientation of the rectangle (i.e., the body-axis alignment), which did not correlate with the internal energy, also affected the strength of animacy perception. They also showed that the subjective ratings of animacy increased with the increase of changes in speed and moving direction. Gyulai further pointed out the moving object with a greater acceleration was more likely to be perceived as animate.¹⁶ These studies showed that the animacy perception depended on the kinematics of moving objects. However, these studies mainly utilized synthetic paths and typically defined the types of motion that trigger animacy perception as “animate motion”. This definition is not strict and is prone to ambiguity. In this study, we defined the motion pattern of living creatures as animate motion, which is an objective definition that is independent of the perception induced. In contrast, the animacy perception is a subjective definition that reflects our brain’s interpretation of target motion, regardless of whether the target itself is animate or not. Therefore, it remains unclear how animate motion (objective physical stimuli) relates to animacy perception (subjective definition).

Animacy perception emerges early in life.^17,18 Both human newborns and visually naive chicks show a preference for biological motion^19,20 and animate motion, such as objects that start to move from rest,^21,22 change speed,^23,24,25 and move in the direction of their main body-symmetry axis as if constrained by the bilateral body plan.^26,27 The inexperienced chicks can also distinguish the temporal contingency between the motion of agents²⁸ and use motion cues to select appropriate responses to avoid danger, fleeing from rapidly approaching objects and slowing down in response to sweeping objects.²⁹ Human infants at three months old can discriminate between chasing and random patterns, and prefer to attend to the former.^30,31 However, human newborns and infants with high risks for autism spectrum disorders showed differently to the biologically relevant motion stimuli.^24,32 Compared to low-risk children, high-risk children looked more at inanimate stimuli when they were born but more at animate stimuli in 4 months. In the same vein, the chick’s preference for biological motion would be impaired if they were exposed to chemicals that increase the risk of autism spectrum disorder in the fetal stage.^33,34,35 In addition, the early preferences for biologically relevant motion have a sensitive period.³⁶ The preferences of chicks disappear after two days of hatching.^37,38 The thyroid hormone T3 can regulate this time window, and it can restore the preference for changing speed in 3-day-old female chicks while inhibit the preference in day-1 chicks.³⁹ These suggest that there are innate brain mechanisms sensitive to moving animacy cues. However, it should be noted that the innate brain mechanisms can be modulated by developmental processes and individual experiences.^17,40 For example, using a natural swim path, researchers found that fish preferred intermittent bout-like motion (discontinuous jumps) with a specific interval that closely tracked their own swim bout frequency at each age, and preferred speed that closely tracked their own speed at each age.⁴¹ In addition, motor training improves the recognition of novel motor patterns.⁴² Furthermore, this animacy perception is stable across cultures, as there is no significant difference in the ability between German adults and Shuar adults.⁴³ Moreover, this ability is common across a wide range of species, including monkeys,⁴⁴ chicks,²⁵ dogs,⁴⁵ cats,^46,47 zebrafishes,⁴⁸ and even invertebrates such as spiders.⁴⁹ These features (innate, cross-cultural, and cross-species) indicate that animacy perception is based on some basic motion feature detection capabilities common to almost all living creatures.

In this study, we aimed to clarify the nature of animate motion and animacy perception, and the relationship between them using actual motion trajectories. We introduced a novel approach to build a dataset containing actual animate motion. Two types of objects were selected to represent animate and inanimate objects: birds and unmanned aerial vehicles (UAVs, also known as drones). Birds flying is one of most common animate motions in the natural environment. This type of motion also has an extremely simple animate form from a distance. Drones are the fair control sample to birds, since they have high behavioral similarities but are non-living objects. We obtained the animate motion by extracting the motion of the object’s centroid (i.e., the center of gravity) from the original videos. By comparing the flight dynamic characteristics of drones and birds, we can characterize the motion features of animate motion in the global reference system.

Moreover, using the actual motion as visual stimuli, we conducted four psychophysical experiments to figure out the relationship between animacy perception and animate motion, and how the brain identifies whether a moving object is animate or not. We proposed two hypotheses. One is the “evidence accumulation” hypothesis, which means that the brain continuously accumulates evidence related to animacy perception and then makes a judgment at the end. This predicates that the longer the accumulation time, the more evidence is accumulated and the more accurate the judgment is. The other is the “feature-matching” hypothesis, which means that there are motion templates of animate motion in the brain, and when the brain notices motion features that match the inner template, the brain determines the moving object is animate. This predicts that the brain can quickly make accurate judgments. To test which hypothesis is correct, we designed two response paradigms: a) delayed response, which requires participants to respond after the motion stop signal, and b) fast response, which requires participants to respond as soon as they determine the category of the moving object. If the accuracy of the delayed response is higher than that of the fast response, then the “evidence accumulation” hypothesis is correct; otherwise, the “feature-matching” hypothesis is correct. In addition, we compared the relative importance of different motion features to animacy perception by stepwise regression method.

Results

Constructing the centroid motion dataset

Thanks to the development of computer vision algorithms, we proposed a new method to build a dataset of actual object motion. This method consists of two steps: (1) record original videos of object motion using cameras with high frame rates without camera motion and zooming; (2) track the position of the object’s centroid using the visual tracking algorithm. This algorithm usually creates a bounding box around the silhouette of the silent moving object in each frame, and the center of the bounding box is defined as the centroid of the target. Therefore, we obtained the centroid’s motion from the original video in this way.

Using this method, we constructed a dataset containing actual animate and inanimate motion. Three kinds of birds (pigeons, canaries, and zebra finches) were chosen based on their different body size and flying characteristics. Three kinds of drones (DJI Phantom 4, Mavic Pro, and Tello) were chosen based on their comparable size to each kind of bird. To better simulate the flying characteristics of birds, an experienced UAV operator imitated the flying trajectories of each kind of bird with the appropriate drone (i.e., pigeon vs. DJI Phantom 4, canary vs. Mavic Pro, and zebra finch vs. Tello). For each kind of flying object, 30 video clips were randomly selected and edited using video editing software (Adobe Premiere Pro CC 2018) to eliminate scenes with obvious motion characteristics that distinguish birds from drones, such as taking off, landing, and hovering. Then, the centroid motion of each flying object in each video clip was extracted using visual tracking algorithms (Figure 1A). Finally, each video clip containing only the centroid motion, which was represented as a single white dot moving on a black background, was cut to 2 s to control the visual motion information provided by each visual stimulus.

Figure 1.

Constructing centroid motion dataset

(A) Centroid motion extraction. The target tracking algorithms created a bounding box around the silhouette of the flying object in each frame, as illustrated by the green rectangles. The center of each bounding box (red and blue dots) was used as the estimated centroid of the moving object.

(B) Two example trajectories to demonstrate the motion kinetic differences between animate (red) and inanimate (blue) flying objects. Each flying trajectory was actually a series of spatial locations over time, and the small black circle represented the first spatial location of each trajectory.

Because the raw trajectories of the objects contain high-frequency noise, such as irregular oscillations caused by the high-speed rotation of the propeller, abnormal positions, and labeling errors, we performed wavelet signal denoising⁵⁰ using the wdenoise function built into MATLAB (see the STAR Methods section for details) for each trajectory before further use. Two examples of the resulting trajectories are illustrated in Figure 1B. The red trajectory was derived from the motion of the animate flying object (bird: Zebra finch), and the blue trajectory was derived from the inanimate object (drone: Tello).

Characterizing centroid motion kinematics of animate motion

Theoretically, a flying object can be in any spatial location and move in any direction regardless of animateness. The motion features distinguishing animate from inanimate flying objects should directly relate to their power source and flying control differences. To identify kinematic features that can be used to define animate motion, we focused on three motion time-domain features of flying objects: speed (the rate of spatial location change over time), acceleration (the rate of speed change over time), and angular speed (the rate of moving direction changes over time). After computing these three kinematic features for each trajectory, we analyzed their differences in the robust central tendency (median) between birds and drones. We found that the medians of the speed ($W=7659,p<0.001$; Figure 2A), acceleration ($W=8099,p<0.001$; Figure 2B), and angular speed ($W=7948,p<0.001$; Figure 2C) were significantly higher for birds than for drones. It is obvious that we cannot reliably separate birds from drones based on their flying speeds. This is reasonable because, technically, the speed of drones could be matched to any type of bird if the data sample was large enough. However, due to essential differences in their flight dynamics, it would be harder for drones to fly as flexibly as birds, as reflected by their lower angular speed and acceleration. Actually, the flight control algorithm for drones tends to make their flight envelope as smooth as possible for the purposes of stability and safety.

Figure 2.

Characterizing motion time-domain features of flying animate and inanimate objects

(A–C) The medians of birds were significantly higher than those of drones. The dots represented individual data. ∗∗∗p < 0.001.

(D) Gaussian fitting results of average amplitude spectra of trajectory fluctuations. The trajectory fluctuation was defined as the difference between the denoised trajectory and the smoothed trajectory in both the horizontal and vertical directions. The smoothed trajectory was obtained by smoothing the denoised trajectory using the moving average filter with an appropriate time window. We first performed FFT on the trajectory fluctuations and averaged them across trajectories. For birds, the average amplitude spectra within specific frequency bands were then fitted using a Gaussian function, and the peak frequencies were marked on the corresponding peak location. The gray area around the average indicates the 95% confidence interval (see STAR Methods for details). The dashed line indicates the wingbeat frequency reported in previous studies.

In addition to the previously described three motion time-domain features, we also analyzed the motion frequency-domain features of trajectory fluctuations. As shown in Figure 1B, the flying trajectory of the animate objects is not as smooth as that of the inanimate objects. The small fluctuations in birds’ flight trajectories might be caused by wing flapping. To sustain flight, birds periodically flap their wings to generate lift force to support their weight and overcome drag on their wings and body,⁵¹ while drones are propeller driven. This results in undulatory flight trajectories (i.e., upward and downward oscillating flight trajectories) for birds, and relatively smooth trajectories for drones. To analyze the periodicity of fluctuations in the flight trajectories of birds, we first obtained the fluctuations by subtracting the smoothed trajectory from the raw trajectory. The smoothed trajectory, which represents the overall movement trend, was computed by smoothing the raw trajectory using the moving average filter with an appropriate time window. Then, we performed a fast Fourier transform on the horizontal and vertical trajectory fluctuations, respectively. The wingbeat frequency was determined by performing a Gaussian fit on the birds’ average amplitude spectrum within specific frequency bands (Figure 2D) (see the STAR Methods section for details). Amazingly, the calculated wingbeat frequency closely matched the number reported in previous studies^52,53 (the gray dash line in Figure 2D, Zebra finch, estimated 24.3 Hz, previously reported, 23.4–27.8 Hz; Canary, estimated 17.6 Hz; Pigeon, estimated, 8.4 Hz, previously reported, 5–6 Hz).

The dependency of animacy perception on features of animate motion

We previously characterized four centroid motion features of birds and drones. To find out how they determine animacy perception, we conducted a series of psychophysical experiments with 24 human subjects. None of the subjects had seen such centroid motion trajectories before and were not informed as to what kind of movements they represented. The experimental procedures were similar across all experiments and are illustrated in Figure 3A. In Experiments 1 and 2, the visual stimulus of each trial was a trajectory randomly selected from the dataset with additional smoothing manipulation. Using trajectory smoothing allowed us to focus on the contributions of speed and acceleration to animacy perception by eliminating trajectory fluctuations caused by wing flapping. The downside of this manipulation was that it reduced angular speed as well. In Experiments 3 and 4, we further included the raw trajectories in the stimuli pool. Participants were asked to report on whether the moving dot represented an animate or inanimate object by pressing the left or right arrow key, respectively (two-alternative forced choice). In Experiments 1 and 3, participants were required to respond after the trial-end signal (Figure 3A, delayed resp.). However, in Experiments 2 and 4, they were instructed to make a decision as quickly as possible (Figure 3A, fast resp.). By comparing the delayed and fast responses, we were able to test whether additional observation time, which allows the participant to accumulate more kinematic features, was beneficial to behavioral performance or not. To ensure that participants make a quick response as requested, we first analyzed the response time in Exp. 2 and Exp. 4. The overall response time consists of two components: the time taken to process sensory information and making a decision (known as the net response time) and the time required to generate and execute the motor command (known as the motor reaction time). To estimate the motor reaction time, we computed the median of response times in a reaction time test task, where participants needed to press the space key immediately when they saw a dot appear. This task consisted of 100 trials (see STAR Methods for details). Then, we computed the net response time by subtracting the motor reaction time from the overall response time. As shown in Figure 3B, the response time of the fast response is significantly shorter than the duration of the moving trajectory (i.e., 2000 ms) in both Exp. 2 ($t\left(23\right)=-16.50,p<0.001)$ and Exp. 4 $\left(t\left(23\right)=-25.13,p<0.001\right)$.

Figure 3.

The dependency of human animacy perception on centroid motion characteristics

(A) Experimental procedures. A white dot was first presented at the starting position of each trajectory for 500 ms to draw participants’ attention and then moved along the centroid’s trajectory. Observers were asked to determine the animacy of the moving dot in each trial by pressing keys after the dot stopped moving (Exps. 1 and 3) or pressing keys immediately after making decisions without waiting for the dot to stop moving (Exps. 2 and 4).

(B) The response time of each experiment. The color represented the response type, with blue indicating delayed response and green indicating fast response.

(C) The mean animacy probability of birds’ and drones’ trajectories in Exp. 1 and Exp. 2. The animacy probability here was defined as the percentage of trajectories perceived as animate.

(D) The overall accuracy of Exp. 1 and Exp. 2.

(E) The mean animacy probability of raw and smooth trajectories in Exp. 3.

(F) The overall accuracy of raw and smooth trajectories in Exp. 3.

The error bar represented the standard error of the mean (SEM), and the dots represented individual data. p values were Holm-corrected. ∗∗∗p < 0.001, ∗∗p < 0.01, ∗p < 0.05, n.s. means p > 0.05. See also Figure S1.

We asked each participant what kind of moving object they thought they saw after the formal experiments. Their guesses were diversely distributed and included swimming, running, and flying objects. However, we found that these participants were able to easily discriminate animate motion from inanimate motion based on only the smoothed centroid motion. We performed a 2 (object category: bird vs. drone) $\times$ 2 (response type: delayed vs. fast) repeated measures analysis of variance on animacy probability of Exp. 1 and Exp. 2. The animacy probability was defined as the percentage of trajectories perceived as animate in each class. We found that the main effect of object category was significant ($F\left(\mathrm{1,23}\right)=301.160,p<0.001,{\eta }_p^2=0.929$), and post hoc analysis showed that the animacy probability of birds’ trajectories was significantly higher than that of drones’ trajectories ($t\left(47\right)=17.354,p<0.001\text{;}$ Figure 3C, bars with red edges vs. bars with blue edges, respectively). The interaction effect between object category and response type was also significant ($F\left(\mathrm{1,23}\right)=10.314,p=0.004,{\eta }_p^2=0.310$). Post hoc analysis showed that for birds’ trajectories, the delayed response was significantly lower than that of the fast response ($t\left(23\right)=-2.411,p=0.041$; see Figure 3C, bars with red edges, filled with red vs. gray); while for drones’ trajectories, there was no difference between the delayed and fast response ($t\left(23\right)=1.4,p=0.169$; see Figure 3C, bars with blue edges, filled with blue vs. gray). We also compared the overall accuracy between Exp. 1 and Exp. 2, and found that the accuracy of the delayed response was significantly lower than that of the fast response ($t\left(23\right)=-3.212,p=0.004$). These results suggested that additional observation time might introduce some negative effects on behavioral performance. Therefore, according to our hypotheses, human observers adopt the “feature-matching” method instead of the “evidence accumulation” strategy, and they only utilized kinematic features within a small time-window, whose length might be limited by the working memory capacity, to reach a decision, instead of using evidence accumulated during the whole trial.

In Experiment 3, we further included the raw trajectories in the stimuli pool. We performed a 2 (object category: bird vs. drone) $\times$ 2 (smoothness: smooth vs. raw) repeated measures ANOVA on animacy probability. We found that the main effect of object category was significant ($F\left(\mathrm{1,23}\right)=460.463,p<0.001,{\eta }_p^2=0.952)$, and post hoc analysis showed that the animacy probability of birds’ trajectories was significantly higher than that of drones’ trajectories ($t\left(47\right)=21.458,p<0.001$). The main effect of smoothness was significant ($F\left(\mathrm{1,23}\right)=34.550,p<0.001,{\eta }_p^2=0.600)$, and post hoc analysis showed that the animacy probability of raw trajectories was significantly higher than that of smooth trajectories ($t\left(47\right)=5.878,p<0.001$). The interaction effect between object category and smoothness was also significant ($F\left(\mathrm{1,23}\right)=34.605,p<0.001,{\eta }_p^2=0.601)$. Post hoc analysis showed that for birds’ trajectories, the animacy probability based on raw trajectories was significantly higher than that based on smooth trajectories ($t\left(23\right)=8.313,p<0.001$; Figure 3E, bars with red edges, filled with red vs. filled with green); while for drones’ trajectories, there was no difference between the animacy probability based on raw trajectories and that based on smooth trajectories ($t\left(23\right)=0.205,p=0.838$; Figure 3E). This might be because trajectory smoothing reduced the fluctuations of raw trajectories, which has a greater impact on birds’ raw trajectories with larger fluctuations than drones’ raw trajectories with small fluctuations. We also compared the overall accuracy of smooth and raw trajectories, and found that the overall accuracy of raw trajectories was significantly higher than that of smooth trajectories ($t\left(23\right)=5.883,p<0.001$; Figure 3F). These results suggest that additional information on trajectory fluctuations (i.e., larger acceleration, larger angular speed, and long-term periodicity) significantly increased the accuracy rate in discriminating birds from drones. It appears that trajectory fluctuations are a critical kinematic feature for the animacy detection of flying objects. The results of animacy probability in Exp. 4 (see Figure S1) were comparable to those in Exp. 3 and thus were not included in the main text (see supplemental information for detailed information).

Next, to better quantify how motion features affect animacy perception, we computed the robust Spearman’s correlation coefficient between each motion feature and animacy perception of raw trajectories in Exp. 3 separately (Figures 4A–4D) (see the STAR Methods section for details). Unsurprisingly, significant correlations were found for all features. Last, to determine the hierarchy of importance of the kinematic features in animacy perception, we further analyzed the dependency of animacy perception on the medians of speed, acceleration, and angular speed by stepwise linear regression in Exp. 1, and on all three motion features and trajectory fluctuations in Exp. 3. In Exp. 1, both speed and acceleration contributed significantly to animacy perception and together accounted for 85.7% of the variability in animacy perception. To quantify the relative importance of each kinematic feature contributing to animacy perception, we computed the standard coefficient B (see the STAR Methods section for details). It showed that acceleration (B = 0.785, p < 0.001) had a more important role than speed (B = 0.158, p = 0.007). The angular speed was not significant in this model, perhaps because it was dramatically reduced by trajectory smoothing. In Exp. 3, all kinematic features except angular speed contributed significantly to animacy perception and together accounted for 92.1% of the variability in animacy perception. The trajectory fluctuations (B = 0.485, p < 0.001) play a more important role than acceleration (B = 0.31, p < 0.001) and speed (B = 0.232, p < 0.001). The effect of angular speed was also not significant, which might be because angular speed is positively correlated with trajectory fluctuations (r = 0.83, 95% CI [0.79, 0.87]). Actually, what the trajectory fluctuations reflect is the periodic changes in the moving direction, which can also be understood as the pattern of angular velocity changes.

Figure 4.

Dependence of animacy probability of raw trajectories on motion kinetic features in Exp. 3

(A–D) The animacy probability is positively correlated with speed, acceleration, angular speed, and trajectory fluctuations, respectively. The red circles represent bird trajectories, and the blue circles represent drone trajectories. The black lines represent the results of linear regression. The gray areas around the regression line indicate the 95% confidence interval (CI) for the regression line. The horizontal dashed line indicates 0.5 AP (animacy probability). Please note that because we were evaluating the correlation between the probability of being perceived as animate and motion kinetic features contained in each trial, the animacy probability here was defined as the percentage of participants who perceived the trajectory as animate.

Discussion

In the current study, we have introduced a novel approach to build a dataset of actual animate motion. Using this approach, we built a dataset containing animate (birds’) and inanimate (drones’) motion. Then, we systematically compared the difference between animate and inanimate motion. We found that the nature of animate motion may lie in changes in the magnitude and direction of speed, i.e., acceleration and angular speed. Furthermore, using the actual animate motion as visual stimuli, we conducted four human behavior experiments to investigate the nature of animacy perception. Overall, this study can improve our understanding of the nature of animate motion, animacy perception, and their relationship.

Centroid motion dataset

We first built a dataset that contains centroid motion trajectories of six kinds of flying objects using our newly proposed method. Motion trajectories can be obtained by tracking radar⁵⁴ and GPS trackers.⁵⁵ However, compared to them, the method using computer vision algorithms was more convenient and low cost, and helped obtain objects’ trajectories which were hard to place the device on, and also did not interfere with the original movement.^56,57 This method helps further investigation on animate motion and animacy perception.

The nature of animate motion

We conducted a systematic analysis to compare the differences in motion time-domain features between animate (birds’) and inanimate (drones’) motion. We found that compared to drones, birds have larger speed, acceleration, angular speed, and trajectory fluctuations. Although we found a significant difference in speed between birds and drones, it is important to note that this difference heavily depends on experimental settings, such as the types of drones and birds used. If we included more samples, drones could theoretically cover all the speed ranges of birds. However, in the present study, we were unable to ensure precise matching of the speed range between birds and drones due to several limitations. Firstly, the drone models and bird types that were commercially available were limited. Secondly, the shooting distance affected the speed data we recorded using the camera. The drones’ flight videos were shot outdoors because drones can fly over a wide area, while birds’ flight videos were recorded indoors since they would fly away when outside. Therefore, speed is not the major feature that distinguishes the motion of birds and drones. Instead, changes in speed magnitude (i.e., acceleration) and speed direction (i.e., angular speed) are less affected by the shooting distance, and they should be the essential kinematic features that distinguish between animate and inanimate motion.

One notable feature of flying birds is that they have better motion flexibility, which is the intrinsic property of body tissues that determines the range of motion achievable without injury at a joint or group of joints.⁵⁸ The power source of birds is unstable because they are powered by the repetitive contractions and extensions of hundreds of muscles. The force exerted by a muscle depends on the shortening rate and its recent history: there is more force after stretching and less force after shortening.⁵⁹ On the other hand, most drones are equipped with electrical motor-driven propellers, whose power source is a battery with a relatively fixed output.⁶⁰ Moreover, the flight control algorithm of drones generally keeps the changes in speed and moving direction as smoothly as possible to avoid accidents. Given these differences, the motion flexibility, which can be quantified by the acceleration and angular speed, should be a good indicator for discriminating between the centroid motion of birds and drones. Indeed, acceleration and angular speed are more important than speed for animals: a predator with a higher acceleration catches prey more easily even if its maximum speed is slower than that of its prey,⁶¹ while a prey with a higher angular speed can escape even if its speed is slower than that of its predator.⁶²

Another notable feature characterizing birds’ flying behavior is wing flapping. Wing flapping serves two purposes during flight: moving against gravity and in different directions. One direct consequence of wing flapping is repetitive fluctuations along the flight trajectory. We found that the fluctuation rates captured by our data analysis method perfectly matched birds’ previously reported wingbeat frequency.

In summary, the acceleration and angular speed might be the defining features of animate motion. The acceleration changes the magnitude of the speed, while the angular speed changes the direction of the speed. These two factors together caused the fluctuations in the trajectory. Assuming there is no acceleration or angular speed in the extreme case, the trajectory will be a straight line. Therefore, in other words, trajectory fluctuation is the defining feature of animate motion.

The nature of animacy perception

In the human behavior experiments, we found that the accuracy of fast response was higher than that of delayed response (Figure 3D), which suggests that the brain adopts the “feature-matching” method to determine whether a moving object is animate, instead of using “evidence accumulation” method. Next, by analyzing the relationship between motion features and animacy perception, we found that animacy perception is positively correlated with four motion features: speed, acceleration, angular speed, and trajectory fluctuations. The animacy probability is higher than 0.5 only when these motion features are higher than certain values. These results confirmed previous studies, which reported that animacy ratings increase with increased changes in speed and acceleration,¹¹ and both actual and illusory faster speeds trigger higher animacy perception.⁶³ Moreover, we ordered the relative contributions of these features to animacy perception, and found that fluctuations and acceleration played a more important role than speed. Since previous studies used synthetic paths, they only focused on one or two motion features. Using actual animate motion, we first demonstrated that some visual cues (e.g., acceleration) are more powerful than others (e.g., speed) when they are simultaneously presented. But we should notice that the weight of each motion feature is not fixed and is related to the properties of the moving object.

These results indicate that the brain is sensitive to acceleration and angular speed. We speculate that there might be specialized neurons and neural circuits in the brain to process them, such as acceleration-sensitive neurons and angular-speed-sensitive neurons. The acceleration-sensitive neurons obtain acceleration through time-delay speed integration, while the angular-speed-sensitive neurons obtain angular speed by integrating motion directions over time. These neurons and neural circuits are the foundations of animacy perception. In addition, using biological motion stimulus (i.e., point light walker), previous studies found that the vertical acceleration pattern of the feet is crucial in identifying animate motion.⁶⁴ To explain these phenomena, Troje and Westhoff (2006) proposed a “life motion detector” theory, which argues that the visual system might contain innate “life detectors” which are sensitive to the life motion signals, especially those from the feet.^65,66,67 The “life detectors” are defined by their functional capabilities, which are to detect the critical features of animate motion and biological motion. Since there are several key features of animate motion, such as acceleration and angular speed, there should be various types of “life detectors” in the brain, such as acceleration-sensitive detectors (i.e., acceleration-sensitive neurons) and angular-speed-sensitive detectors (i.e., angular-speed-sensitive neurons). In summary, these “life detectors” are primarily responsible for detecting features of animate motion and then triggering animacy perception, rather than representing animacy perception themselves.

Motion information is mainly processed by a series of brain areas along the dorsal visual pathway [for review see⁶⁸]. Neurons in the primary visual cortex V1/V2 are capable of extracting local motion features, including speed, motion direction, and spatiotemporal frequency. Neurons in the middle temporal area (MT/V5) integrate the inputs from V1/V2 and exhibit clear feature detection capability of coherent motion direction and speed selectivity of both simple and complex visual motion patterns. A single MT neuron might not be able to encode acceleration, but the pooling responses of a population of MT neurons could estimate visual acceleration.⁶⁹ Neurons in the medial superior temporal (MST) area receive major visual inputs from MT, are selective to complex optic flow patterns,⁷⁰ and are capable of encoding multiple biological motion features (unpublished results of our laboratory), which could also be a type of “life detector”. Furthermore, the intraparietal sulcus (IPS) receives inputs from both the MT and MST and is capable of classifying the stimuli into different categories.⁷¹ A neuroimaging study found that in the task to indicate whether a single moving dot appeared alive, hMT+/V5 was activated, and the activation in the right IPS was positively correlated with perceived animacy instead of animacy cues.⁷² They proposed that the right IPS might be involved in matching stimuli to stored representations of animate motion. In addition, the animacy perception might be represented by the social behavior network (for reviews, see^17,18). Visually naive chicks showed increased activity in the social behavior network of the brain increased when chicks met alive conspecifics.⁷³ Specifically, the septum and preoptic area showed higher activity when chicks were exposed to biological motion⁷⁴ and dynamic features of animate motion (such as changing speed),⁷⁵ while the nucleus taeniae of the amygdala were more active to the static features of typical animate appearance.⁷⁶ Overall, a series of brain areas along the dorsal visual pathway are important in detecting the kinematic features of animate motion, while the social brain network may be involved in representing animacy perception.

To better understand animacy perception, we propose a two-stage processing model of animacy perception, including a sensory information processing stage and a feature-matching stage. In the sensory information processing stage, the motion-sensitive neurons or neural circuits extract various motion features of the moving object. These neurons exhibit greater sensitivity to certain values of motion features (such as gravitational acceleration, and acceleration of animate motion) than to uncommon values. The extracted motion features are then transmitted to higher brain areas to determine the animacy of the object. In the subsequent feature-matching stage, the brain compares the similarity between the extracted motion features of the object with the intrinsic template of animate motion. We assume that there is a multi-dimensional feature space in the brain, and each dimension represents a motion feature, such as acceleration and angular speed. The intrinsic template is a specific cluster in the feature space, which is formed by the distributions of motion features of animate motion. If the motion features of one object are within or close to the cluster, the brain categorizes the object as animate. The template might be innate and can also be influenced by post-natal experience.^21,25,41,42 Overall, this model provides a framework for understanding how the brain processes the information of moving object and determines its animacy.

Limitations of the study

This study also has some limitations. Firstly, this study only compared the motion differences between birds and drones, which may not be representative of all animate and inanimate objects. For example, if we compared the motion of a terrestrial animal (e.g., dog) vs. a robot moving on land (e.g., AIBO), we may find other distinguishing motion features, which are important for animacy perception. Secondly, the study only analyzed the kinematic features of centroid movement to identify the most basic features that defined animate motion. While this is a good point, it is important to note that the relative motion pattern of different body parts of animals should also contain critical information. The difference in the relative motion pattern between animate and inanimate objects is also an inherent feature caused by the fundamental difference in drive mode (muscle vs. motor) and gravity adaptation. Therefore, future studies can investigate the motion of a diverse range of animate and inanimate objects and use more advanced techniques to analyze the relative motion pattern of different body parts to obtain a comprehensive understanding of the motion differences.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Deposited data
Trajectory data	This paper	https://doi.org/10.17632/gr6yp4fnb8.1
Behavioral data	This paper	https://doi.org/10.17632/gr6yp4fnb8.1
Software and algorithms
MATLAB	Mathworks	https://www.mathworks.com/products/matlab.html
NIMH MonkeyLogic	Hwang et al.77	https://monkeylogic.nimh.nih.gov/
JASP	The JASP Team	https://jasp-stats.org/
ECO	Danelljan et al.78	https://github.com/martin-danelljan/ECO
DIMP	Bhat et al.79	https://github.com/visionml/pytracking#dimp
skipped_correlation	Pernet et al.80	https://sourceforge.net/projects/robustcorrtool/

Resource availability

Lead contact

Further information and requests for resources and reagents should be directed to and will be fulfilled by the lead contact, Tao Zhang (taozhang@psych.ac.cn).

Materials availability

This study did not generate new unique reagents.

Experimental model and study participant details

Birds and drones used in video shooting

We used three kinds of birds, including six pigeons (body weight, 194–347 g; body length, 29.5–30 cm), one canary (15–20 g; 12.5 cm), and two zebra finches (7–12 g; 10–11 cm), which were bought from a bird and flower market. We also used three kinds of drones, including one DJI Phantom 4 (weight, 1380 g; symmetrical motor wheelbase, 35 cm), one Mavic Pro (734 g; 33.5 cm), and one Tello (80 g; 13.48 cm). Flight videos of each type of bird were recorded separately. The canary is a very active bird, and we were able to collect enough video data with only one bird. However, zebra finches and pigeons are less active due to captivity, so we need to use two or more birds to capture enough video data for analysis. These birds were given to suitable families for adoption after the experiment.

Subjects in the behaviour experiments

Twenty-nine Chinese college students were recruited. The data of 5 participants were removed due to using the opposite decision strategy from the other participants in judging slow motion as animate and fast motion as insnimate, or exhibiting a particular bias by judging all motion trajectories as animate. The remaining twenty-four students (12 females) aged between 18 and 30 years (M ± SD, 22.71 ± 3.03 years) participated in all four experiments. All participants were right-handed with normal or corrected-to-normal vision and provided informed written consent before the experiments. They were all naïve to the research purposes. The procedures and protocols used in this study were approved by the institutional review board of the Institute of Psychology of the Chinese Academy of Sciences.

Method details

Filming equipment and environment

The video recording device we used was a digital camera (SONY DSC-RX100M5A) that was kept steady on a tripod during shooting. The automatic zooming function was turned off to avoid accidental background movement. To capture high speed movements in good detail, the camera was set to 120 frames per second with a resolution of 1920x1080 pixels per frame. The flying activities of two small size birds (zebra finches and canary) were recorded using a customized filming box (height x length x width: 3x2x1 meters). The inner surface of the filming box was covered by a green curtain except for the transparent front, which was made of organic glass (3x2 meters). The flying activities of pigeons were recorded inside a stadium (height x length x width: 5x18x11 meters). The video clips of flying Tello, Mavic Pro, and Phantom 4 were taken inside an empty parking lot on a breezeless day. We employed an experienced UAV operator to imitate the flying trajectories of birds by observing their motion beforehand. The operator used a hand-operated joystick to control the drones, which made it difficult to precisely control the acceleration rate and frequency. Since the joystick was usually pushed to the bottom, the drones' flying trajectories we obtained should reflect the maximum acceleration and angular speed of drones, though they may not reflect the maximum velocity of drones. Then, for each type of flying object, thirty video clips were randomly selected and edited using video editing software (Adobe Premiere Pro CC 2018) to eliminate scenes with obvious motion characteristics that distinguish birds from drones, such as taking off, landing and hovering.

Behavioural recording techniques

Experimental stimuli and procedures were generated and displayed using MATLAB 2018b and NIMH MonkeyLogic⁷⁷ on Display++ (1920×1080 screen resolution, 120 Hz refresh rate). During the experiment, participants put their heads on the chin rest to keep a viewing distance of 58 cm from the screen.

Stimuli and procedure

In the behavioral experiments, the denoised and corresponding smoothed trajectories from the dataset were utilized. The smoothed trajectories were obtained by smoothing the denoised trajectories using the moving average filter with appropriate window sizes (zebra finch and Tello, 7; canary and Mavic Pro, 9; pigeon and Phantom4, 15). All trajectories were downscaled equally to half of their original size to be presented within the center of the screen.

Each trial began with a white fixation cross ([255, 255, 255], 0.3 ° × 0.3 °) presented at the center of the screen for 300∼500 ms. Then, a white dot ([255, 255, 255], 0.3 ° × 0.3 °) was presented at the starting position of a trajectory for 300 ms to draw participants' attention to that location. Next, the white dot moved along the centroid's trajectory and turned black at the final position of the trajectory. The dot's position was refreshed at 120 Hz. In Experiments 1 and 2, the trajectories were randomly selected from the smoothed trajectory dataset, while in Experiments 3 and 4, they were randomly selected from both the denoised and the smoothed trajectories dataset. Participants were required to identify the category of the moving object (animate vs. inanimate) by pressing the left or right arrow keys, respectively. In Experiments 1 and 3, participants were required to respond within 2000 ms after the dot stopped moving, while in Experiments 2 and 4, participants were instructed to respond as soon as they identified the category of the object, and if they did not respond before the dot stopped moving, they were required to respond within 1000 ms. If participants did not respond in the required time, the trajectory would randomly present again in the remaining trials. The intertrial interval (ITI) was 2000 ms.

In Exp.1 and Exp.2, we presented participants 180 smooth trajectories (30 trajectories $\times$ 6 object types). In Exp.3 and Exp.4, we presented participants 180 raw and 180 smooth trajectories. The order of the presentation of the trajectories was randomized. Participants completed 12 practice trials before the formal experiment. They performed the four experiments in the same order on the same day: Exp.1 (smooth stimuli with delayed response), Exp.2 (smooth stimuli with fast response), Exp.3 (raw and smooth stimuli with delayed response), and Exp.4 (raw and smooth stimuli with fast response). Participants took one minute break after every 60 trials in each experiment and rested for a longer period time between each experiment to ensure they were energetic. No feedback was given during the experiments.

Before performing the animacy judgment task, participants first completed a visual reaction time test task. In each trial, a white fixation cross ([255, 255, 255], 0.3 ° × 0.3 °) was presented at the center of the screen for 300∼500 ms, and then a white dot ([255, 255, 255], 0.3 ° × 0.3 °) was presented at the same location for 500 ms. Participants were required to press the space key immediately when they saw the dot. This trial was repeated 100 times. The median of the response time was computed as the robust estimation of the visual reaction time, which was then used to compute the net response time.

Quantification and statistical analysis

Trajectory pre-processing

To extract the location of the flying object in each frame of the video, we adopted ECO⁷⁸ (for pigeons) or DIMP⁷⁹ (for drones and the two other kinds of birds) algorithms in open-source code provided by the authors' GitHub. The choice of algorithm was evaluated by the experimenter frame-by-frame and was based on the centroid estimation performance of each object. To remove noise from raw trajectories, we first performed the wavelet signal denoising method using the wdenoise function built into MATLAB. The function used an empirical Bayesian method with a Cauchy prior, and the default wavelet was 'sym4'. Various parameters and options of the function were tested, and the small variations of the data did not affect the overall results and conclusions, which suggests that the denoising method is robust.

Motion feature analysis

We computed three motion time-domain features of flying objects: speed, acceleration, and angular speed. The calculation methods of speed ($sp$) and acceleration ($a_t$) are shown in Equations 1 and 2, respectively. The angular speed was defined as the rate of moving direction changes over time, and its detailed calculation method is shown in Equations 3, 4, 5, and 6. $P_t$, $V_t$, and $r_t$ represent the position vector, velocity vector, and angular speed at time t, respectively. $Sign\left(V_{t+1},V_t\right)$ was used to determine the direction of rotation (i.e., clockwise or counterclockwise) and was computed using the direction of the cross product between $V_{t+1}$ and $V_t$. Additionally, to fix oscillation around the π angle, we iteratively normalized $r_t$ using the procedure proposed by Vlachos et al.,⁸¹ which repeated Equation 4 n times or until stability was achieved. In our experiments, we iterated the normalization step 10 times.

$$\begin{array}{c}sp=\frac{\sqrt{{\left(y_{t+1}-y_t\right)}^2+{\left(x_{t+1}-x_t\right)}^2}}{dt}\end{array}$$ (Equation 1)

$$\begin{array}{c}{a}_t=\frac{s{p}_{t+1}-s{p}_t}{dt}\end{array}$$ (Equation 2)

$$\begin{array}{c}{V}_t=\frac{P_{t+1}-P_t}{dt}\end{array}$$ (Equation 3)

$$\begin{array}{c}{r}_t=\frac{sign\left(V_{t+1},V_t\right)·{\mathit{cos}}^{-1}\left(\frac{V_{t+1}·V_t}{\Vert V_{t+1}\Vert ·\Vert V_t\Vert }\right)}{dt}\end{array}$$ (Equation 4)

$$\begin{array}{c}sign\left(V_{t+1},V_t\right)=\{\begin{array}{c}1\left(\left(\left[V_{t+1}\times V_t\right]·{\left[001\right]}^T\right)>0\right)\\ -1\left(otherwise\right)\end{array}\end{array}$$ (Equation 5)

$$\begin{array}{c}{r}_t=r_t-\overline{r}\end{array}$$ (Equation 6)

Then, to reveal the difference in the statistical features of motion features between birds and drones, we performed the independent sample T-test and the nonparametric Mann–Whitney test if the Shapiro–Wilk test suggested deviations from normality.

For each type of flying object, we also analysed the trajectory fluctuations, which were defined as the difference between the denoised trajectory and the smoothed trajectory. The smoothed trajectory was obtained by smoothing the denoised trajectory using the moving average filter with window sizes varying from 3 to 19 with a step size of 2. Then, we performed the fast Fourier transform on the trajectory fluctuations and averaged them across thirty trajectories. For each type of bird, the appropriate window size was the one where a relatively large amplitude of peak frequency was observed. We then performed a Gaussian fit on the average amplitude spectrum of the specific frequency band (see Equation 7 (the actual frequency was obtained from previous research). Therefore, the frequency band was 8–42 Hz for zebra finches, 5.76–30.24 Hz for canaries, and 2.56–13.44 Hz for pigeons. The window size for each type of drone was set to be equal to that of a bird of corresponding size (DJI Phantom 4 for pigeon, Mavic Pro for canary, and Tello for zebra finch).

$$\begin{array}{c}\mathrm{F}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}\mathrm{b}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{s}=\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}\pm 0.68\times \mathrm{a}\mathrm{c}\mathrm{t}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}\end{array}$$ (Equation 7)

Statistical analysis of behaviour results

For the animacy probability of Experiments 1 and 2, we performed a 2 (object category: birds vs. drones) × 2 (response type: delayed (Exp. 1) vs. fast (Exp. 2)) repeated measures ANOVA. For the animacy probability of Experiments 3 and 4, we performed a 2 (object category: birds vs. drones) × 2 (smoothness: raw vs. smoothed) repeated measures ANOVA. Normality for all datasets was tested using the Shapiro–Wilk test. If the normality assumption was violated, we performed a nonparametric test, including a paired samples Wilcoxon signed-rank test and a one-sample Wilcoxon signed-rank test. The p values of post hoc tests were Holm adjusted to correct for Type I error.

The trajectory fluctuation was defined as the area between raw trajectories and smoothed trajectories, and was the sum of small areas determined by two adjacent frames. Since these areas ranged on a large scale across trajectories, they were then logarithmically transferred and added by ten. To determine the relationship between the animacy probability and motion features, we computed the robust Spearman's correlation using the skipped_correlation function in the MATLAB robust correlation toolbox.⁸⁰ This function first identified bivariate outliers by computing all the distances to the center of the bivariate cloud and then checked whether the distances were greater than the given distance.^82,83 This function also computes percentile 95% CIs for each correlation by the bootstrap method (B = 1000).

To further order the relative importance of motion features on animacy probability, we performed stepwise regression using JASP. All statistical tests were two-tailed. After the stepwise regression analysis, to detect the potential collinearity between the motion features in the animacy perception model, the variance inflation factor (VIF) analysis was performed. A feature whose VIF was larger than 10 was considered to have high collinearity with other features and was removed from the main model. The multivariate normality assumption was also tested by investigating whether the residuals of the regression model in a normal quantile–quantile plot lie on a diagonal line. In addition, to identify extreme cases that exerted undue influence on the model, we performed casewise diagnostics and removed outliers whose standardized residuals were larger than 3.

Published: August 17, 2023

Data and code availability

• •Data including trajectories and human behavior experiments have been deposited at Mendely Data and are publicly available as of the date of publication. DOIs are listed in the key resources table.
• •All original code has been deposited at Mendely Data and is publicly available as of the date of publication. DOIs are listed in the key resources table.
• •Any additional information required to reanalyze the data reported in this work is available from the lead contact upon reasonable request.