Attributing social meaning to animated shapes: A new experimental study of apparent behavior

Abstract

In 1944, Heider and Simmel reported that observers could perceive simple animated geometric shapes as characters with emotions, intentions, and other social attributes. This work has been cited over 3000 times and has had wide and ongoing influence on the study of social cognition and social intelligence. However, many researchers in this area have continued to use the original Heider and Simmel black-and-white video. We asked whether the original findings could be reproduced 75 years later by creating 32 new colored animated shape videos designed to depict various social plots and testing whether they can evoke similar spontaneous social attributions. Participants (N = 66) viewed our videos and were asked to write narratives which we coded for indicia of different types of social attributions. Consistent with Heider and Simmel, we found that participants spontaneously attributed social meaning to the videos. We observed that responses to our videos were also similar to responses to the original video reported by Klin (2000), despite being only 13–23 s and portraying a broader range of social plots. Participants varied in how many social attributions they made in response, and the videos varied in how much they elicited such responses. Our set of animated shape videos is freely available online for all researchers to use and forms the basis of a multiple-choice assessment of social intelligence (Brown et al., 2019).

Nearly every social interaction we engage in relies on the ability to understand and reason about the thoughts, emotions, and intentions of other people. The ability to interpret others’ mental states allows us to navigate the complex world of social behavior by making sense of others’ behaviors, interpreting subtle social cues, and making inferences about what other people may be thinking. This set of skills has been referred to as social intelligence (Marlowe, 1986), mentalizing (Frith & Frith, 2003), or Theory of Mind (ToM; Premack & Woodruff, 1978). Its importance has been illustrated in many areas, including studies on social cooperativeness (Paal & Bereczkei, 2007), size of one’s social network (Stiller & Dunbar, 2007), and leadership (Zaccaro, Green, Dubrow, & Kolze, 2018), just to name a few. Despite these indications, much of the research to date has focused on understanding impairments among individuals with clinical diagnoses such as autism spectrum disorder (ASD) and schizophrenia (Chung, Barch, & Strube, 2014; Sprong, Schothorst, Vos, Hox, & Van Engeland, 2007), with few studies focusing on samples of typically functioning adults.

One challenge of research in social intelligence among neurotypical adults is invoking mental state attributions in a suitable way for this population. Traditional methods include the use of written narratives (e.g., Happé, 1994; Stone, Baron-Cohen, & Knight, 1998) and images of facial expressions (e.g., Baron-Cohen, Wheelwright, Hill, Raste, & Plumb, 2001). Not only were many of these tasks designed for use with specific populations (e.g., adults with ASD), but they also may potentially confound social intelligence with verbal reasoning or general intelligence (Greig, Bryson, & Bell, 2004; Olderbak, Semmler, & Doebler, 2019). These methods may also include culturally biased stimuli which could lead to differences in task performance (Dodell-Feder, Ressler, & Germine, 2020).

One relatively underutilized method of modeling social interactions and mental state attributions is the use of animated shape videos as stimuli. This paradigm, based on the classic work of Heider and Simmel (1944), relies on individuals’ ability to make social inferences and judgments from inanimate stimuli, and is a convenient way to elicit social attribution while avoiding some of the limitations of other methods. Not only do these animated tasks typically require less reading or verbal ability, but they have been reported to display little to no cross-cultural differences (Lee et al., 2018). This paper presents a new class of animated shape videos, inspired by the original methods used by Heider and Simmel. Rather than directly replicating the original Heider and Simmel animation, we set out to design a new set of shorter, standardized animations that involve a broader range of social situations and emotions. These new videos were designed to be generally suitable for a variety of social intelligence studies in a variety of research settings and participant populations.

Eliciting Social Attributions using Animated Shapes: Heider And Simmel (1944)

In their studies on social attribution, Heider and Simmel (1944) created a 2.5<H>-min long silent animation which included simple geometric shapes (two triangles and one circle) moving around on a screen. The shapes were not explicitly defined as human figures, nor did they physically resemble humans in any way. Yet when college students viewed the animation and were asked to summarize the events, all but one of the 34 participants in the study (Experiment I) described the video in human-like terms. Many of the scenes were interpreted similarly by participants, suggesting that certain movements by the shapes in the video signaled social attribution in a predictable manner. In another condition of the study (Experiment III), Heider and Simmel played the same animation in a reverse sequence. Again, nearly all participants (42 of 44) described the actions in the video in human-like terms. However, the interpretations showed much more variation, likely because the reverse animation lacks definite organization and participants struggled to construct a social plot consistent with the movements of the shapes.

Heider and Simmel’s (1944) work had several important implications. It demonstrated that even simple geometric shapes can be interpreted as human figures when the shapes seem to be moving on their own accord (shapes not moving on their own are more likely to be interpreted as inanimate objects, e.g. a door or house). Moreover, once a shape is perceived as human, there is a tendency to attribute mental states to the shape and to use these mental states to predict its behavior. According to Heider and Simmel, “as soon as we ascribe a certain movement to a figural unit and consider this unit as an animated being, perception of motive or need is involved” (p. 257). Additionally, their work determined that the cues for ascribing mental states and predicting behavior in a figure are derived from the larger social plot and interactions with other figures, rather than the shapes’ movement patterns in isolation. Heider and Simmel explain that “we cannot read off the motivation from the movements themselves … movements of the other actors, or of preceding or succeeding events, offer cues for the determination of motives” (p. 258). Viewers make sense of the shapes’ mental states by analyzing the preceding events and the shapes’ interactions with other figures. In this way, animated shape videos can be used to depict social plots and human interactions, presenting a unique way of eliciting and studying social attribution under challenging conditions, namely the absence of many typical cues like facial expressions, body language, speech, and tone of voice.

Recent Applications Of Heider & Simmel’s Methods

Many researchers have continued to use the original Heider and Simmel (H&S) video to study social attributions in various fields. Klin (2000) used the H&S animation to evaluate social attributions as an indicator of ToM capacities in a clinical population of individuals diagnosed with high functioning autism and Asperger’s syndrome. Participants watched the video, which had been broken into several short segments, and were asked to provide a series of narratives describing the video’s plot. To quantify the level of social attribution in these narratives, Klin developed a coding system which consisted of several index scores focusing on different aspects of participants’ social and cognitive responses. Index scores measured, among other things, participants’ overall capacity for attributing social meaning to the stimuli, frequency of use of mental state terms, and ability to identify salient social features in the animation. Klin found that participants with autism or Asperger’s were less likely than the typically functioning control group to attribute social meaning to the animations. The clinical participants utilized fewer mental state terms in their narratives and identified fewer social elements in the plot. Klin called the measure he developed the Social Attribution Task (SAT), and it has since been used in follow-up studies with individuals diagnosed with ASD (Klin & Jones, 2006) and schizophrenia (Bell, Fiszdon, Greig, & Wexler, 2010). More recently, researchers have developed a multiple-choice version of the SAT (SAT-MC) which provides scores that correlate positively with measures of adaptive functioning (Burger-Caplan, Saulnier, Jones, & Klin, 2016) and other ToM tasks (Altschuler et al., 2018).

In addition, other researchers have designed their own animated shape tasks to study social attribution in clinical populations (Abell, Happé, & Frith, 2000) and in social neuroscience research (Martin & Weisberg, 2003). In particular, Abell and colleagues (2000) developed a novel set of animated shape videos representing three types of motion: random movement (purposeless movement and no interaction between the shapes), goal-directed action (a shape responds to the physical action or behavior of one another), and ToM (a shape reacts to the other’s mental state). Participants were shown the animations and asked to provide descriptions, which were then rated based on accuracy and type of description. Typically functioning children and adults were accurate in their explanations of the three types of videos. However, children with autism had greater difficulty describing the ToM videos, using mentalizing descriptions less often and referring to mental states that were inappropriate to the animation. Since then, this set (the Frith-Happé animations) has been used to develop a short test of mentalizing ability (White, Coniston, Rogers, & Frith, 2011) as well as to generally observe mentalizing ability among different populations (Corden, Critchley, Skuse, & Dolan, 2006; Moriguchi et al., 2006; Moriguchi, Ohsnishi, Mori, Matsuda, & Komaki, 2007; Russell, Reynaud, Herba, Morris, & Corcoran, 2006). Moreover, the Frith-Happé animations have also been used to detect differences in neural activity (Kirkovski, Enticott, Hughes, Rossell, & Fitzgerald, 2016) or eye movement (Roux, Passerieux, & Ramus, 2013)

Beyond the original H&S and the Frith-Happé animations, most remaining studies have relied on their own uniquely developed sets of animated shape videos. For example, Martin and Weisberg (2003) designed videos to depict one of two different types of shape animation: mechanical action and social interaction. The mechanical action videos used shapes to depict the movement of inanimate objects (e.g., a pinball, a conveyor belt) while social interaction videos used shapes to depict human activities (e.g., sharing, playing on a slide). The videos were presented to typically functioning adults in a functional magnetic resonance imaging (fMRI) experiment. When viewing social interaction videos, participants exhibited activation in different brain regions than when viewing the mechanical action videos, suggesting that these are two distinct cognitive domains. Other studies further examined the effects of different animated stimuli on brain activation and attention (e.g. Isik, Koldewyn, Beeler, & Kanwisher, 2017; Tavares, Lawrence, & Barnard, 2008). Unlike most animated shape tasks, which are typically shown in two dimensions, Visch and Tan (2009) created a three-dimensional shape task involving two interacting shapes. Other researchers have created animated shape videos that were designed by algorithms and not explicitly meant to represent any specific form of social interaction (Wick, Soce, Garg, Grace, & Wolfe, 2019).

While these sets of animations have largely extended the work of Heider and Simmel (1944), they face certain limitations. Although many researchers continue to use versions of the original H&S animation, this animation is relatively long (2.5-min) and depicts a limited set of social behaviors related to aggression and bullying. Many of the newer animations were designed specifically for use in neuroimaging studies (e.g., Ludwig et al., 2020; Weisberg et al., 2014) or to study impairments in clinical conditions (e.g., Abell et al., 2000). For example, the Frith-Happé animations have been used frequently in past research but most of these studies focus specifically on individuals who are diagnosed with ASD or other psychological impairments. This potentially limits their generalizability and application to research in broader populations. Moreover, these sets of videos and the individual participant descriptions of the videos are often not publicly available or easily accessible. This makes it difficult for other researchers to use these stimuli in subsequent research or to determine whether they are equally effective in eliciting social attributions in other participant populations. Others have designed tools to create new animations in the style of the original Heider and Simmel video (e.g. Gordon & Roemmele, 2014), but few studies tested whether they can also be used to evoke social attributions or mental states.

Present Study

We sought to develop a new set of animated shape videos that would provide a more modern and convenient means for assessing social intelligence and which could also be used in many populations and contexts. The custom-developed animations in the measure are a series of short, two-dimensional shape videos designed to mimic human interactions. These videos, which were also utilized in the present study, expand upon and update the work of Heider and Simmel (1944) with modern videos featuring many new social plots and situations. Unlike Heider and Simmel, the focus of our work was not to study the perceptual organization of the shapes’ movements and actions, but rather to develop an updated, publicly available set of videos. An example screenshot of the H&S video and one of our newly developed videos is illustrated in Figure 1. Although our videos were influenced by the original H&S animation, our intent was not to reproduce or replace the original. Instead, our goal was to provide a set of modern, shorter animations which involve a broader range of social and emotional expressions. These videos were designed to be applied to many research settings including (but not limited to) social cognition, social neuroscience, and developmental psychology. We expected that the videos would vary not only in social/emotional content, but also in difficulty of making attributions and in the consistency of such attributions made by different observers. These sources of variability can be regarded as features of this set of videos that may make it useful as a research tool for purposes to which a single video (e.g., the H&S original) is not optimally suited.

Figure 1.

Example screenshots from the original Heider and Simmel animation (A; Figure 1 from Heider & Simmel, 1944) and one of the animated shape videos created in the present study, “Bullying 2” (B).

In prior research, several of these videos were used as stimuli in a multiple-choice test of social intelligence: the Social Shapes Test (SST; Brown et al., 2019). Using these videos, Brown and colleagues observed performance on the SST was positively correlated with performance on emotional recognition and emotion understanding tasks. Compared to other social intelligence tasks, the SST was also less strongly related to measures of verbal ability. These results suggest that the animated shape videos potentially provide a means of evoking social attributions without relying heavily on verbal intelligence.

In the present study, participants watched our custom set of animated shape videos and wrote free response narratives describing the action in the videos. Each participant’s response was coded based on their level of social attribution, using scoring criteria adapted from Klin (2000). Our primary objective was to replicate and extend the findings of Heider and Simmel (1944) and observe the degree to which this new class of stimuli can evoke a broader range of social attributions. We also compared our results to those reported by Klin (2000) for typically developing participants. By omitting any cues to the social nature of our videos, we aimed to investigate whether the videos can elicit spontaneous social attributions with the goal of providing a more modern set of videos that can be utilized for research on social intelligence among typical adults.

Method

Participants

Sixty-six undergraduate students (44 female, 22 male; aged 18–23 years, M = 19.8) at Union College (Schenectady, New York) participated in this study to fulfill a course requirement or for cash ($6). All participants provided written informed consent and the study was approved by the Union College Committee on the Use of Human Subjects.

Materials

An initial set of 77 hand-drawn storyboards was designed to represent various plots or social interactions (Brown et al., 2019). These storyboards were developed based on several rounds of brainstorming sessions and feedback. We provide an example storyboard that was used to create the animation titled “Exclusion” in Figure 2. Several of these storyboards were inspired by social animations used in past research (e.g., Boraston, Blakemore, Chilvers, & Skuse, 2007; Heider & Simmel, 1944; Martin & Weisberg, 2003; White et al., 2011). From this set of 77, we selected 32 storyboards to be developed into digital animations by a professional graphic artist using Adobe Flash. The digital animations underwent several revisions until the desired social plot or interaction was sufficiently depicted (confirmed by agreement of a subset of the authors).

Figure 2.

Example storyboard for the animated shape video “Exclusion.” Individual storyboards were created for 77 potential animations, of which 32 were created and used in this study.

Each of the 32 animated videos was between 13–23 s in length (M = 17 s) and was scripted to follow a social story. For example, one video was designed to depict a shape being “excluded” from a group of other shapes (“Exclusion,” see Figure 3). All videos contained four primary shapes (a yellow plus, purple star, red square, and blue triangle) that moved against a white background. Thirteen of the 32 videos also featured a fifth shape (an orange X). All five shapes were equal in size. In two of the animations, a smaller, orange X was used to represent a child character. The colors and sizes of each shape were consistent across all videos. The color green was not used in order to allow individuals with red-green colorblindness to perceive differences between the shapes. In some videos, shapes also interacted with inanimate objects that were not designed to act as social agents. Among these objects, which were strictly black in color, circles represented mobile objects (e.g., a ball) while rectangles represented fixed objects (e.g., a building). We provide a detailed description of each animation and estimate for the difficulty of identifying the social intention in videos in Appendix A (Tables A1 and A2).

Figure 3.

Six frames from the sample animated shape video “Exclusion” (17 seconds). In this video, a group of three shapes (Blue triangle, Red square, and Purple star) excludes Yellow plus from the rest of the group and moves away every time it comes near.

Several precautions were taken in order to ensure consistency across the 32 animated videos. First, every shape that appears in an animation was required to engage in some degree of movement. Second, we decided that more than one shape needed to perform a unique movement during each video. Both of these guidelines were determined in order to prevent drawing too much attention to any individual shape.

Procedure

The 32 total videos were randomly divided into two randomly ordered sets of 16 (set A and set B). Participants were assigned to watch the videos in one of four conditions: (1) set A in order, (2) set A in reverse order, (3) set B in order, or (4) set B in reverse order. We did not show each participant every video because we thought it would be difficult for participants to write narratives consistent in quality over the length of time required to watch and describe 32 videos. We counterbalanced the video orders in case there were any systematic effects of viewing earlier videos on the descriptions of later videos. Thirty-two people viewed set A (16 in each order) and 34 people viewed set B (17 in each order). Participants completed the study alone in laboratory testing rooms on computers where the videos were presented in an online survey.

The instructions at the beginning of the survey asked participants to pay close attention to each video and watch each video twice. Each video was set to play automatically when the page loaded. Participants watched each video twice by advancing to the next survey page, where the same video appeared again. This was done to place less of a burden on memory, and Heider and Simmel (1944) likewise showed their animation twice to participants. After the video was shown, the following instructions were given: “Write a paragraph about what happened in this video. Please include as much information as you can.” Participants typed a narrative of the video into a text box which appeared on a separate page from the video. This was repeated until participants completed the 16 videos they were assigned. Then participants answered demographic questions and were dismissed. Average time to complete the study was 40 minutes (SD = 13 minutes) and ranged from 16 minutes to 85 minutes.

Scoring Guidelines

Scoring guidelines were developed to code the text provided in the narratives. A subset of guidelines was adapted from the “SAT Coding System” by Klin (2000), which was created to analyze social attributions contained in narratives describing Heider and Simmel’s (1944) social animation. Not all of Klin’s scoring criteria were used because some were specific to the Heider and Simmel video, which was much longer and followed a more specific and detailed social plot than those in the videos we created. Additionally, several of Klin’s guidelines were developed to test a clinical population of individuals with ASD, and therefore did not pertain to our study. Thus, three social attribution indices in total were developed based on Klin’s scoring criteria. A detailed description of each scoring index, along with a list of key words and phrases, is provided in Appendix A. Although each index score was calculated separately and designed to focus on a somewhat separate aspect of social attribution, the indices are likely interrelated as they were all intended to measure facets of social understanding of the videos.

The Cognitive Term Index measured the degree to which participants described the videos using cognitive mental state terms. This included terms denoting shapes’ knowledge, intentions, and beliefs, among other things (e.g., know, think, avoid). The score was determined by calculating the percentage of propositions in participants’ narratives that contained at least one cognitive mental state term. For our purposes, a proposition was defined as a section of a sentence which makes a “claim,” typically when a verb appears. Each sentence in the narratives consisted of one or more propositions.

The Affective Term Index measured the degree to which participants described the videos using affective mental state terms. Affective terms included emotions and feelings (e.g., happy), behaviors which implicitly suggest a shared emotional state between two characters (e.g., celebrating), and emotional states which arise from social situations (e.g., jealous). The score was determined by calculating the percentage of propositions in participants’ narratives that contained at least one affective mental state term.

The Imagination Term Index measured the overall degree to which participants perceived the shapes appearing in the videos as “human” figures. This index was partially based on Klin’s “Animation Index” but amended as that index used a scale to assign level of social attribution rather than counting individual terms. Scores on the Imagination Term Index corresponded to how frequently participants described the shapes in ways that are social or human. Imagination terms included behaviors which do not involve an explicit mental state but which are uniquely human (e.g., talking), situations or environments which are uniquely human (e.g., home), sensory experiences which are not uniquely human but which cannot be experienced by inanimate objects (e.g., seeing), and allusions to a human as described by his or her relationship to another (e.g., friend). The score was equal to the total percentage of propositions in participants’ narratives that contained these terms.

Data Analysis

In total, participants produced one narrative for each of the 16 videos that they viewed. Each narrative was read and scored on the basis of the total number of cognitive, affective, and imagination terms as well as the number of propositions that were included. Additionally, participants’ data from those who viewed the first set of 16 videos in both orders were combined. The same was repeated for the second set of 16 videos. Data were also combined from all participants when calculating the index scores for each video, such that for each video, the sum of relevant terms for that index was divided by the sum of propositions given by all participants.

Results

The written narratives were scored by a single rater (the first author). All narratives were scored during an initial round of coding and then re-scored a second time (approximately ten weeks later) to ensure accuracy of the data. The scoring system was relatively mechanical and mostly consisted of counting words from different categories. The correlation between the first and second ratings (across the 32 videos) were r = .97 for the Cognitive index, r = .97 for the Affective index, and r = .98 for the Imagination index. Data from the second scoring is reported here. We first determined the scores of the three social attribution indices (Cognitive Term Index, Affective Term Index, and Imagination Term Index), the number of propositions, and the word count of the narratives. All participants used at least one cognitive, affective, or imagination term in their narratives in total across the 16 videos they viewed, thereby suggesting that no one viewed the shapes in solely mechanical ways across all videos. Instead, most people seemed to spontaneously attribute social meaning to the clips, replicating the findings of Heider and Simmel (1944) in this new set of animated shape videos. These preliminary results also suggest that participants varied in how “social” their narratives were, i.e., how many cognitive, affective, and imagination terms they used. Figure 4 depicts the variability in the Cognitive, Affective, and Imagination Term Indices for each set of videos.

Figure 4.

Social Attribution Index Score distribution for Sets A and B. This figure illustrates the distribution of Cognitive, Affective, and Imagination Index Scores for Videos in Sets A and B compared to the results from Klin (2000). The whiskers represent the full range of index scores and the boxes represent ±1 SD. A = Set A, B = Set B.

Among the three social attribution indices, participants used imagination terms most often in their descriptions of the videos. On average, 22.6% (SD = 12.5) of the total propositions in participants’ narratives contained imagination terms. Cognitive terms were slightly less likely to be included, with 14.4% (SD = 7.8) of participants’ narratives containing cognitive terms. In comparison, participants rarely used affective terms when describing the videos (M = 4.5%, SD = 4.1). These results are summarized in Table 1. Given the relatively low number of affective terms utilized, it is likely that several of our videos did not contain expressions of emotional states that were easily identified by viewers.

Table 1.

Average Number of Propositions, Word Count, and Results of Index Scores Obtained in Narratives

Videos participants viewed	Word counta M(SD)	Number of propositionsb M(SD)	Cognitive Term Indexc M(SD)	Affective Term Indexd M(SD)	Imagination Term Indexe M(SD)
Videos 1–16	970 (438)	106 (38)	18.4% (8.0)	5.6% (3.5)	23.1% (12.6)
Videos 17–32	1036 (447)	116 (44)	10.3% (5.3)	3.3% (4.5)	22.1% (12.8)
Total average	1004 (441)	111 (41)	14.4% (7.8)	4.5% (4.1)	22.6% (12.5)

^aTotal words produced across all narratives

^bTotal number of propositions across all narratives

^cPercentage of propositions containing cognitive terms across all narratives

^dPercentage of propositions containing affective terms across all narratives

^ePercentage of propositions containing imagination terms across all narratives

We also used the social attribution indices to identify which videos were most likely to be described using social terms (i.e., cognitive, affective, and imagination terms). These results are summarized in Table 2. We found that certain videos were more likely to be described with cognitive terms; for example, “Bullying 1” (for which 36.8% of propositions included cognitive terms) and “Helping” (30.2%). Other videos provoked fewer cognitive terms in the narratives; for example, “Baby” (4.4%) and “Friendship” (3.8%). Likewise, videos such as “SS1” (14.9%) and “Bullying 1” (13.8%) were described with more affective terms while other videos were described with fewer affective terms. In particular, the video “Hurdles” did not elicit any affective terms from any of the participants (0%). Videos which contained the most imagination terms were “Stealing” (51.5%) and “Peer Pressure” (46.9%), while “SS2” (5.8%) and “Line Cutting” (3.1%) had the least number of imagination terms. We also calculated the number of cognitive, affective, and imagination terms after adjusting for video length, but this had little effect on the relative rankings.

Table 2.

Cognitive, Affective, and Imagination Term Indices Compared to Klin (2000)

Video	Cognitive Term Index	Affective Term Index	Imagination Term Index
Klin (2000) Video	13.6%	11.5%	—
Baby	4.4%	2.2%	36.0%
Bullying 1	36.8%	13.8%	16.1%
Bullying 2	12.2%	2.6%	10.6%
Comfort	11.4%	5.9%	10.2%
Couple	20.3%	7.4%	30.4%
Cyberball	12.5%	3.7%	45.8%
Dislike	19.3%	3.0%	7.2%
Distraction	22.2%	2.8%	11.7%
Duck Duck Goose	17.9%	3.3%	34.9%
Exclusion	25.8%	8.2%	18.1%
Family	20.0%	1.4%	18.6%
FB	8.8%	1.5%	16.9%
Friendship	3.8%	0.8%	15.0%
Helping	30.2%	5.8%	26.7%
Hurdles	8.9%	0.0%	39.3%
Juggling	4.6%	0.4%	33.6%
Leader	6.7%	2.4%	24.6%
Line Cutting	9.7%	1.7%	3.1%
Matchmaking	5.3%	0.4%	35.1%
Mocking	19.3%	8.5%	14.2%
Peer Pressure	5.1%	13.4%	46.9%
Sad	17.2%	9.2%	23.6%
Scared	20.6%	5.3%	10.1%
Scaring	10.2%	0.4%	15.0%
Sick	17.4%	3.2%	14.2%
Slow Dancing	5.9%	4.6%	24.2%
SS1	12.5%	14.9%	22.6%
SS2	8.2%	0.5%	5.8%
Stealing	15.9%	4.4%	51.5%
Strangers	13.9%	0.8%	21.7%
Teacher	17.3%	8.6%	14.9%
Tricking	14.6%	2.5%	25.1%

Next, we compared the values on our Cognitive and Affective Term Indices to those of the H&S video in Klin (2000). We found that our mean Cognitive Term Index score (14.4%) was comparable to Klin’s Cognitive Index (13.6%) for the typically functioning sample in the study. Most of our videos had Cognitive Index scores that were within a standard deviation of control group scores in Klin (2000). Moreover, we observed that half of the videos (16/32) had greater Cognitive Index scores compared to Klin’s results (“Strangers,” “Tricking,” “Stealing,” “Sad,” “Teacher,” “Sick,” “Duck Duck Goose,” “Dislike, “Mocking,” “Family,” “Couple,” “Scared,” “Distraction,” “Exclusion,” “Helping,” and “Bullying 1”).

The Affective Term Index score, by comparison, was lower in the present study (4.5%) than in Klin’s (11.5%). Only three of our videos had Affective Term Index scores which exceeded Klin’s (“Peer Pressure,” “Bullying 1,” and “SS1”), with an average Affective Term Index of 14% for these three. Interestingly, the only video which exceeded Klin’s score on both the Cognitive and Affective Term Indices was “Bullying 1,” which is quite similar in content to the Heider and Simmel video. Other videos with high Cognitive Term Index scores and relatively high Affective Term Index scores include: “Exclusion,” “Couple,” “Mocking,” “Teacher,” and “Sad.” These results are summarized in Table 2.

Discussion

We tested whether we could reproduce the observations made by Heider and Simmel (1944), Klin (2000), and others by developing a new set of 32 colored animated shape videos. Each video was designed to portray a different social plot. Using this new class of stimuli, we found that participants often spontaneously attributed social meaning to the videos by using terms that described the mental states of the shapes or described the shapes as “human” characters. People varied in how many social terms they used, which suggests that it may be possible to use our videos to measure differences in tendencies or abilities for social attribution among people. Additionally, our videos varied in how many social terms they were described with and the type of social term they were most likely to be described with. Finally, when comparing our findings to those of Klin (2000), we see that our participants’ scores on the Cognitive Term Index are comparable to the scores of the normally functioning population in that study but that most of our videos evoked fewer affective terms.

Our research suggests that participants spontaneously attributed social meaning to the animations used in our study, with an average of 14.4% of propositions provided by participants in their descriptions of the videos containing cognitive mental state terms, 4.5% containing affective mental state terms, and 22.6% containing imagination terms. However, participants varied widely, with some providing relatively few or no social terms in their narratives, while others used a relatively high number of social terms. Below is a participant’s response that had relatively high scores on the social attribution indices. The narrative is describing the video “Helping,” in which two shapes help a third shape who has been left behind with the task of getting inside a container.

“Here we have a black “u” in the center of the screen and our original four shapes. The shapes all are on the bottom right hand side of the screen until they decide to move into the u. Three of the shapes go into the u but the blue triangle doesn’t follow and then acts upset. Two of the shapes leave the u and the purple shape tries to help the blue triangle. At first he doesn’t seem to want the help. However, eventually he takes the help and all of the shapes are in the u. When they are all in there they get excited and cheer.”

This participant provided a clear social story using the following cognitive terms in their response: decide, follow, tries, help, and want. The following affective terms were used: upset, excited, and cheer. The narrative also contained one imagination term: takes. This participant had high Cognitive and Affective Indices (45.5% and 27.3%, respectively) with a slightly lower score on the Imagination Index (9.1%). The participant spontaneously attributed mental states to the shapes and used those perceived mental states to explain behaviors. In contrast, other participants were less likely to attribute social meaning to the shapes in the videos. Below is a participant’s response which included fewer social terms, also in response to the video titled “Helping.”

“There were four shapes and one “container.” Three shapes went inside the container and one was outside. Then the triangle poked the star a couple of times, and all the shapes went inside the container.”

This participant only used one social term in their response: the word “poked,” which is counted as an imagination term. This participant did not attribute any thoughts, feelings, or intentions to the shapes, and thus scored 0% on both the Cognitive and Affective Indices and a score of 20% on the Imagination Index. Contrasting behaviors like these suggest that people may vary in how well they can understand and attribute social meaning to these videos.

In addition, participants also varied in the way that they actually interpreted the videos and how well their descriptions matched what we intended to depict in creating them. In some videos we intended to depict certain social situations with the shapes’ movements, but participants instead interpreted these movements in a completely different way. For example, in our video titled “Scaring,” we intended to show that a group of shapes is startled by another shape who was hiding behind a wall. Yet not a single participant correctly identified that the group of shapes would be startled or scared by the other shape. Nonetheless, participants still described the video using social terms, just without identifying the theme we believed we were illustrating. Perhaps this video may be harder for people to understand without having any clues or prior knowledge of what is happening.

We observed variability in responses between participants and between the different videos. Some videos were more likely on average to be described with social terms than others. It is possible that our top 16 videos, based on Cognitive Term Index scores, contain social plots that are more easily recognizable by participants. Along these lines, most of the videos with the highest Cognitive Index scores were ones that were directly inspired by existing animated shape tasks. For example, both of the Bullying videos (“Bullying 1” and “Bullying 2”) were inspired by the original H&S animation and evoked a relatively high number of cognitive and imagination terms. Moreover, many of these videos were also designed to represent some form of conflict between the characters (e.g., one shape bullying, mocking, scaring, or excluding others). Our results may suggest that simple forms of conflict can be easily recognized as representing social interactions for most individuals. If most people can identify these social themes, the degree and complexity of the social and mental state attributions people make when viewing the videos can be measured, making them good stimuli for assessing social intelligence.

Further, among our three social attribution indices, participants had the lowest values on the Affective Index. Fewer than 5% of propositions contained affective terms on average. Given that the use of affective and emotional terms to describe the videos was relatively rare, it is possible that our videos did not have expressions of affective states that were easily identifiable by the viewer. A more three-dimensional depiction of shapes (or solids) could permit more motion patterns that might indicate more emotional states. Alternatively, perhaps participants were less likely to spontaneously comment upon the emotional states of the shapes in the videos without instructions directing them to do so.

Even though our videos were substantially shorter than the original Heider and Simmel video, participants made a similar number of social attributions compared to what has been reported in past research. When comparing the scores on the social attribution indices to those reported by Klin (2000), we found that our average scores on the Cognitive Term Index were comparable to the typically functioning group in Klin’s study. Klin found that on average this group used cognitive terms in 13.6% of their propositions. In the present study, we found that participants included cognitive terms in 14.4% of their propositions. Half of our videos (16/32) had Cognitive Term Index scores equal to or greater than what Klin observed, with an average score of 20.5% when just analyzing these videos. On the other hand, we found that the Affective Term Index scores were lower for our videos (4.5%) compared to Klin (11.5%). Moreover, only three of our videos scored equal to or higher than 11.5%. Thus, while the Affective Term Index score across all videos in the present study was lower than that reported by Klin, a few of our individual videos had similar or even higher Affective Term Index scores.

In particular, we observed the most frequent use of ToM terms (either cognitive or affective) among videos that were influenced by previously developed shape tasks (e.g. Heider & Simmel, 1944 or Martin & Weisberg, 2003). Many of these videos involve some form of conflict between the shapes (e.g., one shape bullying, mocking, scaring, or excluding others). This pattern suggests that it may be easier to evoke social attributions from stimuli that are designed to represent conflict. On the other hand, more complex social interactions (e.g., waiting in line or displaying friendship) may require a higher degree of ToM or social intelligence and result in a broader range of attributions.

Limitations and Directions for Future Research

Our study had several limitations that open up potential for future research. One limitation is that in the Klin (2000) study, propositions in participants’ narratives which were not pertinent to the video were eliminated (e.g., vague references). This may have led to higher scores on the indices. Additionally, Klin did not specify which words were counted as affective or cognitive terms. Although the guidelines in Klin’s study were very similar to those in the present study, it is possible that certain terms were coded in different ways. We have made all of the coding data from this study available on the Open Science Framework for future analysis and comparison with other work.

Another limitation is that our instructions asked participants to describe what was happening in the video without giving any clues that there were social connotations. While this method allowed us to determine the spontaneous social attributions of our videos, it is possible that some people did not report social attributions that they did make. Perhaps viewers may have been less likely to contemplate and comment upon the mental states of the shapes and more inclined to describe the actions of the shapes. Future research should examine the differences in responses among instructions which prompt the viewer to view the shapes as social agents and those which give no explicit social attribution instructions, as in this study.

Although the focus of our study was not on the perceptual organization of the shapes, future work should examine how features of the shapes’ movements are related to the interpretations that individuals provide. This would provide a greater understanding of what specific characteristics in the animations, such as speed and perceptual organization, lead individuals to make social judgments. Some past research has sought to identify how these characteristics influence attributions (e.g., Gao, Baker, Tang, Xu, & Tenenbaum, 2019; Rasmussen & Jiang, 2019), but further research using our animations could build on this work.

Finally, without the inclusion of other individual difference measures, it is difficult to determine whether differences among participant responses were due to underlying differences in social intelligence or differences in video content. Future research should examine whether participant’s descriptions of these videos correlate with other standardized measures of social intelligence or ToM. This would help provide further evidence for the construct validity of our animated shape videos as stimuli for social attribution.

Conclusion

In this study, we sought to create a new set of animated shape videos that could elicit social attributions similar to existing tasks (e.g., the original H&S animation) while being shorter in length (ranging between 13–23 s) and containing a broader range of social plots. As they did 75 years ago, college student participants spontaneously attributed social meaning to our videos, as indicated by the use of cognitive and affective terms in their narratives. Our results therefore confirm the findings of Heider and Simmel (1944) and Klin (2000) and demonstrate that those findings were not idiosyncratic to the single particular video used in those studies. Rather, we observed that many of our videos can be used to evoke a similar degree of social attributions while also being much shorter in length. In particular, several videos that depicted conflict between the shapes, like the original H&S video, often elicited the most attributions. However, other forms of social interaction appear to be more difficult to perceive.

We hope these findings inspire future research to better understand how the physical or relational properties of these animations influence the formation of social attributions. Moreover, we developed these videos for use with typically functioning adults in order to enable future researchers to more easily use this methodology for evoking social attributions in studies outside of clinical psychology or neuroscience. Our animated shape videos are freely available for use and can be administered as part of a multiple-choice assessment (Brown et al., 2019). Thus, our animated shape videos may serve as a useful tool for simulating social interactions and invoking social attribution in order to study the collection of cognitive capacities known as social intelligence, mentalizing, and ToM.