Thin-slice perception develops slowly

Abstract

Body language and facial gesture provide sufficient visual information to support high-level social inferences from “thin slices” of behavior. Given short movies of non-verbal behavior, adults make reliable judgments in a large number of tasks. Here we find that the high precision of adult’s non-verbal social perception depends on the slow development, over childhood, of sensitivity to subtle visual cues. Children and adult participants watched short silent clips in which a target child played with Legos, either in the (off-screen) presence of an adult or alone. Participants judged whether the target was playing alone or not, that is, they detected the presence of a social interaction (from the behavior of one participant in that interaction). This task allowed us to compare performance across ages to the true answer. Children did not reach adult levels of performance on this task until age 9 or 10 years and we observed an interaction between age and video reversal. Adults and older children benefitted from the videos being played in temporal sequence, rather than reversed, suggesting that adults (but not young children) are sensitive to natural movement in social interactions.

Introduction

We “decode” the social world with relative ease and automaticity, making rich social inferences from sparse “thin slices” of behavior. For example, seconds-long movies of people performing various activities with no accompanying soundtrack support judgments of diverse characteristics including teaching effectiveness (Ambady & Rosenthal, 1993; Ambady & Gray, 2002), leadership competence (Rule & Ambady, 2008a; Rule & Ambady, 2009), sales effectiveness (Ambady, Krabenhoft, & Hogan, 2006), and sexual orientation (Rule & Ambady, 2008b;). Thin-slice judgments of personality traits typically correlate well with independent subjective measures of the same qualities. In some cases it has also been demonstrated that thin-slice judgments correlate with objective measures of the social “reality” (e.g. sales performance in dollars (Ambady, Krabenhoft, & Hogan, 2006), the grades of medical students (Rosenblum et al., 1994), or clinicians’ success in referring clients (Milmoe et al., 1967). Competent thin-slice perception allows observers to rapidly evaluate a social environment and adjust their own behavior in response to the behavior and characteristics of other social agents. This skill makes it possible for observers to quickly determine appropriate actions to take in novel settings and interpret the actions of other agents within a social context. Thin-slice judgments are thus a powerful tool for comprehending the social world and appear to be employed quickly and automatically (Rule, Ambady & Hallett, 2009).

Here we address two fundamental questions concerning non-verbal social perception. First, how does it develop? We know that adults can make thin-slice inferences from sparse, naturalistic, non-verbal behavior, but the acquisition of this ability has to our knowledge not been studied. Second, beyond simply characterizing the developmental trajectory of non-verbal social perception, we also examine one aspect of what visual information children and adults use to perform these tasks. Thin-slice perception has only been rather coarsely examined for critical features that support task performance. For example, masking manipulations applied to static faces have revealed that multiple facial features provide weakly diagnostic information for sexual orientation, (Rule et al., 2008; Rule, Ambady & Hallett, 2009). Video stimuli have been presented at long and short durations (8 seconds and 1 second) and compared to static-image slideshows sampled from the full sequence to examine the specific contribution of dynamic information (Ambady, Conner & Hallahan, 1999). In general, these manipulations have shown that motion information is critical for thin-slice perception – static images sampled from the full sequence carry significantly less useful information. These results provide a partial characterization of the perceptual cues that support accurate thin-slice perception, but overall few empirical findings are available to characterize the computations underlying robust social inferences drawn from sparse visual data.

In the present study, we examined the temporal direction of face and body gesture as a critical feature for thin-slice perception over development. Specifically, we use temporally reversed videos as a tool to determine whether the order of actions in a thin-slice stimulus and the natural dynamics of human movement carry information for making social inferences. Our use of time reversal here is akin to the use of inverted images in studies of face and object perception (Yin, 1969). Temporal reversal is one way to disrupt the natural movements of the people depicted in a thin-slice stimulus while preserving many basic perceptual properties, including the static frames comprising the sequence.

We developed a novel task to test children’s ability to use non-verbal visual cues to make judgments about social interactions. Compared to many previous thin-slice studies, our task has the benefit of presenting children with depictions of highly natural behavior where the social setting is objectively defined, rather than requiring a subjective measure of the social feature of interest (see Ambady & Rosenthal, 1992 for a discussion of related measurement issues in thin-slice studies). Specifically, we implemented a “detection” task for social interaction that requires our subjects to decide if children filmed while playing with Lego blocks are playing alone or with an unseen playmate. This task provides the observer with multiple non-verbal behaviors including eye gaze, visual speech, and gesture, which can be used to correctly label each video.

Thin-slice stimuli are complex, and therefore multiple processes that develop at different rates may contribute to children’s ability to make thin-slice judgements. Basic social referencing, for example, can be observed behaviorally in infants as young as 10–12 months of age (Gunnar & Stone, 1984), suggesting a relatively early competency for estimating another person’s evaluation of a stimulus or scene and adjusting one’s own behavior accordingly. Children as young as 15–18 months also exhibit behaviors consistent with emerging Theory of Mind competence in “active helping” paradigms (Buttelman et al., 2009) and anticipatory looking studies (Onishi & Baillargeon, 2005) but in general robust Theory of Mind processing appears to emerge later in development. Children exhibit a marked increase in false belief and related tasks between 3 to 5 years of age (Wellman, Cross, & Watson, 2001). Perceptual abilities relevant to processing the behaviors in typical thin-slice stimuli also develop at different rates. Face processing may be mature quite early (Crookes & McKone, 2009) – key signatures of face processing (e.g. the inversion effect) appear to be present in children as young as 4 years old. However, other aspects of facial emotion processing continue developing through late childhood and into adolescence (Batty & Taylor, 2006). Efficient processing of body movements and actions also develops late in childhood (Freire et al., 2006). This partial list of the component processes of thin-slice processing suggests that thin-slice perception may continue to develop over an extended period of time. Making social inferences from limited amounts of observed behavior may require efficient perception of action, intention, emotion, and other aspects of social cognition that are more sophisticated than those tested in infancy and early childhood. If so, thin-slice perception may develop in middle to late childhood just as these related processes do.

Imaging results also support the possibility that thin-slice development might develop slowly during middle childhood. The posterior superior temporal sulcus (pSTS) which is known to be selective for socially relevant movements of the face and body (Puce et al., 1998; Pitcher et al. 2011), exhibits extended development between the ages of 5–10 years (Carter & Pelphrey, 2006;). Also, the right temporo-parietal junction (RTPJ), a cortical area that responds selectively when participants think about another individual’s mental states (Saxe & Kanwisher, 2003), increases its selectivity over this same age range (Saxe et al., 2009). The ongoing development of these cortical areas implicated in high-level social perception suggests that sophisticated social competencies may also develop over an extended period of time.

We therefore hypothesized that thin-slice perception would exhibit a protracted developmental trajectory as children gradually acquire competency at detecting and interpreting the relevant cues in complex social stimuli. We recruited children across a wide age range (3–12 years of age) to examine the developmental trajectory of thin-slice perception for unaltered and temporally reversed stimuli, and directly compared children’s performance to that of adult participants.

Experiment 1

We created “thin slices” of behavior during free play sessions with Lego blocks that included a target child playing alone and then accompanied by a playmate. We presented short segments of these videos played either forward or backward to children and adults to determine the developmental trajectory of non-verbal social perception and to determine the extent to which the temporal direction of motion is a critical feature for task performance.

Methods

Subjects

A total of 32 adults (N=16, forward-video condition, N=16 reversed-video condition) and 98 children (N=49 (15 3–5 y.o., 17, 6–8 year olds, 17, 9–12 year olds) , forward-video condition, N=49 (15 3–5 year olds, 19-6–8 year olds, 15 9–12 year olds)reversed-video condition) were recruited for this study at the Boston Museum of Science. All adults were between the ages of 18 and 42, and children ranged in age from 3 to 12 years of age. All participants reported normal or corrected-to-normal vision and were naïve to the design and purpose of the experiment.

Stimuli

All stimuli were recorded in the Boston Museum of Science as part of the “Living Laboratory” exhibit. We made videos of 5 children (3 female, 2 male) between the ages of 6 and 12 while they played with Lego bricks seated at a large table. Initially, each child was told to play by themselves for a few moments while the experimenter discussed the purpose of the study with their parent. Recording began after each parent completed a video release form. Approximately 2–3 minutes of unaccompanied play were recorded (with the experimenter and the parent some distance away), after which the child’s parent joined them at the opposite end of the table to play together. Behavior was fully unconstrained in each setting.

We selected five 6-second segments (3 accompanied, 2 unaccompanied) from each video removed the audio track. We chose segments that were free of video artifacts, contained the target child completely in the frame, and that were free of accidental interference with the recording equipment (e.g. individuals jostling or walking in front of the camera). We cropped each segment to remove the side of the table opposite the target child from view and rendered each clip in normal, forward order and reverse order (played backwards) for use in the reversed-video condition.

Procedure

We told participants that they were about to see several short videos of children playing with Lego blocks and would be asked to tell the experimenter whether each child was playing alone or with a partner. The experimenter described the physical set-up of the recordings (with accompanying illustrations for young children; see Figure 1) and made clear that there would be no soundtrack and that the target child’s playmate would never be onscreen even if he or she were present during the recording. We also told participants that they would see each child multiple times during the experiment, sometimes playing alone and sometimes playing with someone. Young children viewed the series of pictures presented in Figure 1 and asked two screening questions about the set-up of the recordings before testing began: (1) Is this person’s friend still here even though we can’t see them through this window? (2) Can this person still play with their friend even when we can’t see them? We excluded 3–5 year old children from the task if they were unable to correctly answer these questions (n=8).

Figure 1.

Original videos (top panel) of children playing with Legos either accompanied or unaccompanied were cropped to remove the parent and limit the field of view to the target child (middle panel), yielding 6-second segments of video depicting only one child at play with no soundtrack (bottom panel).

During the task, we presented movies in a randomized order using custom software written for the Matlab Psychophysics Toolbox (Pelli, 1997; Brainard, 1997). Each movie played for 6 seconds, after which a response screen with the words “ONE or TWO?” appeared (asking whether one or two people were present at the table, even though only one could be seen) and remained visible until the subject responded. Participants responded with a large USB switch, or verbally to the experimenter in the case of young children. We did not record response times, due to both the extended duration of the stimuli and the high variability in children’s response latencies in this task. We obtained informed consent from all adult participants and from the legal guardians of the child participants.

Participants were unaware that videos could be presented forward or reversed and were assigned to the “forward video” condition or the “reversed video” condition at random. We varied the video condition between subjects because the inclusion of a sufficient number of forward and reversed videos into a single experimental session was impractical for younger children. Since children in these age groups were generally unable to complete a within-subjects design, we opted to run all age groups in a between-subjects design. Though this weakens our statistical power, it also ensured that young children remained attentive and engaged throughout the task.

Results

We analyzed participants’ accuracy in both the forward and reversed video conditions using a 2×2×4 between-subjects ANOVA with participant sex (male, female), video condition (forward, reversed) and age group (3–5 year olds, 6–8 year olds, 9–12 year olds, and adults) as between-subject factors. This analysis revealed a main effect of age (F(3,112)=45.7, p < 0.0001, χ²=0.55), a main effect of participant sex favoring male participants (F(1,112)=4.7, p = 0.032, χ²=0.041), a main effect of video condition favoring forward playback (F(1,112)=5.06, p=0.026, χ²=0.043) and an interaction between age and video condition (F(3,112)=3.26, p =0.024, χ²=0.080). No other effects reached significance.

To examine the interaction between age group and video condition, we conducted post-hoc t-tests between the forward and reversed condition in each age group. These tests revealed a significant effect of video condition in adult subjects (t(30)=2.98, p = 0.006 and 9–12 year olds (t(30)=2.05, p = 0.046), but no significant effect in either 6–8 year olds (p=0.24) or 3–5 year olds (0.27). The data thus indicate that the effect of reversing the direction of video playback emerges relatively late in childhood.

Finally, we point out that the 3–5 year old subjects do not perform above chance in either the forward or reversed video condition, even though all participants included in our analysis successfully answered the screening questions. To ensure that the chance-level performance of this group was not responsible for the interaction we observed between age and video condition, we conducted an additional ANOVA with the data from this age group removed. This revealed main effects of age (F(2,89)=29.2, p < 0.001), sex (F(1,89)=4.2, p=0.04), and video condition (F(1,89)=3.85, p=0.05) as well as an interaction between age and video condition (F(2,89)=5.28, p=0.007). No other interactions reached significance. This result confirms that our results are not driven by a floor effect in the 3–5 year-old data. To comment further on the poor performance of 3–5 year olds in this task, participants in this age group do significantly exceed the 40% performance level expected if they were answering the question literally by assuming that the absence of the second person from the video indicated that there was only one unaccompanied actor present.

Discussion

Our task reveals several intriguing aspects of non-verbal social perception in childhood and adulthood. First, people are more accurate detecting that a person is engaged in a social interaction if they see that person’s behavior in the normal forward temporal direction than if the video is played backward. This performance benefit is relatively small, but nonetheless the data indicate that observers make use of visual features that incorporate temporal order and/or temporal dynamics to make judgments about social interactions. This result is consistent with observations from non-social object recognition tasks suggesting that so-called “spatiotemporal signatures” (Stone, 1998; Vuong & Tarr, 2006) are computed for moving objects, aiding recognition (Newell, Wallraven, & Huber, 2001) and prediction (Balas & Sinha, 2009a). Our result rules out the possibility that participants perform our task by relying solely on the static frames that make up each movie, the amount of motion in each movie (e.g. motion energy), or static facial expressions. An important challenge for future work is to determine exactly what features are being used in thin-slice tasks. This may require extensive behavioral coding of naturalistic videos or detailed rendering of artificial bodies that can be controlled parametrically (Hodgins et al., 2010).

Second, we find an extended developmental trajectory, with children not reaching adult performance on the forward movies until age 9 or 10. This result is somewhat surprising, because children are capable of making rich judgments of animacy and intentionality from simple geometric stimuli at much younger ages (Dasser, Ulbaek & Premack, 1987; Berry & Springer, 1993; Gergely et al., 1995) and are also capable of performing well in other behavioral tasks that require sustained attention over several seconds (e.g. multiple object tracking (Black & Pylyshyn, 2004; Trick, Jaspers-Fayer, Sethi, 2005)). However, as we noted earlier, this result fits with recent findings that cortical areas supporting social processing also appear to continue developing well into middle and late childhood.

Third, our data show that the benefit of natural motion emerges slowly over the course of development. This suggests that children continue to learn how to use natural movements and gestures for thin-slice perception in middle childhood. One interesting question is whether these developmental effects reflect quantitative change, or qualitative change, in the development of thin-slice perception. Do young children rely on a different set of critical features (perhaps not incorporating motion at all) to make visual judgments, or do they just use the same features as adults and older children less efficiently (Husk, Bennett & Sekuler 2004)? Probing the variant and invariant properties of non-verbal social perception via spatiotemporal manipulations such as sequence scrambling (Vuong & Tarr, 2004) could shed important light on what cues are recruited at different ages.

An important question for future research will be to establish the extent to which the effect of temporal reversal is specific to social thin-slice perception, or reflects development of sensitivity to temporal order and dynamics, more generally. Some prior studies indicate that the effect of reversed motion on object recognition is not universal. Balas & Sinha (2009b) found that video reversal affected fast-moving objects but not slow-moving ones, and Vatakis & Spence (2006) reported that video reversal did not affect visual speech processing, even though they found reversal effects for other actions. A direct comparison of social and non-social perception should use dynamic social and non-social stimuli, with matched temporal dynamics. Although such comparisons have not traditionally been used in the thin-slice literature, they will be particularly interesting in the context of developmental research.

In sum, the current results are consistent with the idea that nonverbal social perception may recruit some perceptual mechanisms that are specific to this task, that operate better on forward than time-reversed samples of behavior, and that have their own distinctive and slow developmental trajectory. Future research should fruitfully explore the visual cues that support these judgments, their developmental trajectory, their brain basis, and their possible disruption in autism.

Figure 2.

Mean accuracy for each age group for forward and reversed videos. Error bars represent +/− 1 standard error of the mean.

Highlights.

1. Thin-slice perception continues to develop over middle childhood – adult levels of performance are not reached until ~9 years of age.

2. Adults and older children (9–12 years old) are more effective with forward videos than reversed.

3. Young children (3–5 years) do not perform above chance.