Actions Speak Louder than Gestures when you are 2 years old

Abstract

Interpreting iconic gestures can be challenging for children. Here, we explore the features and functions of iconic gestures that make them more challenging for young children to interpret than instrumental actions. In Study 1, we show that 2.5-year-olds are able to glean size information from handshape in a simple gesture, although their performance is significantly worse than 4-year-olds’. Studies 2–4 explore the boundary conditions of 2.5-year-olds’ gesture understanding. In Study 2, children of this age have an easier time interpreting size information in hands that are reaching than in hands that are gesturing. In Study 3, we tease apart the perceptual features and functional objectives of reaches and gestures. We created a context in which an action has the perceptual features of a reach (extending the hand toward an object) but serves the function of a gesture (the object is behind a barrier and not obtainable; the hand thus functions to represent, rather than reach for, the object). In this context, children struggle to interpret size information in the hand, suggesting that gesture’s representational function (rather than its perceptual features) is what makes it hard for young children to interpret. A distance control (Study 4) in which a person holds a box in gesture space (close to the body) demonstrates that children’s difficulty interpreting static gesture cannot be attributed to the physical distance between a gesture and its referent. Together these studies provide strong evidence that children’s struggle to interpret iconic gesture may stem from its status as representational action.

When we speak we often gesture. We use our hands to illustrate or emphasize what we say, and even to convey ideas not found in our spoken message. For example, if you go into a bakery and ask for a cake from behind a glass display, you might say, “the chocolate cake with the raspberry filling,” while simultaneously holding your hands in a circle to indicate a round cake. Although the shape of the cake was not explicitly mentioned, your hands make it clear to the shop owner that you are talking about the round chocolate cake, not the square chocolate cake. Now, it would have been just as clear if you had reached toward the two cakes, sitting on an open shelf, with your hands forming a circle in anticipation of grabbing the round cake; that is, if you had used an instrumental action rather than a gesture to indicate which cake you wanted. Here we ask whether young children find gestures as easy to interpret as instrumental actions and, if not, we explore the perceptual features and functions of gestures that might make them more challenging to interpret than instrumental actions.

Although adults have little difficulty extracting the content conveyed uniquely in gesturing hands (Goldin-Meadow & Sandhofer, 1999; Kelly, Özyürek, & Maris, 2010), a broad literature has found that infants and young children have trouble understanding the meaning of gestures, particularly iconic gestures – gestures that acquire meaning through the similarities between their shape or movement and their referent. For example, when shown a gesture pantomiming hammering, 18–22 month olds are no more likely to select a hammer (the correct referent of the gesture) than a bunny (Namy, 2008). It is not until 26 months that children can reliably see the relation between a representational gesture and its referent (Namy, 2008), although this ability improves significantly between the ages of 3 and 5 years (Dimitrova, Özçalişkan, & Adamson, 2016; Goodrich & Hudson Kam, 2009; Hodges, Õzçalişkan, & Williamson in press; Magid & Pyers, 2017; Martenette & Nicoladis, 2011; Novack, Goldin-Meadow, & Woodward, 2015; Stanfield, Williamson, & Özçalişkan, 2014; Tolar, Lederberg, Gokhale, & Tomasello, 2008). Many of these studies used gestures that represent actions, either actions on an object (i.e., handling gestures) or actions by the object (i.e., path gestures). But children have even more difficulty interpreting gestures that represent an object’s shape or perceptual features (Hodges et al., in press; Magid & Pyers, 2017; Tolar et al., 2008). As an example, the ability to interpret a hands held together with fingers extended as if wings referring to a bird emerges later than the ability to interpret a hand flapping movement to refer to a bird (Hodges et al., in press). The protracted development of shape gestures, relative to action gestures, indicates that the mapping between the shape of the hands (in a gesture) and the features of an object may constitute a greater representational leap for children than the mapping between the movement of the hands (in a gesture) and the object on which the action was performed (see Emmorey, 2014; Ortega, Sümer, & Özyürek, 2016; Magid & Pyers, 2017). Given the specific challenges of shape gestures, we make them the target of investigation in the current study.

How do children develop the capacity to interpret gestures? Some theoretical accounts predict that gesture understanding emerges from action understanding (e.g., Vygotsky, 1978; Hostetter & Alibali, 2008). To illustrate, Vygotsky proposed that gesture grows out of action. An infant may reach for an object that is too far away to obtain. When he fails to get the object, his mother brings it to him, thus turning his failed action into a communicative symbol. In a similar vein, Hostetter and Alibali (2008) suggest that thinking about an action activates the motor network associated with that action, which, under certain circumstances, can result in an overt gesture. Although Hostettler and Alibali (2008) do not provide a developmental account for how gesture use first emerges, their view is consistent with the idea that action understanding should support gesture development.

If the development of the action system directly supports gesture development, then we would expect a seamless connection between action comprehension and gesture comprehension. Yet research from two separate literatures – action comprehension development, and gesture comprehension development – suggests a discontinuity between the two. Children’s understanding of other’s actions emerges robustly during the first year of life, well before they display a solid understanding of iconic gestures. A variety of methodologies have found that infants in the first year of life will imitate the goals of another person’s actions (Hamlin, Hallinan & Woodward, 2008), visually anticipate another’s future actions (e.g., Cannon & Woodward, 2012), and exhibit neurophysiological signatures of action processing when observing someone engaged in an instrumental action (e.g., reaching to obtain an object; Southgate et al, 2009). But understanding another’s iconic gestures does not emerge until children are at least 2 years old. Moreover, young children often fail to interpret a gesture that represents an action even though they can interpret the action itself (and even if the action is a failed attempt that did not result in a positive outcome, Novack, et al., 2015). Novack and colleagues (2015) found that 2-year-olds struggle to identify the goal of an iconic gesture demonstration (e.g., gesturing putting a ring on a post), but have no trouble interpreting an incomplete-action demonstration of the same action (e.g., trying, but failing, to put the ring on a post). Importantly, children can produce this action themselves (i.e., they can easily put the ring on the post when given the toy). Thus, in a situation where a child has the ability to produce an action, and to understand an incomplete-action demonstration for that action, the child is nevertheless unable to understand a gesture that represents the action.

What is not clear from these previous studies is why gestures are more challenging for young children to interpret than instrumental actions. Instrumental actions and gestures differ in many ways. Gestures are a type of action in that they involve movements of the body, but they are also representational in that they stand for something other than themselves (see Novack & Goldin-Meadow, 2017). The fact that gestures are representational actions differentiates them from full-blown instrumental actions, whose purpose is to affect the world by directly interacting with it (e.g., grabbing a fork, opening a canister). Note that gestures also differ from actions in terms of intention. An incomplete action (which attempts to achieve an external goal, but fails) may look very similar to a gesture in that it too does not directly manipulate objects in the envionrment. However, the intention of the action is to interact with the physical world, whereas the intention of a gesture is to represent and often communicate information. Gestures are actions whose power resides in their ability to represent actions, objects, or ideas.

Here we ask whether the difficulty that young children have interpreting gesture stems from the fact that it does not have a direct effect on the world (as instrumental action does) and instead has its effect by representing information. We achieve this goal this by making the information conveyed in an instrumental action as comparable as possible to the information conveyed in a gesture. We borrow a paradigm from the action literature in which infants have been shown to successfully interpret information conveyed in the hand when that hand is part of an instrumental action – by 8 months, infants can visually anticipate that a hand with a pincer grip (thumb and forefinger held together) will reach toward a small object, and that a whole-hand grip will reach toward a large object (Ambrosini et at., 2013). Infants are thus able to attend to handshape information when it is embedded in an instrumental action early in infancy, and use that information to make rapid on-line predictions about another person’s actions. We ask whether young children can interpret the same handshape information when the hand is part of a gesture.

Infants’ ability to interpret to a pincer grip vs. a whole-hand grip as a cue to the target of a reaching action provides an ideal test case to compare actions and gestures simply because these handshapes can be used to specify the size of a target object not only in instrumental actions, but also in gestures. In an instrumental reaching action, handshape is functionally linked to obtaining an object of a specific size (one needs a pincer grip to pick up a small object, and a whole hand grip to pick up a large object). But handshape can also be used contrastively in gesture – in this case, to represent either a small or a large object (a gesture with pincer grip refers to a small object, whereas a gesture with a whole-hand grip refers to a large object). We can therefore test children’s relative understanding of instrumental actions vs. gestures by asking them to interpret the pincer and whole-hand grips in a reaching action, compared to the same pincer and whole-hand grips in a gesture.

If action comprehension directly underlies gesture comprehension in development, we would predict that the handshapes that we know infants can interpret when they appear in a reaching action should also be interpretable when the same handshapes appear in a gesture. Alternitively, if gesture comprehension has a protracted development compared to action understanding, requiring a set of skills or abilities beyond the underlying motor program, then handshape cues that children can understand within the context of reaching actions should be harder for children to interpret in the context of gesture.

In Study 1, we compare young children’s interpretations of pincer and whole-hand grips in a very simple gestural paradigm, one in which a hand is placed in between two objects and the child must interpret the handshape information to perform the task correctly; that is, to put a small object in the bucket when a pincer grip is used, and a large object when a whole-hand grip is used. We will find that 2.5-year-olds are able to interpret handshape information of this type in a very simple gesture paradigm, although significantly less well than 4-year-olds. Study 2 explores the boundaries of this understanding by embedding the same handshape information either in an instrumental action (a reach) or in a representation action (a gesture). In Studies 3 and 4, we explore the properties of gesture that make it more challenging for young children to interpret than instrumental action.

Study 1

In Study 1, we present handshape information (a pincer grip vs. a whole-hand grip) in a simple gestural paradigm akin to the paradigms that have been used to examine infants’ understanding of this type of handshape information presented in instrumental actions (e.g., Ambrosini et al., 2013). The experimenter put two objects that differed only in size on the table (i.e., a large duck and a small duck), and produced either a pincer grip (indicating that she wanted the small duck) or a whole-hand grip (indicating that she wanted the large duck), while saying “Can you put this duck in the bucket?” Language of this sort, particularly the deictic this, could be helpful in alerting children to the fact that there is information in the scene that can be used to decide which object should be put in the bucket. To assess potential developmental change in understanding the gestures we chose to examine, we compare the performance of 2.5-year-olds to 4-year-olds, who are known to have relatively advanced representational skills (Hodges et al., in press; Magid & Pyers, 2017; Tolar et al., 2008).

Methods

Participants

All studies were conducted following APA ethical standards, with approval of the Institutional Review Board (IRB) at the University of Chicago (Approval Number: IRB H10193, Protocol Title: Developmental functions of the MNS: Social Anticipation and imitation). Study 1 included eighteen 2.5-year-olds (M age = 31.34 months, range = 30.16 – 33.96 months; 9 females) and 18 4-year-olds (M age = 51.88 months range = 48.9 – 55.0 months; 10 females) recruited from a database of volunteer families in the Chicago area. Prior to starting the study, sample size was established at 18 to meet the counterbalancing needs of the study and to align with conventions for this type of study at the time of testing (cf. Ambrosini et al., 2013, Tolar et al., 2008). This prespecificed sample size was maintained throughout all studies in the current paper. Participants were identified by their parents as falling into the following racial and ethnic groups: 53% Caucasian, 35% African American, 2% Hispanic, and 7% multiracial. An additional three 2.5-year-olds were tested but excluded from the study due to distress (N = 1), experimental error (N = 1), or because the child failed to give only one toy during the warm-up trials (N = 1). All participating families received either a small toy or $10 compensation for participating.

Procedure

Warm-up

Children were seated at a table across from an experimenter. The experimenter had a 76 cm × 23 cm tray in front of her, and the children had a bucket on the table to their right. To introduce the child to the game, the experimenter said, “This is my special bucket. If you help me put toys in the bucket now, we can play with the toys later. We’re going to play a game where I’m going to ask you to put one toy in the bucket. Okay?” After the child consented to playing the game, the experimenter placed two different toys (e.g., a truck and a boat) on the tray approximately 30 cm apart. The experimenter then looked at the toy she wanted while pointing to the toy and labeling it (e.g., “Can you put this truck in the bucket?”). The experimenter then pushed the tray of toys toward the child and allowed the child to select a toy to put into the bucket. Feedback was only provided if the child tried to put two toys in the bucket. If the child did not select a toy, the experimenter encouraged the child and relabeled the toy. The purpose of these warm up trials was to teach children that in this game they can only give one toy, and to demonstrate tht the experimenter can express which toy she wanted by both labeling it verbally and using her hands to indicate it (pointing to it).

Test Trials

Following warm-up, children were given six test trials. Test trials differed from warm-up trials in that the toys placed on the tray were of the same kind and differed only in size (e.g., a small duck and a large duck) (see Appendix for full list of objects). Once the toys were both on the tray, the experimenter used a pincer grip to indicate that she wanted the small toy, and a whole-hand grip to indicate that she wanted the large toy, while using a label that did not distinguish between the toys (e.g., “Can you put this duck in the bucket?”). The experimenter held her hand in front of her chest, half way between the two objects and looked directly down at her hand so as not to provide additional cues that might have indicated which object should be put in the bucket (See Figure 1). No feedback was provided to children. The side of the target (right vs. left), the side on which the first cue was presented (right vs. left), and the size of the first cue (whole-hand grip vs. pincer grip) was counterbalanced across test trials. The size cues presented during test trials alternated in a fixed, pseudo-random order.

Figure 1.

Examples of pincer (left) and whole-hand (right) handshape cues in test trials. The individual who appears here gave signed consent for her likeness to be published in this article.

Analysis Strategy

For all studies, children’s toy choices on each test trial (large or small) were recorded immediately after the trial by looking into the the experimenter’s bucket. A second experimenter confirmed toy choice on each trial for 20% of the participants by watching video recordings of the study. Agreement was 100% between coders on trials that were double coded.

To analyze test trials in all studies, we ran mixed-effect logistic regressions predicting the correct response on each test trial (i.e., selecting the toy that matched the actor’s handshape) with Age group (Study 1) or Condition (Studies 2–4) as fixed factors and subject as a random factor. In our analysis process, we first tested for effects of trial level factors, such as trial number (1–6) to determine if children got better or worse over the course of testing; test item (duck, car, etc.) to determine if certain toys were harder or easier for children to map to handshape; handshape grip (pincer, whole-hand) to determine whether one handshape cue was more challenging; as well as subject-level counterbalanced factors including handshape order (pincer grip first, whole-hand grip first). Models with trial-level factors included by-subject random slopes in addition to random intercepts. Factors were kept in the models if significant, and model comparison was performed via log likelihood ratio tests to find the model of best fit. We also conducted one-sample t-tests (2-tailed) on each condition to evaluate whether perforamnce differed significantly from chance.

Results

Test trials were counted as correct if the child selected the item that matched the actor’s handshape cue. In Study 1, preliminary results found no effects of trial number, test item, handshape grip, or handshape order (all p’s > 0.26). There was a significant effect of age group on performance (β = 1.36, SE = 0.37, z = 3.68, p <.001; model with Age group significantly better than null model: χ², 1 = 21.79, p <.001). Across all 6 items, 4-year-olds were significantly more likely to select the correct toy (M= 5.05 out of 6 trials, SD = 1.3) than 2.5-year-olds (M = 3.55 out of 6 trials, SD = 0.92). However, children in both age groups performed above chance (2.5-year-olds: t(17) = 2.56, p <.05; 4-year-olds: t(17) = 6.68, p<.001).

We now know that when pincer and whole-hand grips are presented in a simple gestural paradigm, 2.5-year-olds, as a group, are able to interpret the handshape information above chance, although their average performance is still significantly worse than 4-year-olds’ performance. These results replicate a developmental pattern found across existing studies in the literature (e.g., Dimitrova, et al., 2017; Goodrich & Hudson Kam, 2009; Hodges et al., in press; Magid & Pyers, 2017; Martenette & Nicoladis, 2011; Novack, et al., 2015; Stanfield, et al., 2014; Tolar, et al., 2008), this time in the context of a very simple gesture. Our next step, in Study 2, is to explore the boundary conditions of the 2.5-year-olds’ ability to interpret pincer grip and whole-hand grip information when they are embedded in an instrumental action vs. a gesture.

Study 2

Previous work from the action literature (e.g., Ambrosini et al., 2013) and our findings from Study 1 suggest that 2.5-year-old children are able to glean information from a pincer vs. a whole-hand grip. Our next question is whether young children are equally able to interpret these handshape grips when they are embedded in a gesture (a handshape grip held static and close to the chest) vs. an instrumental action (the same handshape grip in a reaching hand extended toward an object). If children are able to interpret the handshape information when it is presented in a gesture as well as when it is presented in a reaching instrumental action, we will have support for the hypothesis that gesture understanding shares roots with action understanding. If, however, children have a more difficult time interpreting the same handshape information when it is embedded in a gesture than when it is embedded in an instrumental action, we will have evidence that, at age 2.5 years, children are not able to translate their understanding of action cues into gestures cues.

Methods

Participants

Thirty-six 2.5-year-old children were recruited from a database of volunteer families in the Chicago area. Participants were identified by their parents as falling into the following racial and ethnic groups: 40% Caucasian, 30% African American, 12% Hispanic, and 17% multiracial. 18 children were randomly assigned to the Reaching Condition (mean age = 31.76, range = 30.1 – 34.36 months; 8 females); 18 children were randomly assigned to the Gesture Condition (mean age = 31.78, range = 30.7 – 33.6 months; 9 females). An additional 12 children were not able to finish the study because of distress¹ (N=6; 3 from each condition); experimental error (N = 3; all from Reaching condition); or because they failed to give only one toy during the warm-up trials (N =3; all from Gesture condition). All participating families received either a small toy or $10 compensation for participating.

Procedure

Room set up and Pre-warm-up

Children were brought into a room containing a cardboard “house” and two experimenters (a host and an actor) (See Figure 2). The house was made out of a 48″ × 36″ trifold board and was painted. After getting accommodated to the room and the experimenters by playing a short puzzle game, the child was encouraged to stand on a yellow star approximately 4 feet away from the cardboard house. One of the experimenters, the actor, took a position inside the house so that her upper body was visible; she could extend her hand out of the house, but the front limited her ability to obtain toys positioned in front of the house (See Figure 2). The second experimenter, the host, then told the child a story about the actor – she had just moved into a brand new house and didn’t have any toys to play with. Together, the host and the child were going to give her some toys.

Figure 2.

Screen shot of child giving a correct toy selection. The host (the experimenter on the left) encourages the child to help when the actor (the experimenter inside the red “house”) indicates which toy she wants with her hands (using a pincer grip to indicate a small toy, and a whole-hand grip to indicate a large toy). The experimenters who appear here, as well as the parent of the child shown, gave signed consent for their images to be published in this article.

Warm-up

Following the story, children watched 3 warm-up trials designed to teach the child how the game worked and provide context about how the actor uses her hands in relation to toys. In the first warm-up trial, the host placed a single toy on the ground in front of the actor’s house and labeled it for the child (e.g., “Look, there’s a bear”). The actor in the house looked toward the toy and labeled it herself (e.g., “ooo, a bear!”). Then the actor demonstrated a preference for the toy in one of two ways, depending on condition. In the Reaching condition, the actor reached out from the house and grabbed the toy herself (in this warm-up, the host placed the toy within the actor’s reach). In the Gesture condition, the actor pointed toward the toy with an index finger (her finger always stayed in the gesture space, and the toy was not placed within the actor’s reach). The host then handed the toy to the actor. After 2 single-toy trials, the child watched a two-toy warm-up trial in which there were two different toys (e.g., a sheep and a truck) placed on the ground approximately one inch apart. The actor in the house labeled one of the two toys while either reaching to obtain the toy (Reaching condition), or pointing toward the toy (Gesture Condition). In the Reaching condition the actor obtained the toy herself by reaching; in the Gesture Conditon the host gave the actor the toy that she wanted. This third warm-up trial demonstrated that the actor wanted only one of two toys placed on the ground in front of her.

In warm-up trials 4–6, the child was encouraged to help the actor and give her the toy that she wanted. For these trials, the toys were placed farther away from the house (just out of the actor’s reach) so that the actor in the Reaching condition would not be able to obtain the toy herself. The host placed two toys on the ground, and the actor in the house labeled one of the toys. The actor reached out toward the toy while labeling it, but since it was too far away, she could not obtain it on her own. She held her hand in an extended reaching position while the host said to the child, “oh no! She can’t reach it! Can you give her the one that she wants?” In the Gesture condition, the actor pointed toward one of the toys (both of which were out of reach) and the host said, “Can you help now? Can you give her the one that she wants?” Children who did not consistently give a single toy during the two-toy warm up trials were given up to 3 additional trials to consistently give one toy. The experimenter did not provide feedback if the child gave the wrong toy.

Test Trials

Six test trials followed the warm-up trials. In test trials, the host placed two different sizes of the same kind of toy (e.g., a small car and a large car) out of reach of the actor, labeling them with the common name (e.g., “Look, here are two cars”). The actor then indicated with her hand the toy that she wanted as she produced the common label (e.g., “Ooo, a car!”)². The manner in which the experimenter indicated which toy was the target differed as a function of condition (See Figure 3A). Children in the Reaching condition saw the actor reach out with either a whole-hand grip or a pincer grip positioned between the two toys. Children in the Gesture Condition saw the actor produce either a pincer grip or a whole-hand grip in front of her chest. Importantly, the handshapes were identical in both conditions. Children were then encouraged to give the experimenter “the one that she wants.” In both conditions, the actor focused her eyes on her hand in order draw attention to her hand, and not give cues with her gaze to the intended object. Thus, in test trials, the experimenter’s verbal label could have referred to either object, and only the size of her handshape provided specific information about which toy she wanted. Test trials included 3 big and 3 small trials, with half of the children in each condition seeing big handshape cues first, and half seeing small handshape cues first. The side on which the target object was placed was switched according to a fixed pseudo-random, non-regularized order, and the order of stimuli was randomized.

Figure 3.

Screen shots of whole-hand and pincer grips and Boxes used in all of the studies. The same handshapes were used in the Reaching and Gesture conditions in Study 2 (3A), and the Reaching-Representational conditions in Study 3 (a glass window prevented the actor in the house from obtaining the toy in Study 3; 3B). A similar handshape was used to hold onto the Box in the Distance Control conditions in Study 4 (3C). The individual who appears here gave signed consent for her likeness to be published in this article.

Results

Figure 4 shows the average number of correct test trials by condition. There was a significant effect of condition on performance (β =0.83, SE = 0.31, z = 2.67, p =.007; model with Condition significantly better than null model χ², 1 = 6.91, p = .008). Children in the Reaching condition gave the correct toy on 4.05 out of 6 trials (SD = 1.31), significantly more often than children in the Gesture condition, who gave the correct toy on only 2.88 trials (SD = 1.23). No other trial level counterbalancing factors predicted performance (all p’s >.12), indicating that trial number, test item, handshape grip, and handshape order did not have an effect on performance.

Figure 4.

Average number of correct responses (out of 6) on test trials. Error bars represent standard error of the mean. The dotted line indicates chance performance. Stars indicate significant difference (p < .05) between conditions. Stars that are not linked to brackets indicate a difference from chance.

One-sample t-tests comparing performance to chance (getting 3 correct) indicate that children in the Reaching condition performed significantly better than chance, t(17)=3.21, p=.005. This finding confirms that 2.5-year-olds can use the shape of a hand to accurately identify a wanted object when that hand is reaching. In contrast, on average, children in the Gesture condition did not perform differently from chance (SD = 1.23), t(17) = −0.383, p = 0.70. Thus, the same handshape information that was accessible to children when the hand was extended in a reach was not accessible when the hand was presented in gesture space close to the body.

Discussion

The results from Study 2 demonstrate that 2.5-year-olds are significantly more likely to correctly use shape information in hands that are reaching than in hands that are gesturing. When an experimenter reached out with a whole-hand grip, children handed her a large toy, and when she reached out with a pincer grip, they handed her a small toy. However, when precisely the same handshapes were presented in gesture space, children selected between the toys at random. Understanding gesture thus seems to be more challenging for young children than understanding instrumental action, even when the information contained in the gesture and the reaching action is identical. In other words, a simple handshape taken directly from an action that we know babies can interpret (e.g. Ambrosini et al., 2013) is difficult to understand when presented as a gesture.

Why is the reach easier to understand? The reaching and gesture conditions were matched in terms of the handshape used to indicate the desired object. However, the conditions differed in two respects. First, the reach and the gesture conditions differed in the function that each served. The warm-up trials were designed to help children attribute different functions to the reach and to the gesture. The first few warm-up trials in the Reaching condition demonstrated that the actor can use her arm to successfully obtain objects. These warm-ups thus suggest that the function of the reach is to obtain the object (and reaching attempts on subsequent test trials appeared as failed attempts at an instrumental action). In contrast, the warm-up trials in the Gesture condition demonstrated that the actor was not even trying to reach for the object and was using her hand to indicate, or reference which object she wanted. These warm-ups thus suggest that the function of the gesture is to represent an object. Study 3 explores whether gesture’s representational function is what makes gesture challenging for young children.

A second difference between the reach and the gesture conditions is the placement of the hand. The conditions thus differed in the perceptual information they offered the child. In the Reaching condition, the actor extended her arm toward the object, thus placing the handshape cue close to the objects. In the Gesture condition, the actor held her hand close to her chest (in gesture space), thus placing the handshape cue at a distance from the objects. It may have been easier to interpret the handshape cue when it was physically close to the objects (in the reach) than when it was distant from the objects (in the gesture). Study 4 explores whether the cue’s distance from the objects is what makes gesture challenging for young children.

Study 3

We created a condition in which children saw the reaching action in a context that suggested its function was to represent an object rather than to reach for an object (the Reaching–Representational condition). In this condition, the physical properties of the reaching action were identical to the Reaching condition in Study 2 – the actor used a handshape cue that was presented in an extended arm held near the objects. But the hand was separated from the objects by a transparent window, which means that the extended arm could not be functioning as an instrumental action. Instead, its function was to represent which of the two objects the actor wanted. In other words, we maintained the perceptual features of the reach (the extended arm held near the objects) but manipulated its function to match those of a gesture. We reasoned that if children performed well in the Reaching condition in Study 2 because of the perceptual features of the reaching action, they should perform equally well in the Reaching-Representational condition in Study 3, since it has the same perceptual features. If, instead, children’s ability to use the handshape cue depends on seeing the reach as an instrumental action rather than a representational action, they should perform no better in the Reaching-Representational condition than they did in the Gesture condition in Study 2.

Methods

Participants

A separate group of 18 2.5-year-olds (M age = 31.6, range = 30.03 – 33.93 months; 9 females), with similar demographics, were recruited from the same database as Study 1. Five additional children were excluded because of experimental error (N = 2), because they failed to give only one toy during the warm-up trials (N = 3); or because of parental interference (N = 1). All participating families received either a small toy or $10 compensation for participating.

Procedure

The procedure was identical to Study 2, except that we used a new house, which had a transparent window that prevented the actor from obtaining the toy by reaching for it (See Figure 3B). In introducing the study to the child, the host again told the child the story about the experimenter moving into her new house, but the actor tapped on the window to demonstrate that it was a solid material that she could not reach through. On the initial warm-up trials, the actor pointed with an outstretched arm toward one of the toys (which brought her hand close to the object, separated from it only by the window), and the host then gave the actor the requested toy through a small opening in the side of the house. As in Study 2, after the host demonstrated passing the toys to the actor three times, the child was encouraged to “help give her the one she wants” for another 3 trials. If a child failed to give the actor one toy, the child was given three more warm-up trials.

Test trials followed the structure of Study 2. The actor labeled the toy using a word that referred to both objects while indicating in her handshape the size of the toy that she wanted. When the actor asked for the toy, she used an outstretched arm similar to the arm used in the Reaching condition in Study 2. Importantly, even though the hand cue was close to the toy, the clear window between the hand and the toy meant that the extended arm could not possibly be instrumental. Note that, in this study, the arm and hand were perceptually comparable to the arm and hand in the Reaching condition in Study 2 – the actor used an outstretched arm, the shape of the hand indicated the size of the object that she wanted, and the hand cue was positioned in the object space. However, the function of the movement was comparable to the Gesture condition in Study 2 – because the actor could not physically use her hands to obtain the toy (i.e., because the window impeded the reach), the function of the hand had to be to represent, rather than reach for, one of the objects.

Results

Children in the Reaching-Representational condition gave the correct toy on 3.22 (SD = 1.66) out of 6 test trials, a level of performance that did not differ from chance (t(17) = 0.81, p=.43 (see Figure 4)³. There were no effects of trial number, test item, handshape grip, or handshape order, (p’s >.17). In comparison to performance in Study 2, children in the Reaching-Representational condition performed significantly worse than children in the Reaching condition in Study 2 (M= 4.05), (β =−0.59, SE = 0.29, z = 2.07, p =.04), and no different from children in the Gesture condition in Study 2 (M=2.88) (β =0.23, SE = 0.29, z= −0.81, p = 0.42) (log-likelihood test: χ², 2 = 8.0957, p = .01).

Discussion

In Studies 2 and 3, we find that a reaching action that functions like a gesture is harder for children to understand than a reaching action that functions like a reach. Children in the Reaching-Representational condition performed at chance when required to give the actor a toy that matched her handshape cue. They performed similarly to children in the Gesture condition in Study 2, and significantly worse than children in the Reaching condition in Study 2. These findings suggest that children may have been able to use the handshape cue in the Reaching Condition in Study 2 because the cue was presented within an instrumental action. In contrast, the handshape cue in the Gesture condition in Study 2 and the handshape cue in the Reaching-Representational condition in Study 3 were presented, not within an instrumental action, but within a representational action – the experimenter used her hands to represent object features, rather than to instrumentally obtain an object. Gestures may thus be challenging for young children to interpret because of their status as representational actions.

However, as mentioned earlier, another difference between the Gesture and Reaching conditions was the distance between the handshape cue and the target objects. In the Reaching condition in Study 2, the actor placed the handshape cue right next to the objects, a necessary step in order to make the action seem like an attempted instrumental reach. In contrast, in the Gesture condition in Study 2, the actor placed the handshape cue near her chest in gesture space, quite far from the objects that the cue represented. In the Reaching-Representational condition in Study 3, the actor placed the handshape cue closer to the objects than in the Gesture condition, but not as close as in the Reaching condition – and there was, of course, a transparent plastic barrier between them and the objects. As a result, the distance between the handshape cue and the target objects may have played a role in determining a child’s success – using size information placed at a distance from the target objects may have been what made the two representational conditions challenging. Study 4 tests this hypothesis.

Study 4

In Study 4, we assessed children’s ability to use size cues at a distance, but in the context of an instrumental action (holding out a box that was ready to receive an object), rather than a gesture. The experimenter held a small box (which was ready to receive a small object) or a large box (which was ready to receive a large object) close to her chest in gesture space and distant from the target objects. This manipulation allowed us to test whether children are able to interpret size cues positioned at a distance from the target objects. If children are able to interpret the box cue at a distance, it will be clear that the challenge in interpreting gesture is not its distance from the target. Rather, the challenge may be the fact that gesture is a representational action and not an instrumental action.

Methods

Participants

Eighteen 2.5-year-olds (mean age = 30.99, range = 30.2 – 34.26 months; 9 females), with similar demographics, were recruited from the same database as Study 1. Eight additional children were excluded because they did not finish because of distress (N = 5); 2 because they failed to give only one toy during the warm-up trials (N=2); or experimental error (N = 1).

Procedure

The procedure in the Distance Control condition was identical to Study 2, except that rather than use a handshape cue indicating which object the actor wanted, she held boxes of different sizes signaling the size of the desired object. During the warm-up, the experimenter requested toys while holding a medium size box. During test, the experimenter held either a large box or a small box in gesture space (See Figure 3C).

Results

Children in the Distance Control gave the actor the correct toy on 3.78 (SD = 1.39) out of 6 trials, performance significant above chance (t(17) =2.52, p = 0.02). Thus, 2.5-year-olds can use some spatial cues that are far away from a target object to interpret an actor’s request. In this condition, there were no effects of trial number, test item, handshape order or handshape grip (p’s >.25).

Finally, we compared children’s performance in the Distance Control to the conditons in Studies 2 and 3. Children in the Distance Control condition performed significantly better than children in the Gesture condition in Study 2 (β = .61, SE = .29, z = 2.1, p=.03), and no different from children in the Reaching condition in Study 2 (β = .21, SE = .29, z = 0.69, p=.48). The difference between performance in the Distance Control condition and the Reaching-Representational condition in Study 3 did not reach significance (β =0.39, SE = 0.29, z = 1.31, p =.18) (log-likelihood test: χ², 3 = 9.23 p = .02).

Discussion

The results from Study 4 demonstrate that distance alone cannot account for the performance discrepancy between the Reaching and Gesture conditions in Study 2. It is not the case that children perform worse with any cue that is presented in front of the chest and far away from the target – when the actor held a box in gesture space, children could correctly figure out which object she wanted, despite the physical distance between the box and the toys.

Across studies 2–4, in the two conditions involving instrumental action (reaching for an object, and holding up a box to receive an object), children performed significantly above chance regardless of distance, whereas in the two conditions involving representational actions (gesture and reaching-representational), children’s performance was no different from chance. Together, our results show that 2.5-year-olds have trouble accessing information in gesturing hands, and that this may be because the hands are being used to represent, rather than to instrumentally act on the world.

General Discussion

Previous research has found that young children struggle with interpreting the meaning of iconic gestures (e.g., Namy, 2008; Namy, Campell, & Tomasello, 2004). However, it remains an open question as to why iconic gestures are so difficult for young children to understand. In the current studies, we focused on handshape information that young infants are known to be able to interpret on a rapid timescale when it is embedded in an instrumental action context. We presented this handshape information in a gesture context to determine whether the gestures are as easy to interpret as instrumental actions containing the same handshape information. We reasoned that if an understanding of gestures emerges from an understanding of instrumental actions, then handshape cues that children understand in reaching actions should be equally understandable in gesture. We found that this was not the case. Although 2.5-year-old children were able to interpret pincer and whole-hand grips when they were presented in a simple gestural context (a context containing language that may have guided the child’s attention toward the hand, Study1), they were significantly more successful at using these handshape cues when the handshapes were embedded in an instrumental reach than when they were embedded in a gesture (Study 2). This finding suggests that there are factors that children need to access to interpret gestures, above and beyond the factors needed to interpret instrumental actions.

We also pursued the reason that gestures are challenging for young children to interpret, and found evidence that the difficulty stems from the fact that gestures are representational actions. We embedded the perceptual properties of a reaching action (an extended arm held near the target objects) in a context where the reach served a representational, rather than an instrumental, function; in other words, where it served the function of a gesture (Study 3). We found that, under these conditions, 2.5-year-old children were unable to glean information from the handshape cues in the reach. Thus, when a reach is used to convey information about objects, rather than to grab objects, handshape information in the reach is no longer accessible to young children. These findings suggest that what may make it difficult for young children to interpret gesture is not its form, but rather its function as a representational action. Note that the children must have engaged in top-down reasoning in order to recognize that the extended arm was serving as a representational action when it was situated behind a transparent plastic barrier. Further research is needed to better understand this process.

Finally, we showed that children’s difficulty in interpreting hands in gesture space cannot be attributed to the physical distance between the hands and the target objects since children in the Distance Control condition (involving an instrumental action where there was distance between the hands and the target objects, Study 4) performed no worse than children in the Reaching condition (also involving an instrumental action but where there was no distance between the hands and target objects, Study 2). Moreover children in the Distance Control condition performed significantly better than children in the Gesture condition (involving a representational action, Study 2), even though there was distance between the hands and target objects in both of these conditions. We conclude that children’s difficulty in interpreting iconic gestures resides not in the fact that it is produced in gesture space, but in the fact that it represents information.

It is notable that the gesture we chose is a type of gesture that children have specific difficulty with – iconic shape gesture. Many studies have found that children’s ability to map gestures that reference an object’s perceptual features develops later than their ability to interpret gestures that reference an an action (Hodges et al., in press; Magid & Pyers, 2017; Tolar et al., 2008). In our study, we find that the difficulty in interpreting shape gestures persists even when the gesture is derived from an action that contains precisely the same shape information (in its grip aperture). Children easily understood this handshape in the context of a reaching action, but struggled when the handshape was presented as a static shape gesture. Our findings thus confirm previous work showing that gestures that reference perceptual features of objects are particularly difficult for young children. Our findings also add to the literature by demonstrating that this difficulty is not because children cannot understand the link between the hand and object since they are able to glean information from this link in action contexts.

The unique challenge of shape is reflected in the sign language literature as well. Young deaf children prefer to use iconic action variants for signs (a writing hand for the sign PEN) than iconic perceptual feature variants (upward index finger representing a thin, elongated object for the sign PEN), in cases where both signs are valid labels for a given object (Ortega, Sümer & Özyürek, 2016). This preference shifts over time, with adults demonstrating a preference for shape signs over action signs (Ortega et al., 2016). Thus, across both gesture and sign, young children appear to have more difficulty interpreting shape forms than action forms. Although our results are specific to the feature of size, we suspect that the results would be similar if other shape gestures had been used.

The different performance levels in the Reaching and Gesture conditions in Study 2 contribute to a growing body of literature showing that instrumental actions and representational gestures have distinct effects on cognitive processes across the lifespan (Beilock & Goldin-Meadow, 2010; Kelly, Healy, Özyürek, & Holler, 2014; Lucca & Wilburn, 2016; Novack, Congdon, Hemani-Lopez & Goldin-Meadow, 2014; Novack, et al., 2015). Although gestures are a type of action, and some have argued that gestures grow out of action simulations (see Hostetter & Alibali, 2008), our findings suggest that gestures cannot be reduced to actions in terms of their functions (see Novack & Goldin-Meadow, 2017). It seems that gesture’s status as a representational action leads to specific and unique effects on cognition, which are not the same as the effects of instrumental actions, and which seem to make it hard for 2.5-year-olds to glean information from gesture.

One strength of our design is that we created identical handshapes across the reaching and gesture conditions – allowing tight control over the form of movement. However, there are some inevitable differences between these two conditions, which may also have contributed to the observed effect. First, recall that the warm up portion of the experiment was designed to provde contextual cues about how the experimenter uses her hands – in the Reaching condition she used them to reach and obtain objects (i.e., she used them for instrumental purposes), whereas in the gesture condition she used her hands to point at objects (i.e, she used her hands for referential purposes). This warm up experience conveyed to the children the experimenter’s intention, thus contributing to the contextual factors influencing their processing of the handshape cue. However, one unintended consequence of the difference between these two conditions is that children may have seen the warm up phase and test phase as more distinct in the gesture condition than the reaching condition. That is, the change from a neutral reaching hand to a pincer or whole hand grip may be less distinct than a change from a pointing gesture to an iconic shape gesture, a change that may have been a contributing factor to children’s poorer performance in the Gesture condition.

Second, one might argue that “viewpoint” differs between the Reaching and Gesture conditions in our study. McNeill (1992) has described gestures as having either character viewpoint (a gesture that takes the point of view of an object, animal, or person) or observer viewpoint (a gesture that represents an object, animal, or person), and has found that character viewpoint gestures are more common in young children’s early gesture production than observer viewpoint gestures (see also Õzçalişkan & Goldin-Meadow, 2011). The hand in the Gesture condition in Studies 1 and 2 represents an object and thus takes an observer’s perspective. In contrast, the hand in the Reaching condition is acting on (or at least trying to act on) the object and thus takes the character’s perspective. This difference in viewpoint might account for the children’s poor performance in the Gesture condition, relative to the Reaching condition, in Study 2. Note, however, that the gestures in Study 3 were different from the gestures in Study 2 – they embedded the pincer and whole-hand grips into a reaching action and, in this sense, take the character’s perspective. Nevertheless, the reaching gestures in Study 3 were still more difficult for children than the reaching actions in Study 2, despite both reflecting character viewpoint. It is therefore unlikely that viewpoint can fully account for the differences we find between gesture and action in Study 2.

Even though the 2.5-year-olds in our sample struggled to interpret the handshapes when the actor was gesturing compared to when she was reaching, we know that eventually children will be able to interpret handshape in both contexts. Study 1 shows that in the simplest of situations, 2.5-year-olds can make sense of handshape information in gestures, although their ability is relatively fragile – we found significant improvement between 2.5-year-olds (who were barely above chance) and 4-year-olds (who performed at ceiling). These results raise the open question of how children come to develop the ability to successfully interpret handshape information in gestures.

One possibility is that our ability to understand another’s actions (be they representational or instrumental) develops based on our experience producing those types of actions. Prior work has demonstrated that infants’ understanding of other’s reaches is influenced by their own reaching experiences (Filippi & Woodward, 2016; Gerson & Woodward, 2014; Sommerville, Woodward, & Needham, 2005). The same may be true for gesture. If so, then children’s superior performance in the reaching action condition may be due to their extensive experience producing reaching actions themselves (relative to their limited experience producing gestures). Young children rarely produce iconic gestures, particularly when compared to their production of other gestures like pointing (Iverson, Capirci & Caselli, 1994; Nicoladis, Mayberry & Genesee, 1999; Özçalşşkan & Goldin-Meadow, 2005). Further, feature-based shape gestures (like size gestures), the forms we used in our study, are even less common in children’s own gesture production (Õzçalişkan, Gentner & Goldin-Meadow, 2014). Importantly, there is some evidence to suggest that exposure to seeing, and experience producing, a range of iconic hand forms may support earlier development of iconic mapping. Magid and Pyers (2017) found that by age three, deaf preschool children exposed to American Sign Language were able to interpret novel iconic shape signs, whereas hearing children who had not been exposed to sign langauge could not reliably interpret iconic shape signs until age four. Similarly, Boyatzis and Watson (1993) found that young children were not able to imitate gesture forms that they did not spontaneously produce themselves, highlighting the importance of one’s own gesture production. Studies that directly manipulate children’s experience producing iconic shape gestures, and that then observe whether understanding these gestures improves as a function of the experience, are needed to explore this hypothesis.

Alternatively, the development of children’s gesture processing may be related to non-motor development, such as improvements in representational processing capacities or even language processing. Children’s ability to interpret other types of representational forms (e.g., maps, toy replicas, pictures) undergoes rapid improvement in the early years (e.g., Simcock & DeLoache, 2006; Smith, 2003; DeLoache, 1995), suggesting that a domain-general representational processing capacity may underlie children’s ability to interpret iconic gestures. Indeed, our study used a gesture that requires a representational leap and, as such, may have been particularly taxing for young children. In the iconic shape gestures used in our study, the hands represent a feature of the object (its size) and thus are relatively abstract, particularly compared to action gestures in which the hands represent hands acting on the world (see Ortega & Magid & Pyers, 2017, for similar discussion). The difficulty of the representational task may thus have contributed to children’s relatively poor performance on the gesture interpretation task.

Finally, gesture is part of an integrated system with spoken language (McNeill, 1992), and improvements in this integration over development may contribute to improvements in gesture interpretation. Past research has found that the abilty to integrate spoken language with the information in an iconic gesture develops between the ages of 2 and 5 (Sekine, Sowden, & Kita, 2015; Standfield, Williamson, & Özçalşşkan, 2014). The 2.5-year-old children in our study may have struggled with integrating the gestures they saw with the speech they heard. Indeed, the difference in performance between the 2.5-year-olds in Study 1 and in Study 2 provides some support for this hypothesis. In Study 1, 2.5-year-olds performed slightly above chance upon hearing a co-speech utterance that directly referenced the gesture (e.g., “this duck”). In contrast, in Study 2, children’s performance was not above chance when they heard a co-speech utterance that did not reference gesture (e.g., “ooo, a duck!). Integrating speech that references gesture may be easier (at all ages) than integrating speech that does not directly reference gesture (although, of course, other differences in the experimental designs of Studies 1 and 2 may have been responsible for the difference in performance). Future work should consider this question.

These questions aside, we have shown that the same handshape information can be differentially challenging for young children depending on the function it serves. When children see hands as involved in an instrumental action such as reaching, they can easily access the information in the hands to make inferences about a person’s intentions. However, when children see hands as involved in a representational action such as a gesture, they struggle to interpret the iconicity in the hands. These findings suggest that what makes gesture challenging for young children is its function as a representational action. Overall, our findings deepen our understanding of how children process different types of movement, and open up exciting new questions about action, gesture, and how children come to interpret a world full of informative movements.

Appendix

Warm up example (children saw 4–7 unique sets)	Dimensions in inches (height, width, depth)
	Boat: 2 × 2 × 2.75 Dolphin: 1.5 × 2.5 × 3.25
Test items (children saw all items)	Dimensions in inches (height, width, depth)
	Small Ball: 1.33 × 1.33 × 1.33 Large Ball: 4.25 × 4.5 × 4.25
	Large Duck: 4 × 2.75 × 4.5 Small Duck: 1.75 × 2 × 3
	Small Dog: 6.75 × 2.5 × 8 Large Dog: 1.75 × 2 × 1
	Small Car: 2.75 × 2.75 × 5 Large Car: 1 × 1 × 2
	Small Shoe: 3 × 2.5 × 5 Large Shoe: 1.75 × 1.25 × 2.75
	Small Bottle: 5.75 × 2 × 2 Large Bottle: 3.75 × 1 × 1