Learning through gesture

Abstract

When people talk, they move their hands — they gesture. Although these movements might appear to be meaningless hand waving, in fact they convey substantive information that is not always found in the accompanying speech. As a result, gesture can provide insight into thoughts that speakers have but do not know they have. Even more striking, gesture can mark a speaker as being in transition with respect to a task — learners who are on the verge of making progress on a task routinely produce gestures that convey information that is different from the information conveyed in speech. Gesture can thus be used to predict who will learn. In addition, evidence is mounting that gesture not only presages learning but can play a role in bringing that learning about. Gesture can cause learning indirectly by influencing the learning environment or directly by influencing learners themselves. We can thus change our minds by moving our hands.

What counts as gesture?

People move their hands when they talk — they gesture. The question is why? Most people assume that gesture is mere hand waving, a behavior that has no function and that should be discouraged. But this assumption turns out to be incorrect. Gesturing plays an important role in how we think and learn. The goal of this paper is to explore this function.

Gesture has been an object of scholarly attention for at least 2000 years, across domains as diverse as philosophy, rhetoric, theater, divinity, and language. Gesture came into modern day focus in the semiotic world as one of five nonverbal behaviors catalogued by Ekman and Friesenⁱ: affect displays, whose primary site is the face, convey emotions; regulators are head movements or slight changes in body position that maintain the give-and-take between individuals; adaptors are fragments or reductions of a previously learned adaptive hand movement maintained by habit; emblems have conventional forms and meanings and vary across cultures (e.g., the thumbs-up, okay, shush are all emblems in American culture) and are most often what people mean when they say they are talking about gesture. These four nonverbal behaviors can all be produced with speech, but they need not be.

The fifth nonverbal behavior, which is our focus here, is always produced with speech. This last category, called illustrators by Ekman and Friesen¹, gesticulation by Kendonⁱⁱ, and plain old gesture by McNeillⁱⁱⁱ, is not only tied to speech but often illustrates the message conveyed in the speech it accompanies. For example, a child says that the water level in one container is lower than another and illustrates the point by indicating with his hand first the height of the water in the short container and then the height of the water in the taller container (Figure 1, top pictures). The child’s gestures “match” his speech and thus reinforce the information conveyed in speech.

Figure 1.

Examples of children explaining why they think the amount of water in the two containers is different. Both children say that the amount is different because the water level is lower in one container than the other. The child in the top two pictures conveys the same information in gesture (he indicates the height of the water in each container) — he has produced a gesture-speech match. The child in the bottom two pictures conveys different information in gesture (she indicates the width of each container) — she has produced a gesture-speech mismatch

There are times, however, when gesture conveys information that goes beyond the information conveyed in speech^iv. For example, another child also says that the water in one container is lower than the other but, instead of indicating height in gesture, this child indicates width: She uses two vertically held flat palms to indicate the width of the shorter container and then a C-shaped hand to indicate the width of the taller container (Figure 1, bottom pictures). The child’s gestures “mismatch” her speech and thus add to the information that her speech conveys.

Gesture-speech mismatches of this sort are not uncommon and, in fact, are found in a wide variety of tasks and ages: toddlers going through a vocabulary spurt^v; preschoolers explaining a game^vi, counting a set of objects^vii,viii, or learning to mentally rotate objects^ix; elementary school children explaining Piagetian conservation problems^x, mathematical equations^xi, and seasonal change^xii; children and adults discussing moral dilemmas^xiii; children and adults explaining how they solved Tower of Hanoi puzzles^xiv; adolescents explaining when rods of different materials and thicknesses will bend^xv; adults explaining how gears work^xvi,xvii; adults describing pictures of landscapes, abstract art, buildings, people, machines, etc.^xviii; adults describing problems involving constant change^xix; adults narrating cartoon stories^xx,3,xxi.

We focus in this paper, not on how gesture is produced (that is, its mechanism; see Hostetter and Alibali for a review^xxii), but rather on why gesture is produced — its function. Although gesture can serve a variety of functions, we concentrate here on only one — gesture’s role in learning. From this point of view, the important point about gesture-speech mismatches is that they are often (although not always^xxiii) associated with learning and thus have cognitive significance.

Gesture predicts learning

Learners who are on the verge of making progress on a task gesture differently from those whose knowledge is less advanced. In particular, learners who produce gesture-speech mismatches when asked to explain how they solved a task are more likely to profit from instruction on the task than children who produce only matches^{10,11, xxiv,xxv}. Take, for example, the two children pictured in Figure 1. Both are considered nonconservers — when asked whether water that has been poured from a tall, thin container into a shorter and wider container is still the same amount, both children say “no” and give the same explanation for their beliefs in speech, “it’s different because this one’s lower than that one.” However, the children differ in the gestures that they produce along with their explanations and these gestures make a difference. The child who produces gesture-speech mismatches is more likely to improve on the posttest when given instruction in conservation than the child who produces gesture-speech matches. Figure 2 (left graphs) presents the proportion of children who improved on the posttest after instruction in conservation: Mismatching children were significantly more likely to improve than matching children¹⁰.

Figure 2.

The proportion of children who improved when given a lesson in conservation (left graph, Reference 10) or mathematical equivalence (right graph, Reference 11), as a function of the child’s status as a matcher (white bars) or mismatcher (blue bars) prior to the lesson. The lesson was administered by the experimenter.

The same effect has been found in children learning mathematical equivalence. Children who solve problems such as 5+3+6=__+6 incorrectly but justify their incorrect solution by producing gestures that convey a different problem-solving strategy from their speech (e.g., saying, “I added the 5, the 3, and the 6, and put 14 in the blank,” an add-to-equal-sign strategy, while pointing at the 5, the 3, the 6 on the left side of the equation, and the 6 on the right side of the equation, an add-all-numbers strategy) are particularly likely to profit from instruction in mathematical equivalence — more likely than children who justify their incorrect answers by producing gestures that convey the same information as their speech (e.g., again saying, “I added the 5, the 3, and the 6, and put 14 in the blank,” while pointing at the 5, the 3, and the 6 on the left side of the equation, i.e., producing an add-to-equal-sign strategy in both speech and gesture). Figure 2 (right graphs) presents the proportion of children who improved on the posttest after receiving instruction in mathematical equivalence: Mismatching children were significantly more like to improve than matching children¹¹.

We have seen that when given instruction in a laboratory setting, children who produce mismatches on a task are more likely to profit from instruction in that task than children who produce matches. Do we see the same effect when teachers teach children in a naturalistic tutoring session? Goldin-Meadow and Singer^xxvi asked teachers to individually instruct a series of children in mathematical equivalence. The children were classified into three groups on the basis of the gestures that they produced before and during the lesson: one group of children produced mismatches on the pretest and throughout the lesson; a second group did not produce mismatches on the pretest but began producing them during the lesson; the third group did not produce mismatches at any time during the study. Children who produced mismatches from the beginning solved more problems correctly on the posttest than children who began producing mismatches during the lesson, who, in turn solved more problems correctly than children who never produced mismatches (none of the children solved any problems correctly on the pretest (Figure 3, Reference 26).

Figure 3.

The number of problems children solved correctly after a lesson in mathematical equivalence, as a function of the mismatches the child produced on the pretest and during the lesson: no mismatches at all, mismatches only during the lesson, mismatches on the pretest and throughout the lesson. None of the children had solved any problems correctly on the pretest. The lesson was administered by a teacher who had observed the child explain to the experimenter the solutions he or she gave to the pretest problems (adapted from Figure 1, Reference 26).

When a speaker produces a gesture-speech mismatch, the information conveyed in gesture is, by definition, different from the information conveyed in the accompanying speech. Consider the child who produced an add-all-numbers strategy in gesture while giving an add-to-equal-sign strategy in speech. The add-all-numbers strategy was conveyed uniquely in gesture in that response. However, it is possible that this child is able to articulate the add-all-numbers strategy in speech, and does so in other responses. Alternatively, the information conveyed in gesture in a mismatch may be accessible only to gesture. If so, this child should not be able to articulate the add-all-numbers strategy in speech in any of his responses. Goldin-Meadow, Alibali and Church^xxvii explored these alternatives by examining the entire set of responses children produced on a mathematical equivalence test. They divided the problem-solving strategies children produced into those produced only in speech throughout the problems, those produced only in gesture throughout the problems, and those produced in both gesture and speech (a strategy did not have to be produced in both gesture and speech on the same problem to be a candidate for this category; the strategy just had to appear in gesture and in speech somewhere across the problems).

Figure 4 presents the number of different strategies children produced as a function of modality. The children are divided into those who produced gesture-speech mismatches on the pretest (mismatching children) and those who did not (matching children). There are two interesting results. The first is that very few children of either type produced a problem-solving strategy in speech without also producing it in gesture on some other problem. The second is that mismatching children differ from matching children not in the number of different strategies they produced in speech (with or without gesture), but in the number of strategies that they produce uniquely in gesture — mismatching children produced many, matching children produced few. Thus, mismatching children appear to have more different types of problem-solving strategies in their repertoires than matching children, but all of the “extra” strategies are accessible only to gesture.

Figure 4.

The number of different mathematical equivalence problem-solving strategies mismatching and matching children had in their repertoires, classified according to the modality in which the strategy was produced: uniquely in speech, in both speech and gesture (not necessarily in the same response), uniquely in gesture (data from Reference 27).

Gesture can thus mark learners as being ready to change their knowledge, whether they are taught in the lab or in a naturalistic tutorial. But evidence is mounting that gesture not only reflects readiness for knowledge change but also plays a role in bringing that change about. Gesture can play a role in learning in (at least) two ways: (1) The gestures that learners produce could have a direct on what they are learning. Gestures contain information about whether learners are ready to learn a task. If communication partners are able to “read” those gestures, the partner could then provide input that facilitates learning. In other words, learners’ gestures could change the learning environment. (2) The gestures that learners produce could also have a direct effect on how they are learning. Learners’ gestures could change the learners themselves.

Gesture causes learning through its effects on the learning environment

Do learners elicit different kinds of input as a function of the gestures they produce? To make this argument, we must first show that ordinary listeners, listeners who have not been trained to code gesture, are able to interpret the spontaneous gestures that speakers produce. There is, in fact, evidence that listeners can glean information from the gestures produced by children participating in conservation and mathematical equivalence tasks. These effects have been found in adults and children observing child speakers on a videotape^{xxviii,xxix,xxx,xxxi}; in adults watching children and reacting to them on-line^xxxii; and, most importantly, in adults and children interacting with one another in a naturalistic setting^26,xxxiii. In short, listeners can read the spontaneous gestures that speakers produce.

The next step is to show that listeners change how they respond to a speaker as a function of that speaker’s gestures. Recall that Goldin-Meadow and Singer²⁶ asked teachers to individually instruct children in mathematical equivalence. They observed the kind of instruction teachers gave to children in each of the three groups. Figure 5 presents the data. Teachers used two different strategies in speech significantly more often to instruct children who produced mismatches (either from the beginning of the study or beginning only during the lesson) than to instruct children who never produced mismatches (Figure 5, top). Interestingly, the teachers also produced more of their own mismatches (typically including two correct strategies, one in speech and the other in gesture) significantly more often to instruct children who produce mismatches than to instruct children who never produced mismatches. Thus, teachers do notice the gestures learners produce and they change their instruction accordingly.

Figure 5.

Number of different types of correct strategies (top graph) and gesture-speech mismatches (bottom graph) teachers produced when instructing children who produced no mismatches at all, children who produced mismatches only during the mathematical equivalence lesson, and children who produced mismatches on the pretest and throughout the lesson (adapted from Figure 2, Reference 26).

The final step is to demonstrate that children profit from the input that their gestures elicit from teachers. Singer and Goldin-Meadow^xxxiv designed a mathematical equivalence lesson based on the instruction that teachers spontaneously gave children who produced mismatches. The lesson included either one correct strategy (equalizer) or two correct strategies (equalizer and add-subtract) in speech; in addition, the instruction either contained no gestures at all, matching gestures, or mismatching gestures. There were thus six different training groups. Figure 6 presents the data. Note that the bars on the left (one spoken strategy in the lesson) are all higher than the bars on the right (two spoken strategies in the lesson), indicating that children improved significantly more after the lesson if they had been given one strategy in speech than if they had been given two. Thus including two strategies in speech in the lesson was an ineffective teaching strategy. In contrast, including mismatches in the lesson was very effective. Note that in both sets of bars, children improved significantly more after the lesson if their lesson included mismatching gestures than if it included matching gestures or no gestures at all. The lesson that was most effective contained the equalizer strategy in speech (“to solve this problem, you need to make one side equal to the other side”), combined with the add-subtract strategy in gesture (pointing at the three numbers on the left side of the equation and then producing a ‘take away’ gesture under the number on the right side). In other words, a lesson containing two strategies was particularly effective, but only if the two strategies were produced in different modalities. Including gesture in instruction has, in general, been found to promote learning in mathematical equivalence tasks^xxxv,xxxvi, conservation tasks^xxxvii and tasks involving symmetry^xxxviii.

Figure 6.

Mean number of problems children solved correctly after a lesson in mathematical equivalence, as a function of the instruction children received during the lesson: children were either taught one strategy in speech (left bars) or two strategies in speech (right bars). In addition, the spoken strategies in the instruction were either accompanied by no gesture, matching gesture, or mismatching gesture. None of the children solved any problems correctly on the pretest (adapted from Figure 1, Reference 34).

Taken together, the findings suggest that the gestures that learners produce convey meaning that is accessible to their communication partners. The partners, in turn, alter the way they respond to a learner as a function of that learner’s gestures. Learners then profit from those responses, which they elicited through their gestures. Gesture can thus play a causal role in learning indirectly through the effect it has on the learning environment.

Gesture causes learning through its effects on the learner

Another reason that including gesture in a lesson may be good for learning is that seeing a teacher gesture encourages learners to produce gestures of their own. Indeed, Cook and Goldin-Meadow^xxxix found that children were more likely to gesture during a lesson when their teacher gestured. Importantly, those children who gestured during the lesson were more likely to profit from the lesson than those who did not gesture. Gesturing can help children get the most out of a lesson. But the children in this study were not forced to gesture — they chose to. The children who chose to gesture may have been more ready to learn than the children who chose not to gesture. If so, the fact that they reproduced the experimenter’s gestures may have been a reflection of that readiness to learn, rather than a causal factor in the learning itself. To address this concern, gesture needs to be manipulated more directly — all of the children in the gesture group must reproduce the experimenter’s hand movements during the lesson.

Broaders, Cook, Mitchell and Goldin-Meadow^xl asked children to explain how they solved six mathematical equivalence problems with no instructions about what to do with their hands, and then asked them to solve a second set of comparable problems. Some of the children were told to move their hands as they explained their solutions to this second set of problems; some were told not to move their hands; and some were given no instructions at all. Figure 7 (top) presents the data. Children who were told to gesture on the second set of problems added significantly more new strategies to their repertoires than children who were told not to gesture and than children given no instructions at all. Most of those strategies were produced uniquely in gesture, not in speech, and, surprisingly, most were correct. The children who were told to gesture had been turned into mismatchers — they produced information in gesture that was different from the information they produced in speech.

Figure 7.

The top graph displays the mean number of problem-solving strategies children added to their repertoires on a second set of mathematical equivalence problems, as a function of the instructions given on this set: told to gesture when explaining their solutions, told not to gesture, and given no instructions about their hands (control). The bottom graphs displays a replication of this finding with two groups (told to gesture, told not to gesture; left graph), and the mean number of problems these two groups of children solved correctly after they were given a lesson in mathematical equivalence (right graph) (adapted from Figures 1 and 3, Reference 40).

Were these created mismatchers also ready to learn? To find out, Broaders et al.⁴⁰ gave another group of children the same instructions to gesture or not to gesture while solving a second set of mathematical equivalence problems, and then gave all of the children a lesson in mathematical equivalence. Children told to gesture again added more strategies to their repertoires after the second set of problems than children told not to gesture (Figure 7, bottom left). Moreover, children told to gesture showed more improvement on the posttest than children told not to gesture (Figure 7, bottom right), particularly if the children had added strategies to their repertoires after being told to gesture. Being told to gesture thus encouraged children to express new ideas that they had previously not expressed, which, in turn, led to learning.

Telling children to gesture thus encouraged them to convey previously unexpressed ideas, which, in turn, made them receptive to instruction that leads to learning. Even children who produced no gestures at all at the beginning of the study were likely learn the task if told to gesture. In this way, gesturing can play a causal role in learning. The question that this study does not address is whether the instruction to gesture created new knowledge, or activated knowledge that the children already had. To address this concern, we need to engineer learners’ hand movements so that they instantiate information that the learners do not yet have in their repertoires.

Cook, Mitchell, and Goldin-Meadow^xli taught children words and hand movements that instantiated an equalizer strategy prior to a math lesson, and then asked the children to reproduce those words and/or gestures during the lesson itself. One group of children was taught to say the following words: “to solve this problem, I need to make one side equal to the other side,” an equalizer strategy in speech. Another was taught to make the following hand movements: sweep with the left palm under the left side of the equation, followed by a sweep with the right palm under the right side of the equation, an equalizer strategy in gesture. The third group was taught to say the words and produce the hand movements at the same time, an equalizer strategy in both speech and gesture. All of the children were then given the same lesson; the experimenter taught the children the equalizer strategy using both speech and gesture. The only difference among the groups during the lesson was the children’s own behavior — the children repeated the words and/or hand movements they were taught before and after each problem they were given to solve.

These self-produced behaviors turned out to make a big difference, not in how well the children did at posttest (children in all three groups made equal progress right after the lesson), but in how long they retained the knowledge they had been taught. If children retained the knowledge learned during the math lesson, we should be able to predict their performance on the follow-up test 4 weeks after instruction from their performance on the posttest immediately following instruction. Cook et al.⁴¹ used a regression model to predict follow-up test performance, with posttest performance and condition (speech, gesture, gesture+speech) as factors, and found that regression coefficients differed across the three groups (Figure 8). The unique predictive power was significantly greater for the two gesture groups than for the speech group. Reiterating the instructor’s words did not appear to be particularly effective in helping children retain knowledge they had apparently learned — unless those words were accompanied by gesture. Interestingly, the unique predictive power was not reliably different for the gesture and gesture+speech groups; children in these conditions who improved on posttest tended to maintain their gains on follow-up. Overall, children told to gesture during instruction retained 85% of their posttest gains, on average, compared to 33% for children told to speak and not gesture. Thus, the children’s own hand movements worked to cement what they had learned, suggesting that gesture can play a role in knowledge change by making learning last.

Figure 8.

Regression lines relating performance on the 4-week follow-up to performance on the immediate posttest, as a function of the modality children were told to use when expressing the equalizer strategy during the lesson: in speech alone (β=.33, ns); in speech and gesture (β=.92, p<.0001); in gesture alone (β=.80, p<.0001) (adapted from Figure 2, Reference 41).

But can gesture, on its own, create new ideas? The information that the children produced in gesture in the Cook et al. study⁴¹ (the equalizer strategy) was reinforced by the equalizer information they heard the instructor produce in both speech and gesture during the lesson. Thus, their gestures did not provide information that was unique to gesture. To determine whether gesture can create new ideas, Goldin-Meadow, Cook and Mitchell^xlii also taught children words and hand movements to produce before the lesson began. But in this study the hand movements instantiated a different strategy from the one conveyed in the words they were taught. All three groups were taught to say the equalizer strategy in speech, “to solve this problem, I need to make one side equal to the other side.” One group was taught only these words and no hand movements (no gesture group). One group was taught to say the equalizer strategy while producing a V-hand under the 6+3 in the problem 6+3+5=__+5 and then pointing at the blank, a grouping strategy in gesture (correct gesture group). The third group was taught to say the same words but to produce a partially correct version of the grouping strategy in gestures (partially correct gesture group) — a V-hand under the 3+5 followed by a point at the blank (these movements are partially correct in that the V-hand highlights the fact that two numbers on the left side of the equation can be grouped, and the two gestures together highlight the fact that there are two sides to the equation; the movements are incorrect in that the V-hand isolates the wrong two numbers to be grouped). All of the children were given the same lesson in mathematical equivalence; the experimenter taught them the equalizer strategy in speech and produced no gestures. The children were required to produce the words or words+gestures they had been taught before and after each problem they solved during the lesson.

To analyze the data, Goldin-Meadow et al.⁴² rank-ordered the three conditions from no gesture (−1) to partially correct gesture (0) to correct gesture (+1) and regressed the rank ordering on number of problems solved correct on the posttest (none of the children solved any of the problems correctly on the pretest). The more correct their gestures during the lesson, the better children performed on the posttest (Figure 9, top). Importantly, this effect was mediated by whether a child produced the grouping strategy for the first time in speech after the lesson (Figure 9, bottom). Recall that the experimenter did not use the grouping strategy in either gesture or speech, and the children only produced the grouping strategy in gesture and not in speech. Thus, the strategy had to have come from the children’s own hands, suggesting that gesture can introduce new knowledge into the system.

Figure 9.

The top graph displays the mean number of problems solved correctly on the posttest after the lesson, as a function of the gestures children were told to produce during the lesson: no gesture, partially correct gesture, correct gesture. The bottom displays a mediation analysis demonstrating that the effect seen in the top graph (that is, the effect of gesture condition on posttest performance) disappears when the number of children who added the grouping strategy to their spoken repertoire is included in the analysis (Figures 2 and 3, Reference 42).

The mechanisms responsible for gesture’s learning effects

Gesture can play a causal role in learning, but what are the mechanisms that underlie these effects? And are the mechanisms responsible for the effect that gesture has on learning unique to gesture? Gesture may be special only in the sense that it makes efficient use of ordinary learning mechanisms; for example, cues may be more distinctive when presented in two modalities than in one. On the other hand, it is possible that traditional principles of learning and memory (e.g., distinctiveness, elaboration, cue validity, cue salience, etc.) will, in the end, not be adequate to account for the impact that gesture has on learning; in this event, it will be necessary to search for mechanisms that are specific to gesture.

We explore here one general mechanism by which gesture may promote learning, although there are undoubtedly others (for example, there is evidence that gesturing introduces action information into speakers’ mental representations of a problem, which then impacts how they think about and solve the problem^xliii,xliv). Gesturing can have an impact on thinking by affecting working memory. Gesturing while speaking might require motor planning, execution, and coordination of two separate cognitive and motor systems. If so, gesturing might increase speakers’ cognitive load. Alternatively, gesture and speech might form a single, integrated system in which the two modalities work together to convey meaning. Under this view, gesturing while speaking would reduce demands on the speaker’s cognitive resources (relative to speaking without gesture), and free up cognitive capacity to perform other tasks.

To distinguish these alternatives and to determine the impact of gesturing on a speaker’s cognitive load, Goldin-Meadow, Nusbaum, Kelly & Wagner^xlv explored how gesturing on one task (explaining a math problem) affected performance on a second task (remembering a list of words or letters) carried out at the same time. If gesturing increases cognitive load, gesturing while explaining the math problems should take away from the resources available for remembering. Memory should then be worse when speakers gesture than when they do not gesture. Alternatively, if gesturing reduces cognitive load, gesturing while explaining the math problems should free up resources available for remembering. Memory should then be better when speakers gesture than when they do not. Both adults and children remembered significantly more items when they gestured during their math explanations than when they did not gesture (Figure 10). Gesturing appeared to save the speakers cognitive resources on the explanation task, permitting the speakers to allocate more resources to the memory task.

Figure 10.

The proportion of short and long word lists that children (left graph) or letter lists that adults (right graph) remembered while simultaneously explaining how they solved a math problem and either gesturing or not gesturing while doing so. Children remembered more 1- and 3-word lists while gesturing than while not gesturing. Adults showed a ceiling effect on the 2-letter lists, but remembered more 6-letter lists while gesturing than when not gesturing (adapted from Figure 1, Reference 45).

Why does gesturing lighten cognitive load? Perhaps it is the motor aspects of gesture that are responsible for the cognitive benefits association with producing gesture. If so, the meaning of the gesture should not affect its ability to lighten cognitive load. Wagner, Nusbaum and Goldin-Meadow^xlvi replicated the cognitive load effect on adults asked to remember lists of letters or locations on a grid while explaining how they solved a factoring problem. The adults remembered more letters or locations when they gestured than when they did not gesture. But the types of gestures they produced mattered. In particular, gestures that conveyed different information from the accompanying speech (mismatching gesture) lightened load less than gestures that conveyed the same information as the accompanying speech (matching gesture). Thus, the effect gesture has on working memory cannot be a pure motor phenomenon — it must stem instead from the coordination of motor activity and higher order conceptual processes. If the motor aspects of gesture were solely responsible for the cognitive benefits associated with gesture production, mismatching gestures should be as effective in promoting recall as matching gestures — after all, mismatching gestures are motor behaviors that are physically comparable to matching gestures.

Interestingly, this effect — mismatching gestures lightening cognitive load less than matching gestures — was not found in children who were in transition with respect to the task they were explaining. The adults in the Wagner et al. study⁴⁶ were all experts in solving factoring problems. In contrast, Ping and Goldin-Meadow^xlvii studied the effects of gesturing on cognitive load in children explaining their responses to a liquid conservation task. Most of the children did not know how to solve the problems and many were in transition. Ping and Goldin-Meadow replicated the original findings — gesturing lightened cognitive load. But mismatching gestures lightened cognitive load more than matching gestures for the novice children — the opposite pattern found for the expert adults.

Novices and experts also differ in the nature of their mismatches. As described earlier, when a child novice who is in transition produces a mismatch, the information conveyed in the gestural component of that mismatch is typically not found anywhere in that child’s spoken repertoire — the information is implicit and accessible only to the manual modality (see Figure 4). In contrast, when an adult expert produces a mismatch, the information conveyed in the gestural component of that mismatch can typically be found in speech in another response — the information is explicit and accessible to speech as well as gesture²⁶. Expressing information in gesture that is accessible only to gesture (i.e., implicit knowledge in novices) appears to free up more cognitive resources than expressing information in gesture that is also accessible to speech (i.e., explicit knowledge in experts). The cognitive cost associated with producing mismatches thus may differ in novices and experts because, in novices, the knowledge conveyed in mismatching gestures is not accessible to speech but, in experts, it is.

Whatever the explanation underlying the different effect that mismatching gesture has on working memory in novices vs. experts, it is intriguing that producing mismatching gestures is particularly good for lightening load in learners. Gesturing, in general, and producing gesture-speech mismatches, in particular, allows learners to conserve cognitive resources. Learners might then have more resources available to learn a new task if they gesture while tackling the task than if they do not gesture. Along these lines, recall that children profit more from instruction that contains two different strategies, one in speech and the other in gesture (i.e., mismatches), than from instruction that contains a single strategy expressed in speech and gesture (i.e., matches, Figure 6). Thus, producing mismatches is particularly effective in lightening the novice’s cognitive load, and seeing mismatches is particularly effective in teaching the novice new information — a parallel that warrants additional study.

Both novices and experts produce mismatches which, by definition, instantiate variability — more than one strategy produced in a single response. But the variability in novices’ mismatches serves a different function from the variability in experts’ mismatches. For novices, the information conveyed in gesture in a mismatch is at the cutting edge of their knowledge — the variability in their mismatches can thus serve as an engine of change, propelling development forward^xlviii,xlix. But for experts, the information conveyed in gesture in a mismatch is not new knowledge²⁶ — the variability in their mismatches neither reflects nor creates change, but may instead index discourse instability, a moment when speech and gesture are not completely aligned, reflecting the dynamic tension of the speaking process³ or perhaps the influence that speakers and listeners have on each other^l,li. The expert’s mismatches are best characterized in terms of the kind of variability that comes with expertise: the back-and-forth around a set point that typifies expert (as opposed to novice) performance on a task^lii. These mismatches do not lead to learning in experts, but they do support cognition in other ways (as they also do in novices) — by, for example, facilitating lexical access^liii, helping to package information for speaking^liv, highlighting perceptual-motor information^22,43,44, keeping mental images active^lv,lvi,lvii. As a final caveat, it is important to note that the differences we have been describing between experts and novices are not developmental differences but rather reflect the state of the speaker’s knowledge — adults produce gesture-speech mismatches when they are learning a task, that is, when they are novices^lviii; and children continue to produce gesture-speech mismatches even after they have mastered a task, that is, when they are experts^lix.

In sum, the spontaneous gestures that speakers produce when they talk about a task can serve as a signal that the speaker is in a transitional state and ready to learn that task. Gesture can thus reflect the state of a speaker’s knowledge. But gesture can go beyond reflecting knowledge — it can play a role in changing knowledge, indirectly through its effects on the learning environment or directly through its effects on the learner. One mechanism by which gesture could bring about cognitive change is to lighten the learner’s cognitive load. However, gesturing is not limited to learners, that is, to novices on a task. Individuals who are experts also gesture, but their gestures do not serve a learning function — they support cognition rather than change it. The way we move our hands when we speak is not mere hand waving. Our hands can affect how we think and learn.