Can Traditions Emerge from the Interaction of Stimulus Enhancement and Reinforcement Learning? An Experimental Model

Abstract

The study of social learning in captivity and behavioral traditions in the wild are two burgeoning areas of research, but few empirical studies have tested how learning mechanisms produce emergent patterns of tradition. Studies have examined how social learning mechanisms that are cognitively complex and possessed by few species, such as imitation, result in traditional patterns, yet traditional patterns are also exhibited by species that may not possess such mechanisms. We propose an explicit model of how stimulus enhancement and reinforcement learning could interact to produce traditions. We tested the model experimentally with tufted capuchin monkeys (Cebus apella), which exhibit traditions in the wild but have rarely demonstrated imitative abilities in captive experiments. Monkeys showed both stimulus enhancement learning and a habitual bias to perform whichever behavior first obtained them a reward. These results support our model that simple social learning mechanisms combined with reinforcement can result in traditional patterns of behavior.

A growing body of research indicates that social learning of traditional behaviors is widespread among vertebrate taxa (Galef and Giraldeau 2001; Laland and Hoppitt 2003). Social learning in this context is any process by which an individual modifies its behavior to a stimulus as a result of social experience, while traditions are durable patterns of behavioral similarity maintained in part by social learning (Galef 1995; Perry and Manson 2003). Natural selection can favor such a mode of behavioral development when (a) learning is more adaptive than innate responses (Roper 1983; Boyd and Richerson 1985), (b) individual learning is costly (Slater 1983; Laland and Kendal 2003), (c) social learning mechanisms enable at least moderate fidelity of traditional transmission (Enquist et al. 2007), and (d) the environment is stable enough that the fitness consequences of behaviors are not wholly different from one individual to another (Boyd and Richerson 1985; Laland and Kendal 2003). Examples from primate field data indicate the presence of traditional behaviors in natural populations of primates (Huffman and Hirata 2003; Perry and Manson 2003; Whiten et al. 1999), and captive studies show that at least some nonhuman primates possess social learning abilities (Custance et al. 1999; Snowdon and Boe 2003; Visalberghi and Fragaszy 1990; Westergaard et al. 1995; Whiten and Ham 1992; Whiten et al. 1996; Whiten et al. 2007). The logical inference is that the social learning mechanisms observed in captivity are responsible for the spread and behavioral inheritance of traditions observed in the field, but few explicit causal models have been developed to study exactly how social learning results in an emergent epidemiological property of tradition.¹

Such models are especially interesting because debate still exists in the human and animal literature regarding the extent to which social learning mechanisms can produce durable traditions in the presence of concurrently operating innate biases and individual-reinforcement learning (Cosmides and Tooby 1992; Heyes 1993; Smith 2000). Indeed, a prominent research paradigm, evolutionary psychology, has been predicated on the notion that traditional (i.e., cultural) influences on behavior are negligible and can be largely ignored during evolutionary analysis of human behavior (Aunger 2006; Cosmides and Tooby 1992; Smith 2000).

Responding to an article on cultural evolution by Alex Mesoudi and coauthors (2006), Robert Aunger writes:

In their Introduction, Mesoudi et al. argue that culture is socially acquired information … However, it is possible for genetic or environmental processes to account for what many consider to be cultural behaviour, making cultural explanation superfluous. [Aunger 2006:347]

Aunger (2006) points to no testable predictions from which we could infer his claimed genetic and environmental processes and states only that such processes may account for apparently cultural behavior. While cultural traditions can be inferred reliably if genetic and ecological variation do not predict variation in the trait (Whiten et al. 1999), when all three possibilities—genes, environment, and culture—are all equally predictive, we are essentially ignorant of how the trait in question develops (Matthews 2008). It is in this latter case that Aunger wishes to rule out cultural causation in favor of genes or environment, even when these explanations make no unique predictions that support them to the exclusion of cultural hypotheses. Contrary to the typical inference of field cultural primatologists (van Schaik 2003), evidence that a behavior is locally adaptive does not rule out culture, because all the evolutionary models for social learning and traditions predict they will usually result in local adaptation (Galef 1995; Laland 2004). The relatively nonpredictive views of Aunger and evolutionary psychologists stand in stark contrast to the explicit and testable models developed by many other researchers working within the context of the Neo-Darwinian synthesis (Mesoudi et al. 2006).

Additionally, some evolutionary researchers operating outside the evolutionary psychology paradigm (Richerson and Boyd 2005) question whether durable traditions can be maintained in any species lacking the more cognitively complex social learning mechanisms that current data suggest may be unique to apes and humans. These mechanisms include “rational” motor imitation (Buttelmann et al. 2007), sequence imitation (Byrne 2003; Whiten 1998), teaching (Boesch 1991), and language (possessed by humans solely). The predominant notion in the literature (with some exceptions: e.g., Galef 1995; Reader and Laland 2002) has been that individual learning will consistently erode traditional patterns—and, consequently, that highly faithful and cognitively complex social learning mechanisms are required for traditions to persist in and spread through a social group (Heyes 1993; Richerson and Boyd 2005). For example, in their study of capuchin monkey social learning, Marietta Dindo and coauthors state:

The authors of these studies [captive experiments] have interpreted them as supporting the conclusion that monkeys do not imitate or learn from one another; rather, they simply learn with each other … The results of numerous experimental studies therefore appear in conflict with the inference of field researchers that group-specific behaviours are culturally transmitted in capuchin monkeys, because processes as simple as stimulus enhancement would be insufficient to generate the behavioural variants documented in wild capuchins. [2008:188]

The eroding influence of individual learning on tradition, and the supposed requirement that traditions be transmitted by complex social learning mechanisms, may be notions with unwarranted currency given the lack of studies that explicitly test these hypotheses. For example, in one of the few tests that combined field and experimental research, Joseph Terkel (1996) found that simple social learning maintained a pine-nut-eating tradition in Israeli rats, thus casting doubt on the necessity of complex social learning for maintaining behavioral traditions.

Capuchin monkeys are, indeed, a case example of the apparent paradox presented by species that lack cognitively complex social learning mechanisms but exhibit behavioral traditions. In captivity, they demonstrate stimulus enhancement learning (Custance et al. 1999; Fragaszy and Visalberghi 1990; Fragaszy et al. 2004; Visalberghi 1987; Westergaard et al. 1995; but cf. Dindo et al. 2008) and possibly “final state reenactment” social learning (Custance et al. 1999; Fragaszy et al. 2004:256–257). During stimulus enhancement learning (see Whiten and Ham 1992), an observer does not replicate the actions of the demonstrator (motor imitation), nor does it cognitively model the intentions of the demonstrator and attempt to achieve those intentions (“rational” motor imitation). Rather, the stimulus-enhanced observer merely devotes more of its attention and activities toward an object or part of an object on which it saw the demonstrator act. The form of its subsequent activities with the object are uninfluenced by the demonstration (Fragaszy and Visalberghi 1990; Visalberghi 1987; Westergaard et al. 1995; Whiten and Ham 1992).

The information content of stimulus enhancement is obviously simple and limited relative to imitative forms of learning, and researchers have sensibly questioned whether it can maintain durable traditions. Similar to Dindo et al. (2008),Susan Perry and J. P. Ordonez Jiménez (2006) explicitly argued that the positive evidence for traditions in wild capuchins indicates the monkeys must have greater social learning abilities than the conclusions from captive studies support and that new captive experiments should be designed to discover these undocumented social learning mechanisms.

Newly recognized subtleties in capuchin learning are being studied in captivity. One recent study has shown that capuchin monkeys will engage in stimulus enhancement learning even when the experimenter rewards with food neither the actions of the observed demonstrator animal nor those of the observer test subject (Bonnie and de Waal 2007). Thus, capuchin monkeys are susceptible to social influences on their behavior even when no rewards are present. Kristen Bonnie and Frans de Waal (2007) also showed, however, that the degree of social influence on behavior could be over three times greater when the observer was reinforced for the same action as they observed, which indicates the potential for interaction between social and reinforcement learning.

In this article, we present an experimental model to test the hypothesis that traditions result from an interaction of stimulus enhancement with individual learning. We hypothesize that reinforcement learning may, in at least some contexts, act synergistically with stimulus enhancement to maintain behavioral traditions. Specifically, we predicted that capuchins would experimentally demonstrate: (a) concurrent use of social and individual learning on a novel object manipulation task; (b) “habitual” behavior by consistently recreating fitness-neutral behavioral variations that were discovered in early trials; and (c) more durably habitual behavior when the learning curve for an experiment was more costly.

INTEGRATED LEARNING MODEL

Prior research establishes that capuchin monkeys performing object-manipulation tasks are capable of both reinforcement learning from individual experience and stimulus enhancement effects from social experience. We began by constructing a theoretical reinforcement learning curve that represents how a capuchin monkey would learn two qualitatively distinct methods of manipulating the same object to obtain a food reward. Variation in the performance of similar tasks is observed between individuals in wild settings (Fernandes 1991; Fragaszy et al. 2004; O’Malley and Fedigan 2005; Panger et al. 2002; Phillips 1998; Phillips et al. 2003; Struhsaker and Leland 1977). Assuming the learning of such tasks conforms to the sort of curve commonly observed in manipulation and object choice tasks (Harlow 1949), it can be depicted as we have done in Figure 1.

Figure 1.

Learning curves for alternative solutions (behavioral variants) to the same foraging problem. Shaded region “b” represents the benefit to be gained in a switch from being fully proficient at variant A to proficiency at variant B, while region “a” is the opportunity cost of this switch.

The Y-axis of profit would typically be measured in the field as (food reward)/(time invested), while captive studies typically measure the percentage of responses on a trial that obtain a food reward. Importantly, the curves increase such that early trials are not as profitable as later trials. The curves may also plateau at different profitability levels, meaning that one variant is ultimately more profitable than the other. Assuming the profitability of the behaviors is fitness relevant, individuals who end up with more profitable variants on food-processing tasks will be favored by selection. The issue then is how selection can produce an animal that ends up practicing the more profitable variants given the constraint that the early trials of the learning curves impose.

One such method to arrive at the most adaptive variant might be to use stimulus enhancement learning. As long as the more adaptive behavior tends to be more common in the group than the less adaptive one, then biasing initial attempts at a problem with social information will tend to result in animals taking the path of the more profitable learning curve. At any time, an individual may switch by innovating the alternative behavior. Importantly, however, in this model the innovator does not switch directly from one plateau to the other. Assuming these innovations must be learned, the animal’s newly innovated variant begins at the bottom of the other curve, which could result in an individual switching to a novel behavior only to discover subsequently that the plateau is less profitable than the previous behavioral variant. In the absence of reliable prior knowledge about a plateau’s profitability, innovations may only pay a net profit occasionally, yet the long term payoffs of innovation could be still sufficient to sustain innovative tendencies. In the best case scenario for an individual who switches from a less profitable variant to the more profitable one, the net gain can be calculated as the difference between integrals under the curves, represented as labeled regions “a” and “b” in Figure 1.

$$\text{Net}\text{Profit}=\text{b}-\text{a}$$

Therefore, switching from a practiced behavioral variant to a novel one is selected when Inequality 1 holds:

$$\begin{array}{l}\text{Inequality}1\\ {\int }_{\text{X}\to \text{M}}(\text{Behavioral}\text{Variant}\text{B})-{\int }_{\text{X}\to \text{M}}(\text{Plateau}\text{Y})>{\int }_{\text{X}\to \text{M}}(\text{Plateau}\text{Y})-{\int }_{\text{0}\to \text{X}}(\text{Behavioral}\text{Variant}\text{B})\end{array}$$

Depending on the number of trials remaining for the animal to perform, it may not even be profitable overall to switch from a well-practiced but slightly less efficient technique to a more efficient novel one. Such constraints on trial number may actually exist in natural settings. For example, in a recent study (Matthews 2009), a field group of capuchins was found to variably process a particular Sloanea nut. The intragroup variations in the processing of this nut are indicative of traditional influences, and the tree fruits for roughly three months out of the year. It is conceivable that two months into the nut crop, it may no longer pay to attempt repeated trials on innovations. On the other hand, perhaps the combination of innate biases, individual innovation, and reinforcement learning are sufficient to result invariably in different individuals nearly always converging on the most profitable behavioral variant.

If, in the past ecology of Cebus, the benefits of continuing with a practiced behavioral variant generally have exceeded the costs of learning innovative variants (region a > b for most tasks), then we would predict capuchin monkeys to demonstrate a general bias toward performing actions habitually. That is, we would predict that Cebus will be “creatures of habit” who would faithfully reproduce even arbitrary behavioral variants, because habitual individuals experienced a higher fitness from avoiding the costs of early learning trials for innovations. If, however, practicing innovations has usually had a higher net fitness payout (region a < b for most tasks), then capuchins are not predicted to be particularly habitual. Furthermore, the model predicts that if the difficulty of learning a variant is high (meaning a relatively larger region a), then capuchins should show more durable behaviors with less innovation, because the returns from innovating and practicing innovations are even lower in a task with less profitable initial learning trials.

This model is similar to others that have contrasted selective pressures for innovation or social learning (Boyd and Richerson 1985; Enquist et al. 2007; Galef 1995; Laland 2004; Laland and Kendal 2003; Rogers 1988), but it differs from them in that the alternative “strategies” are innovation or habit, not innovation or social learning. However an animal has come to a successful (reinforced) behavioral variant, we can ask whether investing in innovation would pay a profit given the costs of early learning trials. This principle holds regardless of whether social learning is involved in the acquisition of a behavioral variant. Conceivably, individuals of a species could possess no social learning abilities and yet rarely perform innovations once a given behavioral variant plateau has been reached. Likely all that would be necessary for durable traditions to form in such a species would be the introduction of simple social learning mechanisms.

To test our model, we designed a novel object manipulation task that required animals to use one of two methods to obtain a food reward. Two apparatuses (see Figure 2) were used that allow for tests of the presence of stimulus enhancement learning, individual reinforcement learning, innovation, and the durability of alternative behavioral variants. Thus, this experiment differs from previous experiments with capuchin monkeys, which have focused on understanding the kinds of socially acquired information used to solve object manipulation tasks (Custance et al. 1999; Fragaszy and Visalberghi 1990; Visalberghi 1987). The present study is not designed to test any cognitive hypothesis about the social learning abilities possessed by capuchin monkeys. Rather, we intended to test whether an interaction of stimulus enhancement learning and individual learning would have the properties needed to produce an emergent pattern of durable, socially influenced behavioral variation that is similar to the traditional variations observed in wild capuchins.

Figure 2.

Experimental apparatuses. Apparatus 1 has identical hinged doors on both sides. Apparatus 2 has different doors, a screw top and a pull off plastic lid. (Dimensions: diameter of both apparatuses = 10.2cm; length of Apparatus 1 = 8.6cm, length of Apparatus 2 = 7.9cm.)

METHODS

Study Subjects

The study subjects were nine Cebus apella: four males and five females of subadult or adult age who were socially housed at the National Institutes of Health (NIH), in Poolesville, Maryland. Subjects were not food deprived for this study, and they received daily nutritional supplements of seeds and fresh fruit or nuts. Commercial monkey biscuits and water were available ad libitum. Individuals were spatially and visually isolated from group members for testing. These test sessions lasted for no longer than 15 minutes. On days prior to experimental tests, the study subjects were habituated to the testing cage (150cm × 50 cm × 55 cm; length, width, height), which was accessible through a tunnel-run connection to their main enclosures. The Institutional Animal Care and Use Committee (IACUC) of the NIH approved all captive research performed in this study.

Apparatus

Apparatus 1 had two equivalent doors that swung freely on hinges attached to a clear plastic tube (see Figure 1). It was baited in each trial with a single grape. Apparatus 1 thus provided a learning context in which the two variants (the door on the left and on the right) were equivalent and so should theoretically have very similar learning curves. Also, knowing how to open one door of Apparatus 1 provides all the information needed to open the other door.

Apparatus 2, also baited with a grape, had a screw-top door and a plastic-lid door (see Figure 1). The lid could be removed by prying the edge, as one removes the flexible plastic lid from a food canister. The doors of Apparatus 2 thus require qualitatively different actions to be opened, and the two doors are not necessarily expected to yield the same peak profitability.

Apparatuses 1 and 2 were clamped to the outside of the testing cage’s bars, through which the test subjects reached to interact with the apparatus doors. The apparatuses were baited by placing a solid plastic shield flush with the bars in between the monkey and the experimenter. The shield only blocked the monkeys’ hands from reaching the doors while one experimenter baited the apparatus, and the test subjects could see the experimenter’s actions from over the top edge of the shield and from a hole in the center. A video camera placed about one meter in front of the test cage recorded all trials. One researcher (Matthews) scored all data analyzed from the video recordings.

Procedure

In our experiments, we isolated a single individual and then presented it with eight to ten learning trials with either Apparatus 1 or Apparatus 2. We stopped at eight trials for a given isolation if individuals appeared overly stressed by the experimental procedure. The human experimenter performed an initial demonstration of one door of the apparatus before the first trial by baiting the apparatus. Details are explained in the next paragraph. We isolated each individual four times, once for each door on each apparatus. Only one door was demonstrated throughout a single isolation, regardless of the behavior of the monkey subject. Thus, experimental conditions were consistent across trials within an isolation and varied across isolations. The only exception to the four isolations was one individual who was isolated twice with one door on each apparatus. This individual was tested twice during recovery from an unrelated medical procedure. We used a human demonstrator and isolated monkey test subjects to isolate interactive effects of stimulus enhancement and reinforcement to the exclusion of social motivations or inhibitions that would arise from interactions of test subjects with conspecifics.

During testing, the plastic shield was in place to prevent subjects from interacting physically with the apparatus prior to the demonstration. The human demonstrator (Matthews) opened one door with the subject watching, placed the grape inside, and then closed the door. Thus, the baiting of the apparatus and the demonstration of a door were one in the same. The shield was removed, and the individual had full access through the bars to the apparatus until the food reward was retrieved or until one minute passed. Either condition—food retrieval or a passage of one minute—constituted the end of a trial. At the end of each trial, the human demonstrator approached the cage, replaced the shield, and then rebaited the apparatus through the predetermined door for that isolation in view of the subject. If the subject had not obtained the grape on the previous trial, the demonstrator opened the door, pulled the grape out, held it for one second next to the apparatus, and then replaced the grape and closed the door. We conducted this removal and replacement of the grape to eliminate a confound that would otherwise arise from unsuccessful individuals observing fewer demonstrations of the door being opened and closed, because their grapes would not need to be replaced. We repeated this procedure for eight to ten total trials per isolation. The order of experimental conditions was balanced by males and females and across isolations for order effects in door-opening demonstration and in the type of experimental apparatus. Thus, half of the monkeys had their first trials with apparatus 1 and half with apparatus 2. For the first trial with each apparatus, we conducted half with the right door (hinged-door apparatus 1, plastic-lid door apparatus 2) and half with the left door (hinged-door apparatus 1, screw-top apparatus 2).

Statistical Analyses

Model validation

The model assumes that Apparatus 2, with its more complex doors, will have a more costly initial portion of the learning curve than does Apparatus 1. Given that the food reward is constant (one grape), an increased costliness for Apparatus 2 should be reflected by individuals using more time in initial learning trials to open Apparatus 2 than Apparatus 1 or by individuals being less successful at solving Apparatus 2.

Stimulus enhancement

To test for the operation of stimulus enhancement learning during the experiment, we predicted that individuals would spend more time interacting with the demonstrated door than the undemonstrated door in their first learning trial: that is, prior to any individual experience. We scored any manual or oral interaction with the apparatus doors using point sampling on one-second intervals in the first trial for all subjects. Because the research design was balanced to eliminate order effects, half of the first trials occurred with Apparatus 1 while half were with Apparatus 2, but the apparatus type is irrelevant to the specific predictions of stimulus enhancement. We did not record contacts made with other parts of the apparatus, such as poking or banging at the solid but clear side wall and biting the clamps. We then summed the interactions with each door until the subject fully opened one door or the end of the trial occurred. We subtracted the sum of interactions that did not match the demonstrated door condition from those that matched the demonstration. A positive number thus indicates that individuals direct more interactions toward the side of the apparatus that the demonstrator manipulated. Individual hand preferences of the subjects are unlikely to confound this test for stimulus enhancement because capuchin monkeys do not show a population bias in hand preference (Papademetriou et al. 2005; Westergaard et al. 2000; Westergaard and Suomi 1996), especially when they are allowed to retain a natural crouching posture as was the case in this experiment (Lilak and Phillips 2008).

Reinforcement learning

If reinforcement learning operated in this experiment, then two predictions follow. First, we predicted that there would be a significant negative correlation between trial number and time from first contacting the apparatus to obtaining the food reward; that is, if individuals were learning from experience, they would obtain the grape faster after several trials. This first prediction was tested by scoring the time from contacting the apparatus to obtaining the food reward for all trials during the isolation in which an individual first successfully obtained grapes. We then calculated a Spearman rank correlation for each individual and tested if the mean correlation value across individuals was significantly less than than 0.

Second, we predicted that individuals who innovated (discovered how to open the undemonstrated door) would spend more time interacting with the innovated door in the trial immediately subsequent to the innovation than they did in the first trial for that day. To test the second prediction, we again scored interactions on one-second intervals for the first trial of any day in which an innovation happened and on the trial immediately after the trial in which the innovation was performed. We then subtracted time spent on the innovated door from time spent on the door matching the demonstration for each of these trials and divided the difference by the total number of intervals with interactions. This calculation yielded an index that was comparable between the two trials ranging from 1 (all time spent interacting with the demonstrated door) to −1 (all time spent interacting with the innovated door).

Habitual performance of “neutral” variants

To examine if the monkeys’ actions on the apparatuses were habitual, we tested if behaviors in later trials generally corresponded to the first actions subjects performed on Apparatus 1, which had equivalent doors. If monkeys tend to continue performing seemingly fitness neutral and arbitrary variations on an apparatus, then they are predicted to generally open the same door and remove the grape through the same door as they did initially. Furthermore, a habitual behavioral bias predicts that monkeys will tend to perform behaviors matching early trials even in the presence of the opposite demonstration accessible to social learning. We tested both predictions by calculating the proportion of behaviors that matched the demonstration or the first trial. We scored all instances of a monkey opening a door sufficiently wide to insert its hand as a “door-opening” event. We scored a food-removal event whenever a subject got the food out of the apparatus by any means or with any body part (generally grasping removal with the hands). We then assessed if this proportion was significantly greater than 0.

Relative durability of behavioral variants

We tested if behavior on Apparatus 2, with its higher learning cost, resulted in more durable behavioral variants by comparing the frequency of individuals opening the door that did not match the demonstration to the same frequency for Apparatus 1. Increased habitual biases, and the difficulty of learning the doors themselves, are predicted to reduce the innovation rate for Apparatus 2 if the costliness of learning generally exceeds the profit of alternative behavioral variants (see Figure 1). On the other hand, if the ecology of Cebus has generally favored individuals who consistently found optimal behavioral variants through repeated practice trials of innovations, then innovation is predicted to be higher for Apparatus 2 than for Apparatus 1.

We conducted a second test of the hypothesis that higher learning costs would produce more durable behavioral variants by using data from trials with Apparatus 1. We predicted that individuals who experienced more costly (time consuming) initial learning trials would also display durable behavioral variants (i.e. a lower proportion of innovative actions) in later learning trials during the same isolation on the same apparatus. We made this prediction because we expected individuals to use the costliness of acquiring the behavior they had already learned as a proxy for the general costliness of acquiring behavioral variants for a given task, at least in the absence of other information. To perform this test, we summed the time from contacting the apparatus to obtaining a food reward for the first half of the test session and calculated its correlation with the proportion of door openings in the second half that matched the door opened in trial 1. Because food reward was constant at one grape, the summed time for trials 1–4 is effectively the observed cost invested for the first variant learned. In our model (Figure 1), this value is equal to:

$$1-{\int }_{\text{0}\to \text{3}}(\text{Behavioral}\text{Variant}\text{A})$$

RESULTS

We assessed the intraobserver reliability of one-second interval scoring by having the scorer (Matthews) rescore 20 percent of the one-point samples used to test the stimulus enhancement hypothesis (Pearson correlation = 0.90, p < 0.001, 95 percent confidence interval = 0.87 to 0.93). This rescoring took place ten months after the initial video analysis. We also assessed interobserver reliability by having a qualified observer who was uninformed about the study score the same 20% subset of 1s intervals. The mean Pearson correlation of the second observer’s scores to those of LJM was 0.91. We generated records of door opening events by LJM watching all videos twice and making sure the number of door opening events matched exactly. Time from first contacting an apparatus to obtaining food, which was used in one test for reinforcement learning, was assessed directly from the video play time counter.

Model Validation

Apparatus 2 appeared to have a more costly initial learning curve than Apparatus 1 because individuals were more likely to solve Apparatus 1 and obtain grapes, even though they interacted with both apparatuses for similar amounts of time (Table 1, matched-pair Wilcoxon signed-rank tests, successes one-tailed p = 0.018, time two-tailed p = 0.95). Table 1 shows how some individuals interacted more with Apparatus 2 but still never solved it repeatedly. The reverse—solving Apparatus 2 but not Apparatus 1 despite substantial interaction time—did not occur. Three individuals who successfully obtained multiple grapes from Apparatus 1 were not able to repeatedly open the doors of Apparatus 2 despite having single trials in which they opened one door. These three individuals both opened a door once (one opened the lid and two opened the screw top) but could not open the doors again, despite interactions with the same door. The one individual who did improve in opening the lid door interacted more with the lid than any animal interacted with a hinge door of Apparatus 1 before solving a hinge door.

Table 1.

Successes (Number of Trials When Food Reward Obtained) and Total Time (One-Second Intervals) Interacting with a Door on the First Trial with an Apparatus

	Individual	1	2	3	4	5	6	7	8	9
Apparatus 1	Trials	10	18	20	20	20	19	20	16	20
	Successes*	0	18	19	19	0	19	2	0	18
	Time✧	1	5	2	4	5	3	4	1	6
Apparatus 2	Trials	10	20	20	20	20	18	16	20	20
	Successes*	0	1	1	7	0	0	0	0	1
	Time✧	0	3	30	2	3	8	0	0	10

^*Differences in successes, matched pair Wilcoxon, one tailed, p = 0.018, N = 9

^✧Differences in time, matched pair Wilcoxon, two tailed, p = 0.95, N = 9

Stimulus Enhancement

The capuchin monkeys showed evidence of stimulus enhancement learning during the first trial they had with an apparatus. A nonparametric bootstrap (unweighted sampling with replacement) was significant at the 0.05 nominal level (one tailed, p = 0.047). A standard t-test for a mean interaction with the demonstrated door approached significance (N = 9, one tailed, p = 0.063). Seven out of nine animals interacted more with the door that was demonstrated to them than with the alternative door.

Reinforcement Learning

First, learning by reinforcement predicted that the time used to obtain a food reward would decrease as the trials progressed for both experimental apparatuses. The time from first contacting an apparatus to obtaining food is significantly negatively associated with trial number during the same isolation for all individuals that solved Apparatus 1 in consecutive trials (Figure 3; mean Spearman rank correlation across individuals, r_s = −0.50, t-test for r < 0, N= 5 individuals, p = 0.002). Time decreased reliably with trial number also in the case of the one individual that obtained reinforcement from Apparatus 2 (Figure 3; r_s = −0.91).

Figure 3.

Learning curves for the initial isolation of individual capuchin test subjects. The lines show the first isolation in which the subject obtained food reward. Solid lines are individuals tested with Apparatus 1; dashed line is with the individual that solved Apparatus 2.

Secondly, we predicted that an index reflecting the proportion of interactions with the demonstrated door would decrease after innovations that opened the undemonstrated door. Only four individuals innovated with the door opposite from the one demonstrated to them and then had remaining learning trials in which to test this hypothesis. All individuals had lower index values in the trial immediately subsequent to their innovation as compared to their first trial for that day. (Individual 2 Apparatus 1: 1 → 0.33; Individual 4 Apparatus 2: −0.47 → −1; Individual 9 Apparatus 1: 1 → −0.67; Individual 9 Apparatus 2: −0.17 → −0.63: arrows indicate indices from before and after innovation; nonparametric bootstrap, N = 4, p = 0.048; matched pair t-test, one tailed, p = 0.030).

Habitual Performance of “Neutral” Variants

In their initial isolations in which they obtained a food reward from Apparatus 1, subjects opened the same door on subsequent trials as the one through which they first obtained food significantly more than they opened the opposite door (Table 2, t-test, one tailed, p < 0.001, N = 6 individuals). They were also more likely to remove the grape through the same side as they did initially (Table 2, t-test, one tailed, p = 0.002, N = 6 individuals).

Table 2.

Correspondence of Side through which Food Was Obtained and Doors Opened to the First Trial with Success at Obtaining a Food Reward

Individual	2	3	4	6	7	9
Door openings that matched first trial†	9	10	10	8	2	9
Total subsequent door openings	15	11	10	14	3	11
Food removal side matched first trial*	7	7	9	6	0	8
Total subsequent food removals	7	8	9	6	1	8

^†t-test, one tailed, p < 0.001, N = 6

^*t-test, one tailed, p = 0.002, N = 6

Most individuals retrieved the grape through the demonstrated door on their first success with Apparatus 1 (4 out of 6 individuals), but they tended to continue retrieving the grape through the initial door even in trials in which the opposite door was demonstrated (Table 3, t-test, one tailed, p < 0.001, N = 5 individuals).

Table 3.

Side Removal for Test Sessions When the Demonstration Did Not Match the First Solution Used

Individual	2	3	4	6	9
Removal matched 1st trial*	9	7	9	5	8
Removal matched demonstration	0	1	0	4	0
Total trials after 1st	9	8	9	9	8

^*t-test, one tailed, p < 0.001, N = 5

Relative Durability of Behavioral Variants

The door-opening variants employed on Apparatus 2 appeared to be more durable for the one individual who repeatedly solved it, in that she innovated (opened a different door than she did initially) significantly less than capuchins did who repeatedly solved the Apparatus 1 task (Binomial test, two tailed, p = 0.037, N = 10 trials). For this binomial test, the null probability of innovation was set at the observed average innovation probability among individuals who repeatedly solved Apparatus 1 (expectation = 0.32; average of 6 individuals). The individual who solved Apparatus 2 repeatedly interacted with both doors, then opened the lid door and obtained the food reward, but never opened the screw top door.

More costly opening variants were also more durable in our test with Apparatus 1. Individuals who experienced more costly initial learning trials also more frequently opened the same door in trials five through ten as they initially opened in trial one (Figure 4, Pearson r = 0.81, one tailed, p = 0.047, N = 5 individuals). This result is not an artifact of different overall levels of activity between individuals, because we measured durable door opening as a proportion of total door-opening actions.

Figure 4.

Durability of door opening variants in later trials versus the costliness (summed time) of earlier trials (Pearson r = 0.81, one tailed, p = 0.047).

DISCUSSION

The results indicate that the initial learning curve of Apparatuses 2 is more costly than that of Apparatus 1, a result which validates this key model assumption.

Stimulus enhancement and individual learning both operated during the performance of the subjects during learning trials. A binomial test supported that more individuals devoted more time to interacting with the demonstrated door than to the alternative door. Individual learning is demonstrated by the time reduction for obtaining food rewards and by the individuals who innovated the opposite condition from the one they observed and spent more time on the subsequent trial interacting with the innovated door.

The occurrence of stimulus enhancement followed by reinforcement learning suggests that any patterns of habitualness and durability may result from the interaction of the two learning processes. The capuchins were habitual on the equivalent doors of Apparatus 1, in that they strongly tended to open the same door and obtained the food reward through the same door as they did in their first successful trial. The observation that capuchins were habitual even in the presence of the opposite demonstration (Table 3) shows that the habitual behavior is not solely a product of ongoing stimulus enhancement from the demonstrations. Rather, stimulus enhancement first biases a monkey toward actions involving the demonstrated door, and the monkeys then persist in subsequent trials to open whichever door first gained them food reinforcement.

That capuchins were habitual indicates that their behavior in this experiment conforms to some aspects of the “critical social learning” model (Enquist et al. 2007). During “critical social learning,” an animal biases its first attempts at solving a problem based on social information and then “evaluates” the outcome of the socially learned behavior to decide whether to persist in that behavior. Reinforcement learning can be thought of as a means for “evaluating” the usefulness of the socially acquired information, although it does not conform to all aspects of Magnus Enquist and colleagues’ (2007) model (e.g., the model assumes “evaluation” itself carries no cost). Thus, having found a useful behavioral variant, individuals were not greatly affected by subsequent alternative social information.

As predicted, the behavioral variants (door opening) used on Apparatus 2 were more durable than the variants subjects performed on Apparatus 1. The result from Apparatus 2 is derived from the single individual who repeatedly solved this apparatus. This result is consistent, however, with the observed learning curve and observed variant durability in a field experiment with a similar learning apparatus that was solved repeated by a wild white-fronted capuchin male (Matthews 2008). Clearly, additional data are needed to confirm these findings. Also consistent with our hypothesis is the result that individuals with costly early learning trials for Apparatus 1 exhibited more durable variants in later trials with the same Apparatus (Figure 4). Our model predicts that when individuals experience a more costly learning curve, they will focus more greatly on a variant that has already succeeded with some efficiency for them than on innovative behaviors, because individuals will act on the only information they have and avoid the presumed higher opportunity cost of early trials for alternative variants.

These results are consistent with recent experimental studies in capuchins and chimpanzees showing that innovation does not cause socially learned object-manipulation variants to atrophy even when passed through a chain of different individuals or social groups (Dindo et al. 2008; Whiten et al. 2007). Although additional cognitive processes could be involved (such as a bias for social conformity or action imitation), our results suggest that the simple interaction of stimulus enhancement with reinforcement learning could also be a major factor that promotes the faithful transmission of traditions.

In particular, the results from a social-transmission chain of capuchins in Dindo et al. (2008) are consistent with our results because their control animals (who received no demonstration) exhibited only slightly less fidelity to one behavioral variant than did the experimental subjects that saw one of two alternative behaviors demonstrated (80–100 percent fidelity in controls as compared to 90–100 percent in experimental subjects). While the experimental animals in Dindo et al. (2008) may have behaved slightly more habitually than the controls, the controls were also highly habitual, and this habitualness must be accounted for solely by reinforcement learning from asocial experiences. Dindo et al. (2008) interpreted their transmission chains as resulting from some combination of action imitation, action emulation, or conformity to group norms. The results of the present study and their own control animals indicate that the durability of transmission chains, though clear evidence of a cultural process, could result simply from the interaction of reinforcement with social learning. While some type of action imitation probably caused the form of behavioral act to be initially similar in the transmission chains of Dindo et al. (2008), the durability of the behavior down the chain could arise from reinforcement and without any psychological bias for conformity.

Our model and results also are consistent with the explanation offered for group-typical differences in a similar two-action experiment in wild marmosets (Pesendorfer et al. 2009). Although Mario Pesendorfer and coauthors (2009) found no evidence that the marmosets had a psychological bias toward the most common behavior in a group (conformity bias), they found that groups maintained arbitrary traditions across two test sessions separated by 21 days. Similar to our experiment with individuals, the initial behavioral difference across groups was established through reinforcement learning.

It is clear that more data will be necessary to robustly test the model outlined in Figure 1. In particular, we need more data comparing learning curves for different foraging techniques, whether from captive experiments or field observations. Indeed, we initially envisioned comparing the observed cost integrals both across individuals for an Apparatus (Figure 4), across Apparatuses, and across doors of Apparatus 2. The fact that multiple individuals did not successfully solve Apparatus 2 (Figure 3) precluded the latter two analyses.

If the results of our captive experiments can be generalized to a field setting, then they imply that the traditional behavioral variations observed in wild Cebus (O’Malley and Fedigan 2005; Ottoni et al. 2005; Panger et al. 2002; Perry et al. 2003; Perry and Jiménez 2006; Rose et al. 2003) could result from the interaction of stimulus enhancement and reinforcement learning. Key to this interactive effect is the investment of time and energy in early reinforcement learning trials for a newly innovated or socially acquired behavioral variant. This durable and habitual behavior is predicted by our model if, during the evolutionary history of Cebus, the costs of initial learning trials often have exceeded the benefits of repeatedly practicing innovations to reach and assess an alternative variant’s plateau. This is not to imply that wild Cebus will generally perform suboptimal behavioral variants, because innovations still inevitably happen, just as they did in this experimental study (Table 2). Over enough time, which could be multigenerational, some individuals would undoubtedly reach alternative plateaus with higher profitabilities (Franz and Matthews in press). These individuals would then become demonstrators for other social learners, which is why social learning should eventually yield locally adaptive traditions most of the time (Boyd and Richerson 1985:98–117; Enquist 2007; Galef 1995; Laland 2004). While not perfect, the process may result in higher individual fitness than “going it alone” with reinforcement learning only and similarly would be preferable to uncritical social learning that does not interact with reinforcement.