Abstraction promotes creative problem-solving in rhesus monkeys

Abstract

Abstraction allows us to discern regularities beyond the specific instances we encounter. It also promotes creative problem-solving by enabling us to consider unconventional problem solutions. However, the mechanisms by which this occurs are not well understood. Because it is often difficult to isolate human high-level cognitive processes, we utilized a nonhuman primate model, in which rhesus monkeys appear to use similar processes to consider an unconventional solution to the difficult reverse-reward problem: i.e., given the choice between a better and worse food option they must select the worse one to receive the better one. After solving this problem with only one specific example—one vs. four half-peanuts—three of four monkeys immediately transferred to novel cases: novel quantities, food items, non-food items, and to the choice between a larger, but inferior vegetable and a smaller, but superior food item (either grape or marshmallow), in which they selected the inferior vegetable to receive the superior option. Thus, we show that nonhumans have the capacity to comprehend abstract non-perceptual features, to infer them from one specific case, and to use them to override the natural preference to select the superior option. Critically, we also found that three monkeys had a large learning and performance advantage over the fourth monkey who showed less generalization from the original one and four half-peanuts. This difference suggests that abstraction promoted problem-solving via cascading activation from the two food item options to the relation between them, thus providing access to an initially nonapparent component.

1. Introduction

It is impossible to encounter truly identical situations. The Greek philosopher Heraclitus recognized that the natural world is too dynamic and varied to step in the same river twice, as the particles that constitute it are always in motion. In face of this challenge, we discern regularities beyond the specific incidents we encounter. These regularities arise from inductive abstraction processes that generalize specific events, enabling us to process novel experiences efficiently and react accordingly (Holyoak & Morrison, 2012). Moreover, such inductive processing occurs at multiple levels of abstraction, allowing us to identify a novel sensory input as an instance of, for example, a known object, category, concept, or relation (Badre, Hoffman, Cooney, & D’Esposito, 2009; Herrnstein, 1990; Holyoak & Morrison, 2012; Kowaguchi, Patel, Bunnell, & Kralik, 2016; Kralik, 2012; Kralik & Hauser, 2002; Rosch, 1978; Tenenbaum, Kemp, Griffiths, & Goodman, 2011).

Abstraction also has the power to promote creative problem-solving. Although creativity is difficult to define, it is important to distinguish noncreative and creative problem-solving. Problem-solving in general entails generating a representation of the problem and then solving it by determining the proper sequence of actions to reach the goal state (Bassok & Novick, 2012). Creativity can be introduced into the problem-solving process in one of two places: either in the formulation of the problem itself, or in the delineation of the path taken to solve it. Although there has been considerable research progress examining how agents find solution paths when faced with relatively well-defined problems, less is known about how problem representations are generated and updated (i.e., restructured) (Bassok & Novick, 2012; Sutton & Barto, 1998; van Steenburgh, Fleck, Beeman, & Kounios, 2012). We therefore have focused on the mechanisms of problem formulation and the use of creativity therein.

The curse of dimensionality in real-world problems necessitates a selection process: typical problem-solving involves considering only the most apparently relevant factors to represent the problem. It is up to the observer to determine which factors facilitate a solution. For example, to find a new path to a restaurant one normally considers the most obvious means of transportation (e.g., walk, subway, car), and will take the most direct route available; these solutions are in turn bound by the factors of cost, availability, and intended effort. In contrast, creative problem-solving entails consideration of nonapparent problem components, which at first pass means those outside the scope of the original problem representation: i.e., those not as salient, directly relevant, or learned from experience (Cheng, Ray, Nguyen, & Kralik, 2013; Kralik, Mao, Cheng, & Ray, 2016; Kralik, Shi, & El-Shroa, 2016; Smith & Ward, 2012). A classic example with humans is the 9-dot problem, in which nine dots are displayed in a 3 × 3 square matrix, and the participant must connect all nine dots by drawing only four lines without lifting the pen/pencil (Cheng et al., 2013; Maier, 1930; van Steenburgh et al., 2012). In this case, the highly salient dots and most direct lines that begin and end at the dots define the apparent problem formulation, whereas a solution can only be found when one realizes that the lines can extend past the dots (nonapparent formulation). Thus, in creative problem-solving, an inadequate formulation of the problem based on apparent factors must be replaced by considering ‘outside-the-box’ components (Kralik, Mao, Cheng, & Ray, 2016; Kralik, Shi, & El-Shroa, 2016; Smith & Ward, 2012; van Steenburgh et al., 2012).

Abstraction provides a means to rediscover these nonapparent possibilities: e.g., when one abstracts from a particular instance to a larger class, more instances become available from which a potential problem solution can be identified — a specific-to-general-to-specific access route (Smith & Ward, 2012; Ward, 1994; Ward & Sifonis, 1997; Ward, Patterson, & Sifonis, 2010; Ward, Patterson, Sifonis, Dodds, & Saunders, 2002). However, in principle, abstraction could be even more powerful by leading to further cascades of activation beyond the additional specific instances of the given class: e.g., from specific instances to general class to relations with other classes. Unfortunately, it is difficult to find clear cases of this directed cascading effect of abstraction on problem-solving ability in the human problem-solving literature. This is so because it is sometimes difficult to tease apart the underlying processes and obstacles, such as in the 9-dot problem where, for example, blocking (e.g., by the salient dots) or remoteness (e.g., considering all points on the page) could both underlie the difficulty. The investigation of creative cognition in nonhuman animals provides a complementary approach that may help to isolate and characterize the fundamental underlying cognitive mechanisms (and if successful, the subsequent ability to study neural mechanisms in greater detail). Although human creativity far exceeds other animals, the processes by which nonapparent components are accessed once problems become sufficiently challenging may be shared across broader animal clades, enabling nonhuman studies to help delineate these processes.

To investigate how abstraction may promote creative problem-solving via the cascading activation process, we utilized the reverse-reward problem with rhesus monkeys (Macaca mulatta), in which the monkeys are offered a choice between a less-preferred and more-preferred option, such as one and four quantities of the same food, but are given the option they do not select (Fig. 1A). Thus, they must select the less-preferred option in order to receive the more-preferred one (Albiach-Serrano, Bugnyar, & Call, 2012; Albiach-Serrano, Guillen-Salazar, & Call, 2007; Anderson, Awazu, & Fujita, 2000; 2004; Boysen, Berntson, Hannan, & Cacioppo, 1996; Boysen, Mukobi, & Berntson, 1999; Genty, Chung, & Roeder, 2011; Genty, Palmier, & Roeder, 2004; Kralik, 2005; 2012; Murray, Kralik, & Wise, 2005; Shifferman, 2009; Uher & Call, 2008). Although trivial for humans, this problem is difficult for nonhuman animals — e.g., rhesus monkeys require roughly 1000 trials to solve it (Chudasama, Kralik, & Murray, 2007; Murray et al., 2005). It has generally been assumed that the difficulty stems from the lure of the better reward. However, evidence suggests that this is often not the key issue. First, when choosing between a larger and smaller quantity most subjects inhibit the selection of the larger quantity relatively quickly; however, instead of selecting the smaller quantity, they switch to a side bias (e.g., repeatedly selecting the left option), thus reaching an extended impasse prior to spontaneously solving the problem (Chudasama, Kralik, & Murray, 2007; Murray et al., 2005). This extended impasse and spontaneous problem-solving suggests additional confusion with the task that is eventually overcome (Kralik, Mao, Cheng, & Ray, 2016; Murray et al., 2005). Second, to test the issue of self-control directly, Kralik (2005) first posed an even simpler version of the reverse-reward problem to cotton-top tamarins (Saguinus oedipus), a New World monkey, in which when given a choice between 1 and 3 food items, selecting the 1-item option received the 3 item-option, but selecting 3 received nothing (Fig. 1B). The tamarins were unable to solve this problem even with the strong punishment of receiving nothing when selecting the larger quantity. It was then reasoned that if the problem stemmed from a lower-level impulse to select the larger quantity over the smaller one, the difficulty should continue regardless of any change in outcome as long as the original offer, 1 vs. 3, remained the same. Moreover, if anything, the task should become more difficult if the reward outcome for selecting the smaller quantity was reduced from receiving three food items to receiving only one, with reward thus three times smaller (Fig. 1C). Nonetheless, when keeping the offer the same (1 vs. 3) but reducing the reward outcome for selecting the single quantity from three to one, all four subjects solved the problem, selecting the smaller quantity over the larger one, suggesting that the difficulty lay more in the complexity of the task rather than in the inability to inhibit selection of the larger quantity. The Kralik (2005) study suggests that a critical difficulty in solving the reverse-reward problem stems from the interaction between the two choice options: that is, in recognizing the tertiary relation between the food items (i.e., relation between two things other than oneself). This interpretation is supported by findings from other studies such as the ease with which chimpanzees solve a related accumulation task, in which they readily learn to select a single marshmallow over an accumulating bowl of marshmallows if the chimpanzees directly see that when they select the single marshmallow it is then placed by the experimenter in the bowl (which subjects will ultimately receive) (Beran, James, Whitham, & Parrish, 2016).

Figure 1.

Illustrations of reverse-reward problems. A. The original version of the problem, in which the subject receives the option not selected (response panel with recessed bowls used in current study; offer provided and outcome delivered by Experimenter depicted here with two blue ovals for gloved hands). B. The first modification employed in the Kralik (2005) study with cotton-top tamarins. C. The second modification in the Kralik (2005) study, in which subjects received the single piece of food when selected. D. Illustration of a quality test employed in the current study, in which correct transfer from the original ‘quantity’ version (shown in A) to quality required the rhesus monkeys to select the larger, but lower quality vegetable to receive the smaller, but higher quality food item such as grape. Positions and sizes not to scale—illustrations for general visualization.

Additional studies have also shown that in multiple cases where problem-solving difficulties have been assumed to reflect lower-level (e.g., Pavlovian) influences, they may more accurately resemble cognitive illusions that reflect the constraints/biases of a simple problem-solving system rather than affective-driven prepotent responses (Kralik, Shi, & El-Shroa, 2016; Santos, Ericson, & Hauser, 1999; Wallis, Dias, Robbins, & Roberts, 2001). Indeed, the discontinuous reverse-reward learning curve for rhesus monkeys suggests that, after the extended side-bias impasse, the spontaneous solving of the problem does not occur via simple gradual strengthening over trials, but rather, some change that provides access to the previously inaccessible nonapparent solution (Chudasama et al., 2007; Kralik, Mao, Cheng, & Ray, 2016; Murray et al., 2005). In this light, reverse-reward problem-solving by nonhumans provides a model to study the mechanisms used to find nonapparent solutions, ones that may be shared by people to access remote possibilities that lead to creative solutions. It has in fact been theorized that the ability to solve nonapparent problems may be the key functional advance with the evolution of granular prefrontal cortex in primates (i.e., lateral and frontal polar cortex) (Kralik, Mao, Cheng, & Ray, 2016; Kralik, Shi, & El-Shroa, 2016; Passingham & Wise, 2012; Preuss, 1995; Striedter, 2005; Wise, 2008).

To utilize the reverse-reward problem to investigate how abstraction may promote problem-solving via a cascading activation process, we leverage the fact that in cases where nonhuman subjects learn to solve the reverse-reward problem with the food items present, their solution could be based on a number of different levels of abstraction: e.g., the specific quantities and food items in training (e.g., one and four food pellets), or something more abstract such as the number of items. Previous studies have found evidence for a more general number or size based solution (Albiach-Serrano et al., 2007; Anderson et al., 2000; 2004; Boysen, Berntson, & Mukobi, 2001; Genty et al., 2004; 2011; Kralik, 2012; Uher & Call, 2008). For example, rhesus macaques spontaneously generalized to novel quantities after learning the task with only one and four food pellets in an automated system (Kralik, 2012). However, their actual level of abstraction remains unclear. For example, would they select a less-preferred option (e.g., vegetable) to obtain a more-preferred one (e.g., fruit)? If so, it would show abstraction at the level of a ‘less-preferred➔more-preferred’ solution. Three studies have tested generalization based on food quality using the reverse-reward task, and the results thus far are suggestive but inconclusive, with individual brown (Eulemur fulvus) and black (Eulemur macaco) lemurs showing some evidence for transfer from quantity to quality, but no capuchin monkeys (Cebus apella) doing so (Anderson, Hattori, & Fujita, 2008; Genty & Roeder, 2007; Glady, Genty, & Roeder, 2012).

What is particularly interesting about such a ‘less-preferred➔more-preferred’ solution is that it reflects an internally generated value scale (Kralik, Xu, Knight, Khan, & Levine, 2012; Levy & Glimcher, 2012). This solution therefore would not be tied to the specific perceptual features of the food items since perceptual features are translated to this single value dimension. Yet it has been argued that a unique difference between human and nonhumans may be that the latter is unable to form a rule that is independent of perceptual features, given that even more abstract rules such as same/different require some comparison of the specific perceptual features (Penn, Holyoak, & Povinelli, 2008). Thus, abstraction in the reverse-reward problem potentially enables a test of the strong view that nonhuman animals are unable to comprehend relational rules that are truly independent of perceptual features. In fact, reverse-reward studies were first conducted by Boysen and colleagues with chimpanzees (Pan troglodytes) who initially failed to learn to select the smaller quantity (Boysen et al., 1996; 1999; Boysen & Berntson, 1995). However, the same chimpanzees had previously learned to associate Arabic numerals with quantities, so the experimenters replaced the food items with numerals. When faced with a choice between numerals, the chimpanzees correctly selected the smaller quantity. Thus, the numerals provided a means to reach the solution; however, exactly what enabled success is unclear: e.g., whether the numerals helped reduce an affective-based prepotent response to the larger option or whether they simplified the nature of the problem in other ways (e.g., having selection of the smaller number lead to directly receiving the larger food reward rather than receiving the other choice option, and thus effectively removing the tertiary relationship between the two options). Moreover, although we may assume the chimpanzees’ performance was based on an abstract number understanding to select the smaller numeral, the potential effects of learning vs. generalization remain unclear as subjects had prior training with numerals and first-exposure trials were not noted. A similar study was also conducted with capuchin monkeys that had prior experience with tokens that represented different amounts (Addessi & Rossi, 2011). Three of eight monkeys were able to learn the reverse-reward task with the tokens but not with the food items themselves (or other tokens that each represented one item). Again, however, it remains unclear exactly what led to success for the three monkeys, such as the tokens changing the nature of the problem as described above. In addition, to assess whether the capuchins’ performance was based on an abstract number understanding, only one monkey appeared to successfully generalize to tokens representing novel quantities, although first trial performance is unclear. The results from these studies are nonetheless intriguing and warrant further investigation of potential abstraction ability using the reverse-reward problem.

In addition to testing whether nonhumans have the capacity to abstract to factors beyond perceptual features, our study was also designed to test whether higher levels of abstraction promote creative problem-solving by enabling more significant comprehension of the reverse-reward relationship between the two choice options: specifically, the tertiary relation between the two options. If such promotion is found, it would also provide evidence for the cascading activation process. This is so because if promotion of problem-solving occurs, abstraction of the food items must ultimately have led to activation of something “beyond” the food items themselves, and more specifically, to a detection of the tertiary relation between them.

In the current study we first sought to clarify the level of possible abstraction used to solve the reverse-reward problem by rhesus monkeys and determine whether the level rose above perceptual features of the training stimuli. Then, if individual differences were found — which they were — we examined whether there was a relationship between abstraction level and problem-solving (i.e., abstraction level and solution speed). In Experiment 1, we tested abstraction level with non-rewarded probe trials, which provides the cleanest test, given that feedback could not be used to generalize the probe stimuli. At the same time, the negative feedback of non-reward can instruct them to treat the probe trials differently; and there was some evidence the rhesus monkeys were affected (described below). Additionally, previous reverse-reward studies used rewarded probe trials. We therefore also conducted Experiment 2 in which the probe trials were rewarded to eliminate the negative impact of non-rewarded probe trials, as well as enable a more direct comparison with previous findings. We then examined the individual differences to determine whether abstraction promoted reverse-reward problem-solving via a cascading activation process from the individual choice options to the relation between them.

2. Material and methods

2.1. Subjects

Three male rhesus monkeys, Puck, Hamlet, and Titus, ages 7, 7, and 9 years old, respectively, participated in Experiment 1. Puck and Hamlet were further tested in Experiment 2 and were joined by 8-year old Caesar, who had in fact been tested in Experiment 2 first and therefore could not meaningfully be tested on non-reinforced probe trials in Experiment 1. The monkeys were housed in a homeroom with automatically regulated temperature, ventilation, humidity, and lighting, and were maintained at approximately 95% of their ad libitum weights to ensure good health and sufficient motivation. Environmental enrichment included two or more enrichment items in their home cages at all times, daily playing of radio or videos in the room (the latter via a monitor mounted in view of all individuals), and regular access to a large enrichment area (68 × 38 × 72 inch) in an adjacent room. In addition, the Center for Comparative Medicine and Research (CCMR) at Dartmouth maintains a full-time animal care and veterinary staff that monitored the monkeys’ daily health and wellbeing.

The monkeys were brought individually to the testing room in the laboratory in custom-made chairs. The chairs were designed for maximal comfort and safety (Knight et al. 2013). Because every probe trial was critical, especially first trials, it was imperative to minimize potential errors due to extraneous variables. Thus, the chairs enabled maximal attentional and behavioral control, with reliable positioning of the choice options relative to the monkey, precise timing of the trial sequence, and collection of clear choice data. Animal care and use complied with all current laws and regulations of the United States, the United States Department of Agriculture (USDA), and the Institutional Animal Care and Use Committee (IACUC) of Dartmouth College.

Because a main aim of the study was to have rigorous control over the monkeys’ experience, it is critical to clarify their prior experience. Puck and Hamlet participated in three prior studies (Knight, Klepac, & Kralik, 2013; Kralik, Xu, Knight, Khan, & Levine, 2012; Xu, Knight, & Kralik, 2011), Titus in two (Kralik, Xu, Knight, Khan, & Levine, 2012; Xu et al., 2011), and Caesar was naïve. In all three prior studies the monkeys did make choices based on food preferences, but they never selected between a single fruit and vegetable nor were they tested with a reverse-reward contingency that required selecting the least preferred item — that is, in every case they selected and received the most preferred option. Thus, their prior experience should have had no effects on the outcome of the current experiment. The fact that the results of the fourth monkey who was naïve, Caesar, were similar to Puck’s and Hamlet’s also provides additional support that the findings were not affected by prior experience. Further detail about the prior studies is provided in the Supplementary Material.

2.2. Transfer tests to determine level of abstraction

After solving the reverse-reward problem with only one and four half-peanuts, we designed the probes (in both experiments) to determine what features of the choice options were used to solve the problem. Quantity probes tested whether the monkeys generalized from the original ‘one, four’ pair to other novel quantities. The probe tests in which the smaller quantity was four (called 4-Smaller), i.e., the quantity the monkeys were required to avoid in the original pair but now must spontaneously select, are particularly revealing, with a correct response reflecting an underlying inductive-inference process from ‘select 1, avoid 4′ to quantity in general: i.e., ‘select smaller, avoid larger’. In addition, the quantity probes tested whether the difference between the two quantities influenced performance, with for example, a greater difference between them making the problem easier or harder (Boysen et al., 1996; 1999; Boysen & Berntson, 1995; Brannon & Terrace, 1998; Cantlon & Brannon, 2006; Kralik, 2012). Next, two probes were used to test the extent of content-dependence of the problem solution, with respect to specific features of the original food items themselves, the half-peanuts. Successful generalization to raisins would show a choice option comparison that was not dependent on the original food type (peanuts) nor any of its main perceptual features, such as color, contrast, and texture. And successful transfer to dimes would reveal generalization beyond both the specific perceptual features as well as the content domain of food altogether. To verify that the dimes were not initially treated as potential food items by the monkeys, prior to the non-food test using dimes, we initially acclimated the monkeys to the dimes with the Experimenter standing near the monkey and holding a dime in his hand close enough for the monkey to reach out and take. However, none of the monkeys showed any positive or negative interest in it (e.g., did not attempt to reach out and take it or swat it away). Thus, they were not frightened by the dime stimuli nor did they treat the dimes as food items. In this test, we presented one and four dimes to the monkeys; upon selection, they would receive the other quantity in half-peanuts (i.e., if the one dime was selected, four half-peanuts would be placed in the opposite bowl).

Successful transfer to all of the above probe tests might suggest a problem solution based on a content-free abstract concept of quantity, however, additional quality probe tests examined the level of abstraction even further. By providing a choice between a larger, lower quality food item and a smaller, higher quality food item we could determine whether the solution was more generally based on an internally derived common value comparison of the two choice options (Fig. 1D). The food items used for the quality tests were selected based on previously established preferences of the monkey, in which the laboratory and animal care staff allowed the monkeys to freely choose among multiple items to determine which to give for behavioral enrichment treats. Among the monkeys’ favorites were fruits, especially grapes, and miniature marshmallows. Thus, we used these items for the preferred options (in Experiments 1 and 2 respectively). Vegetables were given as regular dietary supplements by the animal care staff even though they were not highly valued by the monkeys (and thus not given as enrichment treats since they showed little sign of interest in them, taking them reluctantly, eating more slowly, and only after they first ate the fruit when provided together). We therefore used vegetables as the less-preferred food items; however, to identify vegetables they would actually accept and eat across an entire session (and thus maintain low but positive value) we conducted formal 30-trial test sessions in which we simply offered the subject a ½ piece of vegetable. This testing yielded sugar snap peas for Hamlet and Titus, and green beans for Puck (Kralik, Xu, Knight, Khan, & Levine, 2012). In addition, we chose these particular food items to provide a clear test of quality over quantity, with the higher quality items (grape and marshmallow) smaller in length, area and volume than the lower quality ½ and whole vegetables, respectively (Table 1). As can be seen in the table, preference was presumably based on sugar content. Finally, as a sample preference test subsequent to the experiment (and after ‘select the one you prefer’ was reestablished), Puck and Hamlet were available to be tested on a choice between grape vs. whole vegetable, with Puck selecting the grape every time, 30/30 = 100%, and Hamlet, 27/30 = 90% (with errors on the 2^nd, 5^th, 21^st trials).

Table 1.

Quality test food items and their approximate dimensions.

	Experiment 1			Experiment 2**
Specific item	Grape	Pea pod	Green bean*	Mini Marshmallow	1/2 Pea pod	1/2 Green bean*
Type	Fruit	Vegetable	Vegetable	Marshmallow	Vegetable	Vegetable
Color	Deep Red	Green	Green	White	Green	Green
Length (mm)	20	75	102	12.5	37.5	51
Width (mm)	20	20	9	12.5	10	4.5
Depth (mm)	20	10	9	12.5	5	4.5
Area (mm2)	400	1500	918	156.3	750	459
Volume (mm3)	8000	15000	8262	1953	7500	4131
Weight (g)	5	2.8	5.7	0.7	1.4	2.85
Calories	3 (12.5 kJ)	1 (6 kJ)	2 (8.5 kJ)	2 (8.5 kJ)	0.5 (3 kJ)	1 (4.3 kJ)
Sugar (g)	0.8	0.1	0.2	0.4	0.05	0.1
Quality	Higher	Lower	Lower	Higher	Lower	Lower

^*Puck only

^**Both whole and 1/2 vegetable tested

2.3. General procedure

The monkeys were tested individually in a laboratory test room, seated in front of a standard black laboratory table. Because a key aim of the study was to ensure that the monkeys would have exposure to the one and four quantities only, it was necessary to test them by hand, given that current automated systems still make periodic errors, such as pellet dispensers jamming. However, potential inadvertent cuing of the correct response by the experimenters then becomes a critical issue. We thus undertook several measures to minimize this possibility. For potential auditory cues correlated with the correct option, we played white noise in the experimental room to help mask sounds. Potential visual cues correlated with the correct option could derive from any inadvertent positioning or movements by the experimenter (e.g., hand, arm or body positions, and facial expressions). However, multiple precautions suggest that these possible effects were unlikely. First, we used four different experimenters, both male and female, during training and testing to remove any inadvertent idiosyncratic cues (e.g., particular hand position, facial expression). Second, the study was not designed as an all-or-none test of generalization ability, but rather as an attempt to determine the level of generalization along a continuum of possibilities that could very well have differed among the subjects (as it did). Thus, there were no clear outcomes to be anticipated. Third, the laboratory personal protective equipment (goggles over the eyes, medical mask over the nose and mouth, lab coat over the body and arms, and medical gloves over the hands) helped to mask visual cues. Fourth, all experimenters were trained to follow a precise, stereotyped procedure throughout, and at least one other person observed the session outside of the test room via camera (Logitech QuickCam Orbit MP Webcam, Logitech, Newark, CA, USA) and computer monitor to verify that the procedure was carefully followed. Fifth, the simple task of selecting among actual food-item options (other than the non-food dimes tests) honed the monkeys’ attention to the food items themselves, rather than to the experimenter or other stimuli. Sixth, clear threshold lines were marked that produced a consistent and clear criterion for a choice. Finally, for a direct test of whether inadvertent experimenter cues may have influenced the results, we conducted the quantity tests in Experiment 2 (see Section 2.4.3) exactly as was done in the Kralik (2012) study in which automated pellet dispensers were used and cuing by the experimenter rendered impossible. This design replication enabled a direct comparison of the main quantitative transfer results from that study with the current one. No statistical differences were found as reported in the results section below.

A clear Lexan box was constructed to hold two recessed metal bowls to the left and right of the monkey’s midline, but within reach, and with the box surface tilted forward so that the monkeys could see the entire contents of the bowls (see Fig. 1A&D for illustration). At the beginning of each trial, the experimenter would open both hands simultaneously, revealing one half-peanut in one hand and four half-peanuts in the other. The experimenter took care to align both hands over the respective bowls, out of the monkey’s reach but with all half-peanuts clearly displayed. The monkeys were required to keep their hand on the chair’s touch bar mounted right of center and waist high until the moment of selection. After three seconds, the experimenter would move his/her hands forward so that his/her fingertips would reach the edge of the bowl closest to the experimenter (and opposite from the monkey). When this position was reached, the monkey was allowed to make a choice by touching the Lexan box just under the left or right bowl to indicate its choice. The experimenter would then close both hands, turn the hand that the monkey had not selected over, and drop the half-peanut(s) into the bowl. The monkey then collected the reward, and the experimenter set up for the next trial. For testing, the quantities presented on the left and right were pseudo-randomized across trials so that the monkey was never presented with more than three identical trials in a row.

2.4. Experiment 1 & 2 testing procedures

2.4.1. Experiment 1 testing procedure

Specific training procedures are detailed in the Supplementary Material. To determine how the monkeys had solved the reverse-reward problem, we introduced non-rewarded probe trials intermixed with normal reversed-reward trials using the original one versus four half-peanuts exemplar. Although non-rewarded trials significantly reduce the number of probe tests that could be conducted, a critical aim of the study was to ensure that any successful transfer was due to generalization rather than learning from positive feedback. To prepare the monkeys for trials with no reward, prior to the probe tests, we first required performance on the original exemplar pair (one and four half-peanuts) to stabilize with an accuracy of 80% or better in 50-trial sessions in which every 10^th trial on average was not rewarded. Indeed, because of the potential influence from the lack of reward in the probe tests, we consider the first trial of each probe type to be the most indicative of performance.

To minimize the potential influence of the non-rewarded probe trials on subsequent performance, we (a) intermixed the probes with original exemplar trials (i.e., the one and four half-peanuts used in training); (b) withheld reward on up to two original exemplar trials in a session (depending on the length of the session based on their performance, with every 10^th trial on average not rewarded); (c) conducted no more than four probe trials in a given session; and (d) conducted only two trials for each of the seven probes. In addition, it was critical that the monkeys performed the reverse-reward task well, so that the initial probe trials would meaningfully test whether an abstract problem solution was learned and being used. Therefore, we required the monkeys to maintain at least 80% accuracy in a session before any probe trials were conducted. Moreover, to minimize potential errors on probe trials due to extraneous factors, such as a lack of attention, we also required at least 10 consecutive original exemplar trials to be performed correctly prior to any probe trial. In fact, Titus’s performance level dropped below 80% correct per session for 13 consecutive sessions (64.46% ± 3.51 correct), after which he was dropped from the study.

And in attempt to provide the clearest tests possible, we presented the probe tests in order from what we anticipated to be the most difficult (with respect to the level of abstraction) to the easiest, to ensure that the most difficult probes be tested earliest (i.e., with minimal influences of novel experience and lack of reward). Table 2 lists the set of probes in the order in which they were tested (also see section 2.2 for probe test explanations). Because the transfer tests were conducted to determine if performance would remain significantly above chance, one-tailed binomial tests were used to test for significant transfer.

Table 2.

Experiment 1 Probe Types (in order tested)

Inferior offer	Superior offer	Test Explanation
Full pea pod/green bean*	Grape	Transfer to quality, based on internally generated common value scale
Four Dimes	Seven Dimes	A. Non-food item to test generality of value scale B. Most difficult quantity test: i.e., selecting the quantity (four) they learned to avoid during training
Four Raisins	Seven Raisins	A. Different food item and perceptual features B. Most difficult quantity test: i.e., selecting the quantity (four) they learned to avoid during training
Four Half-peanuts	Seven Half-peanuts	Most difficult quantity transfer with same food item: i.e., selecting the quantity (four) they learned to avoid during training
Three Half-peanuts	Eight Half-peanuts	Novel quantities of same food item (larger difference between them)
Two Half-peanuts	Three Half-peanuts	Novel quantities of same food item (smallest difference between them)
Two Half-peanuts	Nine Half-peanuts	Novel quantities of same food item (largest difference between them)

^*green bean for Puck only

Thus, seven different probes were used in Experiment 1. Puck received the seven probe trials, and the same seven again for a total of 14 probe trials. Hamlet received the seven probe trials, and then the first five again for a total of 12. Hamlet then inadvertently received reward after his response to the 12^th (of 14) Experiment 1 probe test. Because the point of Experiment 1 was to test the monkeys on non-rewarded probes to isolate generalization from learning, the monkey was then immediately moved to Experiment 2. Titus received the first six probe trials only, and as stated was removed from the study because he could not maintain performance at the criterion level (≥ 80%) and thus would render the subsequent tests uninformative. Nonetheless, we note that Titus’s performance based on the six probe trials was clear, as shown below.

2.4.2. Experiment 2 testing procedure for quality and non-food item tests

In Experiment 2, we conducted further probe tests that were rewarded based on the reverse-reward contingency. For Puck and Hamlet, accuracy on the original exemplar pair (1 vs. 4 half-peanuts) remained at the criterion of 80% or better at the onset of Experiment 2. The new monkey’s (Caesar) accuracy on the final training session was 98% (100 trials; see the Supplementary Material section for specific training procedures). The probe tests included two 75-trial quantity transfer test sessions (see Section 2.6.), and four additional transfer tests in which the half-peanuts were replaced with other items. For these four additional tests, the monkeys first received 10 trials with the original one vs. four half-peanuts, to verify that the monkeys’ general performance remained stable. After this, 50 test trials were conducted. In the first probe test, the half-peanuts were replaced with raisins (to test an alternative food item with different perceptual features). The next two tests examined whether generalization was based on food amount vs. quality. For the first of these, the one half-peanut was replaced with one half piece of vegetable (sugar snap pea pod for Hamlet and Caesar, green bean for Puck), and the four half-peanuts were replaced with one miniature marshmallow (see Table 1 for food item details). For the next probe test, the one half-peanut was replaced with an entire vegetable, and the four half-peanuts were replaced again with one miniature marshmallow. For the final probe test, the one and four half-peanuts were replaced with dimes (to test the generality of the value scale). The dimes test was conducted last to prevent the dimes condition from potentially influencing the other tests in Experiment 2.

2.4.3. Experiment 2 testing procedure for quantity transfer

To test transfer to multiple quantities, to test for a difference effect (Boysen et al., 1996; 1999; Boysen & Berntson, 1995; Brannon & Terrace, 1998; Cantlon & Brannon, 2006; Kralik, 2012), and to directly compare our results to those of Kralik (2012) in which automated pellet dispensers were used and cuing by the experimenter rendered impossible, we followed the identical procedure of using 2, 3, 5, 6 and 7, and the following 15 combinations: 1 vs. 2, 1 vs. 3, 1 vs. 4, 1 vs. 6, 2 vs. 3, 2 vs. 4, 2 vs. 5, 2 vs. 6, 2 vs. 7, 3 vs. 4, 3 vs. 5, 3 vs. 6, 3 vs. 7, 4 vs. 6, 4 vs. 7. As in the Kralik (2012) study, every combination was tested 5 times per session for a total of 75 trials per session, and two sessions were conducted. Note that the combinations with both novel quantities were 2 vs. 3, 2 vs. 5, 2 vs. 6, 2 vs. 7, 3 vs. 5, 3 vs. 6, 3 vs. 7, for a total of 35 “Novel Only” trials per session; and the combinations in which the quantity 4 was smaller was 4 vs. 6, 4 vs. 7, for a total of 10 “4-smaller” trials per session. In addition, “2-vs” consisted of the quantity 2 versus the following set of quantities, {1, 3–7}, for a total of 30 trials per session; and “3-vs” consisted of 3 versus {1, 2, 4–7} for a total of 30 trials per session.

2.5. Data analysis

To assess performance in the generalization trials we used binomial tests for individuals and t tests for group data. We used t tests to provide the clearest examination of group performance using averages for each individual; in every case, however, the results were further examined with the binomial tests on individual performance as stated. Because the question in every case was to determine whether the monkeys generalized to the new stimuli, one-way tests were used.

3. Results

3.1. Experiment 1

On average, the monkeys required 3213 ± 624 trials (mean ± standard error of the mean—SEM) to learn the reverse-reward task to a stable and sufficient level of performance with one exemplar: one and four half-peanuts. We followed a regime that included remedial trials in which they received nothing when they incorrectly selected the larger quantity, as well as a correction procedure that maintained the food items in the same left-right locations until a correct response was made (see Supplementary Material). Averaging across monkeys, they achieved an accuracy score of 94% ± 3% correct, defined as choosing the smaller quantity.

As Figure 2 shows, on the important first trials, in which the seven probe types (see Table 2) were each experienced for the first time, both Puck and Hamlet attained 100% accuracy (one-tailed binomial test, P < 0.01 for each monkey). Titus, in contrast, was tested on only the first six probe trials (see Materials and methods) and performed at chance level of 50% correct. For all probe trials combined, Puck and Hamlet significantly transferred to the novel trial types (Figure 2; one-tailed binomial test: Puck: 86% correct, P < 0.01; Hamlet: 83%, P < 0.05). Puck was “correct” on the first eight probe trials and “incorrect” on the ninth (4 vs. 7 dimes) and thirteenth (2 vs. 3 half-peanuts), obtaining 12 out of 14 probe trials. Hamlet was “correct” on the first 10 probe trials and then “incorrect” on the next two (4 vs. 7 half-peanuts and 3 vs. 8 half-peanuts), thus obtaining 10 out of 12 (after which he began Experiment 2—see section 2.4.1). Titus was “incorrect” on the first (quality), fourth (4 vs. 7 half-peanuts), and sixth (2 vs. 3 half-peanuts) probe trials out of a total six.

Figure 2.

Experiment 1 probe results. “Percent Correct” is the percent the inferior item was selected. Left side displays the results for the first trial of all seven probe types (six for Titus) (see Table 2); Right side is all probe trials combined. The dashed line at 50% represents chance level performance. *: P < 0.05, **: P < 0.01.

Finally, we examined the relationship between generalization level and the ability to solve the reverse-reward task. Although all three monkeys learned to perform the task, there was a clear relationship between transfer ability (as measured by percent correct on the first six probe trials — see Table 1) and the number of trials required to learn the task (Figure 3).

Figure 3.

The relationship between generalization level (as measured by percent correct on the first six probe trials in Experiment 1; see Table 2) and the number of trials to learn the reverse-reward task.

3.2. Experiment 2

In Experiment 2, we rewarded all trials according to the reverse-reward contingency. Along with Puck and Hamlet, Caesar was added as the third subject in this experiment and thus experienced the probe trials for the first time. He required 1246 total trials to learn the task (98% accuracy on final session) (see Supplementary Material).

Figure 4 shows that the monkeys performed at a high level when presented with different quantities (1–7, combinations listed in section 2.4.3). The overall performance rate of 88% correct was well above chance (one-tail t(2) = 10.94, P < 0.01). More specifically, the monkeys performed well on all trial types: all trials in which both quantities were novel (91%, one-tail t(2) = 12.32, P < 0.01); trials involving the quantity two, i.e., ‘two versus {1, 3–7}’ (89%, one-tail t(2) = 19.18, P < 0.01); trials involving the quantity three, i.e., ‘three versus {1, 2, 4–7}’ (87%, one-tail t(2) = 7.89, P < 0.05); and the most difficult trials in which the quantity ‘four’ that they avoided in the original exemplar choice of one vs. four half-peanuts was now the correct response, i.e., ‘four versus {6, 7}’ (81%, one-tail t(2) = 5.97, P < 0.05).

Figure 4.

Experiment 2 quantity transfer test results (150 trials over two sessions) averaged across the three monkeys (bars) and for each one individually (the three symbols at the top of each bar). Left bar is accuracy (% chose smaller to obtain larger) for original one versus four half-peanut exemplar trials; second bar is accuracy overall; third is for trials with both quantities novel (“All Novel”); fourth and fifth are trials in which one quantity is two (“2-vs”) or three (“3-vs”), respectively; far right bar is accuracy for trials in which the smaller quantity is four (“4-Smaller”). The dashed line at 50% represents chance level performance. *: P < 0.05, **: P < 0.01.

All three of the monkeys performed well above chance (one-tail binomial tests, N = 150, each at a P << 0.0001 confidence level, except where noted). For overall performance, Puck attained 87% correct; Hamlet, 83%; and Caesar, 95%. On novel trials, Puck scored 89%; Hamlet, 86%; and Caesar, 97%. On two versus the other quantity trials, i.e., two versus {1, 3–7}, Puck reached 87%; Hamlet, 88%; and Caesar, 93%; on three versus the other quantity trials, i.e., three versus {1, 2, 4–7}, Puck scored 88%, Hamlet, 79%, P < 1 × 10⁻⁵, and Caesar, 95%; and on the special case of four versus {6, 7}, Puck attained 80%, P < 0.01; Hamlet, 72%, P < 0.05; Caesar, 90%, P < 0.001). Thus, all three monkeys transferred to novel quantity combinations.

To compare these quantity results directly to those reported in the Kralik (2012) study, we conducted two-tail t-tests on performance accuracy and found no significant differences overall (current study: 88%, previous study: 86%; t(4) = 0.53, P = 0.63), with novel stimuli only (current: 91%, previous: 85%; t(3) = 1.14, P = 0.34), and with the most difficult 4-smaller tests (current: 81%, previous: 85%; t(3) = 0.56, P = 0.61). Therefore, the current findings for transfer to novel quantities replicates quite closely those obtained by Kralik (2012) in which an automated version of the reverse-reward task was used, providing a direct comparison across studies and testing paradigms, as well as a test of the possible effects of inadvertent cues from the experimenter in the current study. These statistical results show that no cues significantly influenced the quantity tests and suggest that no cues influenced any of the results from the current study significantly.

As found in other numerosity studies (Boysen et al., 1996; 1999; Boysen & Berntson, 1995; Brannon & Terrace, 1998; Cantlon & Brannon, 2006; Kralik, 2012), there was an overall effect of the differences between the quantities of the choice options, in which performance positively correlated with the numerical difference between quantities, such that a greater difference between them led to generally better performance (R² = .81, P < 0.05); however, no individual monkey exhibited a significant effect of this kind.

The monkeys also performed well on the raisin transfer test overall (92%, one-tail t(2) = 13.75, P < 0.01; Figure 5) and as individuals (one-tail binomial tests, N = 50, Puck, 90% correct, P < 1 × 10⁻⁸; Hamlet, 88% correct, P < 1 × 10⁻⁷; Caesar, 98% correct, P < 1 × 10⁻¹²), with all three monkeys correct on the first raisins trial. Their successful performance showed that they transferred to other food items and across basic perceptual dimensions such as colour and texture. Similar performance levels were observed for the dimes transfer task both overall (89%, one-tail t(2) = 9.7, P < 0.01; Figure 5) and individually (one-tail binomial tests, N = 50, Puck, 82% correct, P < 1 × 10⁻⁵; Hamlet, 96% correct, P << 0.0001; Caesar, 90% correct, P << 0.0001), with all three monkeys correct on the first dimes trial. Successful performance with dimes showed transfer to non-food items (discussed further below), as well as to the unique perceptual features (silver color, shiny texture, inedible).

Figure 5.

Experiment 2 transfer results for raisins, dimes and quality tests averaged across the three monkeys (bars) and for each one individually (the three symbols at the top of each bar). “Percent Correct” is the percent the inferior item was selected (and thus superior obtained). “1/2 Quality” tested a more-preferred mini marshmallow versus a less-preferred ½ vegetable. “Full Quality” tested a more-preferred mini marshmallow versus a less-preferred entire vegetable. The dashed line at 50% represents chance level performance. **: P < 0.01, ***: P < 0.001.

For the quality tests, we used a highly preferred food item (mini marshmallow) and a positively valued but less preferred item (piece of vegetable: sugar-snap pea pod for Hamlet and Caesar, green bean for Puck; see Table 1 for specific item characteristics). The first quality transfer test required the monkeys to choose one half of a vegetable or the miniature marshmallow. The monkeys again performed very well overall (89% correct, one-tail t(2) = 8.29, P < 0.01; Figure 4) and as individuals (one-tail binomial tests, N = 50 trials, Puck, 90%, P << 0.0001; Hamlet, 80%, P < 0.0001; Caesar, 96%, P << 0.0001). Puck and Caesar were correct on the first trial, whereas Hamlet was incorrect. In the second quality transfer test session, the half-vegetable was replaced with a whole one, to further verify that the monkeys were not using ‘size’ to make their choice (i.e., select the smaller amount). Once again, high performance levels were observed both overall (98%, one-tail t(2) = 24, P < 0.001; Figure 4) and with individuals (one-tail binomial tests, N = 50, Puck, 100%,; Hamlet, 100%; Caesar, 94%, all P << 0.0001); on the first trial, all three monkeys correctly selected the whole vegetable to obtain the mini marshmallow.

An important consideration in examining the Experiment 2 results was whether, with initial success, the monkeys were learning across the session rather than simply transferring their knowledge from the initial training with one and four half-peanuts. Within-subject logistic regression analyses conducted on each of the data sessions indicated only one significant result: Hamlet on the half-quality test (P < 0.05). This result reflects the fact that Hamlet did not readily transfer at the beginning of the first quality test session. Interestingly, Hamlet’s improvement after the first trials of the first session was abrupt, suggesting a re-application of the general solution at that time. Once the high level was reached, it was maintained throughout the remainder of testing. Taken as a whole, these tests indicate that no significant learning took place across any of the sessions, with the exception of Hamlet in the half-quality test session. Overall, these findings provide further evidence that the monkeys’ performance reflected transfer of the solution learned in initial training from quantity to quality.

Caesar did not participate in Experiment 1 because his data were gathered before the others. Thus, the initial exposure of each transfer trial type also tests immediate transfer comparable to the probes in Experiment 1. Caesar chose the smaller quantity correctly on every first-exposure to the quantity transfer tests (N=14, 100%) and on the first trial of all other probes: i.e., raisins, dimes, and both quality tests.

4. Discussion

Creative problem-solving is a hallmark of higher intelligence, and it has been theorized that the ability to solve basic problems with nonapparent solutions, a signifier of creativity, may have driven the evolution of granular prefrontal cortex in primates (Kralik, Mao, Cheng, & Ray, 2016; Kralik, Shi, & El-Shroa, 2016; Passingham & Wise, 2012; Preuss, 1995; Striedter, 2005; Wise, 2008). Here we took a comparative approach to examine a nonhuman model of creative problem-solving. We examined how and how generally rhesus monkeys solved the reverse-reward problem to determine: (a) the level of abstraction (if any) used to compare the choice stimuli; (b) whether a nonhuman is capable of abstracting knowledge beyond specific perceptual features; and (c) if abstraction from specific food items to a relation between them might facilitate the problem-solving.

4.1. Level of abstraction

For both experiments, we provided a continuum of probe tests to identify the attributes and level of abstraction used by each individual to select the correct single half-peanut over four half-peanuts. For Experiment 1, we ordered the probe stimuli from most difficult to easiest to provide the most stringent test of generalization. We required a sustained high level of accuracy on the training exemplars before we conducted any probe trial to reduce the potential of inadvertent extraneous effects on the monkey’s choice during the most critical first trials. Subjects were thus comfortable, focused, and had grasped at least a rudimentary solution to the reverse-reward problem. Two of the three rhesus monkeys (Puck and Hamlet) chose “correctly” on the first trial for every probe stimulus. Thus, they generalized their basic reverse-reward solution to novel quantities, notably including the most difficult cases in which the larger training quantity that they had learned to avoid (four) was the smaller and thus “correct” option (Albiach-Serrano et al., 2007; Anderson et al., 2000; 2004; Genty et al., 2004; 2011; Genty & Roeder, 2007; Kralik, 2012; Uher & Call, 2008). Furthermore, their choices did not show dependence on the type of reward offered or even if a chosen item was directly perceived as a reward itself, as the two monkeys transferred to trials of different food items (raisins) and even non-food objects (dimes), the latter of which held no obvious intrinsic value (see below). Beyond this, they spontaneously transferred this understanding to a quality-based problem where they chose between two single food items (a vegetable and fruit) in which the inferior option was now the largest in size.

What could explain the success of the two monkeys on all initial transfer tests? First, it is unlikely to be due to random responding, given that both monkeys went seven for seven on the 1^st trial transfer tests. Overall, Puck obtained the more valuable reward 12 times out of 14, and Hamlet 10 times in a row. For the quality test specifically, Puck and Hamlet were both two for two (as was Caesar on the first trial of both quality tests in Experiment 2). Thus, it was highly unlikely that the monkeys responded randomly. Finally, because the problem was initially presented in only one context (choose between one and four half-peanuts), and the transfer trials were not rewarded, the subjects’ solutions to the novel problems could not have resulted simply from the limited experience with the specific training exemplar nor from reward learning with the subsequent probe tests.

It is highly unlikely that items as diverse as fresh fruit, half-peanuts, raisins, green vegetables, and metallic dimes share perceptual features that govern a learned rule that guided the subjects’ selections. Choices therefore may appear to be based on the abstract concept of “less preferred vs. more preferred”, which is interesting in and of itself, but results of the dimes test suggest a dimension potentially more abstract. When initially shown a dime for acclimation, subjects did not show any interest (e.g., did not reach out and attempt to take it or fixate on it). Thus, they did not treat dimes as food items or items of intrinsic value. Why then did they select four dimes in the transfer trials? This was likely due to the forced choice structure of the task as well as a possible ‘more is better’ effect when identical items are used. Chimpanzees responded in a similar way when presented with rocks instead of food items, although it is possible the rocks had some value as a familiar object with potential uses (Boysen et al., 1996). In our study, if the monkeys applied a ‘more of item x is better than less of item x’ bias to a choice between the dimes, the solution that best explains the entirety of the findings is best characterized as an abstract determination between the inferior vs. superior option.

Experiment 1 was difficult to extend further as unrewarded cases lose their appeal quickly. The negative feedback of non-reward in the transfer tests from Experiment 1 could have caused subjects to treat the probe trials differently, and in fact there was evidence that this started to occur (described below). Moreover, other comparative studies that tested abstraction with the reverse-reward problem typically rewarded probe trials, therefore we conducted Experiment 2, in which probe trials were rewarded. In Experiment 2, the new monkey Caesar applied the abstract problem solution nearly flawlessly in transfer tests by choosing the inferior and thus “correct” option on all first-exposure probe trials and on the first 17 probe trials he experienced (with different quantities) before his 1^st error. Thus, three of the four monkeys in the study appeared to spontaneously use an internally generated value scale that transcends any specific perceptual attributes of the available choices to assess the choice options as inferior or superior (Levy & Glimcher, 2012). The results therefore provide evidence that nonhuman animals can comprehend abstract non-perceptual features, infer them from only one specific case (one vs. four half-peanuts), use them to override the presumably strong natural preference to reach for a superior option and instead select the inferior one, and then apply this abstract ‘select inferior to receive the superior’ rule across a range of forced-choice problems. Although our findings may reflect a unique special case in which a truly non-perceptual attribute can be drawn out via the valuation process, the results nonetheless corroborate other findings for abstract cognitive processing in nonhumans (Addessi & Rossi, 2011; Boysen et al., 1996; Boysen & Berntson, 1995; Brannon & Terrace, 1998; Call & Tomasello, 1997; Cantlon & Brannon, 2006; Hampton, 2001; Kralik, 2012; Kralik & Hauser, 2002; Premack, 1983; Seed & Byrne, 2010; Wallis, Anderson, & Miller, 2001; Wynne & Udell, 2013).

The other two monkeys in Experiment 2 (Puck and Hamlet) also inferred the “correct” solution consistently across all probe tests. The successful monkeys did not always apply the general solution properly, yet these findings are also instructive. In Experiment 2, Hamlet’s difficulty in applying a correct solution to the first quality test (vegetable vs. mini marshmallow) could easily be a residual effect of the unrewarded or ‘punished’ probe trials in Experiment 1, which used similar stimuli. Evidence for this includes his two incorrect responses in Experiment 1 on theoretically easier trials after he had transferred the solution flawlessly on the first 10 probes in a row. A second possibility is that the abstract problem solution was overridden by a stronger affective desire for the very highly valued mini marshmallow. Interestingly, Hamlet’s improvement after the first trials of the first session in Experiment 2 was abrupt, which suggests he was able to quickly re-apply the solution after facing this obstacle. Once he did this, he achieved and maintained a high level of performance.

All four monkeys, then, learned to solve the problem with the one and four half-peanuts. Three of these catarrhine primates transferred a successfully learned rule to all probe trial types, thus exhibiting the formation and application of abstract understanding that selecting the inferior of two choice options would yield the better one. Individual differences notwithstanding, one would also expect species-level differences in abstraction capacity and the ability to apply it. Significant abstraction capacity has been found in many species tested using the reverse-reward problem, but this capacity for generalization appears to expand from prosimians to anthropoids to catarrhines (Albiach-Serrano et al., 2007; Anderson et al., 2000; 2004; 2008; Genty et al., 2004; 2011; Genty & Roeder, 2007; Glady et al., 2012; Kralik, 2012). Whether differences found across studies reflect methodological variance or species-specific ability is yet unresolved, but, assuming the latter, these findings may suggest something about the ancestral conditions that selected for more heightened powers of inference and abstraction. Early catarrhines were probably ecological generalists living in large social groups, like most macaque monkeys (Passingham & Wise, 2012; Striedter, 2005). If so, adaptive advantages could derive from the ability to detect value in a wide range of foods, as well as from diverse foraging strategies. At the same time, the inability of one of the four rhesus monkeys, Titus, to reach the level of abstraction as the others may suggest that this ability lies toward the edge of the species’ cognitive capacity. Indeed, a better understanding of the evolutionary trajectory of cognitive abilities will benefit from a greater focus on species distributions (and thus individual differences) in the abilities.

4.2. Tertiary relation comprehension

The Experiment 1 monkey Titus was “correct” (selecting the inferior reward) on only three of the six first-trial probe tests. He was “incorrect” even with the simpler tests of novel quantities with the original food items; therefore his responses appear to be random. Nonetheless, if there was no generalization whatsoever from his learning, he would have selected the better option exclusively (the default ‘prepotent’ preference), which he did not. His responses suggest that he did recognize the preferred option and learned to avoid it, thus processing it to the level of value, but he did not learn to select the non-preferred (or inferior) one. Rather, he appeared to learn to select the specific single half-peanut, which was not offered in the probe trials, leading to confusion of what to select. His performance thus suggests separable treatment of the two choice options as the preferred (or superior) one and the ‘other’ one, with the superior processed in terms of value, and the inferior only as the specific perceptually bound single half-peanut. This reduced level of processing of the inferior option may have in turn hampered the ability to recognize the tertiary relation between the food item options. In fact, Titus’s eventual inability to consistently apply any solution, abstract or otherwise, to the paradigm (which required us to remove him from the study) further suggests that he either (a) only very weakly responded to a tertiary relation between the specific single half-peanut and the preferred option, or (b) never recognized this relation. For the latter possibility, he may have “solved” the problem via something more akin to simple reinforcement learning: selecting the one half-peanut produces reward. Either way, such weaker solutions to the problem are more susceptible to other influences, such as position biases (i.e., selecting the left or right option repeatedly) that can also be readily strengthened by reward, which continued to occur with Titus.

In contrast, evidence from the other three monkeys suggests that they inferred a meaningful abstract relation between the choice options from the single-example of ’selecting the one half-peanut option yields the four half-peanut option’. The high level of accuracy and reliability of their performance throughout the study across a wide range of stimuli evinces a deeper understanding of the problem that operant conditioning does not easily explain. For example, operant conditioning alone does not convincingly explain how subjects were able to immediately select the “correct” response in probe trials where the most familiar superior option (four half-peanuts) that subjects had learned to avoid from thousands of previous experiences now became the inferior option that they needed to select. Although strong habit effects can produce high reliability, the obvious sensitivity the monkeys showed to the probe trials (conducted in Experiment 1, motivating Experiment 2) shows that they did not respond based on simple habits, which should have produced greater insensitivity and generated a biased response pattern. In addition, the rapid “recovery” for Hamlet in Experiment 2 suggests processing at a level akin to causal reasoning (Kowaguchi et al., 2016; Seed & Byrne, 2010). The three monkeys (besides Titus) also showed strong anticipation for receiving the better option as a reward (i.e., selecting one side then rapidly moving attention and reach to the other), again suggesting awareness of the relation mechanism. Finally, subjects could easily view the entire ‘simple’ procedure (from choice presentation and selection through the delivery of reward); thus it is probable that the highly accurate, reliable, and flexible performance of Puck, Hamlet and Caesar resulted from an awareness of the tertiary relation between the choice options.

4.3. Abstraction promoted tertiary relation comprehension

The three monkeys exhibited a stronger understanding of the relation between the two choice options than did Titus. We therefore found a relationship between the level of generalization on the one hand, and the strength of the tertiary relation comprehension on the other. Titus also took conspicuously longer to solve the reverse-reward problem than the other three (1449 trials more than the next slowest monkey, which was 1.5 times longer). Thus, our study adds to the growing number that highlight individual differences in abilities, and here the differences point to a relationship between abstraction, problem-solving facility, and tertiary relation understanding, which closer consideration of the findings can help delineate further.

As discussed, Titus’s performance showed that he processed the inferior option to a lesser degree than the other monkeys (responding to the specific food item and in any case not to the level of a common value scale). This result suggests that the lesser processing of the inferior option may have hampered Titus’s ability to further recognize the tertiary relation between the food item options, resulting in a weaker grasp of the reverse-reward problem solution. This reasoning thus suggests that greater processing of the inferior option would promote tertiary relation recognition; and indeed all three other monkeys exhibited heightened processing of the inferior option (using a common value scale for both choice options) and solved the problem more quickly and stably than Titus. Of course the opposite causal direction is theoretically possible in that the ability to comprehend the tertiary relation between the two options from the original experience may facilitate abstraction of both choice options to a common value scale, but this possible causal direction (tertiary relation comprehension promoting abstraction) is problematic. If subjects learn the tertiary relation with only the “one vs. four half-peanuts” exemplar, there is no clear plausible mechanism by which attention to the specific perceptual attributes of the food items (e.g., shape, size, and color) would subsequently be eliminated, leaving only the abstract attribute of value, especially with no further feedback or need to motivate the change (i.e., once the problem is ostensibly solved). And yet a simple tertiary relationship based on only those two quantities and food items would be useless on the probe trials in which subjects correctly selected between novel quantities and items. Thus, heightened processing of the inferior option appeared to promote tertiary understanding.

Yet was this ‘heightened processing’ due to the abstraction itself or some other underlying factor that may have led to both the tertiary relation comprehension and the abstraction (i.e., a common cause)? Two possible factors that might drive this heightened processing are (1) higher capacity in general (leading to, e.g., more complete parsing and consideration of the problem components); and (2) heightened attention (resulting in the same). But again, any “common cause” possibility appears unlikely since tertiary relation understanding can occur with the specific food items (i.e., one half-peanut leads to four half-peanuts); thus any cause that promotes comprehending the link between the two food items does not also necessarily lead to higher abstraction, especially with respect to an item valuation scale. There is again no clear mechanism that must lead to further abstraction once the problem is ostensibly solved. The findings thus suggest that abstraction itself played a direct role in promoting the reverse-reward problem-solving.

4.4. Evidence for cascading activation effect

Finally, we consider two possible reasons why the abstraction we found with the three monkeys would promote tertiary relation understanding: an affect-influence-reduction and a cognitive explanation. For the former, abstraction might change the focus from primary affective attributes to secondary ones, allowing subjects to directly associate the two food items and avoid an impulsive, prepotent tendency to select the better one. This would imply turning attention away from primary affective attributes such as calorie to secondary perceptual ones such as color, shape and size; yet our results suggests the opposite — that abstraction was based on value, a more purely primary attribute. Based on this consideration and previous studies that suggest reverse-reward problem difficulties may not derive primarily from affective factors (Kralik, 2005; Kralik, Mao, Cheng, & Ray, 2016; Kralik, Shi, & El-Shroa, 2016), we reject this possibility and conclude that abstraction promoted tertiary relation comprehension via a cognitive rather than an affective change.

Our results thus suggest that abstraction of the food items themselves led more directly to the detection of the tertiary relation between the two items. How exactly this occurs requires further investigation. One possibility is suggested from our recent tool-use study in which the same rhesus monkeys tested in the current study (conducted after this one) chose between two options, one with an otherwise out-of-reach food item resting on top of a support tool and the other with the food item off the tool (Kowaguchi et al., 2016). Initially, they appeared to select the option based on a configurational stimulus of the tool and reward item, i.e., a visual template, and thus utilized an associative process to determine which tool-compound to select. However, once given an additional problem example, they broke from the visual configuration, parsing the food item and tool, recognizing the tertiary relation between them, and using it to make their selection. For the reverse-reward problem, since abstraction enables individuals to consider the specific food items as part of a larger class, it in turn may help to separate them from the rest of the visual scene, which would then be easier to recognize the relation between the food items. In any case, abstraction of the food items to a value-based representation reflects a clear comprehension of both choice options, which is a necessary prerequisite to comprehend the relation between the options. Our results show that three monkeys who indeed processed both choice options to the level of a common value scale solved the reverse-reward problem more readily and handily by comprehending the relation between the choice options, i.e., that selecting the inferior one obtained the superior one. The study thus provides evidence for a cascading effect from the food items to the relation between them. Thus, abstraction can promote problem-solving by helping to provide access to an initially nonapparent problem component.

4.5. Conclusions

Although humans clearly separate themselves from other species with respect to creative problem-solving ability, the mechanisms by which we access previously nonapparent problem components when faced with challenging problems may be shared. For the 9-dot problem, for example, abstraction and a subsequent cascade may be involved in which the problem-solvers realize they can utilize any point on the page (abstraction from 9 dots to the x, y plane), which could subsequently lead to consideration of longer lines beyond the dots themselves, leading ultimately to the solution (Cheng et al., 2013; Kralik, Mao, Cheng, & Ray, 2016). Hence, we believe our study shows how comparative research can provide insights and impetus for further investigation of the specific mechanisms underlying human high-level cognition.

Our study found that rhesus monkeys can reach levels of abstraction that go beyond perceptual features, and this ability facilitates reverse-reward problem-solving, providing a specific example of how abstraction leads to cascading effects that access nonapparent problem components. Indeed, abstraction can help to inhibit prepotent but incorrect solutions, disinhibit otherwise blocked solutions, provide access to remote possibilities, and enable the consideration of complex relationships (Holyoak & Morrison, 2012). These abilities facilitate creative problem-solving, and it will be important to characterize the relationship between abstraction and creative problem-solving more thoroughly in the future, especially the potential potent cascades that can occur. This creative cognition research should take a multidisciplinary approach that includes theoretical and computational development and the examination of neural mechanisms with human and nonhuman animals. In our case, abstraction appeared to enhance reverse-reward problem-solving by promoting the comprehension of the correct solution, rather than simply depressing an affective response to select the better option. Such results reveal an intimate relationship between problem-solving and self-control, which depends on the relative influences of both bottom-up affect and top-down alternative choices. Many maladaptive cases that are attributed to heightened prepotent affect may in fact derive more from exceptionally weak higher-level options (Beran et al., 2016; Kralik, 2005; Kralik, Mao, Cheng, & Ray, 2016; Kralik, Shi, & El-Shroa, 2016; Santos, Ericson, & Hauser, 1999; Wallis, Dias, Robbins, & Roberts, 2001). For those who are able to discern regularities beyond the specific instances encountered, which may cascade to other events these instances take part in, the world is suddenly filled with meaning and possibilities. It has been said that creativity sets us free; but it is through the lens of abstraction that creative solutions are discovered.

• The relationship between abstraction and creative problem-solving remains unclear

• Four rhesus monkeys learned to select an inferior option to receive the superior one

• Three monkeys immediately transferred to a wide range of novel problems

• The abstraction helped comprehend the tertiary relation between the choice options

• Abstraction promotes creativity via cascading activation from objects to relations