Why skill matters

Abstract

Maximizing rewards per unit time is ideal for success and survival for humans and animals. This goal can be approached by speeding up behavior aiming at rewards, and this is done most efficiently by acquiring skills. Importantly, reward-directed skills consist of two components: finding a good object (object skill) and acting on the object (action skill), which occur sequentially. Recent studies suggest that object skill is based on high capacity memory on object-value association. When a learned object appears, the corresponding memory is quickly expressed as a value-based gaze bias, leading to the automatic acquisition or avoidance of the object. Object skill thus plays a crucial role in increasing rewards per unit time.

Evolutionary advantage of skill

The majority of daily human activities, such as lacing shoes, writing with a pencil, riding a bicycle, and using a computer, involve skilled behavior that is carried out with little or no conscious thought. Each kind of skill may be acquired through prolonged and intensive practice across many years [1, 2]. This is not unique to modern human societies. In less labor-divided societies (e.g., hunter-gatherer societies) hunting is a dominant skill among men. The hunting skill is learned gradually with ample experience until it reaches a plateau level toward the end of the life course [3]. Such a protracted course of skill acquisition makes longevity suitable for maintaining and advancing human society and therefore evolutionarily adaptable [4]. Skill also provides evolutionary advantage to many animals [5].

The importance of skills in human and animal societies strongly suggests that the brain has a robust mechanism that controls skills. Indeed, many studies have been aimed at revealing the neural mechanism of skills [6–13]. However, how exactly the changes in behavior associated with skill acquisition benefit humans and animals has not been fully addressed. In this article, we focus on one prominent aspect of skill - speed - and argue that owing to its speed, skill is crucial for success and survival.

An earlier reward is more valued

“You can get $10 now, but if you wait until the next week you will get $15. Which would you choose?” Many people would choose the first option even though the outcome is smaller. This phenomenon is common to humans and animals [14, 15] and is often referred to as ‘temporal discounting of rewards’. Why is an early reward more valued than a late reward? A standard answer in economics is that the value of a future reward is discounted because of the risk involved in waiting for it [16]. An additional reason is that one will receive a larger amount of reward per unit time if the reward is delivered earlier (Figure 1). Indeed, this has been recognized as a key factor in the field of animal foraging research [17, 18] and has been implied in operant conditioning research [19]. So for both humans and animals an early reward is more valued.

Figure 1.

Two kinds of reward delay affecting reward value – hypothetical tasks. A: External delay. In front of you (as a subject) are two buttons, E and L. If you press button E, you will get a drop of juice 1 second later. If you press button L, you will get a drop of juice 5 seconds later. After you have consumed the drop of juice, you can press the button again. Shown here are two extreme cases: you continue to press button E, or continue to press button L. The amount of reward per unit time is larger in the former case. B: Internal delay. In front of you are two buttons, F and S. They are located differently so that you can reach button F in 1 second and button S in 5 seconds. Once you press either button, you will get a drop of juice 1 second later.

One important question concerns the reasons for the reward delay. In the example shown in Fig. 1A, one would simply have to wait for the reward. But this is certainly not the only situation. Another and perhaps more common type of reward delay derives from one’s own behavior (Fig. 1B). Thus, the duration of the behavior acts as a reward delay [20, 21]. If the behavior takes too long a time, it becomes less useful.

There is an important difference between the two kinds of reward delay: unlike the waiting-based reward delay (Fig. 1A), the behavior-based reward delay (Fig. 1B) can be shortened (Fig. 2A) by modifying one’s behavior. From the viewpoint of foraging theory, a quicker behavior enhances one’s fitness to survive because one can get more energy per unit time [20, 22].

Figure 2.

Internal reward delay can be shortened. A: Motivation or skill shortens internal reward delay by shortening own behavior. B: Two kinds of skill that shorten reward delay.

Motivation and skill reduce the duration of behavior

The speed of behavior (and consequently its value) can be increased temporarily by motivation or permanently by skill (Fig. 2A). Motivation is initiated quickly if a reward is expected, but is dissolved quickly if no reward is expected. Its major advantage is flexibility. Skill is acquired slowly by repeating a behavior, but is maintained for a long time. Its major advantage is stability.

Which one is more effective in reducing reward delay? The duration of a behavior (e.g., saccadic eye movement) is shorter if a reward is expected than if no reward is expected after the behavior [23, 24]. This is due to motivation. A similar but larger bias occurs to reaction time (the time until the behavior starts). However, the difference is usually less than twofold [25–27]. The effect of skill is often much larger. If the duration of a behavior is compared between the initial (novice) stage and the late (skill) stage, the shortening could be nearly tenfold [1, 28]. Therefore, in the long run, skill can reduce reward delay more robustly than motivation. Other features of skill enable individuals to reach a goal easily (by automaticity) and securely (by stereotypy and stability). Therefore, we will focus mainly on skill in the following discussion.

Two kinds of skill

Our main message above was “reward delay can be shortened by accelerating own behavior.” But what exactly does ‘behavior’ mean? Human hunters as well as monkeys spend time in capturing an animal as their food resource [3, 29]. But before that, they need to find the animal using various sensory cues. By repeated experience, the humans and monkeys become proficient in both finding good prey animals and capturing them. Thus, the modifiable reward delay consists of two components: 1) the delay before finding a good object and 2) the delay before reaching a reward by acting on the object (Fig. 2B). If the goal of learning is to maximize reward gain [30], both of these delays need to be shortened. Indeed, the delays become a lot shorter by learning. These two types of skill could be called “object skill” and “action skill.”

A similar distinction has been proposed in foraging research [17, 31]. While foraging, animals spend time in two steps - ‘searching’ and ‘handling’ food – before consuming it. Since these two kinds of foraging time are usually spent sequentially, any attempt to reduce each of these times should enhance fitness for survival.

Note, however, that abilities equivalent to object skill and action skill have been studied extensively in the laboratory setting. The acquisition of object skill may depend on attention, perceptual learning, and episodic memory [32–37]. The acquisition of action skill may depend on motivation, procedural learning, and motor learning [10, 38–40]. We feel, however, that these mental abilities have rarely been studied in the integrative manner with a common functional basis, as recent opinion articles argue [41, 42]. This is particularly prominent among abilities supporting object skill. For both humans and animals, finding good objects can be the most challenging and time-consuming process [3, 29]. We thus will focus on object skill in the rest of this article.

Object skill – stable values

In experiments on attention and perceptual learning, the subjects are usually presented with stimuli or objects and are asked to mentally choose one of them. In everyday life, however, there is often no such instruction. We choose a particular object because, by doing so, we have accomplished something rewarding, probably multiple times. We avoid an object because the outcome was less rewarding or punishing. In other words, the proficiency of finding good objects is the result of value-based learning [41, 42].

What is critical about the development of object skill is consistency: an object needs to be associated with a high value consistently. This has been shown repeatedly in various psychological experiments [32, 43, 44]. Actually, such a situation is not unusual in laboratory experiments as well as everyday life. As a result, people may develop amazing abilities to find important objects or their changes [2, 32, 45, 46]. There are many similar situations in everyday life for humans and animals. For example, the visual appearance of a favorable food may not change during one’s entire life. Its value has been high for many years and will remain high for many years. The object has a stable high value.

Object skill – gaze and attention

There are several ways to find a good object. Animals may use different sensory modalities, visual, auditory, somatosensory, olfactory, electromagnetic etc. [47–49]. For primates including humans, visual information is a dominant modality [50, 51]. Indeed, studies on humans and monkeys have suggested that gaze and/or attention is drawn to an object that has a high value [52–58], which is often followed by a manual action performed on the object [59–62]. The values were attached to the objects during previous experiences with the objects. Importantly, such a gaze/attention bias may occur regardless of the subject’s current intention [55, 58]. This implies that the gaze/attention bias can be automatic. This is distractive in the volatile condition where the values of objects change frequently. In the more stable condition, however, the automatic gaze/attention bias is critical for survival, as we will discuss below.

Hikosaka and colleagues have conducted a series of behavioral and physiological experiments using macaque monkeys to address questions about object skill using the value-based gaze bias as a behavioral measure [63–65]. Unlike other mammals and like humans, macaque monkeys have well developed visual cortices, especially the temporal cortex where information on visual objects are processed [50]. They make saccadic eye movements vigorously to orient their gaze to surrounding objects [66, 67]. Such saccades are faster than in humans [67, 68].

The hypothesis underlying this series of experiment was: Object skill develops if the subject experiences objects repeatedly with stable values. The monkeys in these experiments experienced many fractal objects, each with a large or small (or no) reward (Fig. 3). In consequence, half of the objects became high-valued, and the other half low-valued. The monkeys’ eye movements were tested using a free viewing condition, as exemplified in Fig. 4. After several daily learning sessions, the monkey started showing a gaze bias: more frequent saccades to high-valued objects and longer gaze on them [63, 65]. The gaze bias was considered automatic because it occurred reliably and repeatedly even though no reward was ever given after a saccade to any object during free viewing. This is consistent with the data obtained in human subjects described above.

Figure 3.

Fractal objects that one monkey experienced for a long time with biased rewards (n=288). In each row of eight fractals, the left four fractals were high-valued objects (consistently associated with a large reward) and the right four fractals were low-valued objects (consistently associated with a small reward). Reproduced from [65].

Figure 4.

Object skill expressed as automatic gaze bias. A: Procedure for learning stable values of visual objects. Half of the objects were associated with a reward (high-valued objects) and the other associated with no (or a small) reward (low-valued objects). B: Free viewing procedure for testing the value-based gaze bias. On each trial, 4 fractal objects were chosen pseudo-randomly from a set of 8 learned objects (A), were presented simultaneously, and the monkey freely looked at them, but no reward was delivered. Shown here are examples of saccade trajectories. The monkey tended to look at high-valued objects (denoted as ‘H’). (C, D) The percentage of the gaze duration on each object before (C) and after (D) the long-term learning. Red and blue indicated high- and low-valued objects. Reproduced from [63].

Object skill – automaticity

A skill is characterized by automaticity [2, 7, 69]. When human observers have extensively learned to find particular visual objects, they become unaware of their own perceptual distinction or memory retrieval [32, 33, 70]. The automatic nature of object skill is important for the following reasons. Suppose you have experienced all of the fractals in Fig. 3, half of them with a reward and the other half with no reward, many times. Suppose, then, some of these objects are placed in front of you, as in Fig. 4B.

Can you choose the objects with high values and avoid the objects with low values? In fact, this is similar to what the monkeys did. All of the objects shown in Fig. 3 are what one of the monkeys experienced with high and low values many times, and the monkeys’ choice was tested using the free viewing procedure shown in Fig. 4. For any combination of high- and low-valued objects, the monkeys showed a gaze bias similar to that shown in Fig. 4C and D [63, 65].

Such proficient performance would not be possible if humans or monkeys rely on voluntary (not automatic) search. We would have to rely on search images of desired objects in working memory [71]. Note that we cannot hold many items (say, more than seven) in working memory [72]. If we have experienced many objects with different values (as in Fig. 3), it is unlikely that the objects we encounter (as in Fig. 4B) match the search images held in working memory. Then, we cannot choose high-valued objects right away. We have to retrieve a memory associated with each object and judge its value based on the memory. And we have to repeat this memory retrieval-judgment process for all objects we encounter. This is a serial process and hence time-consuming [73, 74]. As we discussed at the beginning, the amount of reward we get per unit time would be very small.

What monkeys experienced in the laboratory (Fig. 3 and 4) is probably similar to what they would experience in the wild. A monkey would experience many objects as they forage foods and as they interact with animated and non-animated objects. On any day, the monkey may encounter some of these objects, but often unexpectedly. The situation may be magnified in human lives. For example, when we read newspapers, there are many pieces of information but we are quickly drawn to important ones even if we have not expected them.

These considerations suggest that the amazing ability of choosing high-valued objects among many others (i.e., object skill) depends on an automatic mechanism which is different from the mechanisms underlying voluntary choice. Neurophysiological experiments have suggested that the neural mechanism of object skill is, at least partly, located in a restricted part of the basal ganglia (the tail of the caudate nucleus and the substantia nigra pars reticulata) [63–65] (Box 1). This is consistent with a long-standing theory that the medial temporal lobe (including hippocampal areas) is essential for declarative memory whereas the basal ganglia are essential for non-declarative memory, specifically skills or habits [75]. Indeed, it was shown that the tail of the caudate nucleus, which receives inputs from temporal visual cortical areas [76], is necessary for learning to choose high-valued objects [77].

Box 1: Neural mechanism of object skill.

Experiments by Hikosaka and colleagues have suggested that object skills are supported by a neural circuit in the basal ganglia, namely, the connection from the tail of the caudate nucleus (CDt) to the superior colliculus (SC) through the substantia nigra pars reticulata (SNr) (Figure). The CDt-SNr-SC circuit represents the characteristics described in the main text, as shown below.

Stable values: Both CDt and SNr neurons encoded stable values of visual objects. During learning, monkeys looked at many fractal objects repeatedly and consistently in association with high- or low-reward values (see Fig. 3 and 4A). After short-term learning CDt and SNr neurons showed little value-based preference. However, after learning across several days, CDt and SNr neurons started showing differential responses to the high- and low-valued objects. CDt neurons are typically excited more strongly by high-valued objects [63]. SNr neurons are typically inhibited by high-valued objects and excited by low-valued objects [65].

Gaze: Many of the SNr neurons were shown to project their axons to the SC [65] which is the key structure for initiating saccadic eye movements [93]. Weak electrical stimulation in the CDt induced saccades [64]. The CDt-induced saccades are likely caused by the disinhibition of SC neurons, since both the direct CDt-SNr connection and the SNr-SC connection are GABAergic and inhibitory [94]. The CDt-SNr-SC oculomotor circuit is necessary for object skill because reversible inactivation of the CDt with a GABA agonist (muscimol) eliminated the gaze bias during free viewing (Fig. 4), but only for objects presented on the side contralateral to the inactivation [86].

Automaticity: Both CDt and SNr neurons responded to high- and low-valued objects differentially even when no reward was delivered after the object presentation. The differential responses occurred either when a saccade was prohibited [65] or allowed [86]. During the free viewing procedure (Fig. 4), CDt neurons were excited before saccades, and did so more strongly when the saccades were directed to objects with their preferred values [86].

High capacity memory: After the monkeys have experienced many objects (up to 300 for each monkey) with biased values, both CDt and SNr neurons reliably classified them based on their stable values [65].

Long-term retention: The neuronal bias based on stable values remained intact even after >100 days of no training, even though the monkey continued to learn many other objects. This was shown systematically for SNr neurons [65].

Object skill – high capacity memory

It is now clear that object skill requires high capacity memory. The association of an object and its stable value makes up a single memory. The number of memories should correspond to the number of learned objects. This is the minimum requirement, however. If a combination of objects has a different value (which is likely), the number of memories would increase drastically.

Moreover, the object-value memory must have a robust access to motor outputs, so that we can physically choose high-valued objects. Importantly, this memory-motor connection must be established for each object. For primates (humans and monkeys), eye movement would be the primary motor output for choice, because a gaze choice precedes or guides a manual choice, as we discussed earlier. Saccadic eye movements during free viewing (Fig. 4) would reflect the robust connection between value memory and eye movement.

In other words, the automatic nature of object skill is enabled only after establishing the memory-motor connections for individual objects, robustly and selectively. This ascertains two important aspects of skill: accuracy and speed. Only if the memory-motor connection is unique to individual objects, can the choice be accurate. Only if the object-value information is quickly transmitted through the memory-motor connection, can the choice be made quickly.

This feature – memory-motor connection – is similar to another popular concept – stimulus-response association – which characterizes skill or habit [78]. In the framework of operant conditioning, a particular response or stimulus-response association is reinforced [78, 79]; ‘stimulus’ itself is assigned no particular value. This may apply to action skill in which the right ‘response’ needs to be chosen and quickened. For object skill, however, it is possible that the representation of ‘stimulus’ is reinforced. According to this scheme, the brain first creates a memory of stimulus-value association, based on which approach or avoid responses are recruited. The neurophysiological data (Box 1) are consistent with this scheme [63, 65]. We thus hypothesize that object skill requires high capacity memory for individual stimuli (i.e., objects), whereas action skill requires high capacity memory for individual responses (i.e., motor actions).

One feature may be missing in the above argument: spatial position. Each object may occupy a particular position and thus the action performed on the object may have a corresponding spatial feature (e.g., saccade with a particular vector). Although it is unclear whether the object-value memory contains spatial information, the neurophysiological data (Box 1) suggest that the basal ganglia system supporting object skill carries spatial information.

Object skill – long-term retention

Human studies on the value-based gaze/attention bias have shown that, once the subjects have acquired a bias, the bias is retained for several days [57, 80]. Studies using monkeys showed that the value-based gaze bias remains robust for more than 100 days [65]. In this specific study [67], the monkeys were not shown some of the value-associated objects for a long time, and then their choices were suddenly tested using free viewing. The monkeys looked at the previously high-valued objects preferentially more often than the previously low-valued objects. This was non-trivial because, during the retention period, the monkeys continued to experience many other fractals, several of which were very similar to the retained fractals. This means that the stable object-value memory is highly object-selective and is hardly interfered with by more recently experienced objects.

Conceptually, this endurance of the object-value memory is necessary for its high capacity. High capacity memory is created only if each memory is retained for a long time. Otherwise, as new memories are acquired, old memories are lost; then, memories would not accumulate. On the other hand, the stable object-value memory is not created immediately. It took about 5 daily learning sessions before the monkeys started preferentially looking at objects that have been associated with high values [65]. To summarize, the stable object-value memory as well as its associated object skill is created slowly, but once established it remains for a long time. This is what characterizes ‘skill’ [40, 81].

Object skill – Limitations

We have emphasized mainly the advantage of skill, especially object skill. However, relying on object skill completely is risky because of its automatic nature. Object skill is basically blind to recent changes in the values of individual objects. If a previously high-valued object becomes toxic, we may not be able to stop reaching for it [53]. This poses a serious problem because motor behavior triggered by object skill (e.g., gaze orienting and reaching) occurs quickly and automatically in response to the appearance of high-values objects. A similar situation happens during drug abuse [82].

A reasonable way to cope with this problem would be to have another system that is sensitive to recent changes in objects’ values and, based on the updated information, change its outputs. This is how voluntary search would work. As we indicated, however, voluntary search itself is slow and capacity-limited. Therefore, the brain should have two systems, one working for voluntary search and the other for automatic search (object skill), as several studies suggest [38, 83–90].

Conclusion

An ultimate advantage of skilled behavior is multi-tasking [91]. Because skills are automatic, more than one can be performed in parallel (including a combination of object and action skills). Object skill enables humans and animals to evaluate objects in any environment and choose a high-valued object quickly and automatically. The choice then allows humans and animals to act on the chosen object quickly so that rewarding outcome can be presented as soon as possible. When one is writing a manuscript, one’s gaze and fingers are moving automatically. The automaticity frees up one’s cognitive resources so that one can focus on the conceptual arrangement of the manuscript. This means that cognitive functions heavily rely on one’s ability to utilize skills [92].

Figure Box 1.

Basal ganglia circuit that supports object skill. Neurons in the monkey CDt receives inputs from the temporal visual cortices and respond to visual objects differentially. Their responses are modulated by the stable (not flexible) values of the visual objects. Neurons in the SNr, which receive inputs from the CDt directly or indirectly, categorize visual objects into high- and low-valued objects. Their stable value signals are then sent to the SC, thereby biasing gaze toward high-valued objects. Arrows indicate excitatory connections (or effects). Lines with circular dots indicate inhibitory connections. Solid and hatched lines indicate direct and indirect connections, respectively.

Box 2. Questions for future research.

In addition to reward values, there are several important factors that influence skill acquisition or are influenced by skills, which need to be analyzed in future research. They include cost, uncertainty, and salience.

Cost

The value of a reward is reduced if the cost for obtaining the reward is high [17, 95]. In this sense, skill is beneficial since the metabolic energy cost decreases as a skill is acquired [96]. More important but less well known is whether or how much the energy spent in the brain decreases with skill acquisition. Activations of cortical and striatal areas (measured in fMRI experiments) tend to decrease with practice [11, 97] except for the early visual cortical areas [98], but it is unclear whether this reflects the general decrease in brain energy expenditure.

Uncertainty

There are two aspects of uncertainty that are relevant to skills, but in different ways. 1) Uncertainty in the environment: Reward outcome is often uncertain. Reward uncertainty, which is detected as reward prediction error [99], may drive learning [41, 100, 101], and consequently facilitate skill acquisition. On the other hand, if there is another option that provides certain information or reward, one may choose the certain option [102, 103], thereby suppressing learning and skill acquisition. 2) Uncertainty of own behavior: Behavior is intrinsically variable [104, 105], which may lead to a random delay of reward, or even no reward [106]. However, the variability is reduced by skill acquisition [107, 108]. This may act as an incentive for establishing skills.

Salience

In addition to something rewarding, humans and animals may need to efficiently deal with something harmful. If a potentially harmful object appears, one would first look at it [109]. One then needs to escape or fight with the object [110]. It would be useful to develop skills to detect such harmful objects quickly (object skill) and escape or fight quickly (action skill). In a broad sense, one needs to find and act on both something rewarding and something harmful. Little is known about whether the brain processes these two kinds of saliency in the same or a different manner in order to develop skills [111].

Highlights.

Skill makes a behavior quicker and thus increases rewards per unit time.

Skill consists of object skill (find good objects) and action skill (act on them).

Object skill is acquired through repeated experiences of objects with stable values.

Object skill allows automatic and quick choices of good objects among many.

Object skill compensates for the limited capacity of voluntary search.