Systematic Exploration and Uncertainty Dominate Young Children’s Choices

Abstract

Organisms need to constantly balance the competing demands of gathering information and using previously acquired information to obtain rewarding outcomes (i.e., the “exploration-exploitation” dilemma). Exploration is critical to obtain information to discover how the world works, which should be particularly important for young children. While studies have shown that young children explore in response to surprising events, little is known about how they balance exploration and exploitation across multiple decisions or about how this process changes with development. In this study we compare decision-making patterns of children and adults and evaluate the relative influences of reward-seeking, random exploration, and systematic switching (which approximates uncertainty-directed exploration). In a second experiment we directly test the effect of uncertainty on children’s choices. Influential models of decision-making generally describe systematic exploration as a computationally refined capacity that relies on top-down cognitive control. We demonstrate that (1) systematic patterns dominate young children’s behavior (facilitating exploration), despite protracted development of cognitive control, and (2) that uncertainty plays a major, but complicated, role in determining children’s choices. We conclude that while young children’s immature top-down control should hinder adult-like systematic exploration, other mechanisms may pick up the slack, facilitating broad information gathering in a systematic fashion to build a foundation of knowledge for use later in life.

Exploration is a critical activity in which organisms seek information that they can use to make effective and rewarding decisions. Although exploration is often advantageous, it is not cost free. The “exploration-exploitation” dilemma (see Hills, et al., 2015 & Mehlhorn, el al., 2015 for reviews) is a tradeoff: gather information and forgo rewards or forgo acquiring information and use existing knowledge to get rewards. Effective decision-making requires appropriately balancing exploration and exploitation, with the ideal balance depending on the environment and the knowledge state of the individual. Higher levels of exploration are optimal when there is greater uncertainty (i.e., when one knows little about the environment, or when the environment is variable or dynamic), and therefore exploration should be especially valuable for young children who often have little knowledge of how the world works.

Indeed, a number of studies support the idea that young children engage in exploration in response to surprising or ambiguous events (Bonawitz, van Schijndel, Friel, & Schulz, 2012; Cook, Goodman, & Schulz, 2011; Schulz & Bonawitz, 2007). Even infants show evidence of exploration, approaching or interacting longer with objects that violated expectations (Sim & Xu, 2017; Stahl & Feigenson, 2015).

Their importance notwithstanding, most previous studies examine exploration as preferences for things that are less known or surprising, while leaving some other important issues unexplored. In particular, little is known about how children mediate the competing demands of exploration and exploitation when faced with a series of decisions. Single choices are typically exploratory or exploitative, but the relative balance achieved between these two types of choices, and the mechanisms supporting that balance, can only be observed across many decisions. By examining children’s patterns of choices across multiple decisions, and comparing them to adults’, we can begin to examine questions such as the relative balance of exploration and exploitation across development, the types of exploration strategies available to young children, and the ways exploration-exploitation strategies change with development.

Theories suggest that children should explore more than adults, or at least more broadly (Gopnik et al., 2017), but this prediction has only very recently received empirical support (Schulz, Wu, Ruggeri, & Meder, 2019; Sumner, Steyvers, & Sarnecka, 2019). Furthermore, this expectation is not uncontroversial: Influential models of decision-making describe exploration as a computationally refined capacity, requiring top-down regulation (Badre, Doll, Long, & Frank, 2012; Daw, et al. 2006; Frank, Doll, Oas-Terpstra, 2009). In addition, mounting evidence points to an important role of executive control process mediated by prefrontal cortex (PFC) in systematic exploration (Blanco, et al., 2015; Badre, et al., 2012; Frank, et al., 2009; Otto, Knox, Markman, & Love, 2014). Ideally, exploration should be systematically directed toward parts of the environment that have greater uncertainty. As things change over time, uncertainty increases in areas that have less recently been checked. This type of directed exploration leads to sequential structure in choice patterns (Knox et al, 2012; Blanco et al., 2015). Given the protracted development of top-down cognitive control and PFC (Bunge et al., 2002; Casey, Giedd, & Thomas, 2000; Sowell, et al. 1999), current theories predict that young children’s exploration should be largely unsystematic, and there is evidence that directed exploration may not emerge until adolescence (Somerville, et al., 2017). Similarly, development is largely conceptualized as a decrease in the stochasticity (or noisiness) of behavior over time. For example, in Gopnik et al. (2017) development is likened to decreasing “temperature” (moving from broader to narrower sampling) in a random sampling process.

In what follows, we present two experiments in which children and adults make a series of choices, and we evaluate the strategies driving their sequences of actions. Our results indicate that early in development exploration is predominant and largely systematic, and that uncertainty plays an important, but complicated, role. We then discuss how such systematic exploration could be implemented in the absence of mature top-down control.

EXPERIMENT 1

Method

Participants

Experiment 1 included thirty-two 4-year-olds (mean age = 54.8 months, 15 girls) and 34 adults (19 women). Four-year-old children were selected because at his age children have known immaturities in selective attention and cognitive control (Bunge et al., 2002; Casey, Giedd, & Thomas, 2000), while being able to understand the task and make independent decisions. Sample-size were chosen to be consistent with previous studies comparing children of this age to adults in cognitive tasks (e.g. Deng & Sloutsky, 2015; 2016). Children were recruited from preschools and childcare centers on the basis of returned parental permission forms. Adults were undergraduate students participating for course credit.

Stimuli and Procedure

The task was a simplified n-armed bandit task framed as a game in which participants collected virtual candy from alien creatures (Figure 1). Four choice options were each marked by a different creature. Each creature gave a set number of candies that remained the same on every trial: 1, 2, 3, and 10 candies, respectively. These values were chosen so that one option was clearly superior to evaluate how often participants exploit it compared to exploring other options. The locations of the creatures and reward values were fixed across the experiment, but were randomly determined for each participant. Note that while reward values were stable, participants were not informed of this fact, and so they should still have subjective uncertainty about the outcomes of their choices and should need to explore to some extent. But, because adults learn and exploit rewarding options even in situations with noisy or changing rewards (e.g. Blanco et al. 2015), they should primarily exploit the best option, with little exploration.

Figure 1.

Trial dynamics. (A) Participants choose one of four creatures. (B) They receive virtual candy for their choice. (C) That candy is added to the meter tracking total accumulated candy, and the next trial begins. Benchmarks on the meter (every 180 candies) indicate the number of stickers earned. In Experiment 2, the reward for one option was hidden behind a door while the other three were visible prior to the choice.

The goal of the game was to earn as much candy as possible. On each of 100 trials, children chose one of the four creatures using a touch screen (adults used a computer mouse). The reward amount was then displayed (3 s), and a meter tracking total accumulated reward was updated. Children earned stickers for each 180 candies, with benchmarks on the meter indicating these goals. Adults did not earn stickers.

Following the main experiment, a subset (N=22) of child participants completed a follow-up memory test¹. The experimenter pointed to each of the four locations in turn and asked the child how many candies they would get if they chose that option.

The primary planned behavioral analyses were to examine (1) the proportion of the highest-value option choices over the course of the experiment as an indicator of exploitation, and (2) the proportion of trials that participants switched between different options as an indicator of exploratory tendencies. Computational modeling analyses were planned to evaluate the strategies that drive children’s and adult’s choices.

Computational Modeling

To investigate participants’ exploration strategies, we developed a Reinforcement Learning (RL) model (Sutton & Barto, 1998) that included both systematic and random exploration. Critically, the choice probabilities were a combination of the expected values of the options and their choice lags. Choice lag was simply the number of trials since an option was last chosen. Choice lag serves as a proxy for uncertainty, because uncertainty increases as a function of time since an option was last checked. Although reward values are stable, because participants are unaware of this fact, their subjective uncertainty should still follow this pattern.

The model learns reward values by updating expected values for each option based on prediction error using the following equation:

$$V_{i,t+1}=\mathrm{ }{V}_{i,t}+\mathrm{ }\alpha \left(R_{i,t}-\mathrm{ }{V}_{i,t}\right)$$ (1)

where V_i,t is the expected value of option i on trial t, R_i,t is the reward on trial t earned for choosing option i, and is the learning rate (a free parameter). Only the chosen option is updated on each trial. The model is initialized with all V_i set to 0. Choice probabilities were determined by the following function:

$$P\left(a_{i,t}\right)=\mathrm{ }\frac{e^{{\beta \mathrm{*}[V}_{i,t}\mathrm{*}\left(1-\varphi \right)+L_{i,t}\mathrm{*}\varphi ]}}{\sum _{j=1}^n{e}^{{\beta \mathrm{*}[V}_{j,t}\mathrm{*}\left(1-\varphi \right)+L_{j,t}\mathrm{*}\varphi ]}}$$ (2)

where P(a_i,t) is the probability of choosing option i on trial t. L_i,t is the lag term encoding the number of trials since option i was last chosen. Two free parameters, ϕ and β, determine the levels of exploitation, systematic exploration, and random exploration. Specifically, ϕ (0 ≤ ϕ ≤ 1) mediated the relative weights of expected value (i.e., exploitation) and lag (i.e., systematic exploration) on choices. Higher values of ϕ indicate greater influence of the lag, and hence more systematic exploration. When ϕ is 0, the model chooses based only on expected value; when ϕ is 1 it chooses only based on the lag.

β was the inverse temperature parameter that controlled the extent that choices were deterministic or stochastic (Sutton & Barto, 1998). Lower values of β capture more “random” choices (i.e., random exploration) and greater values of β captured greater consistency of choices (regardless of whether they are driven by value or by lag). This model is similar to ‘exploration bonus’ models (Daw et al., 2006; Kakade & Dayan, 2002), but with lag serving as a proxy for uncertainty. The model was fit to each participant by finding the set of parameters that maximized the likelihood of the participant’s data given the model. Note that this model is used primarily as an analysis tool to disentangle factors contributing to participants’ decision-making strategies and should not be taken as a direct model of the cognitive processes involved.

Results

Participants’ choices across the experiment are presented in Figure 2A. Adults quickly learned the best option and exploited it heavily (86.3% of trials), with only occasional exploration of other options. Children exploited the best option much less than adults (42.9% of trials), t(64) = 10.05, p < 0.001, 95% CI: 0.348 ≤ (μ₁ - μ₂) ≤ 0.520, d = 2.48. Even within the first 20 trials (Figure 2B), when adults’ exploration should be highest, they exploited the best option more than children (74.3% vs. 30.6%), t(64) = 13.11, p < 0.001, 95% CI: 0.370 ≤ (μ₁ - μ₂) ≤ 0.501, d = 3.23.

Figure 2.

Choice proportions over time. (A & C) Adults exploited the most valuable option on a majority of trials, while children chose it substantially less often in both Experiments 1 and 2. (B) Analysis of the first 20 trials in Experiment 1 indicate that adults shift to high levels of exploitation after a very short period of exploring all options. (D) Adults chose the hidden option in Experiment 2 at approximately chance levels, while children tend to either highly prefer or to avoid the hidden option. (E) Adults tend to only choose the hidden option when it was the highest value option, and largely avoid it otherwise, while most children’s choices are unaffected by whether the hidden option was the highest value or not. Error bars reflect standard errors of the mean.

Interestingly, there were striking individual differences between children (Figure 3). While a minority of children were adult-like (i.e. greatly preferring the highest-value option), most chose all options approximately equally often. Using a lenient criterion of exploiting the best option on > 50% of trials, only 11 out of 32 of children made “adult-like” choices.

Figure 3.

Individual participants’ choices in Experiment 1. Children exhibited striking individual differences in their choices. While some children chose similarly to adults, the majority of children chose all options with approximately equal proportions. Few children were between these two extremes. Almost all adults chose the highest-value option on a large majority of trials.

Memory test analyses confirmed that most children learned the reward values, despite not exploiting the most valuable option. Of the 22 children who completed the memory test, only 2 misidentified the best option. Eighteen correctly recalled the best option and its value, and 2 recalled the best option but misremembered its exact value. Mean accuracy across all memory items was 77.2%. This finding also parallels a recent study in which, despite children accurately indicating the best option following a probabilistic learning task, they were less likely than adults to maximize their choices toward that option (Plate et al., 2018). Together these findings suggest that children’s choices are not motivated by achieving maximum reward to the extent that adults’ are.

Most importantly, children’s choices were far from random. In contrast to adults (who were well below chance), many children switched their responses more frequently than would be expected by chance (Figure 4). These patterns of choices indicate heightened levels of exploration in children, seemingly even at the expense of forgoing exploiting a valuable option. Children’s tendency to explore systematically is supported by modeling results presented in the next section.

Figure 4.

Response switching. The probability of response switching was much higher for children than adults. Dots represent individual participants, and the shaded region indicates the 95% probability density of switch proportions given random responding. The majority of children (17/32) were well above this region, indicating non-random responding and elevated exploration. In Experiment 2 children’s switching was consistent with chance since their choices were primarily determined by which option was hidden on each trial. See also Figure 6 for a breakdown of children’s switching behavior by best-fitting model for Experiment 1.

Modeling Analyses

In order to validate inclusion of systematic exploration in the model, we compared it to a reduced model without systematic exploration (i.e. with ϕ fixed at 0) using the Akaike Information Criterion (Akaike, 1974). The full model better fit 23 of 32 children (total AIC_full = 6391 vs. total AIC_ϕ=0 = 7466) and 21 of 34 adults (total AIC_full = 3058 vs. total AIC_ϕ=0 = 3316), suggesting most participants engaged in systematic exploration to some extent.

We then analyzed the best-fitting parameter values to examine the relative influences of reward value, random exploration, and systematic exploration on participants’ choices. Best-fitting values of all three free parameters were compared using Wilcoxon rank-sum tests because the parameters were not normally distributed (Figure 5A). Children and adults did not differ in the β parameter that controls the level of random exploration, W = 480, p = 0.418, or in the learning rate α, W = 498, p = 0.562.

Figure 5.

Modeling predictions and results. (A) The best-fitting parameter values of the Full model show that adults and children do not differ in the learning rate (α) or the level of random exploration (β ), but that children show a greater relative influence of systematic exploration (ϕ ) than adults. (B) Parameter estimates from the reduced model (with no systematic exploration) suggest that if systematic exploration is not accounted for children appear to be more random (indicated by lower β ), highlighting the importance of including systematic exploration in understanding children’s choices. Vertical bars represent the medians (solid) and means (dashed). (C) The hazard rates of choices, which is the probability of choosing an option given the amount of time since it was last chosen, demonstrates a critical difference in the two single-process models’ predictions. The Lag model predicts an increasing probability of picking an option the longer it’s been since it was last selected. Predictions were produced from simulations using best-fitting parameters from children best-fit by each model. (D) The proportions of participants best-fit by the Value, Lag, and Spatial uncertainty (Exp. 2) models are plotted. All adults were value-based. About half of children relied on choice lag when all options were uncertain (Exp. 1) and most relied directly on uncertainty when it was concentrated on a single option (Exp. 2), but note that this includes both uncertainty seekers and uncertainty avoiders.

Children did, however, have a higher value than adults for ϕ, W = 751, p = 0.007, indicating a greater influence of systematic exploration on children’s choices compared to adults. Importantly, in the reduced model without systematic exploration, best-fitting β was lower for children than adults, W = 264, p < 0.001—highlighting the importance of accounting for systematic exploration. Without it we may have concluded that children’s choices were simply noisier than adults’. Furthermore, the lag-based choices in children are unlikely to be explained by a mere adherence to a spatial pattern (e.g., 1→2→3→ 4; See supplemental analyses at osf.io/6thfn/ for details.)

Single-process models

Because large individual differences between children suggest the presence of distinct strategies, we fit simplified versions of the model that utilized only value (ϕ = 0) or only choice lag (ϕ = 1) to determine the primary factor determining each participant’s choices. α(i.e., learning rate) was set to 1 for this analysis to better equate the two single-process models. A random model was also considered that chose all options with equal probability regardless of rewards or lags, but no participants were best fit by the random model in Experiment 1. As shown in the Figure 5C, the key difference between the two models is that the Lag model predicts that options become more likely to be chosen the longer it has been since they were last selected, whereas the Value model does not.

As shown in Figure 5D, all adults were best fit by the Value model, whereas about half of children (17/32) were better fit by the model capturing systematic exploration–the Lag model. These proportions were significantly different Γ² (1; N = 66) = 26.30, p < 0.001, Cramer’s V = 0.631. To examine whether these models accurately characterize children’s behavior, we compared children best fit by each model to simulations from the models. The results presented in Figure 6 indicate that these single-process models capture children’s choices quite well, further suggesting that children’s responses are not random but instead follow one of two distinct systematic strategies.

Figure 6.

Children’s strategies. Children in Experiment 1 seemed to exhibit two distinct strategies, well captured by the Value and Lag models, respectively. These strategies differed in a few key ways. Value-based participants (N=15) exploited the best option often, explored little, and otherwise showed no particular pattern in their sequences of choices. Lag-based participants (N=17) chose all options equally often, and switched responses at extremely high rates, almost never picking the same option twice in a row. Darker colors represent higher transition probabilities. The predictions for each model are based on simulations using the best-fitting parameters for children that were best-fit by that model.

Together these results indicate that, whereas adults’ choices were driven by value, many children used a strategy that approximates uncertainty-based exploration. To better understand what role uncertainty might play in children’s choice behavior, we conducted Experiment 2.

EXPERIMENT 2

In Experiment 1 children exhibited systematic patterns of choices that suggest they may be engaging in a form of uncertainty-based exploration, with very little exploitation. However, it remained unclear whether their choices were actually driven by uncertainty, and if so, to what extent. In Experiment 2 we manipulated uncertainty directly by hiding only one option’s reward value while exposing the other options. Since the hidden option’s value is unknown, there is more uncertainty surrounding its outcome than the options with visible rewards. This effectively eliminates temporal (i.e. lag-based) uncertainty, while increasing differences in uncertainty between options. If children’s choices are unaffected by uncertainty, the pattern observed in Experiment 1 (i.e., lag-based switching) should persist. In contrast, if their choices are driven by uncertainty, the pattern observed in Experiment 1 should disappear since there is no lag-based uncertainty, and choices should instead revolve around the hidden option.

Method

Experiment 2 included 36 4- and 5-year-old children (mean age = 58.9 months, 22 girls) and 37 adults (20 women). Procedures were identical to Experiment 1, except that the rewards available for choosing three of the options were visible prior to the choice (Figure 1). The reward for the remaining option was hidden by a picture of a door and was revealed only if that option was chosen. The option that was hidden was randomly determined on each trial.

Results

Like Experiment 1, adults chose the best option on almost every trial (93.9% of trials), significantly more often than children (40.3% of trials), t(71) = 15.37, p < 0.001, 95% CI: 0.467 ≤ (μ₁ - μ₂) ≤ 0.606, d = 3.60 (Figure 2C). On average children chose the hidden option on 39.5% of trials, significantly above chance levels, t(35) = 2.15, p = 0.039, 95% CI: 0.258 ≤ μ ≤ 0.532, d = 0.35 (Figure 2D). Adults chose it significantly less often (21.5% of trials) than children, t(71) = 2.63, p = 0.010, 95% CI: 0.044 ≤ (μ₁ - μ₂) ≤ 0.315, d = 0.62. When the hidden option was the highest-value option, adults chose it on the majority of trials (84.3% of trials), but otherwise they almost never selected it (2.2% of trials; Figure 2E). Children chose the hidden option equally often whether it was best or not (42.8% vs. 37.9%, respectively)—less than adults when it was best, t(71) = 4.81, p < 0.001, 95% CI: 0.243 ≤ (μ₁ - μ₂) ≤ 0.587, d = 1.13, but more than adults otherwise, t(71) = 5.29, p < 0.001, 95% CI: 0.222 ≤ (μ₁ - μ₂) ≤ 0.492, d = 1.24.

Children exhibited large individual differences: Some overwhelmingly selected the hidden option, while others avoided it. Binomial tests on each child’s number of hidden option selections showed that 15 out of 36 children were significantly above chance levels, with 9 children choosing the hidden option on almost every trial (≥ 97%). Another 15 children were below chance levels, with 12 choosing the hidden option on less than 3% of trials.

Modeling Analyses

The single-process models were fit to participants’ data to determine the primary factor driving each participant’s choices. In addition to the Lag, Value, and random models, a Spatial uncertainty model was fit. The latter model considers only whether each option’s reward was exposed or hidden. Its choices were determined using the following equation:

$$P\left(a_{i,t}\right)=\mathrm{ }\frac{e^{{\beta }_u\mathrm{*}{U}_{i,t}}}{\sum _{j=1}^n{e}^{{\beta }_u\mathrm{*}{U}_{j,t}}}$$ (3)

where U_i,t was simply coded as 1 if the reward for option i was hidden and 0 if it was visible. Importantly, β_u could be positive or negative, and so it could capture both uncertainty seekers and avoiders. The learning rate parameter α was set to 1 for the Value model.

Similar to Experiment 1, all adults were best fit by the Value model (Figure 5D). Unlike in Experiment 1, the large majority of children were best fit by the Spatial uncertainty model (N=24), while only one was best-fit by the Lag model. Ten children were best fit by the Value model, and one by the random model. This is in sharp contrast to Experiment 1, where more than half of children were best characterized by the Lag model.

These results suggest that when uncertainty was concentrated on a single option, it was the main factor influencing choices for the majority of children, replacing the systematic exploration pattern (i.e. lag-based responding) seen in Experiment 1. That the systematic pattern disappeared when there was no lag-based uncertainty suggests that children’s choice patterns in Experiment 1 were driven at least partially by uncertainty. At the same time, many children were affected by uncertainty in an unexpected way – they avoided the hidden option altogether – something that may require further research.

GENERAL DISCUSSION

Experiment 1 presents novel evidence of heightened exploration early in development, suggesting that children tend to explore more than adults even in situations when they may benefit from exploitation, and that their choices follow systematic patterns which approximate uncertainty-directed exploration. Even in a highly simplified task, where everything was stable with no variability in choice outcomes (which normally results in very little exploration), children explored at extreme levels. These surprising results demonstrate how much there is still to learn about young children’s decision process.

Importantly, when uncertainty was concentrated spatially in Experiment 2 (with no temporal uncertainty) the sequential patterns that dominated children’s strategies in Experiment 1 disappeared. This further suggests that children’s choice patterns in Experiment 1 were motivated by uncertainty-based exploration. Interestingly, though, when uncertainty was concentrated on one option, some children were drawn strongly toward it, whereas others avoided it. This suggests a complex role of uncertainty in children’s choices that warrants further investigation. One possibility is that preference for uncertainty follows an inverted U-shape favoring some intermediate level of uncertainty (and avoidance of higher uncertainty), but that the ideal level varies between children. Another possibility is that being visible increases the appeal of the rewards just enough to outweigh the lure of the uncertain option for some children. An additional (but not mutually exclusive) possibility is that children are not directly motivated to reduce uncertainty, but that various simpler processes (e.g. novelty preference) are at work, together producing broad, but systematic, sampling of the environment. Consistent with this idea, recent work shows that children continue searching after it is unnecessary for their current goals and additional information will not help complete their task (Ruggeri, Lombrozo, Griffiths, & Xu, 2016), suggesting that their exploration/search tendencies are driven by factors beyond task performance or reward seeking. Future work is needed to further clarify the roles and interactions of uncertainty and other mechanisms in motivating children’s choices.

The reported results encourage a reassessment of the role of top-down control in systematic exploration. Some type of systematic exploration clearly dominates choices early in development when control processes are immature. While young children’s immature top-down control should hinder adult-like systematic exploration, other mechanisms seem to pick up some of the slack, facilitating broad information gathering in a systematic fashion. How is this systematic exploration achieved?

We suggest that one particularly promising candidate may be children’s immature attention allocation. Growing evidence suggests that (unless a highly salient stimulus captures young children’s attention, e.g., Robinson & Sloutsky, 2010; Sloutsky & Napolitano, 2003), young children tend to distribute attention broadly, even when focused attention is warranted (Coch, Sanders, & Neville, 2005; Deng & Sloutsky, 2016; Plebanek & Sloutsky, 2017, 2018). Consistent with research suggesting a tight link between attention and decision-making (Gottlieb, 2012; Konovalov & Krajbich, 2016; Smith & Krajbich, 2018), and more specifically between attending selectively to the reward predictor and exploitation (Leong, Radulescu, Daniel, DeWoskin, & Niv, 2017), we posit that young children’s tendency to distribute attention broadly is an important mechanism supporting exploration. For example, it may enable systematic exploration through graded novelty preference (the longer an option was not selected, the more it attracts attention).

Exploration seems to be a major driving force during early childhood—even outweighing the importance of immediate rewards. Starting with broad exploration and progressing to narrower search and increasing exploitation is an effective strategy across multiple domains (e.g. machine learning algorithms or resource foraging), and human development may be no exception (Gopnik, et al. 2017). Put simply, elevated exploratory behavior is advantageous for learning (but not necessarily for performance) as it affords a broad sampling of the environment, which is useful when one knows little. This developmental progression has previously been conceptualized as analogous to simulated annealing (Gopnik, et al. 2017), decreasing temperature over time in a random sampling algorithm. While this idea is generally consistent with our findings, the specific analogy may be somewhat misleading since children’s choices do not conform to random sampling. Further, the temperature parameter in our model did not reliably differ between adults and children.

A salient aspect of the reported results is that there were large individual differences between children. An important avenue of future research will be to examine why some children adopt “adult-like” reward-based strategies and others do not. Some possible factors worth investigating include early academic instruction and differences in maturation of the brain areas involved in decision-making.

One challenge in this research is achieving equal motivation between different age groups. That the prospect of winning stickers may make children more motivated by rewards in the task than adults is a potential limitation of the current studies. It is important to note, however, that this difference alone cannot account for the reported results. In fact, greater motivation should result in greater focus on rewards and higher levels of exploitation, which is the opposite of what we find.

In sum, this research demonstrates that systematic exploratory behavior can stem from other mechanisms than top-down cognitive control, and that these mechanisms may be specifically tuned to achieve broad information gathering in young children. If this is the case, some of the early immaturities of top-down control (e.g., distributed attention) may be advantageous for children: these immaturities may optimize learning (rather than performance) by sub-serving broad information gathering.

Highlights:

• Little is known about how children balance exploration and exploitation across multiple decisions, or how this process changes over development.

• Exploration should be especially useful for young children, who need information to understand how the world works.

• In two reported experiments, adults and 4-year-olds perform a decision-making task in which different options are worth different amounts of reward.

• While adults maximized reward, children exhibited heightened levels of exploration and were characterized by sequential patterns in their choices that approximate uncertainty-based exploration.