Attentional mechanisms drive systematic exploration in young children

Abstract

Exploration is critical for discovering how the world works. Exploration should be particularly valuable for young children, who have little knowledge about the world. Theories of decision-making describe systematic exploration as being primarily driven by top-down cognitive control, which is immature in young children. Recent research suggests that a type of systematic exploration predominates in young children’s choices, despite immature control, suggesting that it may be driven by different mechanisms. We hypothesize that young children’s tendency to distribute attention widely promotes elevated exploration, and that interrupting distributed attention allocation through bottom-up attentional capture would also disrupt systematic exploration. We test this hypothesis by manipulating saliency of the options in a simple choice task. Saliency disrupted systematic exploration, thus indicating that attentional mechanisms may drive children’s systematic exploratory behavior. We suggest that both may be part of a larger tendency toward broad information gathering in young children.

Open Practices Statement

The data reported in this paper is archived and available on Open Science Framework at https://osf.io/ph9kz/.

Cognition changes dramatically in the course of development. Many of these changes stem from developmental changes in allocation and control of attention. Adults are adept at controlling their attention: depending on their goals, they can distribute it broadly or focus selectively on a small subset of stimuli (e.g., Chong & Treisman, 2005). When only some of the available information is relevant, adults tend to selectively focus on that information and ignore the rest (Rehder & Hoffman, 2005; Blair, Watson & Meier, 2009).

In contrast, young children tend to distribute their attention broadly, regardless of task demands, often processing both task-relevant and task-irrelevant information (Deng & Sloutsky, 2015, 2016; Plebanek & Sloutsky, 2017; Smith & Kemler, 1977). This tendency likely stems from immaturities of executive attention (Posner & Rothbart, 2007), resulting in difficulty attending selectively and filtering out less relevant input.

While such immaturities of executive attention may be limiting for learning in academic settings, it is possible that they can be adaptive (Chrysikou, Weber, & Thompson-Schill, 2013; Gopnik, 2010). For example, distributing attention can result in superior performance of children over adults in situations where one has to use information that was previously irrelevant (Plebanek & Sloutsky, 2017; Blanco & Sloutsky, 2019).

Depending on the context, either selective or distributed attention can be advantageous. Selective attention is superior when one is confident that a fraction of the available information is sufficient for their goals. Distributed attention is advantageous when there is more uncertainty about what is important. Therefore, distributing attention may be particularly adaptive early in development, since young children have little knowledge of the world. By facilitating broad information gathering, distributed attention helps reduce uncertainty about the world and build up a rich base of knowledge. Distributed attention early in life may represent a sacrifice of immediate performance in exchange for information. These ideas are consistent with the matched filter hypothesis of cognitive control (Chrysikou, Weber, & Thompson-Schill, 2013), which proposes that less cognitive control may lead to more errors but better learning over time.

In other words, distributing attention may facilitate exploration. Recent research suggests that there is a tight link between attention allocation and choices (Konovalov & Krajbich, 2016; Smith & Krajbich, 2018), and perhaps wider attention allocation also promotes wider distribution in action selection. There are recent reports indicating that children’s choices are, indeed, highly exploratory (Blanco & Sloutsky, 2019 PsyArXiv; Sumner et al., 2019 PsyArXiv; Schulz, Wu, Ruggeri, & Meder, 2019). Interestingly, children’s exploration also appears non-random. This is surprising because decision-making research critically distinguishes systematic and directed exploration from random exploration (Badre, Doll, Long, & Frank, 2012; Daw, O’Doherty, Dayan, Seymour, & Dolan, 2006; Knox, Otto, Stone, & Love, 2012; Somerville, et al., 2017), and converging evidence suggests a crucial role of executive control processes mediated by prefrontal cortex (PFC) in systematic exploration (Badre, Doll, Long, & Frank, 2012; Blanco et al., 2015; Otto, Knox, Markman, & Love, 2014). Given the protracted development of cognitive control and PFC (Bunge et al., 2002; Casey, Giedd, & Thomas, 2000; Sowell, et al. 1999), current theories imply that young children’s exploration should be largely unsystematic (Somerville et al., 2017). However, recent evidence suggests young children perform systematic exploration, although they may not direct their exploration strategically the way that adults do (Blanco & Sloutsky, 2019 PsyArXiv). Specifically, 4-year-olds switched between options at extremely heightened rates, and switched more often to less recently selected options, even when this systematic sampling provided very little information.

These findings raise the possibility that young children’s systematic (i.e., non-random) exploration is driven by different mechanisms than adults’ strategic exploration. We hypothesize that children’s exploratory behavior is tied intricately to their immature attention allocation.

The Current Study

The goal of the study was to test this idea by modulating attention allocation by manipulating salience of a cue linked to a reward. We presented children and adults with a simple decision-making task under three conditions to examine the interplay of attention and choice patterns across development. The conditions differed in terms of the perceptual saliency of the stimuli marking the choice options: a Baseline condition where all options were of equal salience, a Competition condition where a salient option was mapped to the lowest reward (putting reward-seeking and salience in competition), and a Congruent condition where the salient option was mapped to the highest reward.

Our predictions for the study were motivated by the hypothesis that children’s attention allocation is an important factor driving their choices, and in particular that shifting attention broadly across stimuli (instead of focusing selectively) promotes systematically distributing their choices across available options. In the same way that attention shifts over time and is less likely to return to recently focused items, less recently chosen options may become increasingly appealing over time. For example, a type of graded novelty preference may be at work; a recently selected option is no longer novel, but the unselected options have been increasing in relative novelty since last being chosen. This tendency, therefore, may enable systematic (i.e., non-random) exploration. This idea shares some similarity to the predictions of the Hierarchical Competing Systems model of the A-not-B task (Marcovitch & Zelazo, 2009), wherein repeated exposure to A trials actually decreases the likelihood of perseverating on the A location in infants. In adults, the decision process is instead associated with selective attention to (and exploitation of) highly rewarding options.

On the basis of previous work (Blanco & Sloutsky, 2019 PsyArXiv; Sumner et al., 2019 PsyArXiv), we expected that most children in the Baseline condition would engage in systematic exploration, with few children exploiting the best option. The exact role of saliency on children’s choices in the Congruent and Competition conditions could manifest in several ways. One possibility was that children’s choices would be driven largely by salience, leading to selecting the most salient option regardless of its reward value. Another possibility was that saliency would act as a learning cue, leading to faster prioritization of the salient option in the Congruent condition and avoidance of it in the Competition condition.

Finally, it was also possible that altering children’s attentional pattern through bottom-up capture of attention (disrupting attentional distribution) would also change the very exploratory behavior that gives rise to the pattern of choices observed previously in the Baseline condition. If so, we can infer an important connection between attention and exploratory behavior early in development. In contrast, if attention is not a causal factor in children’s exploratory behavior, manipulating attention should lead to little or no change in their systematic exploratory behavior.

While we expected attention allocation to affect exploratory behavior in young children, the exact role of attention is not known. We therefore expected that the second possibility (i.e., salience as a learning cue) was highly unlikely. At the same time, we considered both attentional hypotheses likely (i.e., salience-based responding (Possibility 1) and salience disrupting systematic exploration (Possibility 3). In contrast, small to no effects of saliency were expected in adults, who were expected to maximize reward, exploiting the high-value option in all conditions.

Method

Participants

A total of 110 4-to-5-year-olds (mean age = 57 months; range = 48–67 months; 58 girls, 52 boys) participated: 36 in the Baseline, 37 in the Competition, and 37 in the Congruent conditions. Children did not differ by condition in terms of age, F(2,107) = 2.01, p = 0.139, η² = 0.04, or gender, X²(2; N = 110) = 2.01, p = 0.367. Adults (N = 108) also participated (mean age = 19 years; range = 18–29 years; 60 women, 45 men, 3 other responses): 37 in the Baseline, 34 in the Competition, and 37 in the Congruent conditions. Child participants were recruited from preschools and childcare centers in the Columbus, Ohio area. Adults were undergraduate students participating for course credit.

Procedure

Participants completed a simplified n-armed bandit task, framed as a computer game in which participants collected virtual candy from alien creatures (Figure 1). The goal was to earn as much candy as possible. On each of 100 trials, participants chose one of four creatures and received virtual candy according to their choice. Each creature gave a fixed reward throughout the experiment: 1, 2, 3, and 10 candies, respectively. The locations of these rewards were fixed across the experiment but were randomly determined for each participant. Following the choice, the resulting reward was displayed for 3 s, and a meter tracking the total accumulated candy was updated. Children earned a sticker for every 180 candies earned, with benchmarks on the meter indicating these goals. The experiment took about 10–15 minutes to complete.

Figure 1:

Trial structure. (A) After each choice, (B) the reward earned for the choice was presented for 3 s, (C) then the next trial began. In the Congruent and Competition conditions one option was represented by a colorful image that changed on every trial, while the other three were represented by lower salience images that remained stable across trials. In the Baseline condition, all four options were represented by stable images of equal salience.

Participants were assigned to one of three conditions. In the Baseline condition, all creatures were approximately equally salient, whereas in the Congruent and Competition conditions, salience was unequal. Specifically, three of the creatures were black-and-white stick figures, whereas one was colorful and perceptually rich. Additionally, on each trial the salient image was a different novel creature (Figure 1C). Fifty unique images were used for the salient option, each appearing twice during the experiment. In the Competition condition, the salient option was mapped to the lowest reward (1 candy), whereas in the Congruent condition, the salient option was mapped to the highest reward (10 candies).

Results

Choice proportions

Participants’ choices over the course of the experiment are presented in Figure 2. To assess the effect of saliency on performance, we analyzed the proportion of trials that the highest-valued option was chosen. A 3 (Condition: Baseline, Competition, or Congruent) by 2 (Age Group: Children vs. Adults) ANOVA revealed a main effect of age group, F(1,212) = 391.93, p < 0.001, partial-η² = 0.65, a main effect of condition, F(2,212) = 20.89, p < 0.001, partial-η² = 0.16, and a significant interaction, F(2,212) = 6.17, p = 0.002, partial-η² = 0.06.

Figure 2:

Choice proportions. The proportion of trials on which each option was chosen is presented for blocks of 20 trials. Children in the Congruent condition selected the highest-valued option more frequently than children in both the Baseline and Competition conditions. Interestingly, children in the Competition condition did not select the lowest-valued option (which was salient in that condition) more often than in the Baseline condition. This suggests that simple salience-seeking did not drive children’s choices. Adults exploited the highest-value option in all conditions and selected the lowest option more often when it was salient (in the Competition condition). Error bars reflect standard errors of the mean.

To examine the interaction, we conducted separate ANOVAs within each age group. These analyses revealed significant effects of condition for each group [children: F(2,107) = 15.40, p < 0.001, η² = 0.22; adults: F(2,105) = 10.68, p < 0.001, η² = 0.17]. Specifically, children chose the highest-value option more often in the Congruent condition (M = 0.53) compared to the Baseline (M = 0.28), t(71) = 4.40, p < 0.001, d = 1.03, and Competition conditions (M = 0.30), t(72) = 3.93, p < 0.001, d = 0.91. The Competition condition was not different from Baseline, t(71) = 0.57, p = 0.569, d = 0.13. The effect was different for adults: participants in the Competition condition (M = 0.77) chose the best option less than both the Congruent (M = 0.94), t(69) = 3.86, p < 0.001, d = 0.92, and Baseline conditions (M = 0.91), t(69) = 2.94, p = 0.004, d = 0.70. The Congruent and Baseline conditions were not different from each other for adults, t(72) = 1.50, p = 0.137, d = 0.35.

To further test the effects of salience on choice preferences, we analyzed the proportion of choosing the lowest-valued option (which was salient in the Competition condition) across conditions and age groups. A 3×2 (Condition by Age Group) ANOVA revealed a main effect of age group, F(1,212) = 55.84, p < 0.001, partial-η² = 0.21, a main effect of condition, F(2,212) = 13.68, p < 0.001, partial-η² = 0.11, and a significant interaction, F(2,212) = 3.58, p = 0.030, partial-η² = 0.03. There were also effects of condition within each group, [children: F(2,107) = 5.24, p = 0.006, η² = 0.09; adults: F(2,105) = 10.88, p < 0.001, η² = 0.17]. For children, pairwise tests revealed only that participants in the Congruent condition (M = 0.16) chose the lowest option less than both the Baseline (M = 0.23), t(71) = 3.22, p = 0.002, d = 0.75, and Competition conditions (M = 0.25), t(72) = 2.60, p = 0.011, d = 0.60. The Competition and Baseline conditions did not differ significantly, t(71) = 0.65, p = 0.516, d = 0.15. For adults, participants in the Competition condition (M = 0.17) chose the lowest-value option more often than both the Congruent (M = 0.02), t(69) = 3.40, p = 0.001, d = 0.81, and the Baseline conditions (M = 0.03), t(69) = 3.23, p = 0.002, d = 0.77. The Congruent and Baseline conditions were not different from each other, t(72) = 1.03, p = 0.308, d = 0.24.

In summary, there was a straightforward effect of saliency in adults: in the Competition condition, the highest option was selected less frequently, and the low-valued option was selected more frequently, compared to the other conditions. For children, saliency facilitated choosing the highest option more often (and the lowest option less often) in the Congruent condition compared to other conditions. Although these findings are clear, choice proportions present only preliminary (and ambiguous) evidence because the same choice proportions could be generated by different strategies (e.g., random exploration, systematic exploration, or a combination of exploration and exploitation). To eliminate this ambiguity, in the next two sections, we analyze how participants switch among the options across conditions and age groups, followed by computational modeling of their choice strategies.

Switch proportions

To further examine effects of salience, we analyzed the proportion of trials that participants switched responses, choosing a different option than on the previous trial, as an indicator of elevated exploration (Figure 3). Typically, when outcomes are stable, low levels of exploration are expected (Knox et al., 2012; Otto et al., 2014), however, based on recent evidence (Blanco & Sloutsky, 2019 PsyArXiv), we expected children to switch often, suggesting a high level of exploration. Furthermore, subsequent computational modeling (Blanco & Sloutsky, 2019 PsyArXiv), indicated that this exploration was highly systematic. We, therefore, first focus on participants’ switch responses across conditions, following by modeling of their choice strategies.

Figure 3:

Response Switching. The proportion of trials on which participants made a switch response, choosing a different option than the previous trial, is presented. The pink shaded region represents 95% probability density of switch responses given random responding. Extreme switch proportions for children in the Baseline condition suggest elevated exploration levels. Children’s switch proportions are less than the Baseline in both salience conditions. In contrast, adults rarely switch, instead exploiting the best option. Dots represent individual participants.

A 3×2 (Condition by Age Group) ANOVA on switch proportions revealed a main effect of age group, F(1,212) = 577.83, p < 0.001, partial-η² = 0.73, a main effect of condition, F(2,212) = 17.86, p < 0.001, partial-η² = 0.14, and a significant interaction, F(2,212) = 12.50, p < 0.001, partial-η² = 0.11. For adults, switching was low overall (M = 0.117), and there was no effect of condition, F(2,105) = 1.90, p = 0.154, η² = 0.03.

By contrast, children’s switch proportions were affected by the saliency manipulation, indicated by an effect of condition, F(2,107) = 17.42, p < 0.001, η² = 0.25. Children in the Congruent (M = 0.56), t(71) = 5.57, p < 0.001, d = 1.30, and Competition conditions (M = 0.77), t(71) = 3.22, p = 0.002, d = 0.75, switched substantially less than Baseline (M = 0.91). Additionally, children in the Competition condition switched more than those in the Congruent condition, t(72) = 3.04, p = 0.003, d = 0.71. It is not surprising that switching was relatively low in the Congruent condition since children were exploiting the best option instead. But it is surprising that switching was relatively low in the Competition condition despite no increase in choices of the best option. To examine the underlying choice strategies, we performed computational modeling presented in the next section.

Computational Modeling

To examine the strategies driving participants’ choices across the conditions, we evaluated these choices in relation to a Reinforcement Learning model (Sutton & Barto, 1998) that included both systematic and random exploration (Blanco & Sloutsky, 2019 PsyArXiv). The model used prediction errors to learn expected reward values for each option using the following equation:

$$V_{i,t+1}=V_{i,t}+\alpha \left(R_{i,t}-V_{i,t}\right)$$

where V_i,t is the expected value of option i on trial t, R_i,t is the reward on trial t earned for choosing option i, and α is the learning rate (a free parameter). It then made choices according to the following function:

$$P\left(a_{i,t}\right)=\frac{e^{\beta \ast \left[V_{i,t}\ast \left(1-\varphi \right)+L_{i,t}\ast \varphi \right]}}{{\sum }_{j=1}^n{e}^{\beta \ast \left[V_{j,t}\ast \left(1-\varphi \right)+L_{j,t}\ast \varphi \right]}}$$

where P(a_i,t) is the probability of choosing option i on trial t. L_i,t is the lag term that simply encodes the number of trials since option i was last chosen. The weight parameter ϕ (0 ≤ ϕ ≤ 1) mediates the relative weights of expected values (i.e. exploitation) and lags (i.e. systematic exploration) in determining choices. Greater values of ϕ indicate greater influence of systematic exploration. When ϕ = 0, the model chooses based only on expected value; when ϕ = 1, it chooses only based on the lag. ϕ = 0.5 represents roughly equal contributions of reward value and choice lag. β is the inverse temperature parameter that controls random exploration. At β = 0 choice probabilities become completely random (i.e. equal between all options). As β approaches infinity the model chooses the most favorable option (based on the weighted combination of value and lag described above) on every trial. β and ϕ were free parameters.

The best-fitting parameter values were compared across groups and conditions to determine the influences of reward, systematic exploration, and random exploration on participants choices (Figure 4). A 3×2 (Condition by Age Group) ANOVA on best-fitting ϕ found a main effect of age group, F(1,212) = 82.11, p < 0.001, partial-η² = 0.28, a main effect of condition, F(2,212) = 9.61, p < 0.001, partial-η² = 0.08, and a significant interaction, F(2,212) = 6.92, p = 0.001, partial-η² = 0.06. For adults there was no effect of condition, F(2,105) = 1.62, p = 0.20, η² = 0.03, whereas for children, there was an effect of condition, F(2,107) = 8.80, p < 0.001, η² = 0.14, such that ϕ was higher for the Baseline condition compared to both the Congruent, t(71) = 4.36, p < 0.001, d = 1.02, and Competition conditions, t(71) = 2.47, p = 0.016, d = 0.58. The Congruent and Competition conditions were not significantly different, t(72) = 1.56, p = 0.123, d = 0.36. The high values of ϕ in the Baseline condition and substantially lower values in the other conditions suggest that the salience manipulation dramatically decreased systematic (non-random) exploration in in these conditions compared to Baseline.

Figure 4:

Best-fitting ϕ parameter. Histograms of the best-fitting ϕ parameter for each group are presented. Both salience conditions had a large proportion of children with very low values of ϕ, indicating little systematic exploration, while the Baseline condition had a larger proportion of participants with high values of ϕ, indicating higher levels of systematic exploration. Almost all adults had low ϕ, indicating their choices were largely driven by reward value.

A 3×2 (Condition by Age Group) ANOVA on best-fitting β (Figure 5) found no main effect of group, F(1,212) = 0.14, p = 0.708, partial-η² < 0.001, no effect of condition, F(2,212) = 1.57, p = 0.211, partial-η² = 0.015, and no interaction, F(2,212) = 0.38, p = 0.688, partial-η² = 0.003.

Figure 5:

Best-fitting β parameter. Histograms of the log of best-fitting β (which controls the rate of random exploration) are presented. Log(β) is presented rather than raw values because large outliers make graphing the raw values difficult. There were no significant age or group effect on best-fitting β value.

Discussion

The goal of the current study was to examine the link between children’s exploration and attention by manipulating attention and observing effects on the strategies underlying their choice patterns. The results suggest that attentional manipulation (i.e., exogenously capturing attention through large differences in salience) decreased the level of systematic (i.e. non-random) exploration in young children compared to a Baseline condition—where they were systematic and highly exploratory. Children in the Competition and Congruent conditions switched between options less often, and did so less systematically, than children in the Baseline condition. The difference is particularly striking in the Competition condition where choice proportions were equivalent to the Baseline condition. These differences suggest that the effect of salience on children’s attention caused an important change in how they were making decisions. Conversely, while adults showed effects of saliency on their choice proportions, they were highly exploitative in all conditions, with very little influence of saliency on systematic exploration (Figure 4).

Additionally, while children’s choice preferences were not simply salience-driven (the effect of saliency was dependent on whether or not salience was congruent with reward value), the underlying choice strategy revealed by computational modeling was driven by salience. Whereas high levels of systemic exploration were observed in children in the Baseline condition, these levels were substantially lower in the Congruent and Competition conditions. In the Competition condition we found elevated levels of random exploration, whereas in the Congruent condition, there were higher levels of exploitation, compared to the Baseline condition.

These results suggest a potentially integral role of children’s immature attentional allocation in facilitating systematic exploration—in contrast to adults’ tendency to both selectively attend and maximize reward. Under normal circumstances, systematic exploration is children’s default strategy. Less recently sampled options become more likely to be selected in the future—a pattern that may effective approximate uncertainty-directed exploration in many cases. We suggest that children’s default attention allocation pattern enables systematic exploration through graded novelty preference (wherein novelty steadily increases over time as an option is not attended, becoming more likely to attract attention and be chosen). Manipulating bottom-up attention by altering the relative saliency of the choice options disrupts this process, leading to a reduction in systematic choice patterns.

Unanswered Questions and Future Directions

While the current results indicate there may an important connection between children’s attention and their systematic exploratory behavior, the current study leaves a number of questions to be addressed in future research. For example, we did not directly measure participants’ attention allocation during the task. While children’s tendency to distribute attention broadly and its effects on cognition have been previously shown in other tasks (e.g. Deng & Sloutsky, 2015, 2016), it is possible that children exhibited focused attention during this task. If so, it could be that the mismatch between salience and reward in the Competition condition cause children confusion as to the goal of the task, leading to more random responding (and hence less systematic behavior). In that case, though, we might expect that children would explore the salient option more often to reduce their confusion, which we did not observe. Still it remains possible that the difference in children’s behavior was the result of other mechanisms than a disruption to their default distributed attention allocation pattern. Future work that measures children’s attention directly (e.g. with eye-tracking) will be needed to better understand the exact role that attention contributes to children’s choices. Another potential limitation of the current study is the relatively abstract nature of the rewards. It is possible that children’s behavior may be different in a scenario where rewards are more direct and immediate, such as directly foraging for food rewards, rather than collecting virtual points that eventually earn real stickers. Investigating exploratory behavior in children in that type of situation could be an insightful area for future research. Finally, while the results implicate an important role of attention, it is possible that other factors also contribute to children’s exploratory behavior. For example, exploring may also be more intrinsically rewarding to children compared to adults. Such intrinsic enjoyment of exploration coupled with graded novelty preference would also result in systematic exploration. However, in this case, it is not clear why salience would disrupt the intrinsic value of exploration.

Conclusions

Attention and decision-making may be intricately linked in both children and adults (Konovalov & Krajbich, 2016; Smith & Krajbich, 2018), and developmental changes in attention may be an important factor in the development of decision-making. In particular, children’s tendency to distribute attention broadly may be tied to their tendency to explore systematically. Children’s increasing ability to attend selectively may then coincide with an increase in maximizing their choices toward rewarding actions. Younger children’s choices seem to be geared toward learning rather than maximizing reward, a tendency which may be directly linked low cognitive control (Chrysikou, Weber, & Thompson-Schill, 2013), and which is supported by several recent findings showing that, despite knowing the best option, children are less likely than adults to maximize their choices toward it (Plate et al. 2018; Blanco & Sloutsky, 2019 PsyArXiv). Understanding the critical and complex interaction of attentional mechanisms and decision-making is an exciting area of future research and may be particularly important factor in understanding cognitive development.