Cognitive Mechanisms Underlying Recursive Pattern Processing in Human Adults

Abstract

The capacity to generate recursive sequences is a marker of rich, algorithmic cognition, and perhaps unique to humans. Yet, the precise processes driving recursive sequence generation remain mysterious. We investigated three potential cognitive mechanisms underlying recursive pattern processing: hierarchical reasoning, ordinal reasoning, and associative chaining. We developed a Bayesian mixture model to quantify the extent to which these three cognitive mechanisms contribute to adult humans’ performance in a sequence generation task. We further tested whether recursive rule discovery depends upon relational information, either perceptual or semantic. We found that the presence of relational information facilitates hierarchical reasoning and drives the generation of recursive sequences across novel depths of center embedding. In the absence of relational information, the use of ordinal reasoning predominates. Our results suggest that hierarchical reasoning is an important cognitive mechanism underlying recursive pattern processing and can be deployed across embedding depths and relational domains.

1. Introduction

Humans are powerful generalizers capable of inferring, predicting, and generating patterns across many domains (Dehaene, Al Roumi, Lakretz, Planton, & Sablé-Meyer, 2022; Fitch, Friederici, & Hagoort, 2012; Frank & Tenenbaum, 2011; Gomez, 2002; Lake & Piantadosi, 2020; Miller, 1956; Saffran, Aslin, & Newport, 1996; Shepard, 1987). Recursive patterns, which are often observed in human behavior, consist of structures embedded within other structures of the same kind. They are seen across domains like natural language (e.g., structures like “They don’t know that we know” where one sentence is embedded within another), music (e.g., repeating melodic phrases within repeating melodic phrases), computer programming (e.g., processes such as loops nested within other loops), and mathematics (e.g., fractals, recursively defined functions, repeating fractions) (Friederici et al., 2011; Hauser, Chomsky, & Fitch, 2002; Hofstadter, 2001; Kilpatrick, 1985; Koelsch, Rohrmeier, Torrecuso, & Jentschke, 2013; Martins, Gingras, Puig-Wald Mueller, & Fitch, 2017; Odifreddi, 1992; van der Hulst, 2010). These types of recursive patterns are rarely, if ever, generated by non-human animals, leading some to speculate that perceiving, predicting, and producing recursive patterns is the key feature that distinguishes human cognition from animal cognition (Chomsky, 2014; Corballis, 2007; Dehaene, Meyniel, Wacongne, Wang, & Pallier, 2015; Fitch, 2014; Fischmeister et al., 2017; Hofstadter, 1979; Martins, 2012). However, the origins and mechanisms underlying recursive pattern processing are not yet known.

Cognitive scientists often use sequencing tasks to investigate the mechanisms underlying recursion (Kirov & Frank, 2012; Lakretz et al., 2021; Lashley, 1951; Miller, 1958; Terrace & McGonigle, 1994). A popular approach for measuring recursive patterns in sequencing tasks focuses on center-embedded sequences. Center-embedded sequences of the form A_nB_n (e.g., “The cat the dog chased meowed”) are recursive patterns characterized by mirrored symmetrical relations and nested long-range dependencies between sequence items (de Vries et al., 2008; Gibson & Thomas, 1998). Linguists have argued that, unlike other types of sequences, center-embedded sequences arise from context-free grammars that are represented with recursive tree structures (Chomsky, 1956).

Many scientists have investigated whether center-embedded sequences are accessible to humans, non-human species like monkeys, birds, and artificial neural networks (Abe & Watanabe, 2011; Ferrigno, Cheyette, Piantadosi, & Cantlon, 2020; Fitch & Hauser, 2004; Gentner et al., 2006; Jiang, Long, Cao, Li, Dehaene, & Wang, 2018; Liao, Brecht, Johnston, & Nieder, 2022; Winkler, Mueller, Friederici, & Männel, 2018). Some studies used grammaticality judgments where participants were shown two sequences (one center-embedded, the other non-center-embedded) and forced to make a choice about the “correctness” of the two alternatives (McCoy, Culbertson, Smolensky, & Legendre, 2021). Other studies like Ferrigno et al. (2020) used open-ended sequence generation tasks that required participants to actively generate sequences as opposed to passively recognizing them. This tests not only whether an individual recognizes a rule violation but also whether an individual can use a rule to generate information. The ability to create (not just recognize) novel recursive patterns is central across domains of hierarchical logic. Ferrigno et al. (2020)’s sequencing task provides a stronger test of generalization compared to grammaticality judgments because it taps into the production part of the comprehension/production distinction. This production task requires participants to generate a sequence of responses from a large set of possibilities, whereas recognition tasks are often constrained to picking one of two alternatives. The active generative (as opposed to passive discriminative) aspect of this task is, therefore, much more comparable to everyday rule use in human language and motor routines, thus making the task ideal for studying recursive pattern learning (Dedhe, Clatterbuck, Piantadosi, & Cantlon, 2023).

One limitation of existing research on recursive rules in humans is that prior studies did not test generalization of center-embedded rules to new sequence depths (though see McCoy et al., 2021; Shin & Eberhard, 2015). The ability to generalize to novel, unseen depths is central to the “unboundedness” claim about language and other hierarchical behavior. This purported “unbounded” nature of hierarchical behavior involves making “infinite use of finite means” (Chomsky, 1965, quoting von Humboldt, 1836), that is, generalizing complex patterns beyond the finite bounds of previously observed input. However, center-embedded utterances of more than two levels of embedding depth are rarely seen in everyday natural language and are especially difficult to process (Gibson & Thomas, 1999; Karlsson, 2010). Furthermore, McCoy and colleagues (2021) showed that evidence of depth generalization in humans is currently lacking, and it remains unclear whether humans extrapolate recursive rules across depths. For example, Ferrigno et al. (2020) showed that adults across cultures generalize recursive rules to novel center-embedded sequences of a fixed length (four items) but did not test whether they generalize the rule to longer sequences. Our experiments here target whether humans generalize novel center-embedded rules from one to two levels of depth. These experiments are critical towards establishing the unbounded nature of hierarchical psychological processes.

A second limitation of existing research is that it has not explored the types of information that facilitate recursive rule learning. Natural human behaviors that commonly show recursive patterns are domains like language, music, and math that contain meaning. However, common paradigms for testing recursive rules often strip meaning away to test artificial grammars (strings of nonsense syllables like bo-pi-ku) and arbitrary shape sequences (strings of shapes like square–circle–triangle; McCoy et al., 2021). Stripping meaning away from stimulus items is problematic because it can exclude features that are crucial to learning (Goldberg, 2010; Johnson & Jusczyk, 2001). Thus, it is important to test whether and what types of semantics might aid learning. Here, we compared center-embedded rule learning across three types of information: perceptual, semantic, and arbitrary. For perceptual support, the sequence stimuli were brackets: [] {} () in which shape, color, and orientation provided information about relations between units. For semantic support, the stimuli were words like lamb sheep kitten cat puppy dog where word meaning (categories) provided information about relations between units. Finally, in the no support condition, the stimuli were arbitrary shapes like ▲ ⬟ ◾ ♥ ★ ⬮. If relational information between units is important for recursive rule learning, then participants will generate center-embedded rules significantly more often in the perceptual support and semantic support conditions compared to the no support condition.

In the current study, we used an open-ended sequence generation task to test the breadth of human recursive rule learning across embedding depths and sequence content. Beyond analyzing rule accuracy, we tested specific hypotheses about the psychological basis of recursive processing. Previous studies could not rule out that participants were using alternative strategies like attending to item order, bigrams, or counting (McCoy et al., 2021) and simpler cognitive mechanisms like associative chaining or ordinal reasoning (de Vries et al., 2008; Ferrigno et al., 2020; Rey, Perruchet, & Fagot, 2012). We used a novel Bayesian data analysis model to quantify the extent to which each participant used three different cognitive mechanisms to generate sequences: associative chaining, ordinal reasoning, and hierarchical reasoning. This approach allows us to disentangle the contributions of different cognitive mechanisms to recursive rule use in humans.

2. Methods

2.1. Participants

A total of 197 participants took part in this experiment. Participants were English-speaking U.S. adults and were compensated $2.50 each. Guidelines of our university’s institutional review board (IRB) were followed for all procedures. All research was conducted via an online behavioral task hosted on Amazon MTurk. We restricted the participant pool to those with a 95% approval rate and over 5,000 approved human intelligence tasks, under the Mechanical Turk blog’s recommendations for improving the quality of participants (Amazon Mechanical Turk, 2012).

2.2. Overview

Participants learned two training sequences: each four items in length (Fig. 1). After learning to reliably generate these sequences, participants received open-ended transfer trials. During these trials, participants received two novel testing sets of items that were not presented during training: a set of four items (maximum depth possible = one level of embedding) and another set of six items (maximum depth possible = two levels of embedding). Each set was tested in eight trials. Participants could generate any sequence and were not selectively rewarded for generating center-embedded sequences during the open-ended transfer trials. There was no “right” answer: participants could potentially generate any of the 24 possible permutations of the four items and 720 possible permutations of the six items. Along with these eight novel transfer trials per set, participants also received four familiar trials with each of the two old training sets. These eight familiar trials included differential feedback (positive or negative feedback, as appropriate) and served as checks to ensure that the participants retained the training sequences and were motivated to respond correctly. Participants performing below chance (50%) on these familiar check trials were excluded from subsequent analysis. The order in which novel and familiar sets appeared was randomized.

Fig. 1.

The two training sets and two testing sets of items used across three relational support manipulations: perceptual support, semantic support, and no support. The perceptual support contains visual relational information along two dimensions: form (color/shape) and orientation (left or right). The semantic support contains linguistic relational information along two dimensions: species and age (young or adult). In the no support condition, the stimuli are completely arbitrary and unrelated. The red numbers illustrate the correct sequence of the training items that is, the order in which they need to be clicked. Testing trials were non-differentially reinforced so that subjects could freely generate sequences without feedback as a guide to center embedding. The stimulus items appeared scrambled-up on the testing screen at random spatial locations which changed in each trial.

2.3. Experimental conditions

There were three experimental conditions (Fig. 1). In the first condition (perceptual support), participants were shown “bracket” items: [] {} (). These bracket stimuli could be organized along two perceptual dimensions: orientation (right-ward or left-ward facing) and form (color/shape). Twenty-nine participants were assigned to this condition. Twenty-four participants (ages 26–70 years; 10 female) performed above chance on the check trials and were included in subsequent analysis. In the second condition (semantic support), participants were shown words: LAMB SHEEP KITTEN CAT PUPPY DOG. These words could be organized along two semantic categories: age (young or old) and species. Fifty-six participants were assigned to this condition. Thirty-eight participants (ages 32–66 years; Twenty-four female) performed above chance on the check trials and were included in subsequent analysis. In the final condition (no support), participants were shown arbitrary shapes: ▲ ⬟ ◾ ♥ ★ ⬮. These shapes had no underlying relations or structure. Fifty-seven participants were assigned to this condition. Thirty-two participants (ages 28–68 years; 12 female) performed above chance on the consolidation check trials and were included in subsequent analysis. All other parts of the experimental procedures were identical across conditions.

The perceptual support condition described above used “bracket” stimulus items. To avoid the possibility that participants may be tapping into their real-world knowledge of the expected order of parentheses (say from mathematics or computer programming), we included an additional perceptual support condition using different stimulus items. These items included two dimensions of perceptual support: shape and color. In this perceptual support condition, participants were shown colored shapes: ▲ ▲ ▲ ● ● ●. All other parts of the experimental procedures were identical to those described above (Fig. S2). Fifty-five participants were assigned to this condition. Participants were English-speaking U.S. adults. Forty-one participants (ages 24–66 years; 13 female) performed above chance on the check trials and were included in subsequent analysis. All further details about this additional perceptual support condition can be found in the Supporting Information.

2.4. Training

The experiment began with an instruction screen where participants were told that they would see pictures that they had to click in the correct order. They were told that while they may not know the order to begin with; they would discover it as they proceeded through the experiment. To initiate each trial, participants clicked on an icon at the center of their fixation screen. This was followed by the appearance of four images—the stimulus items—scattered on the screen. Images appeared at random spatial locations which changed on each trial (Fig. 1). Participants were required to learn a sequence, that is, click on the stimulus items in the correct order. If the participant clicked on the correct item, it would turn opaque (fade into the screen) and be accompanied by positive auditory feedback (a ding). Participants would then move on to click further items to complete the sequence. If the participant clicked on the wrong item, they would receive negative feedback—-both visual (a red screen with a sad face) and auditory (a buzzer). The trial would end there, and the participant would be shown a 1-s black time-out screen. They would then start over on the fixation screen. If the participant clicked on all four items in the correct order, they would immediately receive positive feedback—both visual (a green screen with a happy face) and auditory (a chime). The trial would end, and the participant would be shown a 1-s intertrial interval screen. This process would continue until the participants crossed an 80% accuracy threshold (getting at least four accurate out of five consecutive trials) and showed that they had successfully learned the sequence of items. Participants then went on to learn the next sequence. A total of two sequences were used for training in each experimental condition (Fig. 1). The order in which these sequences appeared was randomized.

2.5. Testing

Like training, the testing phase began with an instruction screen where participants were told that they would be seeing pictures that they had to click on in the correct order. They were additionally told that while they may not know the correct order, they should nevertheless take their best guess. To begin each trial, participants clicked on a start icon at the center of their screen. This was followed by the appearance of either four images or six images—the stimulus items—on the screen. As before, these images appeared at random spatial locations which changed in each trial. Upon clicking a stimulus item, it would turn opaque (fade into the screen) and be accompanied by positive auditory feedback (a ding). Because these were open-ended transfer trials, participants received positive feedback regardless of the sequence generated. After clicking on all the items, participants would immediately receive positive feedback—both visual (a green screen with a happy face) and auditory (a chime). The trial would end, and the participant would be shown a 1-s intertrial interval screen. They would then start the next trial. During testing, participants received novel sets of items that were not presented during training. Each set was tested in eight trials. Along with these eight novel transfer trials per set, participants also received four familiar trials with each of the two old training sets. These eight familiar trials included differential feedback (positive or negative feedback, as appropriate). Thus, these eight familiar trials served as checks to ensure that the participants retained the training sets and were motivated to respond correctly. Participants performing below chance (50%) on these familiar consolidation check trials were excluded from subsequent analysis. The order in which novel and familiar sets appeared was randomized.

2.6. Behavioral data analysis

We analyzed three types of sequences in participants’ responses: center embedded, crossed, and tail recursive (Table 1). Center-embedded sequences are behavioral markers of recursive pattern processing. Crossed sequences can be learned from the training data by paying attention to item order alone. Tail-recursive sequences contain bigrams that preserve learnt adjacent pair associations (i.e., transition probabilities) from training. We coded the frequency of each type of sequence as a proportion of total transfer trials (eight). We hypothesized that if subjects used hierarchical reasoning, they would produce more center-embedded sequences than crossed or tail-recursive or other sequences. The generation of these sequences was modeled in accordance with three cognitive mechanisms: associative chaining, ordinal reasoning, and hierarchical reasoning. We calculated the conditional probability of generating each type of sequence assuming each cognitive mechanism was being used (Table 2) that show values for the Bayesian likelihood term: p(response | cognitive mechanism).

Table 1.

Some of the possible behavioral responses in the perceptual support condition across depths or levels of embedding

	One Level of Embedding		Two Levels of Embedding
Sequence	Example	Structure	Example	Structure
Center embedded	[ { } ] or { [ ] }	A1A2B2B1	( [ { } ] ) or { [ () ] }	A1A2A3B3B2B1
Crossed	[ { ] } or { [ } ]	A1A2B1B2	( [ { ] } ) or { [ () } ]	AxAyAzBiBjBk
Tail recursive	[ ] { } or { } [ ]	A1A2B1B2	() [ ] { } or { } () [ ]	A1B1A2B2A3B3

Note. There are 24 possible sequences (each being a string of four stimulus items) in the one level of the embedding condition and 720 possible sequences (each being a string of six stimulus items) in the two levels of the embedding condition. The letters (A,B) in the structure columns represent one relational dimension (orientation in the perceptual support condition; age in the semantic support condition). The subscripts (_{1, 2}) represent the other relational dimension (form in the perceptual support condition; species in the semantic support condition). The crossed sequence structure uses alphabetical subscripts instead of numerical ones to convey that the relative internal order of the A elements and the relative order of the B elements does not matter as long as all the As occur before all the Bs (see Table S1 in the Supporting Information for the full list of possible center-embedded, crossed, and tail-recursive sequences in the two levels of embedding condition). Note that all crossed sequences where x = k and y = j and i = k are counted as center-embedded sequences.

Table 2.

Conditional probability matrix showing p(response | cognitive mechanism) or the likelihood of generating different types of response sequences across one and two levels of embedding given a particular cognitive mechanism. These probabilities represent an ideal, noise-free scenario where each cognitive mechanism faithfully produces mechanism-appropriate responses

	One Level of Embedding				Two Levels of Embedding
Sequence	Associative Chaining	Ordinal Reasoning	Hierarchical Reasoning	Random (Noise)	Associative Chaining	Ordinal Reasoning	Hierarchical Reasoning	Random (Noise)
Center embedded	0	$\frac{1}{2}$	1	$\frac{2}{24}$	0	$\frac{1}{6}$	1	$\frac{6}{720}$
Crossed	0	$\frac{1}{2}$	0	$\frac{2}{24}$	0	$\frac{5}{6}$	0	$\frac{30}{720}$
Tail recursive	1	0	0	$\frac{2}{24}$	1	0	0	$\frac{6}{720}$
Other	0	0	0	$\frac{18}{24}$	0	0	0	$\frac{678}{720}$

We considered three cognitive mechanisms drawn from psychological theory. Associative chaining involves forming links between adjacent units in a spatial or temporal sequence (Crowder, 1968; Santolin & Saffran, 2018, Skinner, 1957). Associative chaining is akin to statistical learning and preserves the learnt associations or transition probabilities between the stimulus items during training. Associative chaining users would successfully extract the bigrams seen in training and would thus generate mostly tail-recursive sequences. For example, an associative chaining user in the perceptual support condition might learn from the training sequences that {is followed by} and is followed by]. This would predict the subsequent generation of tail-recursive sequences like []{}or{}[] that preserve these learnt associations. Ordinal reasoning involves understanding that items in a sequence can be ordered along an underlying dimension along with knowledge about an item’s sequence position or index (D’amato & Colombo, 1990; Inhelder & Piaget, 1964; Terrace, 1993). It preserves the relative ordinal positions of the stimuli learned during training. Ordinal reasoning users would generate a mixture of center-embedded and crossed sequences—all sequences that preserved learnt ordinal position knowledge would be valid. For example, an ordinal reasoning user in the semantic support condition might learn from the training sequences that SHEEP and DOG occur at position #2 and that LAMB and PUPPY occur at position #3. This would predict the subsequent generation of sequences where SHEEP and DOG always occupy an earlier ordinal position compared to LAMB and PUPPY, thus preserving the learnt ordinal positions. Hierarchical reasoning deals with nested, embedded hierarchical tree structures where lower level units are combined to form higher level ones (Green-field, 1991). Hierarchical reasoning users would generate mostly center-embedded sequences because they depend on two-or-three-tiered relations between non-adjacent stimulus items.

Previous research that models the learnability of the hierarchical structure of language used formal Bayesian frameworks that capture the relation between observed behavior and the computational processes (or “grammars”) proposed to underpin this behavior (Perfors, Tenenbaum, & Regier, 2011). Perfors and colleagues (2011) formulated a model that makes inferences about underlying generative grammars based on observed language data. On similar lines, we implemented a novel Bayesian mixture model to infer likely strategies from behavior. Our model incorporated the entire set of possible responses and explicitly accounted for different sequences generated by associative chaining, ordinal reasoning, and hierarchical reasoning. We assumed that any given participant responded using a mixture of these three mechanisms. We also included a fourth catch-all noise parameter or “random” cognitive mechanism to capture mistakes that people make and anything else not directly captured by the three cognitive mechanisms. The noise strategy models a high entropy response, predicting that all of the 4! = 24 possible sequences in the one level of embedding condition and all the 6! = 720 possible sequences in the two levels of embedding condition are equally likely to be generated.

We considered it likely that the participants used a mixture of cognitive mechanisms. To handle this, our model inferred a distribution of mechanisms for each individual participant. Assuming the fixed “loading” matrix α of how each strategy relates to responses (Table 2), we inferred a probability vector β over cognitive mechanisms, representing the probability a person used each of the three possible cognitive mechanisms or p(cognitive mechanism | response). For example, one participant might be inferred as 80% hierarchical and 20% ordinal; another may be 50% ordinal and 50% associative. β was given a Dirichlet(0.2) prior which encourages sparsity among different mechanisms and the prior over strategies was assumed to be uniform. The likelihood in the model computed the probability of human responses as α^Tβ, and the inference provided samples of the posterior distribution p(β | responses). The Bayesian model was run using the Stan package in R (Stan Development Team, 2022) with the default gradient-based Monte Carlo Markov chaining algorithm NUTS, i.e., the No U-Turn Sampler (Hoffman & Gelman, 2014). It was run for 10,000 iterations. We ran four chains to test convergence, which we confirmed using standard diagnostics.

3. Results

Participants generated significantly more center-embedded responses compared to crossed responses in the presence of perceptual and semantic relational support across one and two levels of embedding (Fig. 2). Our data analysis model (see the Supporting Information for more details) quantified the extent to which each participant used three different cognitive mechanisms of increasing complexity: associative chaining, ordinal reasoning, and hierarchical reasoning. Results from the model show that participants in both the perceptual and semantic support conditions preferred hierarchical reasoning over other mechanisms across both levels of embedding. In the absence of underlying structure, participants showed a preference for using ordinal reasoning over both hierarchical reasoning and associative chaining (Fig. 3).

Fig. 2.

The proportion of each response sequence generated across three experimental conditions: perceptual support (N = 24), semantic support (N = 38), and no support (N = 32). Chance for all three types of response sequences is 8% for one level of embedding and <5% for two levels of embedding. The error bars represent standard error of the mean. *represents a significant difference (p < .05) between center-embedded responses and crossed responses using a Wilcoxon signed rank test (see the Supporting Information for detailed statistical results). The bars do not add up to 100% because all other responses apart from center-embedded, crossed, or tail-recursive sequences have not been shown in the figure. Such “other” responses constitute a minority of observed behavior across all conditions.

Fig. 3.

The average inferred posterior probability from the Bayesian model (averaging across participants) of engaging in each cognitive mechanism across three experimental conditions: perceptual support (N = 24), semantic support (N = 38), and no support (N = 32). Chance is 25%. The error bars represent the standard error of the mean, across participant model values. * represents a significant difference (p < .05) between hierarchical reasoning and ordinal reasoning using a Wilcoxon signed rank test (see the Supporting Information for detailed statistical results).

An analysis of individual data (Fig. S1) showed that in the presence of perceptual support, most participants used hierarchical reasoning, followed by the semantic support condition where more than half of the participants engaged predominately in hierarchical reasoning across both levels of embedding. While most individuals in the no support condition were ordinal reasoners, about a fifth of them selectively relied upon hierarchical reasoning. These individuals demonstrate that relational information facilitates but is not mandatory for hierarchical reasoning. We tested whether individual participants maintained their reasoning strategy across depths. The likelihood of engaging in hierarchical reasoning was highly correlated across one and two levels of embedding all conditions (Fig. 4). These results imply that participants use consistent cognitive mechanisms across levels of embedding. A small number of individuals did not show any identifiable cognitive mechanism; these individuals were high in noise, that is, the random cognitive mechanism. Generally, noise was significantly higher in the two levels of embedding condition compared to the one level of embedding condition (Fig. 4).

Fig. 4.

The inferred posterior probability from the Bayesian model of engaging in each cognitive mechanism across three experimental conditions for one and two levels of embedding: perceptual support (N = 24), semantic support (N = 38), and no support (N = 32). (a) The pie charts show the proportion of participants who predominantly use each cognitive mechanism. Two participants in the semantic support condition, and two participants in no semantic support condition did not show an identifiable dominant cognitive mechanism; they have been excluded from the pie charts. (b) The average inferred noise from the Bayesian model across three experimental conditions. The error bars represent the standard error of the mean. *represents a significant difference (p < .05) between one level of embedding and two levels of embedding using a Wilcoxon signed rank test (see the Supporting Information for detailed statistical results). (c) The inferred posterior probability from the Bayesian model of engaging in hierarchical reasoning across levels of embedding for each individual (points). The correlation between one level of embedding and two levels of embedding is significant (p < .05) across all three relational support conditions using the Pearson’s correlation coefficient (see the Supporting Information for detailed statistical results).

The results from the additional perceptual support condition agree with those from the perceptual support condition with the bracket stimuli described above (Figs. S3 and S4). All further details about this additional perceptual support condition can be found in the Supporting Information.

4. Discussion

We used an open-ended sequence generation task to study whether human adults use robust recursive rules. We found that human adults generalize recursive rules across novel depths of embedding. This ability to extrapolate counts as strong evidence of the unbounded nature of recursive pattern processing and was often not demonstrated in prior studies (McCoy et al., 2021). Participants especially generated recursive center-embedded sequences in the presence of relational information that is either perceptual or semantic. This finding is a novel demonstration of the domain generality of recursive reasoning across perception and semantics. Furthermore, we showed that human adults can use perceptual relational information to generate recursive center-embedded sequences with diverse stimuli—both “bracket” stimuli that may potentially be scaffolded by real-world knowledge of the expected order of parentheses (say from mathematics or computer programming), and shape stimuli where previous real-world knowledge is unlikely to have contributed. The comparable performance across varied visual-perceptual affordances points to the robustness of recursive rule use even in the absence of experience. Most participants found recursive pattern processing difficult in the absence of relational information. Thus, using relational information between stimulus items is important for hierarchical reasoning. Hierarchical reasoning is therefore dependent on general relational reasoning that involves identifying perceptual or conceptual links between two or more constituent objects, items, or units of information (Holyoak, 2012; Holyoak & Lu, 2021). Our findings highlight the central role of real-world contexts, meanings, and informational affordances to pattern processing.

We quantified the extent to which three cognitive mechanisms contributed to the generation of center-embedded sequences: hierarchical reasoning, ordinal reasoning, and associative chaining. Participants predominantly used hierarchical reasoning in the perceptual and semantic conditions but used ordinal reasoning in the no support condition. Individual participants maintained their preferred strategy across depths—participants who used hierarchical reasoning at one level of depth also used hierarchical reasoning at two levels of depth. Hierarchical reasoning users have a proclivity to generate nested, embedded hierarchical tree structures where lower level units are combined to form higher level ones (Greenfield, 1991). However, it is unclear how the tiered relations between non-adjacent stimulus items are represented and manipulated during hierarchical reasoning. Implementing hierarchical logic may depend either on push-down stacks characteristic of context-free grammars (Fitch, 2014) or on queues characteristic of context-sensitive grammars (Jiang et al., 2018; Malassis et al., 2020). More research is needed to precisely characterize the mental representations and computations implicated in hierarchical logic.

Notably, some participants preferentially generated center-embedded sequences even in the absence of relational information. This finding highlights the potential of the human mind to spontaneously discover recursive patterns even without relational support. Relational structure facilitates but is not mandatory for recursive pattern processing. However, most participants did not implement hierarchical reasoning in the no support condition. Instead, participants preferentially used ordinal reasoning in this condition. Thus, when unable—or unwilling—to use hierarchical reasoning, participants defaulted ordinal reasoning. The inability to use hierarchical reasoning may be nested in performance issues (Lakretz & Dehaene, 2021). For example, hierarchical reasoning may be subject to higher executive function (working memory, attention, motivation, etc.) demands compared to ordinal reasoning, and thus the few participants in the no support condition who successfully use hierarchical reasoning may simply be more devoted to the task.

However, the difficulty of using hierarchical reasoning may be due to other reasons such as ordinal reasoning (a linear cognitive mechanism) being more primitive compared to complex, abstract hierarchical reasoning. Ordinal reasoning involves “linear” (Coopmans, De Hoop, Kaushik, Hagoort, & Martin, 2022; Shi, Emond, & Badri, 2020) or “flat” (Matthei, 1982) mental representations. On the contrary, hierarchical reasoning relies on two or more dimensions of mental representations. Prior research showed that children solve certain hierarchical tasks by simplifying them; they “flatten” or “chain” items rather than using hierarchical reasoning (Greenfield, 1991; Matthei, 1982; Shi et al., 2020). Recent work has found that although 3-to-4-year-old children can use hierarchical reasoning, they are more likely to use ordinal reasoning compared to adults (Dedhe, Piantadosi, & Cantlon, 2022; Ferrigno et al., 2020). Non-human primates also spontaneously use ordinal reasoning rather than hierarchical reasoning during comparable sequencing tasks (Ferrigno et al., 2020). Furthermore, connectionist models tested with comparable sequencing tasks show limited capacity for recursive rule implementation and often struggle with long-range dependencies (Elman, 1990; Kirov & Frank, 2012; Lakretz et al., 2021; but see Yang & Piantadosi, 2022). Connectionist models could simulate hierarchical behavior but in ambiguous circumstances strongly favored linear structures over hierarchical interpretations (Coopmans et al., 2022). Thus, linear ordinal reasoning may be more primitive than hierarchical reasoning because human children, non-human primates, and artificial neural networks all recognize ordinal structures earlier in development and with more ease and accuracy compared to hierarchical structures.

Our findings reveal robust recursive rule generation by adult humans in tasks that provide ground structure—in this case, perceptual and semantic relations. When no relational support is available, adults do not easily generate recursive rules. A strong version of “dendrophilia” (Fitch, 2014) would imply that humans generally find recursive structure wherever it exists, even when there is no relational information among elements—this was not the pattern we observed. When adult humans cannot or do not see relations among sequence items, they revert to ordinal reasoning, much like children, non-human primates, and neural networks who are first learning to sequence items into meaningful patterns. Ordinal reasoning may, therefore, be a crucial precursor of recursive pattern processing.