Building hierarchically nested structure by rapid neural sequences

Significance

The human mind excels at building complex, multilevel structures, which we use daily from constructing a sentence to planning a multistep project. Yet, how the brain actually builds these mental hierarchies has remained largely unknown. This study reveals the underlying neural process using magnetoencephalography (MEG). We found that the brain achieves this domain-general capacity through two rapid neural operations, which separately identify an item’s correct level in the hierarchy and arrange that item into its correct order within that level. These findings shed light upon the core neural “rules” that allow us to flexibly generate complex thoughts from simple building blocks, a cornerstone of language, problem-solving, and abstract reasoning.

Abstract

Hierarchically nested structures are fundamental to human cognition, enabling complex behaviors across domains including language, planning, and mathematics. However, the neural mechanisms that enable the flexible construction of these hierarchical structures are poorly understood. Here, we designed a task where participants mentally built sequences with nested, multidepth structures by recursively applying a fixed set of rules. Using magnetoencephalography, we find that the brain constructs nested hierarchies through rapid neural sequences that perform two recurring generative operations. The first operation identifies the hierarchy depth of a symbol and is associated with increased ripple-band power; while the second arranges the symbol into its correct order at that level, a process that scales with the number of depths, also positively correlated with planning time. These results reveal a fundamental neural computation for transforming sensory information into structured representations, which is essential for higher-order cognition.

In daily life, people often use a few basic rules to organize complex tasks. For example, when planning a grocery trip, we might first group items by category (e.g., produce, dairy, grains) and then sequence the whole shopping list according to the store layout. This approach reflects the construction of a hierarchically nested structure, where categories and subcategories are applied recursively (e.g., fruits within produce), creating multiple levels of organization. Similarly, classic problems like the Tower of Hanoi can be solved by breaking them down into simpler subproblems, each solved with the same set of rules (1, 2). These examples highlight our remarkable ability to build multilevel structures from simple principles.

The ability to generate hierarchically nested structures is a domain-general capacity that transcends various cognitive functions (3–6), such as organizing knowledge, decision-making, language processing, and mathematics. A few basic elements or “primitives” (e.g., numbers, words, or musical notes) can be flexibly combined into a vast diversity of complex structures across domains (e.g., equations, sentences, and chords). However, despite the ubiquity and the combinatorial power of hierarchical structures, the neurocomputational mechanisms for constructing such structures remain unclear.

The defining characteristic of a nested structure is the existence of multiple levels of embedding (or “depths”) that organize elements into a hierarchy. Such hierarchical depth allows motifs to be embedded within self-similar motifs (as in a family tree or a fractal pattern), which is a feature absent in simple linear structures. Neuroscientific evidence suggests that the brain is sensitive to these levels of structure. For instance, activity in the inferior frontal gyrus (IFG) is modulated by the number of depths or the complexity of a hierarchical structure in tasks involving language syntax (7–17), musical (18, 19), and motor sequences (20, 21). Moreover, recent neural decoding studies have also demonstrated the compression of spatial and sound sequences into nested repetitions of primitives (22–24), as well as the hierarchical reorganization of complex sequences in working memory (25). However, it is still unknown how the brain actively constructs hierarchically nested structures, depth by depth, in real time.

One potential neural computation for building nested structures is through rapid sequential binding of “structural roles” and “fillers” at each level of the hierarchy. In other words, the brain might assign a generalizable role (i.e., a placeholder for a certain depth level) to a specific item (i.e., the content to fill that role) by activating their neural representations in a particular order (26). For example, to place an object at a given depth of the hierarchy, the neural code for that depth level could be activated immediately before the neural code of this object. This role-filler binding operation would then repeat for each new level, reusing the same set of rules at each depth. Thus, carrying out such binding across successive levels likely requires the brain to rapidly reactivate the relevant neural codes (i.e., structural roles and their fillers) in the correct order and context for each step of the construction process.

In essence, this is reminiscent of “rapid neural sequences” which are sequences of accelerated reactivations observed in rodents (27, 28) and humans (29–31). Notably, these rapid neural sequences do not merely repeat past experiences; instead, they can reorganize objects based on previously learned knowledge or rules (30). That is, the brain can factor out abstract information from specific inputs and use those abstract codes to guide novel sequences. This ability to use abstract codes is thought to be crucial for flexibly binding objects to structural roles in new situations (32), enabling the construction of complex nested hierarchies beyond rote memorization. A recent magnetoencephalography (MEG) study demonstrated that rapid neural sequences could assemble elements into novel combinations, supporting novel problem solving through compositional inference (33). Together, these findings lead to the hypothesis that rapid neural sequences might underlie the construction of hierarchically nested structures in the human brain.

To test this hypothesis, we designed a symbolic sequence generation task where participants applied prelearned rules to build a multidepth hierarchical structure of symbols and produced a linear sequence of pictures mapped onto those symbols. We used MEG to track the neural activity underlying this process, allowing us to observe the depth-by-depth construction of the nested structure at millisecond timescales. Specifically, we predicted this construction process would be implemented by two distinct rapid neural sequences: depth-to-symbol coordination, which adds a new level to the hierarchy by reactivating a depth code precisely before its corresponding symbol, and symbol-to-symbol sequencing that reactivates symbols within the same level in rule-defined order. To examine these predictions, our analysis proceeded in three stages. First, we verified that participants successfully built the required structures by identifying neural evidence of the target picture sequence. Second, to characterize the building blocks of these structures, we tested whether the brain represents task-related knowledge (e.g., depth and symbol) independently of specific pictures, a prerequisite for flexible rule-based structure construction. Finally, we directly tested the two predicted neural sequences by analyzing the relative reactivation timing (lags) of depth and symbol codes during the planning stage, which probes the real-time neural operations the brain uses to construct a hierarchically nested structure.

Results

Rules for Generating Sequences with Nested Structures.

Before presenting the neural findings, we first outline the rules of the sequence generation task. We defined four primitives, each denoted by one of four solid shapes (Fig. 1 A, Left). Each primitive consists of two specific symbols in a fixed order, which are neither interchangeable nor replaceable by other symbols. There are four unique symbols, which are identical to the primitives in shape but are hollow instead. Participants can use these primitives recursively to build multidepth nested structures and generate linear sequences accordingly (Fig. 1 A, Right). The crucial ingredient is that each symbol yielded by a currently “active” primitive can either be emitted as an observable object in the output sequence (increasing the sequence length by one), or can itself be turned into a new primitive (with the same shape). This latter option delays the symbol’s appearance in the output sequence, and expands the depth of the nested structure by one level.

Fig. 1.

Rules for generating sequences with nested structures. (A) The rules for generating sequences with nested structures consist of four primitives (i.e., solid shapes). Each primitive consists of two specific symbols (i.e., hollow shapes) in a fixed order. The sequence is generated through the recursive use of primitives, whereby a nested structure is built depth by depth. Importantly, to facilitate neural decoding, the structure unfolds based on the constraint that it only grows with the symbol whose shape is different from the primitive above. Moreover, symbols are mapped to distinct pictures, which results in a sequence of symbol-bound pictures. Note that while we illustrate a depth-3 structure that produces a length-4 sequence, these rules can be applied to generate sequences of arbitrary length (or depth). (B) Two operations repeated at each depth for structure construction. The depth-to-symbol coordination arranges symbols to create a new depth (i.e., green arrows), and the symbol-to-symbol sequencing orders the two symbols at the same depth (i.e., red arrows) according to the primitives. A solid green arrow points to a symbol that is not further expanded from the current depth (i.e., final symbol), while a dashed green arrow points to an intermediate symbol that will be turned into a new primitive and dropped from the current depth to add a new depth. In principle, the generation process of such a nested structure can be expressed as S(N_depth+1) = DS[S(N_depth)], where S(N) is a depth-N structure, and DS[] denotes the depth-to-symbol coordination that expands the structure by one depth.

To facilitate subsequent decoding analyses, we added a constraint to ensure that the sequence generation process was fully determined once the starting primitive and the total number of depths were specified. This constraint specifies that the nested structure could only grow along a symbol with a shape distinct from that of the primitive above, which also prevents trivial repetitions of the same primitive at consecutive depths. For example, in the depth-3 structure shown in Fig. 1A, starting with the “square” primitive at Depth I, it is the “triangle” symbol rather than the square symbol that becomes a new primitive which governs Depth II. Despite this constraint, the generation process retained flexibility by allowing a random starting primitive or a varying total number of depths, or both (as in the flexible-depth sequence generation task below). We use Roman numerals to denote levels of depth (e.g., Depth III, larger numbers indicating greater depth from the starting primitive) and Hindu-Arabic numerals to denote the total number of depths (e.g., a “depth-3” structure has three levels of depth). Finally, because each symbol is uniquely mapped to a distinct picture, a depth-N structure produces an output sequence of N+1 pictures (Fig. 1 A, Right). Importantly, the same symbol can be associated to different pictures across sequences, enhancing the flexibility of sequence generation and enabling the decoding of abstract codes (e.g., depth, symbol) factorized from specific picture stimuli.

We hypothesized that constructing hierarchically nested structures in the brain could be achieved by two recurring operations that are applied to create each new depth (Fig. 1B). First, along the vertical direction, depth-to-symbol coordination serves as the core operation that precisely locates appropriate symbols at each depth. As shown in the depth-3 structure in Fig. 1B, starting from a given depth, a solid green arrow points to an observable “final symbol” that remains at the current depth, while a dashed green arrow points to an “intermediate symbol” which becomes a new primitive and is dropped from the current depth (e.g., the triangle symbol, initially at Depth I, becomes a primitive which governs Depth II). Second, along the horizontal direction, symbol-to-symbol sequence (depicted by red arrows in Fig. 1B) organizes the order of two symbols allocated to the same depth, as defined by the corresponding primitive (Fig. 1 A, Left). Together, these two operations add a new depth to the nested structure at a time. By leveraging recent advances in detecting rapid neural sequences in humans using MEG (34), we designed our analyses to find neural evidence of these two hypothesized operations during nested structure construction.

Experiment Procedure and Behavioral Results.

To verify that participants could indeed learn and use these sequence generation rules, they were required to perform both a fixed-depth and a flexible-depth sequence generation task. We conducted a 2-d experiment with 26 participants. On Day 1, participants learned the rules for generating sequences with nested structures through three phases (Fig. 2 and SI Appendix, Methods). During this learning process, they were not exposed to any picture stimuli and only needed to correctly generate one length-4 sequence to ensure that they learned the rules without overtraining.

Fig. 2.

Experimental design and procedure. On Day 1, participants learned the rules for generating sequences with hierarchically nested structures. On Day 2, participants conducted sequence generation tasks in MEG. To utilize neural decoding, participants learned a mapping between symbols and pictures which was used to output a picture sequence. These pictures were first presented randomly during the functional localizer prior to any other task, which ensures that picture classifiers trained from the functional localizer do not contain any task-related knowledge (thereby do not bias the following neural sequence analyses). In each of the three runs of the fixed-depth sequence generation task (i.e., three depths), the output sequence consists of four distinct pictures mapped to different symbols. Each run ended up with three knowledge probe blocks where participants were quizzed on the symbol, depth, and order of the four pictures in this run separately. There was a 5-min resting state scan before the functional localizer (i.e., PRE rest), used as a control for the 5-min planning during fixed-depth sequence generation. Finally, in the flexible-depth sequence generation task, the symbol–picture mapping learned in the 3rd run of the fixed-depth task was used to generate sequences flexibly given a random combination of a total depth number and a starting primitive in a self-paced manner.

On Day 2, participants performed the sequence generation tasks inside the MEG scanner (Fig. 2, Right). This session began with a 5-min rest (i.e., PRE rest) for baseline measures, followed by a functional localizer during which participants passively viewed 12 distinct pictures presented repeatedly in random order. Importantly, this localizer was done before sequence generation, in order to train picture classifiers based on brain responses with no task-related knowledge.

After the localizer, participants completed three runs of a fixed-depth sequence generation task. In each run, participants learned the one-to-one mapping between four symbols and four specific pictures (note that the 12 pictures were randomly and equally divided into three sets, each used in one run). Then, a randomly selected starting primitive was presented, and participants mentally generated a depth-3 structure and produced a length-4 picture sequence based on the learned symbol–picture mapping. After a 5-min planning stage, participants were quizzed on the symbol, depth, or order for each picture in the sequence, presented in three separate knowledge probe blocks. The accuracy of probing symbol, depth, and order codes across runs was 98.4% ± 0.3% (mean ± SE), 94.8% ± 1.1%, and 98.1% ± 0.3%, respectively, suggesting participants successfully generated the required depth-3 structure and the corresponding picture sequence (see SI Appendix, Fig. S1 for more behavioral results).

Next, participants performed a flexible-depth sequence generation task, which demanded greater generalization. Based on the symbol–picture mappings learned in the final run of fixed-depth sequence generation, participants were asked to generate picture sequences given a random combination of starting primitive and number of depths (i.e., 3, 4, or 5), including a total of 12 trials (4 starting primitives × 3 total depths). In each trial, participants mentally generated the required picture sequence during a self-paced planning stage, then provided the output sequence via button press. Participants correctly generated 11.0 ± 0.3 sequences and spent 51.1 s ± 5.0 s for planning, with longer times required for sequences with more depths (one-way ANOVA, F(2, 71) = 10.3, P = 1.2 × 10⁻⁴). These results confirm that participants learned the nested structure rules and could flexibly construct new sequences of varying depths.

Neural Evidence of Mentally Generated Sequences with Nested Structures.

The successful behavioral performance in the fixed-depth sequence generation task indicates that participants correctly constructed the required structures. Therefore, we proceeded to search for the neural evidence of the sequence of pictures corresponding to the constructed structure, i.e., sequential reactivations of pictures determined by the underlying nested structure. To this end, we first trained a binary one-vs.-all classifier for each picture using sensor-level MEG data from the functional localizer (SI Appendix, Methods). These classifiers captured each picture’s distinct evoked neural responses. The stimulus code of each picture is operationally defined as the classifier coefficients over all sensors (i.e., a 1 × N_sensors coefficient vector) (Fig. 3A). The peak cross-validation decoding accuracy of 35.1% ± 1.8% was achieved at 190 ms after picture onset (Fig. 3 B and C; see SI Appendix, Fig. S2 for performance of all classifiers), which was significantly above the permutation threshold. Thus, classifiers trained at this time point were used to obtain a time-course of reactivation probability for each picture during the 5-min planning stage of fixed-depth sequence generation.

Fig. 3.

Multivariate decoding of pictures and their sequential reactivations determined by a multidepth nested structure. (A) Examples of whole-brain neural activity evoked by a picture (Left) and its decoding performance (Right). A binary classifier was trained for each of the 12 pictures. The predicted probability of the presented picture is plotted against those of the other pictures. (B) Temporal generalization plot of mean decoding accuracy for all participants. (C) Decoding accuracy obtained when classifiers were trained and tested on the same time point (i.e., diagonal pattern of the temporal generalization). The vertical dashed line indicates peak decoding accuracy at 190 ms post stimulus onset. (D) Examples of sequential reactivations of pictures (i.e., picture sequence determined by a depth-3 structure) during the planning stage of fixed-depth sequence generation are shown from one participant for visualization purposes. Each row depicts reactivation probabilities of the four states at a given time point. (E) No significant picture sequence was observed during PRE rest. (F) During fixed-depth planning, the target picture sequence was detected in the forward direction with its peak effect at 150 ms state-to-state time lag. (G) A positive time effect of picture sequence was detected during fixed-depth planning. A suprathreshold positive beta estimate indicates that the sequence strength increases with time, and vice versa for a negative beta estimate. The red line represents the true data, while the dashed lines denote separately the 2.5th and 97.5th percentiles of the null distribution obtained from permutations. In (A, C, E, and F), the shaded areas represent SE, the horizontal dashed line is the permutation threshold (SI Appendix, Methods).

Using the temporally delayed linear modeling framework (34), we assessed the degree to which picture reactivations followed the expected output sequence during the planning stage in each run of the fixed-depth sequence generation task. For the length-4 picture sequence in each run (e.g., A→B→C→D), we evaluated the sequence strength of all permitted pairwise state-to-state transitions (i.e., A→B, B→C, C→D) across various time lags (i.e., the time interval between the reactivations of two successive states), ranging from 10 ms to 600 ms, in both forward and backward directions. The overall sequence strength was estimated by averaging across all pairwise transitions and runs. To control for multiple comparisons due to the various time lags tested, the significance of sequence strength was determined via a nonparametric permutation test where state-to-state transitions were shuffled (SI Appendix, Methods).

Our results showed a significant effect of picture sequence in the forward direction, peaking at 150 ms time lag (see Fig. 3D for results from one participant, Fig. 3F for the group-level results, and SI Appendix, Figs. S3 and S4 for supplementary results). In addition, we tested whether the strength of this picture sequence changed over the 5-min planning period. By modeling the time effect using a “reactivation × time” regressor (34), we evaluated time modulation on sequence strength within the time lag ROI where a significant sequence effect was found. We observed a positive time effect, indicating that sequence strength increased as time progressed (Fig. 3G). As a control, no evidence of the correct picture sequence was found during the PRE rest period, when participants had not yet been exposed to any pictures (Fig. 3E). Neural evidence of the output picture sequence confirmed that participants successfully built the intended nested structure. Moreover, the neural representation of the output sequence becomes more robust as the nested structure is progressively built during the 5-min planning stage.

Neural Representation of Task-Related Knowledge.

After identifying the sequential reactivations of pictures ordered by the underlying nested structure, we investigated how the brain represents the abstract task-related knowledge associated with each picture. In our task, each picture is mapped to a symbol, which in turn has a specific depth in the hierarchy and an order in the output sequence. Successful picture sequence generation requires participants to correctly map pictures to symbols, allowing them to arrange pictures according to their associated symbols in the nested structure. This means that task-related knowledge of each symbol, such as its depth within the nested structure and its order in the output sequence, was also assigned to the corresponding picture. We asked whether the brain encodes these abstract codes (i.e., symbol, depth, and order) for each picture, in addition to encoding the picture’s sensory properties. Such encoding would imply that participants have factorized task-related knowledge from the visual input, facilitating them to treat new pictures according to learned rules (26, 30).

To investigate this, we compared neural responses to pictures during the functional localizer (before any picture–symbol mapping was learned) with those during the knowledge probe after sequence planning (i.e., a conjunctive representation of sensory features and task-related knowledge). Intuitively, if two different pictures share the same abstract code (e.g., placed at the same depth), their neural representations should become more similar than they were in the localizer when such knowledge was absent. We formalized this analysis using representational similarity analysis (RSA) (35). The neural similarity between each pair of pictures in the functional localizer was used as a baseline accounting for their difference purely in sensory features, which was then subtracted from those calculated in the knowledge probe (36). Hence, the change of neural similarity could be attributed to task-related abstract codes. That is, neural similarity between two pictures is expected to increase if they are mapped to the same symbol, are placed at the same depth, or share the same order in output sequences.

We calculated the difference between the representational dissimilarity matrices (RDMs) derived from the neural data of 12 pictures in the functional localizer and each of the three knowledge probes (i.e., symbol, depth, and order) separately, yielding, for each probe, a 12 × 12 RDM for each time point after stimulus onset. Then, for each probe and time point, we regressed the change of neural dissimilarity (i.e., RDM _{knowledge probe} – RDM _{functional localizer}) on three model RDMs that separately captured differences in symbol, depth, and order codes assigned to these pictures. We found significant effects of all three abstract codes when they were explicitly probed (e.g., symbol code effect when symbol was probed), with symbol code and depth code leading order code in time (Fig. 4A). Crucially, we simultaneously included the three model RDMs as regressors to account for the contributions of all abstract codes, regardless of which one was probed.

Fig. 4.

Neural representations of task-related knowledge. (A) During the knowledge probe after fixed-depth sequence generation, pictures sharing the same task-related knowledge (i.e., symbol, depth, or order) showed an increased similarity in their evoked neural responses (from all MEG sensors) when participants were explicitly probed on such knowledge (in contrast to the neural similarity in the functional localizer). Colored horizontal lines denote periods of significant effects identified by permutation tests corrected with cluster-wise P < 0.05. The shaded areas represent SE. (B) Effect of task-related knowledge across different probing tasks. The increased neural similarity due to each abstract code was calculated in all three knowledge probes, as shown by each set of three bars (mean ± SE) based on the same temporal ROI where a significant effect of this code was observed, indicated by horizontal lines in (A). Each dot indicates a result from one participant, *P < 0.05, **P < 0.01. (C) Multivariate decoding of symbol, depth, and order. For each type of abstract code, one-vs.-all binary classifiers were trained and tested at the time point when the most significant effect was identified by RSA (i.e., highest t-value at 580 ms for symbol, 570 ms for depth, 820 ms for order).

Next, we asked whether an abstract code was represented even when it was not probed explicitly (e.g., effect of symbol code when depth was quizzed). During the knowledge probe of each abstract code, we examined effects of the other two codes (e.g., effects of depth and order codes during the symbol probe) within the time window where these two codes exhibited significant effects when they were probed explicitly (e.g., effects of depth and order codes when they were separately probed, as indicated by the relevant horizontal bars in Fig. 4A). We found that increased neural similarity due to the same symbol or depth code remained in other probing tasks. However, the effect of order code was absent when participants were quizzed on depth or symbol code (Fig. 4B).

These results indicate that task-related knowledge, instantiated by the abstract codes, was factorized from sensory features. This is essential to apply the learned rules to novel pictures for generating new sequences with nested structures, wherein generalizable structural roles are flexibly bound to symbols and their associated pictures. Intriguingly, neural representation of the underpinning nested structure, as characterized by the symbol and depth codes, seems to be prioritized over that of the order code in the output sequence which might be needed only for the linearization of the nested structure.

Coordination between Depths and Symbols Guiding Nested Structure Construction.

Having revealed the neural evidence of successful picture sequence generation and the abstract codes underpinning the nested structure, we turned to our core hypothesis that the construction of hierarchically nested structures could be achieved through rapid neural sequences. We first asked whether the brain builds each new depth of the hierarchy via depth-to-symbol coordination. Specifically, does the neural code for a depth reactivate immediately before the code for a symbol when the nested structure extends by one level? This depth-to-symbol coordination is supposed to be the backbone of constructing the hierarchy, repeatedly binding a structural role (depth) to a filler (symbol) at each level.

In the context of depth-to-symbol coordination, symbols can be classified into two types. “Intermediate symbols” are those with a different shape from their primitives above (e.g., the hexagram symbol at Depth I in Fig. 5A). They become new primitives and function as scaffolds along which the structure incrementally unfolds. In contrast, “final symbols” are those that remain at the same depth throughout and are eventually linearized to produce the output sequence (e.g., the triangular symbol at Depth I in Fig. 5A).

Fig. 5.

Coordination between depths and symbols for building nested structures. (A) An example of transitions from depths to symbols and their associated pictures (i.e., depth-to-symbol coordination), assigning each depth to appropriate symbols to generate a depth-3 structure and produce a length-4 picture sequence. Intermediate symbols refer to those that are later turned into primitives for further expansion, while final symbols are those that remain at the same depth throughout the generation process. (B) Examples of depth-to-symbol coordination during the planning for a depth-3 structure are shown from one participant for visualization purposes. Each row depicts reactivation probabilities of all states at a given time point. (C) Depth-to-symbol coordination was not observed during PRE rest when participants were not engaged in any task. (D) During the construction of a depth-3 structure in the fixed-depth task, significant effect of depth-to-symbol coordination (including both intermediate and final symbols) was detected with its peak effect at 80 ms time lag (see SI Appendix, Fig. S5 A and B for depth-to-symbol coordination for final and intermediate symbols separately). (E) The coordination between depths and intermediate symbols decreased with time, as indicated by a negative beta estimate. The red line represents the true data, the dashed lines denote separately the 2.5th and 97.5th percentiles of the null distribution obtained from permutations. (F) Depth-to-symbol coordination was found for each depth during the generation of a depth-3 structure. (G) Compared to the baseline (100 to 50 ms before event onset), ripple band power (90 to 150 Hz) increased at the onset of depth-to-symbol coordination events when the depth was activated. Black contour denotes the significant cluster (cluster forming threshold P < 0.01, cluster-wise P < 0.05). See SI Appendix, Fig. S7A for results in an extended epoch and distribution over sensors. (H) Within a 200-ms time window following the depth-to-final-symbol coordination events during the fixed-depth task, the target picture that matches the final symbol is significantly more likely to reactivate compared to each of the three mismatched pictures (two-tailed paired t test, FDR corrected P < 0.001 for all comparisons). In (C and D), the dashed line is the permutation threshold after controlling for multiple comparisons (SI Appendix, Methods), the shaded areas represent SE. In (F and H), each dot indicates a result from one participant. *P < 0.05, **P < 0.01.

Similar to the picture sequence analysis, we investigated the coordination between depths and symbols by searching for neural evidence of sequential reactivations of a certain depth code and the symbol assigned to this depth, i.e., a “depth-to-symbol sequence” in which a depth code is reactivated before the symbol at this depth. To this end, using MEG data during the knowledge probe, we trained classifiers for the abstract codes of depth and symbol separately at the time point when their peak effects were identified by RSA (Fig. 4A). The training procedure closely mirrored that of the picture classifiers, with one key difference – we redefined picture labels to represent abstract codes (e.g., depth or symbol) instead of the stimulus identity labels (e.g., “dog” or “car”; see SI Appendix, Methods for details). As depicted in Fig. 4C, all abstract codes were successfully decoded from neural activity. Crucially, prior to calculating the reactivation of abstract codes from continuous MEG signals, the stimulus codes of pictures sharing the same abstract code were regressed out from that abstract code, as in a related study (30). This step reduced potential confounds from sensory features by making abstract codes orthogonal to conjunctive sensory representations.

Depth-to-symbol coordination was quantified by averaging all pairwise transitions permitted in the depth-3 structure (Fig. 5A) across various time lags for each run in the fix-depth sequence generation task. During the 5-min planning, we found significant depth-to-symbol coordination when both intermediate and final symbols were included (Fig. 5D; see also Fig. 5B for example results from one participant). Reactivation of a depth code regularly preceded that of symbols at this depth with its peak effect identified at 80 ms time lag. Crucially, this depth-to-symbol coordination was found when we examined each depth separately (one-tailed one-sample t test, Depth I: t(24) = 2.53, P = 0.009; Depth II: t(24) = 2.54, P = 0.009; Depth III: t(24) = 2.45, P = 0.011; see Fig. 5F), suggesting that this operation was repeated each time the structure expanded one level deeper. While the decreasing trend across depths was not statistically significant, it implies that initial depths may be rehearsed more frequently during the incremental construction of the nested structure, potentially because participants might restart from Depth I each time a new depth was added. Besides, depth-to-symbol coordination was also detected when intermediate and final symbols were modeled separately (SI Appendix, Fig. S5 A and B). However, no depth-to-symbol sequence was detected during the PRE rest (Fig. 5C).

As depth-to-symbol coordination occurred to build the nested structure over time, the scaffolding between depths and intermediate symbols is expected to decay since an intermediate symbol is dropped and becomes the final symbol at the depth one level deeper. Indeed, we found a negative time effect on depth-to-intermediate-symbol coordination (Fig. 5E), reflecting a declining demand for such temporary transitions as the structure was gradually completed. In contrast, no time effect was found for the coordination between depths and final symbols, implying that it remained relatively stable throughout the nested structure construction.

In both rodent and human studies (30, 37), a power increase of neural oscillations in the ripple band was observed during rapid neural sequences. Here, we examined power changes in ripple frequency band during rapid neural sequence events such as depth-to-symbol coordination, compared to an event-free baseline (from 100 to 50 ms before sequence event onset). We identified rapid neural sequence events by searching for moments with high reactivation probability of one state (e.g., a depth code) followed by the next state (e.g., a symbol) in the sequence with a specific time lag (SI Appendix, Methods). As shown in Fig. 5G, we observed a significant power increase in ripple frequency band (90 to 150 Hz) averaged across sensors, at the onset of depth-to-symbol coordination events when the depth code was reactivated (including both intermediate and final symbols; see SI Appendix, Fig. S7A for relevant results in an extended epoch and distribution of ripple-band power increase over sensors). In contrast, no significant power increase was observed in the same frequency band at the onset of picture sequence events (SI Appendix, Fig. S7B).

Finally, to generate the output picture sequence, one must know where each picture is placed in the multidepth structure, as pinpointed by the symbol mapped to this picture. This could be achieved by reactivating the associated picture immediately after the coordination between a depth code and the final symbol, effectively stitching this picture onto the multidepth structure. If so, a picture mapped to the preceding final symbol would be more likely to reactivate compared to mismatched pictures in the same output sequence. For example, as shown in Fig. 5A, following “Depth II → Hexagram”, “scissors” (a match) is expected to reactivate more frequently compared to the other three pictures (Fig. 5H, Left). To test this possibility, we counted the number of reactivation events for each picture within a 200-ms time window following the offset of a depth-to-final-symbol event (SI Appendix, Methods). Indeed, after the reactivation of a final symbol, the reactivation frequency of the matched picture was significantly higher than that of each mismatched picture (two-tailed paired t test, FDR corrected P < 0.001 for all comparisons) (Fig. 5H, Right).

These results support depth-to-symbol coordination as the core operation of hierarchical construction. At each depth, the brain enacts a rapid neural sequence in which the depth code precedes the symbol code. This self-similar operation was observed across all depths, which is effectively a role-filler binding that places the correct symbols at each depth. Moreover, immediately after a final symbol is assigned, the matching picture is preferentially reactivated, integrating that item into the emerging structure, thereby producing the output picture sequence.

Symbol-to-Symbol Sequence during Nested Structure Construction.

In addition to the precise coordination between depths and symbols, another hypothesized operation needed to complete the hierarchically nested structure is the ordering of two symbols at each depth ruled by the primitives (Fig. 1B). To examine this process, we investigated neural evidence for such symbol-to-symbol sequences (Fig. 6A). The following analyses used a 60- to 90-ms time lag ROI where stronger forward than backward symbol-to-symbol sequencing was observed during the fixed-depth sequence planning (one-tailed paired t test P < 0.05; see SI Appendix, Fig. S6 for results across more time lags); this is also consistent with the significant time lags identified for depth-to-symbol coordination (Fig. 5D). During the 5-min PRE rest, no symbol-to-symbol sequence was observed (Fig. 6 B, Left). In contrast, during the 5-min fixed-depth sequence planning, we found symbol-to-symbol sequence in the order defined by the primitives (one-tailed one-sample t test, t(24) = 1.72, P = 0.049) (Fig. 6 B, Middle). Similarly, symbol-to-symbol sequence was also observed during the planning stage in flexible-depth sequence generation task (one-tailed one-sample t test, t(24) = 2.11, P = 0.023) (Fig. 6 B, Right), where each trial involved a random combination of starting primitive and total depth.

Fig. 6.

Symbol-to-symbol sequence during construction of nested structures with fixed and flexible depths. (A) Illustration of symbol-to-symbol sequences, i.e., the sequential reactivations of two symbols defined by each primitive. (B) Symbol-to-symbol sequence in the order defined by primitives (i.e., yellow arrows in A) was found during the planning stage of both fixed-depth and flexible-depth sequence generation tasks (uncorrected for the three periods tested). No symbol-to-symbol sequence was observed during PRE rest. Note that symbol-to-symbol sequence was evaluated by considering symbols at all depths in a nested structure together across runs or trials. Error bars represent SE. (C) Effect of depth on symbol-to-symbol sequence strength during flexible-depth planning, i.e., sequence strength tended to increase with the number of depths in the nested structure. (D) Effect of depth on the time effect of symbol-to-symbol sequence during flexible-depth planning. The extent to which sequence strength increased with time was greater for nested structures with more depths. (E) The mean strength of symbol-to-symbol sequence was positively correlated with the planning time during flexible-depth sequence generation. (F) The strength of the “scaffolding” depth-to-intermediate-symbol coordination during fixed-depth sequence generation was positively correlated with planning time during flexible-depth sequence generation. (G) The strength of depth-to-final-symbol coordination during fixed-depth sequence generation did not correlate with planning time of flexible-depth sequence generation. In (B and E–G), each dot represents a result from one participant. In (C and D), each dot represents a result from a single trial from a participant in the flexible-depth sequence generation task, horizontal bars indicate the depth effect estimated by linear mixed-effects modeling, solid black lines denote linear fits with error bars representing 95% CI. In (E–G), the shaded areas represent 95% CI. *P < 0.05.

Next, we asked whether symbol-to-symbol sequence strength was modulated by the number of depths in the nested structure. Using linear mixed-effects modeling, with participants as a random variable, we found that both the sequence strength and its time effect tended to increase with the number of depths (F = 4.74, P = 0.03, Fig. 6C; F = 4.81, P = 0.028, Fig. 6D). Post hoc analyses revealed significant symbol-to-symbol sequences for depth-4 and depth-5 structures only (one-tailed one-sample t test, depth-4, t(23) = 3.4, P = 0.001; depth-5, t(24) = 2.04, P = 0.027), suggesting a greater demand for this operation in these two novel conditions with more depths. Here, the absence of a significant effect for depth-3 structures might be due to potential adaptation to the depth-3 structures constructed in the prior fixed-depth task. Besides, the self-paced planning times in the flexible-depth task (SI Appendix, Fig. S1E) was substantially shorter than the fixed 5-min planning in the fixed-depth task. This reduced planning time might cause signal-to-noise ratio limitation for symbol-to-symbol sequence detection.

During the flexible-depth sequence generation, we found that the strength of symbol-to-symbol sequence was positively correlated with planning times across participants (Pearson’s correlation r = 0.576, P = 0.003) (Fig. 6E). Controlling for depth with a trial-level linear mixed-effects model, we found that stronger symbol-to-symbol sequence strength was still significantly associated with longer planning times (β = 346.22, P = 0.03). This suggests that participants who were less efficient in constructing nested structures with flexible levels of depth relied more on symbol-to-symbol sequences for rehearsal during the self-paced planning. Additionally, the mean planning time during flexible-depth sequence generation was correlated with the coordination between depths and intermediate symbols (Pearson’s correlation r = 0.541, P = 0.005) (Fig. 6F), but not with the coordination between depths and final symbols (Pearson’s correlation r = 0.21, P = 0.314) (Fig. 6G), during fixed-depth sequence planning. This suggests that less efficient participants, who made greater use of the scaffolding depth-to-intermediate-symbol, also required more time to complete the challenging flexible sequence generation task. Note that we did not evaluate depth-to-symbol coordination in the flexible-depth task since there were no probing data for training classifiers of deeper structural levels (i.e., Depths IV and V).

Overall, these results provide neural evidence for another operation of hierarchical construction, that is, rapid symbol-to-symbol sequencing that organizes symbols at the same depth, with its engagement increasing with the number of depths in the nested structures. Together, these symbol-to-symbol sequences operate alongside depth-to-symbol coordination to ensure that each new depth is added correctly and that symbols at each depth are ordered for accurate structure construction.

Source Localization of Rapid Neural Sequences.

We also conducted source localization to explore brain regions associated with the rapid neural sequences for the nested structures and corresponding picture sequences. Epoched data were created by aligning continuous MEG data, recorded during the fixed-depth sequence planning, to the onsets of either picture sequences or depth-to-symbol coordination events. Time-domain beamforming was used to source-localize these epoched data (SI Appendix, Methods).

To reveal the differences between potential neural sources related to the representation of output picture sequences and those involved in the core operation for building nested structures, we compared the source-localized results of picture sequences and depth-to-symbol coordination within a time window from −50 ms to 50 ms relative to sequence onset (see SI Appendix, Fig. S8 for their main effects). We found picture sequences exhibited stronger activation in the bilateral primary visual cortex (at ~10 ms post sequence onset) (Fig. 7A), while depth-to-symbol coordination shows stronger reactivation in bilateral IFG (at −33 ms before sequence onset) and the left motor cortex (at 10 ms post sequence onset) (Fig. 7B). All results survived nonparametric spatiotemporal cluster-based permutation tests for multiple comparison correction (voxel-wise P < 0.01 and cluster-wise P < 0.001; see SI Appendix, Methods).

Fig. 7.

MEG source localization at the onset of rapid neural sequences. Contrasts of (A) picture sequence > depth-to-symbol coordination and (B) depth-to-symbol coordination > picture sequence. Statistical analyses were conducted within a time window from −50 ms to 50 ms relative to sequence onset. The peak t-value denotes the maximum t-statistic observed within this interval, and peak time indicates its latency. Significance was determined by nonparametric spatiotemporal cluster-based permutations (voxel-wise P < 0.01 and cluster-wise P < 0.001).

It is important to note that MEG source localization results should be interpreted cautiously due to the challenges of the ill-posed inverse problem. To mitigate these issues, we used high-resolution structural images of individual participants to create more accurate forward models. The identified anatomical sources are primarily located on the cortical surface (e.g., IFG, motor, and visual cortex), which is closer to MEG sensors and generally results in more reliable reconstruction (38, 39).

Discussion

The ability to build nested structures is essential for diverse cognitive functions, from comprehending sentences with embedded clauses to formulating multistep solutions to complex problems. This study investigated how the brain might achieve this ability, and whether it could be implemented via rapid neural sequences. We asked two primary questions: 1) Do rapid neural sequences underlie the depth-by-depth construction of a hierarchical structure? 2) If so, what are the specific neural representations and operations involved, and how do they support flexible, rule-based generation of novel structures? To address these questions, we used a sequence generation task and MEG decoding methods. Our results provide evidence that rapid neural sequences serve as a plausible neurocomputational mechanism for building nested structures, achieved through precise coordination between the neural representations of structural roles (i.e., depths) and their corresponding fillers (i.e., symbols). The depth-to-symbol coordination and symbol-to-symbol sequencing create each depth in the nested structure, which manifest as rapid sequential reactivation of depth and symbol codes and unfold at the scale of tens of milliseconds.

During the construction of a hierarchically nested structure, depth-to-symbol coordination is the core operation that forms the backbone of the multidepth structure. The reactivation of a depth code (representing the current level of nesting) guides that of specific symbol codes (representing the objects at that level), potentially binding structural roles with their role-fillers. Such a coordination is facilitated by the factorization of abstract codes (i.e., depth and symbol) from the stimulus code (26), which allows efficient construction of various nested structures based on a small set of rules, and more broadly, flexible generalization of schematic knowledge to new instances, such as generating picture sequences with novel stimuli in this study.

Interestingly, during the fixed-depth sequence planning, the coordination between depths and intermediate symbols began to diminish as time progressed, in contrast to the strengthening of the output picture sequence over the same period. These findings imply that, as the nested structure is gradually established, the focus shifts from intermediate symbols (along which the structure grows) to final symbols at the terminals of the structure, which directly inform the order of associated pictures in the output sequence. Furthermore, participants who exhibited greater scaffolding depth-to-intermediate-symbol coordination during the fixed-depth task also took longer to complete the flexible-depth task (Fig. 6F), indicating consistent efficiency in applying the learned rules across both sequence generation tasks.

The symbol-to-symbol sequence, which orders two symbols for each new depth expanded by a primitive, is prominent in the flexible-depth sequence generation task where the same primitive may be used repeatedly (i.e., depth-4/5 structures). This is evidenced by increased strength of symbol-to-symbol sequence in scenarios with more depths (Fig. 6C), which also tends to increase over time (Fig. 6D). The symbol-to-symbol sequencing occurs in tandem with depth-to-symbol coordination, collectively completing the nested structure underlying the linear output sequence. Taken together, these two operations are both implemented through rapid neural sequences. They provide a plausible neural mechanism for the dynamic, depth-by-depth construction of hierarchically nested structures.

When comparing the plausible neural sources related to picture sequence and depth-to-symbol coordination, we found that the latter elicited stronger activity in the bilateral IFG and the left motor cortex (Fig. 7B), both of which are associated with the complexity of encoding linear sequences with nested structures (24). During the structure construction, working memory is likely involved in maintaining the rules and holding the intermediate representations. Thus, IFG may track the rules to be applied at the current depth and the symbols already placed within the nested structure (25). Notably, the neural representations of hierarchical structures have been extensively studied in the context of syntactic processing in natural language, with converging evidence highlighting a crucial role for the IFG (7–17). However, the multifaceted nature of natural language, such as the interplay between syntax and semantics (40–42), makes it challenging to isolate the neural computations of the domain-general structure-building processes (43, 44). Thus, we designed a symbol sequence generation task based on an artificial grammar (45) to avoid the confounds of natural language and directly investigate the core neurocomputational processes underlying nested structure construction.

The increase in ripple band power identified at the onset of depth-to-symbol coordination aligns with recent MEG studies on rapid neural sequences in humans (30, 36), which resembles hippocampal neural replay in terms of sequential reactivations (37). The propagation of neural signals between the hippocampus and cortical areas is crucial for cognitive functions during both sleep and awake states (37, 46, 47). Recent research has identified a role for generative replay in the hippocampal-prefrontal circuit in human constructive inference (33). However, due to inherent limitations of MEG source localization, particularly for high-frequency neural oscillations and subcortical regions (39, 48), it remains technically challenging to confirm whether the observed ripple-band activity originated in the hippocampus. Future research is needed to further investigate this possibility.

Although recent rapid neural sequence studies in humans have shown that objects are spontaneously organized based on learned rules (30, 36, 49), these studies have mostly focused on linear, nonhierarchical sequences. Our study extends this by demonstrating that rapid neural sequences also support the efficient and flexible construction of nested structures. This aligns with the idea that rapid neural sequences facilitate compositional computation by binding objects to generalizable roles and chaining role-bound objects into novel complex structures (32). Our findings reveal that symbols are precisely organized at each depth through rule-based operations (e.g., assigning a symbol the role of “a placeholder at Depth II”), potentially enabling the remarkable productivity required for composing unbounded structures (50, 51).

The “language of thought” hypothesis posits that mental representations possess a language-like structure, enabling the composition of complex thoughts from simpler concepts (52, 53). A recent study decoded neural representations of primitives from the simplest mental program encoding a spatial sequence in human working memory (23), supporting both pattern compression (54) and next-location prediction as the sequence unfolds. In line with this, our findings provide further neural evidence for the repetitive embedding of self-similar representations to generate nested structures of varying depths and contents.

The nested structure construction in our sequence generation task reflects core principles of recursive computation, including self-reference, hierarchical structure, and repeatability (53). However, we acknowledge that this task simplifies the complexity of real-world recursive processes. The limited number of depths and the relatively structured generation process are due to constraints such as working memory capacity and experimental feasibility. Crucially, these simplifications allow us to establish a clear decoding target for the neural sequence analysis. While we cannot completely exclude the possibility of rote memorization during sequence generation, particularly for simpler depth-3 sequences, the successful construction of novel flexible-depth structures, together with increased planning times and neural sequence strength for structures with greater depths, provides converging evidence that participants actively constructed nested structures based on rules.

In summary, our findings reveal a direct neurocomputational answer to the questions regarding how the brain builds hierarchically nested structures. We found compelling evidence that rapid neural sequences serve as a plausible neural mechanism for nested structure construction. Specifically, we show that depth-to-symbol coordination and symbol-to-symbol sequencing are orchestrated through precise sequential reactivations according to learned rules. These rapid neural sequences support efficient, rule-based binding of structural roles and fillers, advancing our understanding of hierarchical construction beyond linear sequences. More broadly, these findings offer insights into the neural basis for the flexible use of primitive rules that underlies complex human behaviors.

Materials and Methods

Twenty-six healthy right-handed participants with no history of psychiatric or neurological disorders were recruited for this study. One participant was excluded due to technical issues during MEG recording. The final sample consisted of 25 participants (22.0 ± 0.5 y old, mean ± SE; 15 females). All experimental procedures were approved by the Peking University Institutional Review Board, and all participants provided written informed consent. MEG and structural MRI data were collected at the Center for MRI Research, Peking University, Beijing, China. Further details on experimental procedures and methods are provided in SI Appendix, Methods.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

The preprocessed data and code related to this study is available from the OSF repository (https://osf.io/q8s4b/) (55).