Emergent representations of graphical structure in mechanistic neural models of causal judgment

Abstract

Humans have a remarkable ability to judge causal relationships from a limited number of unreliable observations. Past work on causal cognition has largely focused on normative accounts of human behavior, leaving unknown how biologically plausible neural systems could learn causal relationships from observations and update their representations of causal structure with additional evidence. Here, we leverage task-optimized recurrent neural networks to discover candidate implementation-level neural mechanisms of causal judgment. We propose a novel cognitive task in which a subject observes stochastic samples from an unknown causal structure (e.g. among variables A, B, and C with unknown causal relationships), and must judge whether a specific causal relationship is present given a query (e.g. “true”). We found that, after training, recurrent neural networks perform the task with high accuracy, adopt strategies that incorporate the behavior of non-queried variables to form their judgments, and, despite being trained only on pairwise queries (“true”, “true”, etc), form implicit beliefs about the complete graphical structure underlying the observations. Lastly, we use dynamical systems analysis to identify a set of low-level neural mechanisms that implement causal judgment and representation of causal graphical structure. Together, these findings lay the groundwork for a “true” approach to causal cognition, providing a potential basis for subsequent experimental study in the brain.

1. Introduction

Humans routinely make judgments about causality based on observations of the world. For example, noticing whether a medication leads to symptom relief or whether a food type leads to stomach discomfort are intuitive judgments about cause and effect. These examples also highlight an important aspect of causal judgment in that they often require multiple observations: what if a headache resolved on its own rather than because of the medication? This process of reasoning about causation from a series of sparse and uncertain observations is ubiquitous in daily life [1], and has long been thought to be a cornerstone of human cognition [2]. Although we might often focus our attention on only a pair of variables (e.g. headaches vs medication), causal relationships are typically not judged in isolation. Humans commonly seek to understand the structure of causal relationships between several variables at once in order to consider possible alternative causes of an effect [3]. Importantly, this ability to consider causal interactions between multiple variables is thought to underlie a range of cognitive functions, including imagining possible future or counterfactual outcomes [3–10], guiding explanations and decisions [11–15], and learning in variable environments [16–18].

Currently, the most widely used formalization of causal knowledge is the directed acyclic graph (DAG), in which directed edges between nodes represent causal relationships between objects or events in the world [19]. In particular, Bayesian networks – a probabilistic variant of DAGs – have become highly influential models in both psychology and machine learning. For example, there has been extensive research on estimating causal structure from observational data using Bayesian networks [20–23], and such approaches form the basis of leading cognitive theories of human causal induction [4,24–30] and causal inference in multisensory perception [31–35]. In the Bayesian framework, the brain is hypothesized to perform posterior inference over a limited number of possible causal structures, typically selecting the structure that best explains the observations, and potentially also updating its posteriors with additional evidence [36].

Although Bayesian approaches have been successful in accounting for a wide variety of human behaviors [37], it is unclear whether and how biologically plausible neural systems could realistically implement the computations required by Bayesian causal model selection [33, 38, 39]. More broadly, it is currently not known how a population of neurons should coordinate its neural activity to infer causal relationships from experience, represent causal structure, and update such representations based on any further experience. Additionally, due to implementation-level constraints, neural systems might adopt algorithmic approaches outside of those that have thus far been considered in cognitive science, which have instead focused largely on normative models of human behavior. Therefore, investigating the possible neural mechanisms of causal judgment could yield novel hypotheses at the “true” of analysis [40], in addition to the implementation level.

Flexibly judging different causal relationships from a sequence of observations requires both integrating evidence over time [41–43] and performing context-dependent computations [44] (e.g. switching from judging whether $A$ causes $B$ to whether $B$ causes $A$). Recurrent neural networks (RNNs) are among the simplest neural architectures with these abilities, and can be related to recurrent computations in the brain [44, 45]. Thus, RNNs provide a natural model for investigating biologically plausible mechanisms of causal judgment. Here, we study RNNs that have been trained on a novel causal judgment task in which the network observes stochastic samples from an unknown DAG and must judge whether a queried cause-effect relationship is present. We find that RNNs are capable of performing this task, invoke non-queried variables to resolve causal relationships, and form implicit beliefs about the entire graphical structure underlying a sequence of observations in the task despite only ever being trained on a single cause-effect query for any sequence in the training data. Dynamical systems analysis of the RNNs [46–55] reveals putative neural mechanisms and geometric principles used to judge and represent causal relationships, together providing novel hypotheses that could guide future experiments probing the neural basis of causal cognition.

2. Training RNNs to make causal judgments

2.1. Preliminaries

Our analysis is inspired by the structural causal models framework [6, 19, 56]. For our purposes, a causal model consists of (1) a graph $G$ with nodes $\{A,B,C,\dots \}$ (representing distinct variables, objects, or events in the world) and directed edges $\{(A,B),(C,B),\dots \}$, together with (2) functions that assign each node a value (or activity) based on the values of the other nodes and any exogenous sources of variability. We say that a variable (represented by $X$) causes another (represented by $Y$) if there is a corresponding directed edge $(X,Y)$ in $G$. A node $Y$ for which $(Y,X)$ is a directed edge in $G$ is referred to a parent of $X$, and the set of all parent nodes of $X$ is represented by pa $\left(X\right)$. Finally, we focus on causal models whose directed edges do not form cycles (i.e., DAGs).

2.2. The DAG task

A canonical model for human causal judgment in cognitive science is the “true” parameterization of causal DAGs, where causes have independent chances of producing their effects [4, 25, 57, 58]. Formally, for a DAG $G$ whose nodes $n=A,B,C,\dots$ have activities $v_{nk}$ at a sequence of observation times $k=1,\dots ,K$, samples from $G$ are generated via ancestral sampling in the following manner. A node $n$ can spontaneously activate with probability $p_{\text{spont}}$ (the “true”), or can otherwise be activated by a parent node in $G$ (if it has any) with probability $p_{\text{cause}}$. Thus, the probability for node $n$ to be active in observation $k$ is a monotonically increasing function of the number of active parent nodes, given by

$$p\left(v_{nk}\mid \text{pa}(n)\right)=1-\left(1-p_{\text{spont}}\right){\left(1-p_{\text{cause}}\right)}^{a_{nk}}$$ (1)

where $a_{nk}={\sum }_{m\in \text{pa}(n)}{v}_{mk}$ is the number of parent nodes of $n$ active on observation $k$. While in general DAGs are not uniquely identifiable without interventions [56,59], the noisy-or parameterization introduces asymmetries in the observation statistics that improve identifiability of the underlying structure from observations alone while still being capable of expressing complex statistical dependencies [60]. For example, in a two-node DAG with one edge $(A,B)$, the child node $B$ is more active than the parent $A$ – information that can be leveraged to resolve a causal relationship.

Here we propose to recast noisy-or structure learning as a cognitive task: a subject observes a sequence of activity from a collection of nodes with an unknown causal structure, together with a query regarding a specific causal relationship (“does $A$ cause $B$?”), and must judge whether the query is true or false given the observations (Figure 1a). This task goes beyond simple binary evidence accumulation of instances of $A$ and $B$ because the subject can reason about the role of other variables $\text{(}C,D,\dots \text{)}$ that might confound a putative causal judgment, and because there are many more possible contexts in which the same observations should be integrated differently (compared to standard context-dependent decision making tasks with two contexts, e.g. [44]).

Figure 1: Causal judgment task and neural network model.

a, Schematic of task. Observations are stochastically generated from a DAG whose true edges are unknown. The subject is queried (instructed or cued) on whether a causal relationship exists between a specific pair of nodes. b, Task implementation in neural networks. Inputs are a concatenation of three components: the node observations, the cause query (one-hot encoded), and the effect query (also one-hot encoded). RNNs are trained to report whether the queried causal relationship is true. Further details shown in Figure S1.

2.3. Recurrent neural networks

To discover putative mechanisms for how neural systems can form causal judgments, we used an approach in which RNNs were trained to solve the DAG task, then reverse-engineered to identify their constituent dynamical computations. Importantly, this approach is agnostic regarding the process responsible for forming the causal judgment system itself (be it e.g. evolution, lifetime learning, or development [61]).

For the DAG task on $N$ graph nodes, we considered nonlinear RNNs with hidden units that obey the following standard dynamical updates:

$$\tau \frac{d\mathbf{\text{x}}(t)}{dt}=-\mathbf{\text{x}}(t)+f(\mathbf{\text{Wx}}(t)+\mathbf{\text{Bu}}(t)),y(t)=\sigma (\mathbf{\text{Cx}}(t))$$ (2)

(e.g. see [52, 62, 63]; Figure 1b). Here $\mathbf{\text{W}}\in {ℝ}^{J\times J}$ is the recurrent weight matrix, $\mathbf{\text{u}}(t)\in {ℝ}^M$ is the input vector at time $t$, $\mathbf{\text{B}}\in {ℝ}^{J\times M}$ are the feedforward weights, and $f=\text{tanh}$ is a nonlinear activation function. Each ${\mathbf{\text{x}}}_j$ represents e.g. the firing rate of a neuron or putative collection of neurons. The RNN output at time $t$ is represented by $y(t)$, and computed from the hidden unit activity via output weights $\mathbf{\text{C}}\in {ℝ}^{J\times 1}$ followed by the logistic sigmoid function $\sigma$.

Inputs $\mathbf{\text{u}}(t)$ to the RNN consist of $M$-dimensional vectors made up of three concatenated segments (Figure 1b): the observed activity of the $N$ graph nodes, the representation of the queried cause (one-hot encoded), and the representation of the queried effect (also one-hot encoded). While node activity varies stochastically over the time course, the query inputs are tonic. The RNN output represents its judgment as to whether the queried cause-effect relationship exists in the observation sequence. Note that the processing of the observation sequence is context-dependent: the RNN must be able to flexibly reconfigure how it integrates the observations depending on the causal query. Further, it is worth emphasizing that for a given observation sequence in the training data, the RNN only ever judges a single cause-effect query before those observations are discarded and the next observation sequence and query is generated (see Figure S1 for further details). Thus, RNNs are explicitly not trained to judge multiple different queries for any sequence of observations.

3. Results

3.1. RNNs can accurately perform the DAG task

We first focused on analyzing RNNs trained to make pairwise causal judgments on observations between three variables $(A,B,C)$, as this is the smallest number of nodes in which non-trivial graph structures are possible: chains, forks, colliders, and mediators represent distinct causal relationships among three variables (Figure 2a) that do not appear in two-node systems. Further, three-node DAGs are capable of expressing an extremely rich range of possible sequences of observations due to the diversity of graph structures and the stochastic nature in which nodes interact with each other.

Figure 2: Trained RNNs can perform the DAG task and use non-queried variables to resolve causal ambiguities.

a, The six categories of DAGs on three nodes. b, Accuracy and loss (on out-of-sample data) during training. Dark lines show averages over 10 randomly initialized RNNs. Faint gray regions show means ± 1 standard deviation. c, Accuracy of trained RNNs as a function of DAG category. Each data point represents the average performance of an RNN on 100 randomly generated DAGs of the listed category. d, Accuracy of an example trained RNN as a function of all possible queries on a chain DAG. While the RNN achieves high performance on five out of six queries, query $AC$ (“true”, red arrow) is at chance level (dotted line). e, The reduced accuracy on chain DAGs arises from the RNN often inferring that $A$ causes $C$ directly when in fact $A$ causes $B$ and then $B$ causes $C$ (and similarly for other permutations of $A$, $B$, and $C$ in chain DAGs). f, Accuracy of the RNN on judging whether $A$ causes $B$ for a fork DAG (right), under normal conditions (“true”, black circles) vs. when the activity of $C$ is randomized (teal circles). Randomizing $C$ significantly degrades RNN accuracy ($p$ < 10⁻¹⁷, Wilcoxon signed-rank test), often causing it to report that $A$ does cause $B$ since the covariance of $A$ and $B$ can no longer be explained by another variable. g, Similar to (f), but where $A$ does cause $B$. Small but statistically significant difference between control and randomized conditions (p < 10⁻¹⁵).

After training, RNNs solved the task with high overall performance, achieving a mean accuracy of 0.95 (out of 1) on novel observations sampled from three-node DAGs (Figure 2b). This indicates that RNNs can successfully judge cause and effect under the specific conditions of the DAG task, and thus constitutes a suitable starting point for analyzing mechanisms of causal judgment.

3.2. Judgment strategies and failure modes in trained RNNs

Despite achieving high accuracy, RNNs occasionally made errors, prompting us to investigate possible behavioral biases. To do so, we categorized all DAGs on 25 nodes into six distinct categories: the empty DAG, DAGs with a single edge, fork DAGs, collider DAGs, chain DAGs, and mediator DAGs (Figure 2a). Of these categories, RNNs had markedly lower accuracy on chain DAGs in particular (Figure 2c), and, more specifically, tended to erroneously judge that $A$ causes $C$ in circumstances where in fact $A$ causes $B$ and $B$ then causes $C$ (Figure 2d,e). Thus, trained RNNs displayed a transitivity bias, similar to human causal judgment in some conditions [64,65].

Next, we sought to probe the underlying strategy adopted by the RNNs. If an RNN is judging whether $A$ causes $B$, to what extent does it care about $C$? To evaluate this capacity, we considered a fork DAG in which the only causal effects are that $C$ causes $A$ and $C$ causes $B$, and then queried the RNN as to how it judges if $A$ causes $B$. This case is potentially insightful because the RNN might ignore $C$ and selectively integrate instances of $A$ without $B$. Alternatively, it could leverage the activity of $C$ to correctly identify it as a common cause of both $A$ and $B$, despite the RNN not being explicitly queried on role of $C$.

To resolve this ambiguity, we randomized the activity of node $C$ (via circular shuffles in time) while leaving the activity of $A$ and $B$ intact. This led to a substantial decrease in RNN performance (Figure 2f), indicating a critical role for observations of $C$ when judging whether $A$ causes $B$. Further, we also simulated observations from a DAG in which $A$ acted as a mediator between $C$ and $B$ (i.e. $C$ causes $A$, $C$ causes $B$, and $A$ causes $B$). We found that the RNN’s ability to discriminate the causal effect of $A$ on $B$ had a small but statistically significant improvement when $C$ was randomized (Figure 2g), presumably due to $C$ presenting less as a potential confounder under randomization. Together, these results show emergent utilization of information about non-queried variables to judge causal relationships, as has been postulated for human causal reasoning [57].

3.3. RNNs trained on single cause-effect pairs implicitly represent entire DAGs

To further probe the RNNs’ beliefs about causal structure underlying observations, we performed an analysis where we sampled a single sequence of observations from an example DAG and then cycled through every possible causal query with the observations kept fixed. This revealed that, despite only ever being trained on a single cause-effect query per observation sequence, RNNs form accurate judgments about the complete graphical structure (Figure 3a–c, Figure S2).

Figure 3: Probing RNNs’ beliefs about the complete graphical structure underlying observations.

a, Example DAG on which behavior was tested across queries for a single fixed observation sequence. b, Adjacency matrix corresponding to the DAG in (a). c, Top: Evolution of an RNN’s judgments on all causal relationships over the course of a trial. Bottom: Sequence of observations used to form the judgments made in top panel. Red ticks correspond to the times sampled. d, Control analysis for probing RNN judgment. Left: shuffling (via cyclic permutations) the marginal activity of each node considerably alters the inferred graphical structure (single example shown). Right: similarity between the graphical structure inferred from the actual vs. shuffled observations (distribution over 100 random shuffles shown). Similarity defined as the fraction of graph edges inferred from the shuffled observations that are equal to the graph edges inferred from the actual observations. e, Correlation coefficients (derived from observations in (c)) between every pair of nodes shows that RNNs do not simply report correlations. Additional examples shown in Figure S2.

Next, we wondered whether the RNN was simply integrating the marginal statistics of each node’s activity to make its judgment. As noted in subsection 2.2, the noisy-or parameterization can induce signatures of causal relationships in the marginal statistics of each node. For example, a node with no parent nodes can only activate via the leakage probability – a rare occurrence compared to a node with several parent nodes, each capable of activating the child node. We conducted a shuffle analysis by performing a circular permutation independently for each node’s observation sequence, such that the marginal activity statistics were all preserved, but the covariance between node activity was eliminated. Under this analysis, we found that RNNs failed to recover the true DAG structure, indicating that they were not simply making judgments via the marginal statistics (Figure 3d). Similarly, we confirmed that RNNs were not simply reporting the correlation structure of the observations (Figure 3e), which could show similarity to the causal structure in some circumstances. Thus, trained RNNs formed their judgments through both the marginal and covariant activity.

3.4. RNN activity shows an emergent geometry reflecting graphical structure

Towards understanding how neural systems can perform causal judgments, we next examined the low-dimensional activity trajectories of a representative RNN. For visualization, we projected the trajectories of the hidden units onto the top three principal components (PCs) , which together accounted for 75% of the variance for this RNN (Figure S3). There are six possible causal queries for a three-node DAG ($A$ causes $B$, $A$ causes $C$, $B$ causes $A$, etc.). As such, we first simultaneously plotted the trajectories for each of the six possible queries in the absence of any observations. These plots showed distinctive structure: trajectories radially diverged from a common starting point near the origin before settling at query-dependent locations that were approximately equidistant from their neighbors (Figure 4a). Hidden states showed minimal movement thereafter, suggesting that recurrent dynamics guided trajectories towards possible fixed points associated with each query.

Figure 4: Emergent representations of graphical structure in trained RNNs.

a, Neural trajectories generated by a trained RNN without any observations (but with tonic query input enabled). Colored paths correspond to different queries. Right panel rotated 90°. b, Same as (a), but with observations sampled from an example DAG. c-e, Average neural trajectories corresponding to DAGs with increasing numbers of edges. Averages taken over 500 samples. Black arrows in (d) and (e) indicate trajectories corresponding to queries that are true in corresponding DAG. All panels and trajectories plotted in common reference PC space.

Next, we sampled a single sequence of observations from an example DAG ($B$ causes $C$, $C$ causes $A$), constituting a single trial of the DAG task, and performed a similar visualization. The corresponding trajectories were similar to the observation-free case in terms of diverging from a common initial point, but were considerably more irregular due to the stream of stochastic inputs (Figure 4b).

We then began investigating the representation of specific causal graphs. We started by sampling observations from the empty DAG (such that nodes only activate spontaneously) and visualizing the average trajectory over 500 trials. (Note that by a trial we mean a sequence of observations from a DAG, along with a query input). This recapitulated the radial divergence seen in the observation-free case, but with elongated trajectories associated with “true” RNN outputs (Figure 4c). However, as we introduced edges into the underlying DAG and further examined activity across trials, we found that the associated trajectories traced paths in the opposite direction in state space compared to when the edge did not exist (Figure 4d,e). Averaging trajectories with causal relationships being present (edge exists) or absent (edge does not exist) yielded an effective decision axis that ran parallel to the RNN trajectories and orthogonal to a decision plane (Figure S4).

These results are suggestive of a systematic implementation for representing arbitrary causal graphs, where query-specific trajectories converge on either side of a decision plane depending on how the RNN integrates the input sequence.

3.5. RNNs use query-specific line attractors to represent causal relationships

To go beyond visualization of the hidden unit trajectories, we next sought to uncover the specific dynamical mechanisms underlying the formation of causal judgments. To do so, we adopted the “true” approach of Sussillo et. al. [44,46,47]. Briefly, the approach consists of identifying fixed points in the RNN dynamics (i.e. neural states where $d\mathbf{\text{x}}(t)/dt\approx 0$), characterizing how identified fixed points relate to RNN output, and understanding how the neural state is driven between fixed points via interactions between the RNN’s external inputs and its recurrent dynamics. In particular, we looked for fixed points while observations were absent but the query input was on, thereby simulating observation-free periods in the task (see Appendix B for details).

We found that fixed points were organized into approximate line attractors specific to each query, all of which were aligned with the decision axis (Figure 5a,b). Further, this fixed point structure was conserved across RNNs trained to judge causal effects in DAGs with two, three, or four nodes (Figure S5). Linearization of the RNN dynamics around each fixed point revealed that the corresponding largest real-valued eigenvalues were approximately equal to 1 (Figure S6), consistent with a stable integrating mode (Figure S7). Moreover, line attractors were clustered in space according to the queried cause: the $A\to X$ line attractors were closer together and had visibly more similar dynamics compared to the $B\to Y$ and $C\to Z$ line attractors (Figure 6, cf. Figure S5), suggesting a systematic geometric organization of population activity relative to the causal graph.

Figure 5: Formation of causal judgments via line attractor dynamics.

a, Example DAG used to generate observations. b, Fixed points (colored crosses) are organized into an approximate line attractor for each query (specified above each panel). Vector field shows recurrent dynamics (via top two PCs) for each query. White circles: initial states; white stars: final states; black lines: average trajectories over 500 trials sampled from DAG in (a); faint gray lines: example trajectories from three trials. See Figure S5 and Figure 6 for examples for other node counts. c, Left: color legend for all eight possible observations ($A$ alone, $B$ alone, $AB$, $BC$, etc). Right: input vectors ${\left\{\mathbf{\text{B}}{\mathbf{\text{u}}}_k\right\}}_{k=1}^8$ corresponding to all possible observations, projected onto top three PCs. d, Interaction between input vectors and recurrent dynamics differentially moves neural state along the line attractor (represented by thick colored line interpolating between fixed points). Colored squares show instantaneous effect of corresponding input. Colored triangles show neural state after 200 timesteps. e, Repeated observations of $A$ induces the neural state to traverse line attractor from end to end. f, Similar to (e), but for observing $A$ and $B$ together. Note dependence of outcome on initial state.

Figure 6: Clustering of line attractors (“judgment axes”) across queries.

a, Visualization of all line attractors simultaneously for an RNN trained on the 3-node DAG task. Line attractors colored by query. Crosses represent fixed points; dashed line, output axis; white circle, coordinates of initial state. b, Same as (a), but for an example RNN trained on the 4-node DAG task. c, Euclidean distances between line attractor centroids across RNNs trained on the DAG task with 2, 3, and 4 nodes. Each data point represents an average over 10 randomly initialized RNNs. First three panels ordered to highlight that clustering occurs primarily via the queried cause. Right-most panel re-ordered to show secondary clustering occurs via the queried effect.

3.6. Probing input-specific dynamics of updates to causal judgments

We next sought to understand how different observations moved the neural state along the line attractors. As a motivating example, we first focused on the $A\to B$ query, though the core mechanisms hold more generally. We selected an example fixed point in the middle of the line attractor and visualized the effect of every possible input ($A$ alone, $A$ and $B$ together, $B$ and $C$ together, etc). Different input combinations displaced (“true”) the neural state in different directions according to the instantaneous effect of the input vectors (Figure 5c,d). Then, depending on the input-driven repositioning, recurrent dynamics carried the neural state back to the line attractor, but further towards the true or false ends depending on whether the observation supported or contradicted the query (Figure 5d).

This “true” activity pattern suggests a mechanism through which the RNN can traverse the line attractor in order to make its judgment. For example, when querying whether $A$ causes $B$, repeatedly observing $A$ alone iteratively kicks the neural state towards the “true” end of the line attractor (it is unlikely that $A$ causes $B$ if one often observes $A$ without also $B$), regardless of where the neural state starts (Figure 5e). On the other hand, observing $A$ and $B$ together supports the query, and therefore typically moves the neural state towards the “true” end of the line attractor (Figure 5f, middle and right). Interestingly however, this effect is state-dependent: observing $AB$ supports $A$ causing $B$ unless the RNN already “true” $A$ does not cause $B$, in which case all observations (including $AB$) move the neural state even further towards the “true” end (Figure 5f, left). Finally, we comprehensively quantified the extent to which every possible observation moves the neural state along each query’s line attractor starting from every fixed point, revealing a diversity of state-dependent input effects that together describes how an RNN makes its judgments (Figure S8).

3.7. A neural algorithm for forming causal judgments

From our collective results we derived a mechanistic model of how recurrent neural circuits can judge and represent causal relational structure (Figure 7). Our model is based on the linearized RNN dynamics: first, using a first-order Taylor series, the local dynamics around a fixed point $x^{*}$ on the line attractor can be approximated as a linear dynamical system ${\mathbf{\text{z}}}_{t+1}\approx {\mathbf{\text{J}}}_{\text{rec}}{\mathbf{\text{z}}}_t+{\mathbf{\text{J}}}_{\text{inp}}{\mathbf{\text{w}}}_{t+1}$, where ${\mathbf{\text{J}}}_{\text{rec}}$ and ${\mathbf{\text{J}}}_{\text{inp}}$ are the Jacobians of the RNN update function with respect to the neural (hidden) state and the input, and where ${\mathbf{\text{w}}}_t$ is an input term (see Appendix C for details). The long-term behavior of the neural state (e.g. for $t=1,\dots ,T$) in response to a single observation $\widehat{\mathbf{\text{w}}}$ can be approximated in the neighborhood of $\mathbf{\text{x}}\ast$ in terms of the slowest eigenmode as ${\mathbf{\text{z}}}_T\approx \mathbf{\text{r}}\left({\ell }^{\top }{\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}\right)$, where $\mathbf{\text{r}}$ and ${\ell }^{\top }$ are the right and left eigenvectors associated with the largest eigenvalue of ${\mathbf{\text{J}}}_{\text{rec}}$ (in our RNNs this is typically ${\lambda }_1\approx 1$). The expression ${\ell }^{\top }{\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}$ is a scalar that describes how far the neural state will move in direction $\mathbf{\text{r}}$, as a function of how the input vector ${\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}$ aligns with left eigenvector ${\ell }^{\top }$.

Figure 7: Population dynamics and geometry for a neural implementation of causal judgment.

a, Stylized line attractors for each causal query, clustered by the queried cause. Colored crosses represent fixed points; dashed line, trained output weights plotted as direction in PC space; white circle, initial coordinates of neural trajectory. b, Local dynamics underlying how neural state transitions between fixed points on a line attractor. For each query, associated fixed points and their right eigenvectors define a stable “true” ranging from “true” to “true”. Left eigenvectors determine input selectivity: an input aligned with a left eigenvector will drive the neural state towards “true”, whereas an anti-aligned input will drive state towards “true”. c, Schematic of how confirmatory vs contradictory evidence shifts neural state towards true vs false ends of the line attractor. d, Direction of movement along the line attractor (corresponding to network’s change in judgment) is determined by how the input vector aligns with the left eigenvector. As the query changes (e.g. from $A\to B$ to $A\to C$), the eigenvectors are reconfigured: the orientation of the left eigenvector changes to become aligned with observations of $AC$, and anti-aligned with observations of $AB$. Visual perspective represented by cartoon eye in (e). e, Three examples of how local recurrent dynamics are reconfigured to enable opposing effects of inputs under different queries. f, Representation of graph edges is implemented via flexible reconfiguration of left and right eigenvectors across queries. g, Left eigenvectors also reconfigure within queries, as a possible mechanism to consolidate judgments by becoming orthogonal to the inputs.

For a given query, a line attractor is composed of a collection of such fixed points (Figure 7a), each of which is associated with a right eigenvector $\mathbf{\text{r}}$ specifying a direction of movement in state space. Collectively, these points define a “judgment axis” that spans from “false” to “true” (Figure 7b). By orienting the corresponding left eigenvectors such that they align specifically with observations of the cause and effect together, but are anti-aligned with observations of the cause without the effect, the neural state will move appropriately towards the true or false ends of the judgment axis.

However, when the queried graph edge changes (e.g. from $A\to B$ to $A\to C$), the recurrent dynamics must change accordingly: an observation of $AC$ switches from being contradictory evidence to confirmatory evidence (Figure 7c). Thus, using the same connectivity matrix, a change in query causes a reconfiguration of the (locally linear) recurrent dynamics such that the left eigenvector switches from being anti-aligned with observation $AC$ to being aligned with it (Figure 7d).

Further, because observing the queried cause without the effect has a profoundly negative effect on judgment, clustering line attractors by the queried cause (Figure 6) enables the RNN to partially reuse the same local dynamical motif to carry the neural state towards the false end of the line attractor whenever the queried cause is observed alone. Finally, the state-dependent effect of an observation within queries (Figure 5f) is consistent with a reconfiguration of the left eigenvectors within the line attractors, such that at the extreme ends the left eigenvectors become orthogonal to (or even anti-aligned with) observations they were previously aligned with (Figure 7e). This could constitute a potential mechanism to consolidate a judgment.

4. Conclusion

Here we identified putative neural mechanisms of causal reasoning by reverse-engineering RNNs trained on a novel cognitive task. Our results show that rather than explicitly performing model selection over candidate causal graphs [25], neural systems could express population dynamics capable of judging and emergently representing causal relational structure, offering a mechanistic alternative to normative behavioral accounts of human causal judgment [38]. Importantly, the geometry of the resulting dynamics are shaped by several factors, most notably the multi-tasking aspect of the DAG task [66–68]: the number of queries that RNNs must be able to solve grows quadratically with the number of nodes in the graph, requiring numerous query-dependent reconfigurations of how evidence is integrated. Line attractors should thus be spaced far enough apart to limit interference between queries. However, symmetries in the DAG task facilitate partial reuse of dynamical motifs [52] (e.g. observing $A$ alone should affect judgment similarly across queries $A\to B$ and $A\to C$), encouraging certain line attractors to be closer together. These competing pressures appear to yield a hierarchical geometry where judgment axes are clustered primarily by cause and secondly by effect (Figure 6), in turn enabling compositional reuse of dynamical motifs across causal queries.

Appendix

A. Recurrent neural network training details

A.1. Network and optimization parameters

We trained RNNs with $J=128$ hidden units equipped with tanh nonlinearities using the cross-entropy loss function (Figure S1). We used a time constant corresponding to $\tau =13$ time steps. RNNs were trained to hold their output for a period of 179 time steps. An RNN output with $y(t)\ge 0.5$ represents it making a “true” judgment, and otherwise a “true” judgment. Both the recurrent weights and the hidden unit activity were regularized using L2 penalties with coefficients ${\lambda }_{\text{rec}}={10}^{-4}$ and ${\lambda }_{\text{hid}}={10}^{-5}$. RNNs were trained with a batch size of 1024. During training, hidden unit activations were corrupted with Gaussian noise (mean, 0; standard deviation, 0.025) to improve out-of-sample performance. Sets of 10 RNNs were trained for 25,000, 50,000, or 75,000 epochs for DAG tasks with 2, 3, or 4 nodes respectively, which was sufficiently many epochs to consistently solve the task with high accuracy using interpretable dynamical structures. RNNs were trained using the Adam optimizer with a learning rate of 4 × 10⁻⁴.

A.2. Generating training data

In our training data, a sequence of observations was made up of 75 samples from a DAG, separated in time by an interstimulus interval of 5 time steps. Nodes in the DAG probabilistically activated each other under the noisy-or rule described in subsection 2.1, where $p_{\text{cause}}=0.8$ and $p_{\text{spont}}=0.2$. To construct a DAG, we first initialized its adjacency matrix with all zeroes, and then (to ensure acyclicity) filled in the upper triangle only, by sampling binary values uniformly at random. Node labels were then randomly permuted. Consequently, for a fixed pair of nodes $(A,B)$, the marginal probability that query $A\to B$ is true across all randomly sampled DAGs in the training data is 0.25. During training, subjects (networks) encounter all possible $N$-node DAGs, but only a small fraction of all possible sequences of observations. For example, although there are only 25 possible DAGs on three nodes, we can generate extremely large amounts of training and test data by stochastically sampling different observation sequences from those nodes. In our case, since we trained our RNNs using observation sequences comprised of 75 temporally-spaced samples, there are 2²²⁵ possible sequences for each DAG.

B. Fixed point analysis

To identify fixed points, we minimized a discrete-time analogue of the “q function” from ref. [46]:

$$q(\mathbf{\text{x}})=\frac{1}{2}\Vert \mathbf{\text{x}}-F(\mathbf{\text{x}},\mathbf{\text{u}}){\Vert }^2,$$ (3)

where $F$ represents the one-step update resulting from discretizing the continuous RNN dynamics in Equation 2 (see Equation 6 below). Here we have suppressed the dependence on time to simplify notation. We focused our analysis on minimizers $x^{*}$ of Equation 3 such that $q\left({\text{x}}^{*}\right)<1\times {10}^{-5}$, and thus we typically identified approximate fixed points, or “slow points”, rather than fixed points per se [69].

Based on the behavior of the RNN when simulated without observations (subsection 3.4), we hypothesized that any fixed point structure would be query-specific (particularly as the query input is always on, providing a tonic context signal). When searching for fixed points, we therefore provided a static input vector ${\widetilde{\mathbf{\text{u}}}}_k\in {ℝ}^M$ for which the node activity is zero but for which query $k$ is active; i.e. ${\tilde{u}}_{ki}=0$ for $i=1,\dots ,N$, and ${\tilde{u}}_{ki}={\tilde{u}}_{kj}=1$ if $i$ and $j$ correspond to the queried cause and effect indices respectively, and ${\tilde{u}}_{ki}=0$ otherwise.

To seed the fixed point finding algorithm, we used the following procedure to select initial points. When the number of nodes was three or less, we enumerated all possible DAGs, simulated RNN trajectories on each DAG, and sampled initial points at three time steps along each average trajectory, as well as from two randomly selected single trials. When the number of nodes was larger than three, the number of possible DAGs grew extremely quickly, making the number of possible seeds far too large to test. Instead, we sampled initial points from the empty DAG, a complete DAG (where there is an edge of some orientation between every pair of nodes, e.g. $\{(A,B),(A,C),(A,D),(B,C),(B,D),(C,D)\}$), the complete DAG with all edges reversed, as well as the singleton DAGs (i.e. the DAG with exactly one causal edge, as in Figure 2a, for every possible edge). This ensured that we seeded the fixed point finder from a variety of “true” and “true” judgments for every possible causal query.

C. Linear dynamical systems analysis

In this section we derive the linear approximation to the query-specific RNN dynamics, much of which follows the derivations from refs. [44,50]. First, recall that the continuous-time RNN dynamics are given by

$$\tau \frac{d\mathbf{\text{x}}(t)}{dt}=-\mathbf{\text{x}}(t)+f(\mathbf{\text{Wx}}(t)+\mathbf{\text{Bu}}(t))$$ (4)

where $f$ is the tanh activation function, and where the RNN output is subsequently obtained as $y(t)=\sigma (\mathbf{\text{Cx}}(t))$. In practice, we run the RNN by approximating the continuous-time dynamics via the following discrete-time update rule:

$${\mathbf{\text{x}}}_{t+1}=(1-\alpha ){\mathbf{\text{x}}}_t+\alpha f\left(\mathbf{\text{W}}{\mathbf{\text{x}}}_t+\mathbf{\text{B}}{\mathbf{\text{u}}}_{t+1}\right),$$ (5)

where $\alpha =1/\tau$. (Note that Equation 5 is obtained from Equation 4 via the finite difference approximation).

Next, we wish to understand the dynamics of the discrete-time RNN around a fixed point $x^{*}$ associated with query $q$ (where $q$ indexes the causal queries $A\to B$, $A\to C$, etc.). To that end, we define the RNN update function $F$ as

$$F(\mathbf{\text{x}},\mathbf{\text{u}}):=\alpha \mathbf{\text{x}}+(1-\alpha )f(\mathbf{\text{Wx}}+\mathbf{\text{Bu}})$$ (6)

and set ${\text{Δx}}_t={\text{x}}_t-{\text{x}}^{\ast }$ (i.e. the neural state relative to the chosen fixed point) and $\text{Δ}{\mathbf{\text{u}}}_t={\mathbf{\text{u}}}_t-{\mathbf{\text{u}}}^{\ast }$. Since we are interested in query-specific dynamics, we choose ${\mathbf{\text{u}}}^{\ast }\in {ℝ}^M$ to be the input corresponding to query $k$ but without observations, analogous to the input used during fixed point finding. Then (following e.g. ref. [50]) we perform a first-order Taylor approximation about the pair $\left({\text{x}}^{\ast },{\text{u}}^{\ast }\right)$:

$${\mathbf{\text{x}}}_{t+1}=F\left({\mathbf{\text{x}}}_t,{\mathbf{\text{u}}}_{t+1}\right)$$ (7)

$$=F\left({\text{Δx}}_t+{\text{x}}^{\ast },\text{Δ}{\mathbf{\text{u}}}_{t+1}+{\mathbf{\text{u}}}^{\ast }\right)$$ (8)

$$\approx F\left({\mathbf{\text{x}}}^{\ast },{\mathbf{\text{u}}}^{\ast }\right)+{\mathbf{\text{J}}}_{\text{rec}}{\text{Δx}}_t+{\mathbf{\text{J}}}_{\text{inp}}\text{Δ}{\mathbf{\text{u}}}_{t+1}$$ (9)

where ${\left[{\mathbf{\text{J}}}_{\text{rec}}\right]}_{ij}=\partial F{\left({\mathbf{\text{x}}}^{\ast },{\mathbf{\text{u}}}^{\ast }\right)}_i/\partial {\mathbf{\text{x}}}_j$ and ${\left[{\mathbf{\text{J}}}_{\text{inp}}\right]}_{ij}=\partial F{\left({\mathbf{\text{x}}}^{\ast },{\mathbf{\text{u}}}^{\ast }\right)}_i/\partial {\mathbf{\text{u}}}_j$ are the Jacobians of the dynamical update function with respect to the neural state and the input respectively. Next, because ${\mathbf{\text{x}}}^{\ast }$ is a fixed point of $F$ associated with query ${\mathbf{\text{u}}}^{\ast }$, $F\left({\mathbf{\text{x}}}^{\ast },{\mathbf{\text{u}}}^{\ast }\right)\approx {\mathbf{\text{x}}}^{\ast }$. Hence we obtain the following local linear approximation to the RNN dynamics:

$$\text{Δ}{\mathbf{\text{x}}}_{t+1}={\mathbf{\text{J}}}_{\text{rec}}\text{Δ}{\mathbf{\text{x}}}_t+{\mathbf{\text{J}}}_{\text{inp}}\text{Δ}{\mathbf{\text{u}}}_{t+1}.$$ (10)

Finally, setting ${\mathbf{\text{z}}}_t:=\text{Δ}{\mathbf{\text{x}}}_t$ and ${\mathbf{\text{w}}}_t:=\text{Δ}{\mathbf{\text{u}}}_t$, we have the linear dynamical system from subsection 3.7,

$${\mathbf{\text{z}}}_{t+1}={\mathbf{\text{J}}}_{\text{rec}}{\mathbf{\text{z}}}_t+{\mathbf{\text{J}}}_{\text{inp}}{\mathbf{\text{w}}}_{t+1}.$$ (11)

A powerful approach to evaluating the long term behavior of a linear dynamical system is in terms of its eigendecomposition. In our linearized RNNs, the recurrent dynamics matrices are generally non-normal [70–72]; i.e., ${\mathbf{\text{J}}}_{\text{rec}}^{-1}\ne {\mathbf{\text{J}}}_{\text{rec}}^{\top }$. Importantly, for non-normal matrices the eigendecomposition is defined in terms of both left and right eigenvectors,

$${\mathbf{\text{J}}}_{\text{rec}}=\mathbf{\text{RΛL}},$$ (12)

where $\mathbf{\text{Λ}}\in {ℝ}^{J\times J}$ is a diagonal matrix whose diagonal terms are the eigenvalues ${\lambda }_1,\dots ,{\lambda }_J$ (ordered from largest magnitude to smallest), $\mathbf{\text{R}}\in {ℝ}^{J\times J}$ is a matrix whose columns ${\mathbf{\text{r}}}_1,\dots ,{\mathbf{\text{r}}}_J$ are the right eigenvectors of ${\mathbf{\text{J}}}_{\text{rec}}$, and $\mathbf{\text{L}}={\mathbf{\text{R}}}^{-1}$ is a matrix such that each row ${\ell }_j^{\top }$ is a left eigenvector associated with the right eigenvector ${\mathbf{\text{r}}}_j$. To evaluate the long term behavior of $\mathbf{\text{x}}$ (e.g. for some large $T$) in response to a single observation $\widehat{\mathbf{\text{u}}}$ near fixed point ${\mathbf{\text{x}}}^{*}$, we consider the linearized system with $\widehat{\mathbf{\text{w}}}=\widehat{\mathbf{\text{u}}}-{\mathbf{\text{u}}}^{*}$,

$${\mathbf{\text{z}}}_T={\mathbf{\text{J}}}_{\text{rec}}^{T-1}{\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}$$ (13)

$$=\mathbf{\text{R}}{\mathbf{\text{Λ}}}^{T-1}\mathbf{\text{L}}{\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}$$ (14)

$$=\sum _{j=1}^J{\lambda }_j^{T-1}{\mathbf{\text{r}}}_j{\ell }_j^{\top }{\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}.$$ (15)

In our RNNs, the leading eigenmodes are either associated with a single real-valued eigenvalue near 1, or they come in complex conjugate pairs (Figure S6). In the latter case, these modes generate partial oscillatory activity patterns within two-dimensional planes to induce movement along the line attractor. Here, we restrict our focus to the former case (i.e. the real eigenmodes) as these provide a simple and sufficient implementation-level description of how a linear recurrent neural system can judge causation within the context of a specific query. Under this assumption, the eigenmodes with eigenvalues < 1 decay rapidly and thus, for sufficiently large $T$, the long term behavior of $\mathbf{\text{z}}$ is simply given by the expression

$${\mathbf{\text{z}}}_T\approx \mathbf{\text{r}}\left({\ell }^{\top }{\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}\right),$$ (16)

where we have dropped the subscripts on the left and right eigenvectors since we assume that they correspond to the eigenvector with largest eigenvalue. Further, we suppress reference to the eigen-value itself since it is assumed to be approximately equal to 1, and therefore ${\lambda }_1^{T-1}\approx 1$.

The term ${\ell }^{\top }{\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}$ is a scalar that describes how far $\mathbf{\text{z}}$ will move in direction $\mathbf{\text{r}}$ as a function of how aligned the input vector ${\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}$ is with the left eigenvector ${\ell }^{\top }$. This is the key insight underlying the putative neural implementation of causal judgment. If the input vector is aligned with the left eigenvector, then ${\ell }^{\top }{\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}>0$, and $\mathbf{\text{z}}$ will move in direction $\mathbf{\text{r}}$ (i.e. towards the “true” end of the judgment axis)¹. If the input is anti-aligned with the left eigenvector, (i.e. ${\ell }^{\top }{\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}<0$) then $\mathbf{\text{z}}$ will move in direction $-\mathbf{\text{r}}$ (towards “true”). Additionally, the RNN can selectively ignore an input by having the left eigenvector oriented such that ${\ell }^{\top }{\mathbf{\text{J}}}_{\text{inp}}\widehat{\mathbf{\text{w}}}=0$ (i.e., oriented orthogonally). See Figure 7d for visual schematics providing further intuition about the interaction between left eigenvectors and inputs.

Because an observation can switch from being confirmatory to contradictory when the query is changed, the left eigenvectors must be reconfigured (i.e. reoriented) by the query input to facilitate the neural state moving in the opposite direction along the judgment axis. At the same time, some observations should have similar effects on judgment across queries (e.g. observing the queried cause without the effect should have a strongly negative change in judgment regardless of what the queried effect is), and thus the alignment of the left eigenvector relative to some of the inputs should be similar across queries. These two competing demands induce the geometric organization of the line attractors shown in Figure 7 and Figure 6.

D. Limitations

This study represents a first step towards understanding neural mechanisms of causal reasoning. As such, several important limitations remain. For example, the variant of the DAG task we have analyzed only considers facilitatory causes, leaving open how neural systems might also reason about inhibitory or mixed causal effects. It also assumes causal interactions follow the disjunctive “true” parameterization, as opposed to e.g. conjunctive interactions where multiple causes must co-occur to elicit an effect. Further, causal structure learning in cognitive science is believed to also occur through interventions, which we did not explore in this study. Addressing these limitations will be important directions for future work.

E. Causal reasoning in large language models

The rapid development of large language models (LLMs) has generated growing interest in evaluating their causal reasoning abilities [73–77]. For example, Lyu et. al. [73] proposed to probe causal judgments by constructing competing natural language causal narratives for pairs of variables $X$, $Y$ (corresponding to $X\to Y$ versus $Y\to X$), and then comparing an LLM’s ability to perform a zero-shot prediction task when prompted under each framing. Early studies of this kind have also motivated the development of systematic benchmarks to evaluate causal reasoning via natural language [78,79]. However, such approaches differ from our work in several important respects. First, unlike the RNNs studied here, which can be directly analyzed as biologically plausible recurrent dynamical systems, the mechanisms underlying causal judgments in LLMs are not straightforwardly aligned with recurrent computations in the brain. Second, causal reasoning in LLMs is typically elicited through natural language prompts, which likely engage rich semantic and linguistic representations that make it much more difficult to isolate the specific mechanisms that facilitate causal judgments. By contrast, our proposed DAG task abstracts away from natural language scenarios, enabling a direct analysis of how causal judgments can emerge from recurrent neural dynamics.

F. Computational resources

Models were trained on the Axon computer cluster at the Zuckerman Institute at Columbia University using nodes comprised of two Xeon E5–2660 v4 CPUs, eight GTX 1080 Ti GPUs, and 125GB RAM.

G. Code availability

Code used to train the causal RNN models (using PyTorch [80]) is available at: https://anonymous.4open.science/r/causalRNN-D6CC .