Spiking world model with multicompartment neurons for model-based reinforcement learning

Significance

Dendritic computation is key to the brain’s ability to integrate information over long timescales. Inspired by this, this study proposes a spiking neural network model that embeds dendritic mechanisms to enhance long-term memory and planning. This biologically grounded approach enables the construction of a spiking world model for model-based reinforcement learning, bridging microscale neural computation with macroscale cognitive function. By applying this model to a challenging decision-making task, we show that it can match the performance of traditional artificial intelligence systems while using brain-like mechanisms. Our work not only improves the capabilities of spiking neural networks but also highlights how complex biological processes, like dendritic integration, can inspire more powerful and efficient computing systems.

Abstract

Brain-inspired spiking neural networks (SNNs) have garnered significant research attention in algorithm design and perception applications. However, their potential in the decision-making domain, particularly in model-based reinforcement learning, remains underexplored. In reinforcement learning, a world model refers to a predictive model that learns the environment’s dynamics and enables agents to simulate future trajectories in a latent space, thereby improving sample efficiency and long-horizon planning. The difficulty lies in the need for spiking neurons with long-term temporal memory capabilities, as well as network optimization that can integrate and learn information for accurate predictions. The dynamic dendritic information integration mechanism of biological neurons brings us valuable insights for addressing these challenges. In this study, we propose a multicompartment neuron model capable of nonlinearly integrating information from multiple dendritic sources to dynamically process long sequential inputs. Based on this model, we construct a spiking world model (Spiking-WM), which integrates a spiking state-space model, a spiking convolutional encoder, and a fully connected spiking network for policy learning, to enable model-based deep reinforcement learning with SNNs. We evaluated our model using the DeepMind Control Suite, demonstrating that Spiking-WM outperforms existing SNN-based models and achieves performance comparable to artificial neural network-based world models employing Gated Recurrent Units. Furthermore, we assess the long-term memory capabilities of the proposed model in speech datasets, including Spiking Heidelberg Digits dataset, Texas Instruments/Massachusetts Institute of Technology Acoustic-Phonetic Continuous Speech Corpus, and LibriSpeech 100h, showing that our multicompartment neuron model surpasses other SNN-based architectures in processing long sequences.

Brain-inspired computing has emerged as a powerful paradigm for developing novel algorithms that draw on principles of neuronal dynamics and spike-based signal transmission in the brain. Within this paradigm, spiking neural networks represent a biologically inspired class of models that process information using discrete spikes rather than continuous activations. Spiking neural networks (SNNs) exhibit energy efficiency and computational advantages (1, 2), and have been effectively utilized across a wide range of applications (3–6). Current SNN models predominantly employ Leaky Integrate-and-Fire (LIF) neurons as a trade-off between biological plausibility and computational simplicity. However, recent advancements in neuroscientific research have revealed the significant role of dendritic structures in neuronal information processing (7–9). The multicompartment neuron (MCN) model refers to a mathematical framework that treats different parts of a biological neuron such as the axon, soma, and dendrites as independent computational units. The multicompartment SNNs captures the morphological characteristics of biological neurons, utilizing dynamic equations to model dendritic information processing, membrane potential propagation, and somatic spike generation (9, 10). Various implementations of this model, such as the two-compartment neuron model (11, 12) and the multicompartment model with apical and basal dendrites (13, 14), have demonstrated applications in tasks like MNIST classification and target-based learning (15). Additionally, the MCN models have been integrated into neuromorphic chips for energy-efficient simulation of neural circuits (16, 17). Recent studies have shown that multicompartment SNNs outperform LIF-based SNNs in both temporal sequence classification tasks (18) and neuromorphic datasets (19).

Unlike static image recognition tasks, reinforcement learning necessitates that an agent collect data and refine models through dynamic interactions with the environment. In particular, model-based reinforcement learning methods (20, 21) construct a predictive model based on data gathered from the agent’s interactions with the environment. This predictive model is subsequently employed to generate synthetic data through simulated interactions with the agent, thereby optimizing the agent’s policy. By reducing the number of direct interactions required with the real environment, these methods enhance sample efficiency. For example, the Dreamer model (22) uses the recurrent state-space model to construct a world model, enabling latent imagination in long-horizon tasks. This predictive model is commonly referred to as a world model. Conceptually, a world model serves as a compact latent representation of the environment’s dynamics, allowing agents to generate imagined rollouts without direct interaction with the external world (23). By learning such a representation, agents can perform long-horizon reasoning and policy optimization more efficiently. For example, the Dreamer framework demonstrates how world models can enable effective latent imagination for continuous control tasks. In this work, we extend this paradigm into the spiking neural network domain by introducing a spiking world model.

Early studies employing brain-inspired approaches to reinforcement learning (24, 25) explored the critical role of reward models in neural decision-making processes (26). Some preliminary progress has been made in combining SNNs with reinforcement learning (27). However, such efforts have primarily focused on model-free RL (28). In contrast, model-based RL requires network models with robust long-term memory capabilities, and realizing memory-enabled architectures, such as Recurrent Neural Networks (RNNs) (29), through spiking neuron still presents significant challenges (30). Despite emerging efforts to realize model-based reinforcement learning using spiking neural networks (31), the implemented network designs are comparatively rudimentary and continue to lag behind standard deep reinforcement learning approaches in both task complexity and performance.

Biological studies have demonstrated that the complex computational functions of dendrites enhance neuronal pattern separation (32), significantly improve the temporal sequence memory of neurons (33, 34), facilitate rule learning and the formation of long-term memory (35), and enable neurons to perform more complex perceptual decision-making tasks (36, 37). Inspired by the dendritic computational mechanisms of biological neurons, this study presents a multicompartmental neuron model with sequential nonlinear dendritic integration capabilities. Leveraging this model, we develop a spiking world model to enable a model-based deep reinforcement learning framework using spiking neural networks. Our Spiking-WM explicitly consists of a spiking state-space model, a spiking convolutional neural network (SCNN) for visual encoding, and a fully connected spiking network for policy optimization, forming a full-fledged spiking counterpart to classical deep world models. The proposed approach achieves performance comparable to conventional RNN-based DRL methods on DeepMind visual control tasks. Moreover, it outperforms existing spiking recurrent neural network models on speech sequence datasets, including Spiking Heidelberg Digits dataset (SHD), Texas Instruments/Massachusetts Institute of Technology Acoustic-Phonetic Continuous Speech Corpus (TIMIT), and LibriSpeech 100h. Overall, the main contributions of this work are as follows:

• We propose a multicompartment neuron model that incorporates dendritic computations, inspired by the structural characteristics of biological neurons. Our spiking MCN model exhibits long-term sequential memory capabilities by leveraging the biophysical dynamics of neuronal compartments, providing enhanced temporal processing capacity for spiking neural networks.

• Building upon the multicompartment neuron framework, we introduce a spiking world model and develop a fully spiking model-based DRL framework. Specifically, we implement an SNN-based Dreamer model, offering a biologically plausible alternative for model-based reinforcement learning in spiking neural networks.

• We evaluate the proposed Spiking-WM and MCN models on DeepMind visual control tasks and sequential speech datasets, including SHD, TIMIT, and LibriSpeech 100h. Experimental results demonstrate that our models surpass existing state-of-the-art spiking recurrent neural networks with the same parameter budget, establishing a benchmark for SNN-based sequence modeling and control tasks.

1. Related Work

Spiking neural networks, which model the dynamical processes of biological neurons and their spiking activity, are considered to represent a more efficient class of neural network models (38). Most spiking neural network research utilizes point-neuron models to represent biological neurons, such as the widely used LIF (39) neuron model, whose computing process relies solely on the somatic membrane potential. The ALIF (40) model proposes an adaptive spiking recurrent neural network to process time-domain datasets classification. Adaptive Exponential LIF (41) has been shown to accurately predict spike timing and replicate in vitro electrophysiological classes, especially in high-conductance states typical of in vivo cortical activity. The improved recurrent spiking neuron (42) modified LIF neurons to enhance their inherent recurrence for sequential learning, demonstrating that the improved SNN architecture achieves comparable accuracy to LSTMs on speech recognition tasks.

Point neuron models lack structural detail in neuronal modeling and are insufficient for accurately capturing the complex information integration and long-sequence processing capabilities of biological neurons. Consequently, there is growing research interest in computational modeling of dendritic processes and the development of multicompartmental neuron models. A two-compartment neuron model with distinct soma and dendrite components is introduced by (11, 12), which fine-tunes the dendritic synapse weights to facilitate dendritic predictions of somatic spike firing patterns. Recent studies have shown that burst-dependent synaptic plasticity, when integrated with apical dendritic activity and feedback pathways, enables hierarchical neural networks to achieve effective credit assignment and support deep learning mechanisms (43). The Bursting Cortico-Cortical Network (BurstCCN) (44) integrates bursting activity, short-term plasticity, and dendrite-targeting interneurons to propagate backpropagation-like error signals across multiple layers, enabling single-phase deep learning in cortical-inspired networks. The TC-LIF (18) built a two-compartment spiking neuron model to enhance long-term sequential modeling in SNNs. Target-based learning (15, 45) leverages the apical dendrite to receive the plasticity signal, enabling local modification of neuron weights.

However, the aforementioned multicompartmental neuron models provide only a simplified representation of dendritic computations, with most implementations incorporating a basic neuronal structure containing a single dendrite. Additionally, they largely overlook the nonlinear interactions and memory storage mechanisms involved in dendrite-to-soma signal transmission. To address these limitations, this study considers the distinct computational roles of dendrites at different locations within biological neurons, such as the apical and basal dendrites of pyramidal neurons. Furthermore, we model the cooperative computational processes that occur during signal transmission across dendrites at different locations.

Currently, spiking neural networks have widespread applications in various areas, including visual information processing (46), decision-making (47), predictive coding (48), sequential data recognition (49, 50), and solving constraint satisfaction problems (51). In addition, within the computational neuroscience domain, SNNs have been extensively employed to model cortical circuits and cognitive processes, including decision-making dynamics (52), working memory (53), and neural population coding (54). These studies demonstrate that SNNs can serve not only as engineering tools but also as biologically grounded models for understanding brain function. Nevertheless, the application of SNNs with deep architectures in complex decision-making tasks still remains comparatively limited. One major challenge for implementing RL algorithms using SNNs is the difficulty in training and converging the model, owing to the dynamic nature of RL environments. Some studies have applied spiking neural networks to reinforcement learning tasks. Using the ANN-to-SNN conversion method, the convolutional layers and fully connected layers in the DQN network are converted to SNN formulation applied to the Atari Breakout game (55, 56).

In the area of continuous control tasks, SNNs based on R-STDP have been applied to intelligent vehicle lane-keeping tasks (57) and training hexapod robots to walk (58). PopSAN (59) uses a population coding method with spiking neurons to implement an SNN-based reinforcement learning Actor-Critic algorithm, which is used for continuous control tasks in MuJoCo simulation environment. A two-module spiking neural network model is introduced that leverages biologically inspired mechanisms, such as “dreaming” (offline learning in a model-based simulated environment) and “planning” (online learning), to enhance reinforcement learning efficiency without requiring the detailed storage of past experiences (31). The MCS-FQF (60) presents a brain-inspired deep distributional reinforcement learning algorithm based on multicompartment neuron model SNN and population coding method, which significantly enhances computational power and performance in Atari games.

Nevertheless, existing research on reinforcement learning with spiking neural networks has primarily focused on implementing model-free RL methods, while the application of SNNs in model-based RL remains largely unexplored. In this study, we develop a spiking world model based on a multicompartmental neuron model with nonlinear dendritic computations, enabling the application of SNNs in model-based deep reinforcement learning.

2. Preliminary and Methodology

In this section, we first formulate the reinforcement learning paradigm as a Markov Decision Process and introduce the foundations of model-based reinforcement learning, which enables policy optimization through a learned world model. We then describe the spiking neuron model, emphasizing its biologically inspired membrane potential dynamics and temporal processing capability. Building upon this, a multicompartment neuron model with nonlinear dendritic integration is proposed to simulate hierarchical signal modulation between basal and apical dendrites. Finally, we develop a spiking world model that integrates convolutional, recurrent, and multicompartment spiking components, jointly optimized via Spatio-Temporal Backpropagation to achieve efficient and biologically plausible sequential decision-making.

2.1. Model-Based Reinforcement Learning.

Reinforcement learning can be formed as a Markov Decision Process (MDP), which can be described by element matrix: $<\mathcal{S},\mathcal{A},{\mathcal{P}}_{s{s}^{\prime }}^a,{\mathcal{R}}_{s{s}^{\prime }}^a,\gamma >$. At discrete time step $t\in [1; T]$, the agent takes action with probability $a_t\sim p(a_t|s_t)$ with $a_t\in \mathcal{A}; s_t\in \mathcal{S}$. The state transiting to $s_{t+1}\sim p(s_{t+1}\in \mathcal{S}|s_t,a_t)$ when agent taking action $a_t$ at state $s_t$ with probability ${\mathcal{P}}_{s_t{s}_{t+1}}^{a_t}$, and the agent receives the reward $r_t\sim {\mathcal{R}}_{s_t{s}_{t+1}}^{a_t}$ generated by the unknown environment. Typically, the main objective of reinforcement learning is to maximize the expected cumulative reward $E_p(\sum _{t=1}^T\gamma ·r_t)$.

Model-based reinforcement learning represents the world environment with latent dynamic model, using probability functions $q(s_t|s_{t-1},a_{t-1})$ to fit the state transition process ${\mathcal{P}}_{s_t{s}_{t+1}}^{a_t}$. Similarly, the rewards from the environment can also be predicted by $q(r_t|s_t)$. Compared to model-free reinforcement learning, model-based reinforcement learning can reduce the number of interactions with the environment by learning policies through the world model. This alleviates the trial-and-error cost of reinforcement learning algorithms and increases the feasibility of deploying these algorithms in real-world environments.

2.2. Spiking Neuron Model.

The spiking neuron model simulates the propagation of electrical potentials and the firing processes of biological neurons. Taking the LIF neuron as an example, which is designed primarily for balancing the intricate complexity of biological neuron details with computational tractability, and is widely used in the construction of spiking neural networks and brain simulations. The membrane potential dynamics of the LIF neuron are described by Eq. 1, and the parameter τ is the somatic membrane time constant. When the membrane potential $U_t$ exceeds a threshold $V_{\mathit{th}}$, the neuron fires a spike $S_t$ and the membrane potential is reset $U_t\leftarrow V_{\mathit{res}}; U_t>V_{\mathit{th}}$.

$$\begin{array}{c}\hfill \tau \frac{d{U}_t}{\mathit{dt}}=-U_t+x_t; U_t<V_{\mathit{th}}\end{array}$$ [1]

$$\begin{array}{c}\hfill S_t=\left\{\begin{array}{c}1\hspace{1em}{U}_t>V_{\mathit{th}}\hfill \\ 0\hspace{1em}{U}_t<V_{\mathit{th}}\hfill \end{array}\right.\end{array}$$ [2]

Although the membrane potential dynamic process of LIF spiking neurons has the inherent ability to process sequential information, it only considers the passive updating of membrane potential and spike firing process. This approach ignores the morphological and structural characteristics of neurons and models neurons as point-like structures without taking into account the local computational capabilities and functional enhancements provided by different parts of biological neurons, such as dendrites and soma.

2.3. Multicompartment Neuron with Sequential Nonlinear Dendritic Integration.

Spiking neurons, by modeling the dynamic biophysical processes of biological neurons, possess the inherent ability to process sequential information due to the mathematical form of their dynamic differential equations. In this work, we propose a method for sequential information processing based on multicompartment neurons and apply it to model-based deep reinforcement learning. The MCN model is inspired by the hierarchical structure of biological neurons, where different parts of the dendritic tree, specifically the basal and apical dendrites, play distinct roles in processing incoming signals. While the basal dendrites receive input from synapses and contribute to the local computation of membrane potential, the apical dendrites are involved in modulating the overall neuron behavior by influencing the somatic membrane potential.

The apical dendrite, in particular, has a modulatory effect on the soma, a mechanism that is crucial in many biological models (61, 62). The apical input tends to regulate the sensitivity of the neuron to inputs received at the basal dendrites, allowing for more flexible and context-dependent processing of information. Specifically, the input from the apical dendrite can either enhance or suppress the activity of the soma, depending on the synaptic weight and the dynamics of the input signals. We formulate the MCN model with local membrane potential computation processes of the basal and apical dendrites as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\tau }_b\frac{d{V}_t^b}{\mathit{dt}}=-V_t^b+x_t^b\hfill \\ x_t^b=W^b{S}_t^{\mathit{in}}+W^{h,b}{S}_t^{h-1}\hfill \end{array}\right.\end{array}$$ [3]

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\tau }_a\frac{d{V}_t^a}{\mathit{dt}}=-V_t^a+x_t^a\hfill \\ x_t^a=W^a{S}_t^{\mathit{in}}+W^{h,a}{S}_t^{h-1}\hfill \end{array}\right.\end{array}$$ [4]

$$\begin{array}{c}\hfill \tau \frac{d{U}_t}{\mathit{dt}}=-U_t+\sigma (V_t^a)(\frac{g_B}{g_L}(V_t^b-U_t)+W^s{S}_t^{\mathit{in}}),\end{array}$$ [5]

where the $V_t^b$ and $V_t^a$ is basal and apical dendritic potentials, respectively. Both change dynamically based on the external sources of spike signals $S_t^{\mathit{in}}$ and internal signals $S_t^h$ representing hidden states. The basal and apical dendrites are equipped with synaptic weights $[W^b,W^{h,b}]$ and $[W^a,W^{h,a}]$, which are adjustable parameters providing the dendrites with local learning capabilities. The modulatory influence of the apical dendrite is reflected in the term $\sigma (V_t^a)$ in the somatic potential equation.

This term effectively scales the input from the basal dendrite based on the apical dendritic voltage, illustrating how the apical dendrite can adjust the neuron’s response to inputs depending on the current state of the apical dendritic potential. This mechanism enables the neuron to perform context-dependent processing, a feature that is crucial for learning and decision-making in more complex environments, as explored in biological neural systems (61). Same as the LIF model, the MCN fires a spike when the somatic potential $U_t$ exceeds the threshold. In order to simplify the neuronal computation model, when a MCN fires a spike, both the dendritic potential and the soma potential of the MCN are reset simultaneously.

2.4. Spiking World Model.

To build a biologically plausible world model capable of sequential decision-making, we design a spiking world model that integrates three major components:

• a spiking states space model based on the proposed multicompartment neuron,

• a spiking convolutional neural network (SCNN) for visual feature extraction and spike-based encoding, and

• a fully connected spiking layer that fuses latent state representations and actions to generate the sequential model inputs.

This architecture goes beyond the structure of classical deep networks by combining these components into a unified spiking computation framework. The membrane potential dynamics of spiking neurons enable intrinsic temporal integration, while the hierarchical structure allows the world model to learn both spatial and temporal representations.Concretely, the SCNN encoder processes high-dimensional visual inputs $O[t]$ and produces a spike-encoded feature representation $S^{\mathit{ob}}[t]$. The fully connected SNN module then integrates this observation with the previous latent state $Z^{h-1}[t]$ and previous action $A^{h-1}[t]$, producing a sequential spike signal $S^{\mathit{in}}[t]$ as input to the state-space model. The MCN-based SSM further captures temporal dynamics and generates the latent state trajectory $Z^h[t]$, which is subsequently used to predict rewards $\widehat{R}[t]$, continuation signals $\widehat{C}[t]$, and future observations $\widehat{O}[t]$. This design results in a comprehensive spiking world model that integrates spatial feature extraction, temporal modeling, and decision-relevant latent state inference in a single biologically inspired architecture.

$$\begin{array}{c}\hfill S^{\mathit{ob}}[t]=SCNN(O[t]); S^{\mathit{in}}[t]=SNN(Z^{h-1}[t],A^{h-1}[t])\end{array}$$ [6]

We used the Euler method to numerically expand the Eqs. 3–5 MCN formulation, proposing the following sequential model:

$$\begin{array}{c}\hfill V^b[t]=V^b[t-1]+(W^b{S}^{\mathit{in}}[t]+W^{h,b}{S}^{h-1}[t]-V^b[t-1])\mathrm{\Delta }t/{\tau }_b\end{array}$$ [7]

$$\begin{array}{c}\hfill V^a[t]=V^a[t-1]+(W^a{S}^{\mathit{in}}[t]+W^{h,a}{S}^{h-1}[t]-V^a[t-1])\mathrm{\Delta }t/{\tau }_a\end{array}$$ [8]

$$\begin{array}{c}\hfill h[t]=\frac{g_B}{g_L}(V^b[t]-U[t-1])+W^s{S}^{\mathit{ob}}[t];\hspace{1em}z[t]=\sigma (V^a[t])\end{array}$$ [9]

$$\begin{array}{c}\hfill U[t]=U[t-1]+(z[t]\odot h[t]-U[t-1])\mathrm{\Delta }t/\tau \end{array}$$ [10]

$$\begin{array}{c}\hfill S^h[t]=\mathrm{\Theta }(U[t]-V_{\mathit{th}}),\end{array}$$ [11]

where $\mathrm{\Theta }(·)$ is the Heaviside step function. When firing a spike, the MCN resets its membrane potential as $U[t]=U[t](1-S^h[t])$. We use the Sigmoid function with parameter β as the gating function for the apical dendritic membrane potential with

$$\begin{array}{c}\hfill \sigma (x)=\frac{1}{1+e^{-\beta x}}\end{array}$$ [12]

As illustrated in Fig. 1A, the computational process of the MCN is unfolded in both temporal and spatial dimensions. At time steps $t=1,2,3,4$, the input spikes $S^{\mathit{in}}[t]$ together with the recurrent state spikes $S^{h-1}[t]$ are projected to the basal and apical dendrites, generating the dendritic membrane potentials $V^b[t]$ and $V^a[t]$. These potentials further produce the membrane potential update term $h[t]$ and the modulatory signal $Z[t]$ (Eq. 9), which are then integrated at the soma to update the neuronal state. The state spiking encoding $S^h[t]$ of the multicompartment neuron model, along with the dynamic updates of the neuron’s dendritic and somatic membrane potentials, endows it with long-short term memory capabilities. The observation encoding and sequence model output are used to generate the model state representation $Z^h[t]$:

$$\begin{array}{c}\hfill S^h[t]=MCN(S^{\mathit{in}}[t],S^{h-1}[t])\end{array}$$ [13]

$$\begin{array}{c}\hfill Z^h[t]\sim q_{\theta }(Z^h[t]|S^{\mathit{ob}}[t],S^h[t]))\end{array}$$ [14]

Fig. 1.

The spiking world model based on MCN. (A) Biological structure of pyramidal neurons (Left) and the multicompartment neuron model (Right). (B) The architecture of spiking world model.

Similar to the Dreamer model, we use the SNNs to output distributional state parameters instead of the direct values of the state representation. The model state $Z^h[t]$ and the spiking information output $S^h[t]$ from the MCN are used to predict the reward $\widehat{R}[t]$ and the continuous signal $\widehat{C}[t]$ for each episode. The decoder uses a spiking deconvolutional network to obtain predictions of observation $\widehat{O}[t]$ from the model’s latent state. The predictive outputs of the world model are used to select actions and evaluate states.

$$\begin{array}{c}\hfill {\widehat{Z}}^h[t]\sim p_{\theta }({\widehat{Z}}^h[t]|S^h[t])\end{array}$$ [15]

$$\begin{array}{c}\hfill {\widehat{R}}^h[t]\sim p_{\theta }({\widehat{R}}^h[t]|S^h[t],Z^h[t])\end{array}$$ [16]

$$\begin{array}{c}\hfill {\widehat{C}}^h[t]\sim p_{\theta }({\widehat{C}}^h[t]|S^h[t],Z^h[t])\end{array}$$ [17]

$$\begin{array}{c}\hfill \widehat{O}[t]\sim p_{\theta }(\widehat{O}[t]|S^h[t],Z^h[t])\end{array}$$ [18]

Fig. 1B shows the spiking world model and action selection process. We use an SNN model to construct the policy model, which includes both actor and critic networks. The actor network integrates the spiking sequence representation of the world model state $[S_t^h,Z_t^h]$ at spiking time step t to determine the action selection $a^h$ at the h-th step within the environmental context. The critic network generates environmental state values $v^h$ based on the spiking sequence representation of the model as an estimate of the expected returns obtainable from the current state.

$$\begin{array}{c}\hfill \begin{array}{cc}\text{Actor:}\hfill & a^h\sim SNN(S_t^h,Z_t^h)\hfill \\ \text{Critic:}\hfill & v^h\sim SNN(S_t^h,Z_t^h)\hfill \end{array}\end{array}$$ [19]

We train the actor and critic using the bootstrapped method. Within the predicted horizon of length H, the λ-returns are calculated based on the current reward predictions ${\widehat{R}}^{1:H}$ and environmental condition predictions ${\widehat{C}}^{1:H}$, as shown by Eq. 20. As in the Dreamer method, SNN based actor and critic are trained by discrete regression and maximum returns.

$$\begin{array}{c}\hfill {\mathcal{R}}^{\lambda ,h}={\widehat{R}}^h+\gamma {\widehat{C}}^h[t]((1-\lambda )v^{h+1}+\lambda {\mathcal{R}}^{\lambda ,h+1}) \lambda {\mathcal{R}}^{\lambda ,H}\doteq v^H\end{array}$$ [20]

The loss function of the world model consists of the model prediction loss ${\mathbb{L}}_{\mathit{pred}}(\theta )$ and the posterior estimation loss ${\mathbb{L}}_{\mathit{model}}(\theta )$, as shown in Eq. 21. The objective of the model prediction loss ${\mathbb{L}}_{\mathit{pred}}(\theta )$ is to optimize the maximum likelihood of the decoder, reward predictor, and continue predictor under the true observations, rewards, and continue condition values of the environment. The model dynamics prediction loss ${\mathbb{L}}_{\mathit{model}}(\theta )$ aims to improve the sequence model MCN’s prediction of the next state by minimizing the KL divergence between the prior predictions and the posterior representations using Bayesian estimation methods.

$$\begin{array}{c}\hfill {\mathbb{L}}_{\mathit{wm}}(\theta )=E_{q_{\theta }}[{\displaystyle \sum _{h=1}^H}({\mathbb{L}}_{\mathit{pred}}(\theta )+{\mathbb{L}}_{\mathit{model}}(\theta ))]\end{array}$$ [21]

$$\begin{array}{c}\hfill {\mathbb{L}}_{\mathit{pred}}(\theta )={\displaystyle \sum _{x^h}}-ln{p}_{\theta }(x^h|,S^h[t],Z^h[t]) x^h\in \{O^h,r^h,c^h\}\end{array}$$ [22]

$$\begin{array}{c}\hfill {\mathbb{L}}_{\mathit{model}}(\theta )=KL[q_{\theta }(Z^h|S^{\mathit{ob}}[t],S^h[t])||p_{\theta }({\widehat{Z}}^h[t]|S^h[t])]\end{array}$$ [23]

Given that the long-short term memory mechanisms in SNNs and MCNs rely on the temporal integration of spike sequences, we optimize the parameters of SNNs using the Spatio-Temporal Backpropagation (STBP) method (60). STBP integrates surrogate-gradient techniques with backpropagation through time (BPTT), thereby producing stable and scalable gradient estimates for nondifferentiable spike events. In contrast to purely local or hand-crafted plasticity rules, STBP facilitates the direct optimization of task-level objectives (e.g., reconstruction, reward prediction, actor/critic losses) across multiple layers and extended temporal horizons. This capability is crucial for attaining competitive performance on the large-scale control and sequence benchmarks employed in this work. Empirically, STBP has demonstrated its scalability to deep SNN architectures and has achieved state-of-the-art results on various temporal processing tasks by effectively preventing vanishing and exploding gradient problems during temporal credit assignment via BPTT-style propagation, and by addressing spike nondifferentiability with carefully designed surrogate gradients (63). Due to its effectiveness in temporal credit assignment, compatibility with complex loss functions required by world models, and proven practicality for deep network architectures, STBP was selected as the primary optimization strategy in this study. At the discontinuous points of spike firing, we use a surrogate function $g^{\prime }(x)$ as the gradient of spike firing moments:

$$\begin{array}{c}\hfill \frac{\partial S^h[t]}{\partial U[t]}⟵g^{\prime }(x)(U[t]-V_{\mathit{th}})\end{array}$$ [24]

$$\begin{array}{c}\hfill g^{\prime }(x)=\left\{\begin{array}{cc}0,\hfill & |x|>\frac{1}{\alpha }\hfill \\ \hfill \\ -{\alpha }^2|x|+\alpha ,\hfill & |x|\le \frac{1}{\alpha }\hfill \end{array}\right.\end{array}$$ [25]

3. Experiments

To evaluate the performance of the proposed spiking world model, we conducted experiments on the DeepMind Visual Control Suite. Additionally, to investigate the long-sequence memory capability of the MCN model, we tested the performance of different neurons on the speech datasets SHD, TIMIT, and LibriSpeech 100h. Furthermore, to assess the impact of MCN parameters on model performance, we employed grid search and adaptive parameter learning methods to analyze the task performance and long-sequence memory capacity of the Spiking-WM model under different dendritic parameter configurations of the MCN.

3.1. DeepMind Visual Control Experiments.

We conducted experiments on visual input control tasks in DeepMind Control (DMC) Suite, comparing the performance of MCN-based spiking world model across 19 continuous control tasks. For comparison, we evaluated the capabilities of the traditional GRU-based world model algorithm Dreamer (22). In addition, to demonstrate the critical role of the MCN model in the spiking world model and its advantages over other point-neuron models and simple two-compartment neuron models, we constructed equivalent spiking world models based on several alternatives. These included the aLIF (40) model, formed by augmenting the baseline LIF (64) neuron for processing long-sequence spiking data, the LIF-RNN (42) model, which enhances internal recurrence, the parallelized spiking neuron (PSN) (65) and the two-compartment neuron (TC-LIF) (18) model. Experiments were conducted on the DeepMind Visual Control suite using identical model parameters and training settings. Detailed model and experimental parameter settings can be found in SI Appendix, Table S1.

DMC tasks based on visual input require the model to accurately represent the features of the current observation images and make correct control actions based on visual perception. In the experiment, we use a 4-layer spiking convolutional network to process image observations from the environment, encoding the image observation into the spiking sequence space. All compared models use the same SCNN model structure and parameter settings to process visual inputs and employ the same SNN-based Actor-Critic policy learning.

In the DMC experiments, the sequence prediction model spiking-WM consists of 512 multicompartment neurons. For the sake of fairness in comparison, we increased the number of neurons and the dimensionality of weights in the other baseline models to ensure consistency in the total number of parameters. Detailed parameter statistics and experimental results are provided in Table 1. The decoder uses a 4-layer spiking transposed convolutional network, and the predictor is a fully connected LIF neuron SNN.

Table 1.

Scores for DMC visual-input control experiments at 1M frames. Values in bold indicate the best performance for each task or cases in which two models achieve comparable top scores

Task	Dreamer (GRU) (22)	LIF (64)	aLIF (40)	PSN (65)	LIF-RNN (42)	TC-LIF (18)	Spiking-WM (Ours)
Parameters (M)	19.10	19.11	18.85	18.84	19.11	18.84	18.84
Env steps (M)	1.0	1.0	1.0	1.0	1.0	1.0	1.0
Acrobot swingup	219.1	104.0	79.1	95.8	125.5	106.5	113.7
Cartpole balance	970.7	663.0	502.4	788.6	957.7	965.3	973.3
Cartpole balance sparse	991.2	991.5	748.3	832.6	718.2	992.4	989.6
Cartpole swingup	810.1	774.1	614.0	575.9	772.6	667.7	791.0
Cartpole swingup sparse	480.2	261.3	217.1	240.6	211.1	265.5	420.1
Cheetah run	720.1	515.5	558.1	283.9	510.8	517.8	577.2
Cup catch	958.2	768.2	494.2	915.8	523.0	938.7	960.3
Finger spin	637.3	416.2	433.5	472.4	509.3	657.8	682.0
Finger turn easy	763.5	489.2	542.9	592.7	706.1	688.5	727.5
Finger turn hard	397.2	193.6	201.3	289.1	269.4	369.1	406.3
Hopper hop	373.4	104.9	145.7	114.4	215.6	205.4	234.9
Hopper stand	913.6	650.6	704.1	493.1	715.1	864.2	905.3
Pendulum swingup	800.4	660.9	665.0	556.4	669.2	746.2	796.5
Quadruped run	690.4	440.7	425.3	303.9	485.9	372.1	515.3
Quadruped walk	387.7	259.7	260.6	280.3	320.3	302.2	350.7
Reacher easy	736.2	403.4	422.0	511.5	503.1	613.2	701.3
Walker run	705.5	320.4	325.8	264.1	352.2	460.8	541.6
Walker stand	965.9	799.8	724.6	790.2	784.6	916.7	960.5
Walker walk	955.1	795.2	871.5	706.0	924.0	916.1	938.5
Mean	709.2	505.9	470.2	479.3	540.7	608.7	662.4
GRU%	–	67.0	63.2	64.3	73.8	81.9	90.4

The final evaluation results were obtained after training for one million (1M) frames. The results in Fig. 2 and Table 1 demonstrate that our proposed Spiking-WM model outperforms other spiking neuron baselines in terms of learning speed and final scores, achieving results comparable to the ANN-based Dreamer model. On the average score across 19 tasks, the Spiking-WM model achieves 90.4% of the performance of the ANN-based Dreamer(GRU) model, surpassing the TC-LIF model, which reached only 81.9% of Dreamer(GRU) scores, and outperforming point-neuron models like LIF and its derivative models by 31%38%.

Fig. 2.

Evaluation scores for DeepMind visual control experiments.

Additionally, the Spiking-WM model exceeds the performance of Dreamer(GRU) in four tasks: Cartpole Balance, Cup Catch, Finger Spin, and Finger Turn Hard. In contrast, other spiking neuron-based world models generally do not outperform Dreamer(GRU) in terms of task performance, with the exception of TC-LIF on the Cartpole Balance Sparse task. These results demonstrate the advantages of our proposed MCN over existing simple neuron models in implementing the Spiking-WM model and highlight the potential of biologically plausible brain-inspired spiking neural networks for model-based reinforcement learning algorithms. Using the trained model, we sampled the predictions of world models with different neurons for the first 50 steps of the Walker Run task, as shown in SI Appendix, Fig. S1. The T = 0 represents the input state, and T = 5 to 50 represents the model’s Decoder prediction outputs. The results show that our proposed MCN-based world model, Spiking-WM, achieves better predictive capabilities, leading to higher final task scores compared to other neural world models.

3.2. Sequential Speech Data Experiments.

To validate the potential of the proposed multicompartment neuron model in handling sequence models, we conducted tests on the speech datasets SHD, TIMIT, and LibriSpeech; the results are shown in Table 2. In contrast to the DMC task, where the proposed multicompartment neuron was integrated into a full spiking world model, here we design a simpler network architecture to directly evaluate the intrinsic long-sequence memory capability of the MCN itself. Specifically, we construct a two-layer fully connected spiking network using MCN neurons, without incorporating the additional modules required for model-based reinforcement learning. Different neural models share an identical network architecture and training parameter configurations when applied to the same dataset, enabling a fair comparison of the memory capacity endowed by different neuron models. For the SHD dataset, the network adopts a fully connected structure comprising two hidden layers, each containing 128 neurons, with the STSC (66) spike processing method implemented at the input layer, which has been demonstrated to significantly enhance the performance of SNNs in handling spike information (67, 68).

Table 2.

Comparison of different neuron types on long-sequence datasets

Dataset	Method	Neuron type	Params (M)	Acc(%)
SHD	GRU	Rec	3.5	89.92
	LIF (64)	FF	2.4	72.06
	aLIF (40)	Rec	2.5	88.64
	PSN (65)	FF	2.0	85.86
	LIF-RNN (42)	Rec	2.1	80.34
	TC-LIF (18)	FF	2.1	89.45
	MCN(Ours)	Rec	2.5	89.57
TIMIT	GRU	Rec	14.1	83.75
	LIF (64)	FF	13.9	78.35
	aLIF (40)	Rec	11.4	82.10
	PSN (65)	FF	11.8	80.05
	LIF-RNN (42)	Rec	19.6	81.28
	TC-LIF (18)	FF	11.9	80.90
	MCN(Ours)	Rec	14.0	83.01
LibriSpeech	GRU	Rec	5.1	90.22
100h	LIF (64)	FF	3.9	80.16
	aLIF (40)	Rec	3.7	87.08
	PSN (65)	FF	2.6	87.56
	LIF-RNN (42)	Rec	3.8	88.25
	TC-LIF (18)	FF	3.9	88.59
	MCN(Ours)	Rec	3.9	88.77

In the “Neuron Type” category, “FF” denotes neurons that primarily compute information in a feedforward manner, while “Rec” represents neurons with an enhanced design for recurrent information processing within the neuron itself.

We conducted comparative experiments on the TIMIT and LibriSpeech 100h datasets using a two-layer neural network with recurrent connections, following the experimental settings in paper (42). The TIMIT dataset is a widely used benchmark for speech recognition, containing phonetically diverse sentences spoken by multiple speakers, while LibriSpeech 100h is a subset of the larger LibriSpeech corpus, consisting of approximately 100 h of transcribed English speech derived from audiobook recordings. The network architecture follows a two-layer recurrent structure, maintaining the same parameter configurations as in the referenced study LIF-RNN (42). Experimental results demonstrate that our proposed MCN outperforms other spiking neuron models on the SHD, TIMIT, and LibriSpeech 100h datasets, achieving performance comparable to that of GRU, which is consistent with the results of the DeepMind visual control experiments.

3.3. Analysis of MCN Parameters.

3.3.1. Membrane time constant.

Compared to the LIF neuron model, the multicompartment neuron model introduces more parameters through the refinement of dendritic structures, thus providing more adjustable components for the neuron model. In our experiment, we investigated the impact of MCN parameters on model performance. For the membrane time constants of the soma (τ) and dendritic membrane (${\tau }_a,{\tau }_b$), we applied the approach from PLIF (69), treating them as learnable parameters and conducting experiments on the DMC task. The results are shown in Fig. 3A. In the four experimental tasks, the learnable membrane potential time parameters in the Finger Spin and Reacher Easy tasks led to performance improvements. However, in the Cup Catch task, the learnable τ did not yield higher scores compared to the fixed membrane time constant, and even resulted in a negative performance gain in the Walker Walk task.

Fig. 3.

Experimental results on the impact of MCN parameters on model performance. (A) Comparison results of experiments on MCN with and without learnable membrane potential time constant. (B) Effects of dendritic inputs from different parts of neurons on performance. (C) The distribution of soma, basal dendrite, and apical dendrite time parameters after learning. (D) Grid search of basal dendrite conductance parameters and apical dendrite gate parameter on SHD dataset. (E) Grid search of basal dendrite conductance parameters and apical dendrite gate parameter on Walker Walk. (F) Performance comparison of neuronal numerical simulation methods.

The statistical distribution of the trained membrane potential time constant for dendrites and the soma is presented in Fig. 3C. Although the final distributions of the optimized membrane time constant exhibit slight variations across the four tasks, a significant portion of the distribution regions remain overlapping. Combined with the results in Fig. 3A, this suggests that adjustments to different membrane time constants in the MCN model do not substantially impact overall model performance. Instead, the key contribution of the MCN model lies in the incorporation of dendritic computational processes.

3.3.2. Ablation study on dendritic inputs.

To investigate how different dendritic compartments contribute to the computational performance of the proposed multicompartment neuron model, we conducted an ablation study by selectively removing inputs to either the basal or apical dendrites. Specifically, we compared three configurations:

1. Full: the recurrent state projects to both basal and apical dendrites (the default setting in this work).

2. w/o Basal: the recurrent state projects only to apical dendrites.

3. w/o Apical: the recurrent state projects only to basal dendrites.

The results are shown in Fig. 3B. Across multiple DeepMind Control tasks (Cup Catch, Finger Spin, Hopper Stand, Quadruped Run, and Walker Walk), the Full configuration consistently achieves the best performance. In contrast, removing either basal or apical inputs leads to a clear decline in performance. For instance, in the Hopper Stand and Finger Spin tasks, the absence of apical inputs results in a dramatic performance drop, while in Quadruped Run and Walker Walk tasks, performance is significantly impaired without basal inputs.

These results highlight the synergistic role of basal and apical dendritic inputs: basal dendrites provide essential local integration of recurrent signals, while apical dendrites modulate neuronal excitability and context-dependent processing. Only when both dendritic compartments receive inputs can the model maintain optimal long-term prediction and control capabilities. Consistent with the formulation in Eqs. 3 and 4, the basal and apical dendrites play complementary roles in recurrent information integration, and their cooperative processing is indispensable for achieving the best performance.

Furthermore, we analyzed the impact of the basal dendritic conductance parameter $\frac{g_B}{g_L}$ and the apical dendritic nonlinear function parameter β on the MCN model. Using a grid search approach, we tested the model performance with different values of $\frac{g_B}{g_L}$ and β on the SHD dataset and the DMC Walker Walk task. The parameter search results indicate that the synergistic interaction between apical and basal dendrites significantly impacts the performance of the MCN model. On the SHD dataset, higher accuracy is observed in the Upper-Left region of Fig. 3D ($\frac{g_B}{g_L}\le 1.8$, $\beta \le 1.8$). In the Walker Walk task, $\frac{g_B}{g_L}$ and β exhibit a more distinct adaptation relationship, with high scores concentrated in the Upper-Right region of Fig. 3E. Moreover, better performance is more likely to be achieved when β falls within the range of 0.5 to 1.0 and $\frac{g_B}{g_L}$ within the range of 1.0 to 1.5.

3.3.3. Comparison of numerical simulation methods.

As noted in prior work, a clock-based Euler discretization of continuous spiking neural dynamics may introduce systematic numerical errors (70). To examine whether more advanced integration schemes can improve the performance of the Spiking-WM model, we conducted additional experiments comparing three numerical simulation methods: Euler, Heun, and Runge–Kutta 4 (RK4) (71). The results across representative DeepMind Control tasks are shown in Fig. 3F. The detailed derivation of the Heun and RK4 numerical simulations for the MCN model can be found in SI Appendix.

From the comparison, we observe that although Heun and RK4 methods provide higher-order accuracy in numerical integration, the overall model performance does not improve. In fact, both methods generally yield lower scores compared to Euler. This phenomenon arises because advanced integration schemes require more intermediate computations at each simulation step, which substantially increases the depth and complexity of the computational graph. When training spiking neural networks with surrogate gradient-based backpropagation, such additional intermediate states amplify gradient approximation errors and hinder effective parameter optimization.

In contrast, the Euler method, despite its lower numerical precision, produces more stable gradients and enables more effective end-to-end optimization. Consequently, models simulated with Euler discretization achieve higher task performance in practice. These findings highlight a trade-off between numerical accuracy and trainability, while advanced solvers reduce local discretization error, their computational complexity can degrade the optimization dynamics of STBP-trained spiking networks. Our results suggest that Euler discretization strikes a favorable balance for the current learning framework, making it a practical choice for large-scale SNN training.

3.4. Analysis of MCN Spiking Activity.

To analyze the impact of multicompartmental dendritic potential on neuronal spiking activity, we recorded the dendritic membrane potential of MCN in Spiking-WM during model prediction on the Walker Run task in DMC, as shown in Fig. 4 A and B. Furthermore, we perform Pearson correlation analysis and Phase Locking Value (PLV) analysis between the basal and apical dendritic potentials and the somatic membrane potential of MCN to investigate the relationship between dendritic membrane potentials and the somatic membrane potential. The results of these analyses are shown in Fig. 4 C and D, respectively. Fig. 4C presents the Pearson correlation statistics for the dendritic (basal and apical) membrane potentials with the somatic membrane potential, computed across 512 MCNs. The results indicate a strong positive correlation between basal dendritic membrane potential and the somatic potential (Basal/Soma), with an average correlation coefficient around 0.75. In contrast, the apical dendritic membrane potential shows a negative correlation with the soma (Apical/Soma), with correlation coefficients mostly between −0.25 and −0.5. This suggests that basal and apical dendrites have different roles in modulating the somatic membrane potential.

Fig. 4.

Statistics of spike firing activity and correlation analysis of dendritic/somatic membrane potentials. (A) The apical dendritic potential of MCNs. (B) The basal dendritic potential of MCNs. (C) Pearson correlation statistics between dendritic and somatic membrane potentials. (D) Phase locking value analysis between dendritic and somatic membrane potentials. The orange solid line represents the median, while the green dashed line represents the mean. (E) The spike sequential of MCNs during the model prediction process. (F) Details of the dendritic membrane potential and spike firing of the 200th neuron.

In addition to the Pearson correlation, we performed a Phase Locking Value (PLV) analysis to quantify the synchronicity between dendritic and somatic membrane potentials. As shown in Fig. 4D, both basal and apical dendritic potentials exhibit synchronization with the somatic potential. However, the basal dendrites demonstrate a stronger synchronization, as indicated by the higher median PLVs compared to the apical dendrites. This finding further emphasizes the greater role of basal dendrites in driving somatic membrane potential activity. During the model prediction phase, we observed synchronized activity between the dendritic membrane potentials, leading to burst firing patterns of the neuron, as shown in Fig. 4E.

The Fig. 4F presents the recorded dendritic membrane potential, somatic membrane potential, and spike firing of a single neuron (Neuron 200) within the 300 ms to 450 ms interval. To facilitate observation of the regulatory effect of apical dendrites on neuronal activity, we have included the gating signal $z[t]=\sigma (V^a[t])$ (the blue line) of the apical dendrite in the figure. The neuron exhibits significant spike firing activity during the 350 ms to 380 ms period, during which the gating signal of the apical dendritic membrane potential remains within the range of 0.5 ∼ 1.0 for an extended duration. At this time, the apical dendritic membrane potential is greater than zero. In contrast, although the basal dendritic membrane potential exhibits larger fluctuations, it has a lesser influence on the neuron’s spiking activity. Even during periods of high fluctuation in the basal dendrite, the neuron does not fire if the apical dendritic gating signal is below 0.5, preventing spike generation. These results support the hypothesis that in the MCN model, the basal dendrite plays a critical role in processing input information and providing the foundation for neuronal spike firing. The apical dendrite, on the other hand, exerts a modulatory effect that is crucial for the complexity of information processing and spike firing patterns. While the basal dendrite facilitates the basic activation of the neuron, the apical dendrite’s modulation of the somatic potential significantly influences the complexity and dynamics of neuronal activity. This underscores the complementary roles of the basal and apical dendrites in enabling both the basic and more intricate aspects of neuronal computation and firing behavior.

3.5. Energy Efficiency.

One of the notable advantages of spiking neural networks lies in their low-power computational characteristics. Since the output and signal transmission of spiking neurons are expressed in a binary “0-1” form, synaptic operations in SNNs can be implemented as accumulated (AC) operations. In contrast, ANNs rely on the more energy-intensive multiply-and-accumulate (MAC) operations during input computation. In this work, we evaluate the energy efficiency of the proposed MCN model in comparison with GRUs and the simpler LIF neurons. The results are presented in Table 3. We measured the average neuronal firing rate $\overline{r}$ during the testing phase of the trained model on the DMC task. The computational cost of a neuron is derived under the condition where the Fan-in is set to m and the Fan-out is set to n.

Table 3.

Energy efficiency comparison of GRU, LIF, and MCN neuron models, with m and n denoting neuron fan-in and fan-out, r the input spiking rate, and T the spiking period

Neuron	$\overline{r}$	Computational cost	SynOps*/MACs# (million)	Energy cost (μJ) (neuron/model)
GRU	–	3n(m+n)	1.57#	7.24/139.88
LIF	0.26	r*T*n(m+n)	1.56*	2.11/25.86
MCN	0.12	r*T*n[2(m+n)+n]	1.21*	1.96/12.94

Energy cost includes the consumption of GRU, LIF, and MCN models, as well as the total energy expenditure of the reinforcement learning framework.

Energy consumption is estimated based on the 45 nm CMOS energy benchmark, where the energy cost of an AC operation is $E_{\mathit{AC}}=0.9 \text{pJ}$, and that of a MAC operation is $E_{\mathit{MAC}}=4.6 \text{pJ}$ (72). Compared with the GRU model, the spiking neuron model achieves a substantial reduction in energy consumption. Although the proposed MCN model exhibits higher theoretical computational complexity than the LIF neuron, its enhanced internal capacity for information retention and processing effectively suppresses the neuronal firing rate, thereby yielding lower overall energy consumption than the LIF-based model. Relative to the GRU model, the energy consumption of the MCN is reduced by an order of magnitude.

4. Conclusion

This work introduces a multicompartment neuron-based spiking world model, demonstrating its effectiveness in model-based SNN deep reinforcement learning and long-term memory tasks. Through comprehensive evaluations on DeepMind Control suits and large-scale speech datasets, our approach consistently outperforms conventional SNN models and achieves competitive performance relative to ANN-based world models. Moreover, our analysis of neuronal membrane potentials, spiking behavior, and dendritic parameter optimization underscores the significance of dendritic information processing. We show that coordinated dendritic activity plays a crucial role in shaping neuronal responses and improving overall model performance. These findings provide insights into the functional importance of dendritic computation in SNNs, paving the way for more biologically inspired and efficient neural architectures in future research.

4.1. Limitation.

A limitation of the current model lies in the handling of the somatic spike and reset. For simplicity, both somatic and dendritic membrane potentials are reset simultaneously after a spike. However, in biological neurons, a backpropagating action potential (bAP) typically depolarizes the dendrites, rather than causing hyperpolarization through the reset mechanism used in our model. This simplified approach may not fully capture the complex interaction between somatic and dendritic potentials. Future work could incorporate more biologically plausible models that consider bAPs, thereby enhancing the realism of dendritic dynamics and neuronal computation.

In addition to the neuronal dynamics, another limitation concerns the training method. We adopt STBP for optimizing the deep spiking world model, which is effective for long-horizon tasks but not biologically plausible, as it relies on nonlocal backpropagation-through-time for credit assignment. We chose STBP for its stability and scalability, yet acknowledge this compromises biological fidelity. As a next step, we plan to explore hybrid strategies that combine the optimization advantages of STBP with more biologically inspired mechanisms, such as STDP or Hebbian plasticity, target-based learning schemes, and online approximations like e-prop. Such approaches may trade some efficiency for greater interpretability and plausibility, and represent promising directions for extending our framework.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

The model of this research is one of the core and part of BrainCog Embot (73). BrainCog Embot is an Embodied AI platform under the Brain-inspired Cognitive Intelligence Engine (BrainCog) framework, which is an open-source Brain-inspired AI platform based on Spiking Neural Network. Codes and data have been deposited in GitHub https://github.com/Brain-Cog-Lab/Spiking-WM.