Adaptively biased sequential importance sampling for rare events in reaction networks with comparison to exact solutions from finite buffer dCME method

Abstract

Critical events that occur rarely in biological processes are of great importance, but are challenging to study using Monte Carlo simulation. By introducing biases to reaction selection and reaction rates, weighted stochastic simulation algorithms based on importance sampling allow rare events to be sampled more effectively. However, existing methods do not address the important issue of barrier crossing, which often arises from multistable networks and systems with complex probability landscape. In addition, the proliferation of parameters and the associated computing cost pose significant problems. Here we introduce a general theoretical framework for obtaining optimized biases in sampling individual reactions for estimating probabilities of rare events. We further describe a practical algorithm called adaptively biased sequential importance sampling (ABSIS) method for efficient probability estimation. By adopting a look-ahead strategy and by enumerating short paths from the current state, we estimate the reaction-specific and state-specific forward and backward moving probabilities of the system, which are then used to bias reaction selections. The ABSIS algorithm can automatically detect barrier-crossing regions, and can adjust bias adaptively at different steps of the sampling process, with bias determined by the outcome of exhaustively generated short paths. In addition, there are only two bias parameters to be determined, regardless of the number of the reactions and the complexity of the network. We have applied the ABSIS method to four biochemical networks: the birth-death process, the reversible isomerization, the bistable Schlögl model, and the enzymatic futile cycle model. For comparison, we have also applied the finite buffer discrete chemical master equation (dCME) method recently developed to obtain exact numerical solutions of the underlying discrete chemical master equations of these problems. This allows us to assess sampling results objectively by comparing simulation results with true answers. Overall, ABSIS can accurately and efficiently estimate rare event probabilities for all examples, often with smaller variance than other importance sampling algorithms. The ABSIS method is general and can be applied to study rare events of other stochastic networks with complex probability landscape.

INTRODUCTION

Many critical events in biological processes occur rarely within the relevant physical time scale. Bacteriophage λ in E. coli^{1, 2, 3} can maintain a stable dormant lysogenic lifestyle when integrated into the E. coli genome, but can spontaneously transit to the lytic lifestyle of phage outburst^{2, 4, 5, 6, 7, 8} with a small probability (∼4 × 10⁻⁷ per cell cycle⁹). Crossing barrier in the free energy landscape in some slow-folding protein may be rare, but critical.¹⁰ In tumorigenesis, cells experiencing normal growth rarely transit spontaneously to uncontrolled tumor growth.^{11, 12} However, environmental changes, e.g., those resulting in the accumulation of DNA hypermethylation in promoter CpG islands,^{13, 14} can accelerate such transitions. Multi-stable cellular states of endogenous molecular-cellular networks and rare stochastic transitions between them may offer a general framework to study human diseases.^{15, 16} Accurate assessment of rare event probabilities is, therefore, important for understanding the machineries behind many critical biological processes.

It is challenging to study rare events from the viewpoint of mechanistic theory.^{17, 18} Here we study networks of biochemical reactions. In principle, the transition probability rates between two states can be calculated exactly, if the state space of biochemical reaction networks are completely accounted for, e.g., when the underlying discrete chemical master equation can be solved exactly.^{8, 19} However, when the state spaces and the transition matrices are too large to be efficiently computed, a widely used approach to study stochastic behavior of biochemical reactions is that of Monte Carlo sampling, first formulated as the stochastic simulation algorithm (SSA).²¹ However, the original SSA²¹ is ineffective for studying rare events, as most computing time is spent on following high-probability paths.^{22, 23}

The techniques of importance sampling and reweighting can improve sampling efficiency significantly. They have been widely used in equilibrium sampling where the condition of detailed balance holds.^{24, 25} However, stochastic processes in reaction networks are generally not time reversible and the condition of detailed balance is not valid. Kuwahara and Mura developed the weighted SSA (wSSA) algorithm by applying the importance sampling technique to study stochastic reaction networks, in which each reaction rate is biased by a pre-determined constant, with the overall summation of reaction rates unchanged.²² As the probability for reaction selection can be biased such that rare events are sampled more frequently while the time scale of the underlying reactions is maintained, significantly improved sampling efficiency for rare events was reported.^{22, 23, 26} However, the choice of bias constants strongly affects the effectiveness of wSSA. When there are many reactions and the network is complex, the heuristic approach of determining bias constants by examining the reactions does not work.²² As there is no general guidance in how bias constants should be chosen, poor choices may lead to estimations that are less accurate than the original SSA.²³

Daigle et al. developed the doubly-weighted SSA (dwSSA) algorithm, in which a multilevel cross-entropy (CE) method is used iteratively to provide estimates of bias constants.²³ This is achieved by running long trial simulations until a fraction of the sampled trajectories reaches the target states.²³ With this automated estimation, both reaction selection and the underlying time scale of reactions can be biased.²³

A drawback of methods using constant biases such as wSSA and dwSSA is that the bias coefficients are global and state-independent, and are not influenced by the concentrations of molecules, which evolve with time. As the apparent rate of a reaction can vary dramatically depending on the copy number of molecules, the degree of bias for a reaction therefore needs to be adjusted according to the available copy numbers of reactants. With globally fixed bias constants, a network with reactions of a wide range of rates will have over- and under-biased reactions, depending on the states of the system. As a result, estimated properties of a network will have large variance, making these methods unsuitable for complex networks.²⁷

Roh et al. developed a state-dependent biasing wSSA method (swSSA).²⁷ By empirically classifying reactions into groups of favored, disfavored, and neutral reactions, biases in selection probability for reactions in the first two groups are calculated in a state-dependent fashion. The swSSA method can have better estimation accuracy and efficiency than the wSSA method,²⁷ at the expense of about twice as many biasing parameters as that of the wSSA.²⁷ Roh et al. further developed the state-dependent doubly weighted SSA method (sdwSSA), where reactions are further grouped into bins according to their selection probabilities, and are assigned different bias constants, which are automatically estimated using the cross-entropy method.²⁶ However, the number of parameters to be estimated using sdwSSA is much larger than that of wSSA, dwSSA, and swSSA. For example, about 20 bias constants need to be estimated for a simple reversible isomerization system with only two reactions.²⁶ Estimating a large number of bias constants needed for complex networks becomes difficult.

In this study, we describe an algorithm named adaptively biased sequential importance sampling (ABSIS) for efficient sampling of rare events. Based on the principle of sequential importance sampling, our approach adopts the look-ahead strategy, a technique well-established in polymer and protein studies,^{28, 29, 30, 31} to gather future information for design of bias parameters to enable effective barrier crossings.^{28, 29, 30, 31, 32, 33} By enumerating short paths from the current state, bias coefficients are generated based on analysis of these short paths. Unlike the dwSSA and sdwSSA methods, in which biases are fixed constants after parameter estimation, the biases in ABSIS for each reaction is dynamically determined based on exact calculation of the total probability of short κ-step forward- and backward-moving reaction paths, without the need of binning reaction rates. Reactions with higher probability of forward-moving are then encouraged, and reactions with higher probability of backward-moving are discouraged. Regardless of the number of reactions in the networks, we only need to assign two bias parameters for the whole network: the degree to encourage forward-moving reactions and the degree to discourage backward-moving reactions, which both can be estimated through an efficient parameter estimation algorithm.

We also take advantage of the recent development of a method that directly solves the discrete chemical master equation.^{8, 19} With a finite buffer, the rare event probability of a stochastic network of modest size can be computed exactly using this method, allowing us to have a gold standard to objectively assess the accuracy of estimated rare event probabilities through sampling. With errors computed based on exact numerical solutions, we show with four biological examples that the ABSIS method have improved or comparable accuracies compared to other methods (the dwSSA method, and the swSSA and sdwSSA methods when data available), at overall significantly reduced computational cost and much higher success rate.

This article is organized as follows: We briefly discuss the theoretical framework of reaction networks, the principle of sequential importance sampling, and details of the ABSIS method. We then apply our method to study four biological problems, namely, the birth-death process, the reversible isomerization model, the bistable Schlögl model, and the enzymatic futile cycle, and compare the accuracies of estimations and the success rates in generating reaction paths reaching the target states with the SSA and dwSSA methods. We conclude with remarks and discussions.

MODEL FRAMEWORK

Reaction networks

We assume a well-mixed biochemical system with constant volume and temperature. There are n different molecular species: $\mathcal{X}=\{X_1,X_2,\cdots ,X_n\}$. We use x_i(t) to denote the copy number of molecular species X_i at time t. There are m possible different reactions in the system: $\mathcal{R}=\{R_1,R_2,\cdots ,R_m\}$. Each reaction R_k has an intrinsic reaction rate constant r_k. The microstate of the system at time t is represented by a non-negative integer column vector: $\mathit{x}\left(t\right)={(x_1\left(t\right),x_2\left(t\right),\cdots ,x_n\left(t\right))}^T$, where ^T denotes the transpose. An arbitrary reaction R_k (k = 1, 2, ⋯, m) with intrinsic rate r_k takes the general form:

$$c_{1k}{X}_1+c_{2k}{X}_2+\cdots +c_{nk}{X}_n\stackrel{r_k}{\to }{c}_{1k}^{\prime }{X}_1+c_{2k}^{\prime }{X}_2+\cdots +c_{nk}^{\prime }{X}_n,$$

which brings the system from a microstate ${\mathit{x}}_i$ to ${\mathit{x}}_j$. The difference between ${\mathit{x}}_i$ and ${\mathit{x}}_j$ is the stoichiometry vector ${\mathit{s}}_k$ of the reaction R_k: ${\mathit{s}}_k={\mathit{x}}_j-{\mathit{x}}_i={(s_{1k},s_{2k},\cdots ,s_{nk})}^T={(c_{1k}^{\prime }-c_{1k},c_{2k}^{\prime }-c_{2k},\cdots ,c_{nk}^{\prime }-c_{nk})}^T\in {\mathbb{Z}}^n.$ The stoichiometry matrix S for the reaction network is defined as: $\mathit{S}=({\mathit{s}}_1,{\mathit{s}}_2,\cdots ,{\mathit{s}}_m)\in {\mathbb{Z}}^{n\times m},$ where each column represents a single reaction. The rate $A_k({\mathit{x}}_i,{\mathit{x}}_j)$ of reaction R_k that transforms microstate from ${\mathit{x}}_i$ to ${\mathit{x}}_j$ is determined by the intrinsic rate constant r_k and the combination number of relevant reactants in the current microstate ${\mathit{x}}_i$: $A_k({\mathit{x}}_i,{\mathit{x}}_j)=A_k\left({\mathit{x}}_i\right)=r_k{\prod }_{l=1}^n\left(\begin{array}{c}{x}_l\\ c_{lk}\end{array}\right),$ assuming the convention $\left(\begin{array}{c}0\\ 0\end{array}\right)=1$.

State space and probability landscape

The state space S of a reaction network is defined as the set of all possible microstates that the system can visit from a given initial condition: $\mathcal{S}=\left\{\mathit{x}\right(t\left)\right|\mathit{x}\left(0\right),t\in (0,\theta )\}$. We denote the probability of each microstate at time t as $p\left(\mathit{x}\right(t\left)\right)$, and the probability distribution at t over the whole state space as $\mathit{p}\left(t\right)=\left\{\right(p\left(\mathit{x}\right(t\left)\right)\left|\mathit{x}\right(t)\in \mathcal{S})\}.$$\mathit{p}\left(t\right)$ is also called the probability landscape of the network.⁸ It can be visualized as a time-evolving scalar surface over the state space, with the value at each state x taken to be $p\left(\mathit{x}\right(t\left)\right)$. The volume integral under the surface at any arbitrary time t is always 1:

$${\int }_{\mathit{x}\in \mathcal{S}}p(\mathit{x},t)d\mathit{x}=1.$$

In general, there is no assumption of detailed balance. For a reaction network with arbitrary stoichiometries and a specific initial state $\mathit{p}\left(0\right)$, its probability landscape is governed by the discrete chemical master equation (dCME)

$$\frac{d\mathit{p}\left(t\right)}{dt}={\mathit{A}}^T\mathit{p}\left(t\right),$$ (1)

from which the time-evolving probability landscape $\mathit{p}\left(t\right)$ and its steady state can be directly obtained.^{8, 19} Here A is the transition rate matrix $\mathit{A}=\left\{A_k({\mathit{x}}_i,{\mathit{x}}_j)\right\}\in {\mathbb{R}}^{\left|\mathcal{S}\right|\times \left|\mathcal{S}\right|}$.

Transition paths and transition probabilities

A transition path π_{(S, T)} consists of a sequence of states: $\mathbf{S}=({\mathit{x}}_0,\cdots ,{\mathit{x}}_N)$, starting from ${\mathit{x}}_0$ and ending at ${\mathit{x}}_N$, along with a sequence of time points T = (t₀, ⋅⋅⋅, t_N) when each of these states are visited. Here N is the length of the transition path. When the beginning state ${\mathit{x}}_0$ and the ending state ${\mathit{x}}_N$, as well as the sequence of states S and time points T are unambiguous from the context, we use π_{(0, N)} to denote the transition path π_{(S, T)} for convenience. This transition path is understood to move from state ${\mathit{x}}_0$ to state ${\mathit{x}}_N$ through a total of N steps following the specific sequence of states S and sequence of time points T. The sequence of time points can be alternatively specified by the corresponding sequence of time intervals: {τ₀, τ₁, ⋯, τ_{N − 1}} = {t₁ − t₀, t₂ − t₁, ⋯, t_N − t_{N − 1}}. In implementation, these time intervals are not predefined but are small random values generated by sampling Poisson processes, whose rates are governed by the underlying chemical reaction rates (see below). We assume that there is a unique reaction connecting each neighboring pair of microstates ${\mathit{x}}_{i-1}$ and ${\mathit{x}}_i$ along the reaction path. The probability p(π_{(S, T)}) of a given transition path π_{(S, T)} can be calculated as the product of the probability of the initial state ${\mathit{x}}_0$, and the probabilities of all subsequent transitions between neighboring states $p\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1}$,

$$\begin{array}{cc}\hfill p\left({\pi }_{(0,N)}\right)=p\left({\pi }_{(\mathbf{S},\mathbf{T})}\right)& =p\left({\mathit{x}}_0\right)\prod _{i=1}^{N}p\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1}.\hfill \end{array}$$ (2)

Assuming a Poisson process, the probability $p\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1}$ of each transition occurring during an infinitesimally small dτ_{i − 1} after τ_{i − 1} can be calculated as^{22, 34}

$$\begin{array}{ccc}\hfill p\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1}& =& A_0\left({\mathit{x}}_{i-1}\right)e^{-A_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}d{\tau }_{i-1}·\frac{A_k\left({\mathit{x}}_{i-1}\right)}{A_0\left({\mathit{x}}_{i-1}\right)}\hfill \\ & =& A_k\left({\mathit{x}}_{i-1}\right)e^{-A_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}d{\tau }_{i-1},\hfill \end{array}$$ (3)

where $A_0\left({\mathit{x}}_{i-1}\right)={\sum }_{k=1}^m{A}_k\left({\mathit{x}}_{i-1}\right)$ is the sum of rates of all reactions that could happen at the state ${\mathit{x}}_{i-1}$, and $A_0\left({\mathit{x}}_{i-1}\right)e^{-A_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}d{\tau }_{i-1}$ is the probability that there is exactly one reaction occurring in next time interval τ_{i − 1}.³⁵ The subscript k denotes the reaction R_k that connects state ${\mathit{x}}_{i-1}$ to state ${\mathit{x}}_i$. The fraction $\frac{A_k\left({\mathit{x}}_{i-1}\right)}{A_0\left({\mathit{x}}_{i-1}\right)}$ is the probability that the kth reaction R_k occurs during τ_{i − 1}.^{22, 34} Taking together, the overall probability of the transition path π is

$$p\left({\pi }_{(0,N)}\right)=p\left({\mathit{x}}_0\right)\prod _{i=1}^N{A}_k\left({\mathit{x}}_{i-1}\right)e^{-A_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}d{\tau }_{i-1}.$$ (4)

Macrostates and probability of rare transitions between macrostates

We define a macrostate B as a set of microstates: $\mathcal{B}=\left\{\mathit{x}\right|\in \mathcal{S}\}\subset \mathcal{S}.$ Here we are interested in biologically motivated macrostates. For example, in a bistable genetic switch system, most microstates belong to either the “on/off” or the “off/on” metastable states, each of which can be regarded as a macrostate. The probability of a macrostate B can be written as: $p\left(\mathcal{B}\right)={\sum }_{\mathit{x}\in \mathcal{B}}p\left(\mathit{x}\right).$

For a stochastic network, if a destination macrostate D can be reached from a beginning macrostate B, the probability of the system transiting from B to D is 1 if given infinite amount of time. However, we are interested in the probability of transition from B to D within a finite period of time θ. That is, we wish to estimate $p\left(\mathcal{D}\right|\mathcal{B},t\le \theta )$, which may be small (Fig. 1),

$$p\left(\mathcal{D}\right|\mathcal{B},t\le \theta )=\sum _{{\pi }_{(\mathbf{S},\mathbf{T})}}p\left({\pi }_{(\mathbf{S},\mathbf{T})}\right|{\mathit{x}}_0\in \mathcal{B},\mathbf{S}\cap \mathcal{D}\ne \varnothing ,t_N-t_0\le \theta ).$$

Figure 1.

Distribution of rare event transition paths. Rare event transition paths traveling from state B to D within time θ are of interest.

Calculating exact transition probability

The finite buffer dCME method can be used to enumerate the state space S of stochastic networks of modest size, and is optimal both in time complexity and in space requirement.^{8, 19} For these networks, we can directly solve the dCME. The transition probabilities of specific paths connecting two macroscopic states can therefore be calculated exactly. In this study, the probabilities of rare events in all examples are computed both by the finite buffer dCME method and by sampling methods. The results of the former are regarded as exact solutions, against which results from sampling methods are compared.

Weighted SSA and doubly-weighted SSA

There are potentially an enormous number of transition paths connecting two macrostates. In general, if enumeration is infeasible, exact calculation of the transition probabilities is not possible. One can estimate the probabilities through Monte Carlo sampling.

A number of sampling methods for rare events have been developed based on the principle of importance sampling. Kuwahara and Mura developed the wSSA algorithm,²² in which the rate of each reaction $A_k\left(\mathit{x}\right)$ is biased by a pre-selected predilection constant α_k, which will increase or decrease the rate of a specific reaction. This may affect the fraction of sampled paths reaching the target states. These paths are generated from the biased reaction rate $B_k\left(\mathit{x}\right)={\alpha }_k{A}_k\left(\mathit{x}\right)$. The biased probability $p^{\prime }\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1}$ of the reaction in the time step starting at state ${\mathit{x}}_{i-1}$ is calculated as

$$\begin{array}{ccc}& & p^{\prime }\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1}\hfill \\ & & =A_0\left({\mathit{x}}_{i-1}\right)e^{-A_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}d{\tau }_{i-1}·\frac{B_k\left({\mathit{x}}_{i-1}\right)}{B_0\left({\mathit{x}}_{i-1}\right)},\hfill \end{array}$$ (5)

where reaction R_k leads ${\mathit{x}}_{i-1}$ to ${\mathit{x}}_i$, and $B_0\left(\mathit{x}\right)\equiv {\sum }_{k=1}^m{B}_k\left(\mathit{x}\right)$. A weight for correcting the bias is also kept for this reaction:

$$w(k,{\mathit{x}}_{i-1})=\frac{A_k\left({\mathit{x}}_{i-1}\right)B_0\left({\mathit{x}}_{i-1}\right)}{B_k\left({\mathit{x}}_{i-1}\right)A_0\left({\mathit{x}}_{i-1}\right)}=\frac{1}{{\alpha }_k}\frac{B_0\left({\mathit{x}}_{i-1}\right)}{A_0\left({\mathit{x}}_{i-1}\right)}.$$ (6)

The true probability $p\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})$ is then recovered as $p\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})=w(k,{\mathit{x}}_{i-1})·p^{\prime }\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1}).$ The biased probability p^′(π_{(0, N)}) for the full path is:

$$p^{\prime }\left({\pi }_{(0,N)}\right)=p\left({\mathit{x}}_0\right)\prod _{i=1}^N{p}^{\prime }\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1},$$

with the weight:

$$w\left({\pi }_{(0,N)}\right)=\prod _{i=1}^{N}w(k,{\mathit{x}}_{i-1})=\prod _{i=1}^N\frac{1}{{\alpha }_k}\frac{B_0\left({\mathit{x}}_{i-1}\right)}{A_0\left({\mathit{x}}_{i-1}\right)},$$

The true probability of the path is then: p(π_{(0, N)}) = w(π_{(0, N)}) · p′(π_{(0, N)}).

In wSSA, bias is introduced through the second factor $\frac{B_k\left({\mathit{x}}_{i-1}\right)}{B_0\left({\mathit{x}}_{i-1}\right)}$ in Eq. 5, which represents the biased probability in selecting the next reaction in the wSSA scheme. The time scale of the Poisson process underlying the reaction, namely, the first factor in Eq. 5, remains unchanged.

In the doubly-weighted SSA method (dwSSA),²³ both the selection probability and the Poisson time scale are biased, and the biased probability for each step in a dwSSA sampling path is:

$$\begin{array}{ccc}& & p^{\prime }\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1}\hfill \\ & & =B_0\left({\mathit{x}}_{i-1}\right)e^{-B_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}d{\tau }_{i-1}·\frac{B_k\left({\mathit{x}}_{i-1}\right)}{B_0\left({\mathit{x}}_{i-1}\right)}\hfill \\ & & =B_k\left({\mathit{x}}_{i-1}\right)e^{-B_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}d{\tau }_{i-1},\hfill \end{array}$$ (7)

where $B_k\left({\mathit{x}}_{i-1}\right)={\gamma }_k{A}_k\left({\mathit{x}}_{i-1}\right)$ is the biased reaction rate. The weight for the kth reaction occurring at step i − 1 is obtained from dividing Eq. 3 by Eq. 7:

$$\begin{array}{ccc}\hfill w(k,{\mathit{x}}_{i-1})& =& \frac{A_k\left({\mathit{x}}_{i-1}\right)e^{-A_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}}{B_k\left({\mathit{x}}_{i-1}\right)e^{-B_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}}\hfill \\ & =& \frac{1}{{\gamma }_k}\exp [(B_0\left({\mathit{x}}_{i-1}\right)-A_0\left({\mathit{x}}_{i-1}\right)){\tau }_{i-1}].\hfill \end{array}$$

The biased probability for a full dwSSA path π_{(0, N)} is:

$$p^{\prime }\left({\pi }_{(0,N)}\right)=p\left({\mathit{x}}_0\right)\prod _{i=1}^N{p}^{\prime }\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1},$$

and its weight is:

$$\begin{array}{ccc}\hfill w\left({\pi }_{(0,N)}\right)& =& \prod _{i=1}^{N}w(k,{\mathit{x}}_{i-1})\hfill \\ & =& \prod _{i=1}^N\frac{1}{{\gamma }_k}\exp \left[(B_0\left({\mathit{x}}_{i-1}\right)-A_0\left({\mathit{x}}_{i-1}\right)){\tau }_{i-1}\right].\hfill \end{array}$$

The true probability of the path p(π_{(0, N)}) can be recovered as: p(π_{(0, N)}) = w(π_{(0, N)}) · p′(π_{(0, N)}).

A key component of the dwSSA method is an automatic method to estimate the bias constant γ_k for each reaction R_k. A large number (typically 10⁵) of full-length trial simulations are run, with some of them reaching the target macrostate. The number of each reaction that occurred in those simulations that reached the macrostate are counted and compared to the expected number of occurrence if one were to follow a Poisson process in the same given time under the same initial condition. Those reactions occur more frequently than expected are biased towards. Reactions that occur less frequently than expected are biased against. This procedure is repeated with the biases updated iteratively, until a predefined fraction (e.g., 2%) of full-length trial paths reaches the target macrostate. In the further developed swSSA and sdwSSA methods, the bias coefficient is not a constant, but depends on the copy numbers of molecules of the current state.^{26, 27} In order to assign more effective bias coefficients, a refined scheme of bias assignment is used in sdwSSA, in which each reaction is divided into multiple bins according to its probability to be chosen, with each bin assigned its own bias coefficient.²⁶

There are a number of issues with these methods. First, estimation of the bias parameters relies on counting the number of occurrence of a reaction, which may not be possible or the estimate may not be reliable if reactions happens rarely. For example, gene binding and unbinding reactions in a toggle switch system bring the system from one metastable state to another. But this happens only once or twice during an extended time. It is challenging to sample these binding/unbinding reactions adequately using trial simulations, where limited runs are carried out. Second, as the estimated bias parameters are either constant or based on current state, no considerations for possible future barriers in the probability landscape is incorporated. This becomes problematic for complex systems, for example, those with multistabilities, where steep barriers need to be crossed. In these systems, the desirable bias may be quite different depending on the neighborhood where the system is currently located in the landscape. Third, there is a proliferation of adjustable bias parameters, for example, on the order of O(βm) for the sdwSSA method, with the number of bins β = 5–20 for each of the m reactions, making the assignment of bias coefficients a challenging task.²⁶ As a result, often the overall amount of computation involved is substantial, the variance of samples generated is high, and the accuracy is still unsatisfactory.

Adaptively biased sequential importance sampling

Here we describe a new method called ABSIS for estimating rare transition probability between macrostates. Our approach is based on the look-ahead strategy and the principle of sequential importance sampling,³¹ which have found wide applications in studies of polymers and protein biophysics,^{29, 30} where challenging problems such as RNA loop entropy calculation, generation of protein folding transition state ensemble, and protein packing have been investigated.^{30, 33, 36} In ABSIS, bias for each reaction is calculated based on present and future information, and is adaptively adjusted automatically, resulting in more efficient sampling of rare events. It can be applied to stochastic networks with complex probability landscapes.

Perfect path sampling

Assume we wish to reach the macrostate D from the microstate ${\mathit{x}}_0$. We can classify paths π_{(0, N)} starting at ${\mathit{x}}_0$ and ending at ${\mathit{x}}_N$ into two sets ${\mathcal{P}}_{\mathcal{D}}$ and ${\mathcal{P}}_{\overline{\mathcal{D}}}$: those that reach the macrostate D before time θ form the set of paths ${\mathcal{P}}_{\mathcal{D}}=\left\{{\pi }_{(0,N)}\right|\mathbf{S}\cap \mathcal{D}\ne \varnothing ,t_N-t_0\le \theta \}$, and those that do not form another set of paths ${\mathcal{P}}_{\overline{\mathcal{D}}}=\left\{{\pi }_{(0,N)}\right|\mathbf{S}\cap \mathcal{D}=\varnothing ,t_N-t_0>\theta \}$.

Our goal is to assess the transition probability $p\left(\mathcal{D}\right|{\mathit{x}}_0)$ from the microstate ${\mathit{x}}_0$ to the macrostate D. It can be calculated as

$$p\left(\mathcal{D}\right|{\mathit{x}}_0)={\int }_{{\pi }_{(0,N)}}\mathbb{I}\left({\pi }_{(0,N)}\right)·p\left({\pi }_{(0,N)}\right)d{\pi }_{(0,N)},$$

where $\mathbb{I}\left({\pi }_{(0,N)}\right)$ is an indicator function such that $\mathbb{I}\left({\pi }_{(0,N)}\right)=1$ if ${\pi }_{(0,N)}\in {\mathcal{P}}_{\mathcal{D}}$, and 0 otherwise. Namely, it is 1 if a path π_{(0, N)} starting from state ${\mathit{x}}_0$ reaches the macrostate D in time, and 0 otherwise. Perfect path samples for calculating $p\left(\mathcal{D}\right|{\mathit{x}}_0)$ then can be drawn as

$${\pi }_{(0,N)}\sim \mathbb{I}\left({\pi }_{(0,N)}\right)·p\left({\pi }_{(0,N)}\right).$$

In general, if our goal is to estimate certain property of the reaction paths, which is expressed as a scalar function $f\left(\mathit{x}\right):{{\mathbb{Z}}^{+}}^n\mapsto \mathbb{R}$ of the microstate x, perfect sampling of the reaction paths for this estimation problem is then:

$${\pi }_{(0,N)}\sim \mathbb{I}\left({\pi }_{(0,N)}\right)·\widehat{f}\left({\pi }_{(0,N)}\right),$$

where $\widehat{f}\left({\pi }_{(0,N)}\right)=p\left({\mathit{x}}_0\right)f\left({\mathit{x}}_0\right){\prod }_{i=1}^{N}p\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})f\left({\mathit{x}}_i\right)$, and $({\mathit{x}}_0,\cdots ,{\mathit{x}}_N)$ forms the path π_{(0, N)}.

Optimal bias strategy and future-perfect adaptive weighting

Similarly, the probability $p\left(\mathcal{D}\right|{\mathit{x}}_i)$ that future paths after a reaction connecting the microstate ${\mathit{x}}_{i-1}$ to ${\mathit{x}}_i$ will reach the destination macrostate D in time is:

$$p\left(\mathcal{D}\right|{\mathit{x}}_i)={\int }_{{\pi }_{(i,N)}}\mathbb{I}\left({\pi }_{(i,N)}\right)·p\left({\pi }_{(i,N)}\right)d{\pi }_{(i,N)}.$$

To estimate the transition probability $p\left(\mathcal{D}\right|{\mathit{x}}_{i-1})$, the next state ${\mathit{x}}_i$ can be sampled future-perfectly if we draw ${\mathit{x}}_i$ as

$${\mathit{x}}_i\sim p\left(\mathcal{D}\right|{\mathit{x}}_i).$$

If our goal is to estimate the property f(·) of the reaction path that reaches the macrostate D, ${\mathit{x}}_i$ can be sampled optimally as

$${\mathit{x}}_i\sim {\int }_{{\pi }_{(i,N})}\mathbb{I}\left({\pi }_{(i,N)}\right)·\widehat{f}\left({\pi }_{(i,N)}\right)d{\pi }_{({\mathit{x}}_i,N)},$$

where $\widehat{f}\left({\pi }_{(i,N)}\right)=p\left({\mathit{x}}_i\right)f\left({\mathit{x}}_i\right){\prod }_{l=i+1}^{N}p\left({\mathit{x}}_l\right|{\mathit{x}}_{l-1},{\tau }_{l-1})f\left({\mathit{x}}_l\right)$.

κ-Step look-ahead bias strategy and adaptive weighting

As it is usually impossible to enumerate and examine all paths up to time θ to calculate $p\left(\mathcal{D}\right|{\mathit{x}}_i)$ exactly, we approximate it by adopting a κ-step look-ahead strategy. Briefly, we analyze statistics of exhaustively generated short paths, and design biases based on estimations made on these short paths. We first classify κ-step paths π_{(i, i + κ)}, which all have the first step following a specific reaction connecting ${\mathit{x}}_{i-1}$ to ${\mathit{x}}_i$, into three types: forward-moving paths ${\mathcal{P}}_F$, backward-moving paths ${\mathcal{P}}_B$, and non-moving paths ${\mathcal{P}}_N$ (Fig. 2):

$$\begin{array}{cc}\hfill \text{Forward-moving:}{\pi }_{i,i+\kappa }\in {\mathcal{P}}_F& \text{if}d({\mathit{x}}_{i+\kappa },\mathcal{D})<d({\mathit{x}}_{i-1},\mathcal{D}),\hfill \\ \hfill \text{Backward-moving:}{\pi }_{i,i+\kappa }\in {\mathcal{P}}_B& \text{if}d({\mathit{x}}_{i+\kappa },\mathcal{D})>d({\mathit{x}}_{i-1},\mathcal{D}),\hfill \\ \hfill \text{Non-moving:}{\pi }_{i,i+\kappa }\in {\mathcal{P}}_N& \text{if}d({\mathit{x}}_{i+\kappa },\mathcal{D})=d({\mathit{x}}_{i-1},\mathcal{D}).\hfill \end{array}$$ (8)

Here $d({\mathit{x}}_{i+\kappa },\mathcal{D})$ and $d({\mathit{x}}_{i-1},\mathcal{D})$ are the distances between the states ${\mathit{x}}_{i+\kappa },{\mathit{x}}_{i-1}$, and the target macrostate D, respectively. We define the distance $d(\mathit{x},\mathcal{D})={\min }_{{\mathit{x}}_l\in \mathcal{D}}d(\mathit{x},{\mathit{x}}_l).$ For convenience, we use 1-norm distance. The forward-moving, backward-moving, and non-moving probabilities after κ-steps, given that the first reaction connects state ${\mathit{x}}_{i-1}$ and ${\mathit{x}}_i$, can be calculated as

$$p_{\mathsf{F}}\left({\mathit{x}}_i\right)=\sum _{{\pi }_{(i,i+\kappa )}\in {\mathcal{P}}_F}p\left({\pi }_{(i,i+\kappa )}\right)$$

$$p_{\mathsf{B}}\left({\mathit{x}}_i\right)=\sum _{{\pi }_{(i,i+\kappa )}\in {\mathcal{P}}_B}p\left({\pi }_{(i,i+\kappa )}\right)$$

and

$$p_{\mathsf{N}}\left({\mathit{x}}_i\right)=1-[p_{\mathsf{F}}\left({\mathit{x}}_i\right)+p_{\mathsf{B}}\left({\mathit{x}}_i\right)].$$

We can then have the approximations $p\left(\mathcal{D}\right|{\mathit{x}}_i)\approx p_{\mathsf{F}}\left({\mathit{x}}_i\right),$ and $p\left(\overline{\mathcal{D}}\right|{\mathit{x}}_i)\approx p_{\mathsf{B}}\left({\mathit{x}}_i\right)+p_{\mathsf{N}}\left({\mathit{x}}_i\right),$ which will be used to construct the bias functions for accelerating/decelerating the reaction rate and for selecting reaction. We can now have the approximation:

$$\begin{array}{ccc}\hfill p\left(\mathcal{D}\right|{\mathit{x}}_i)& =& {\int }_{{\pi }_{(i,N)}\in {\mathcal{P}}_{\mathcal{D}}}\mathbb{I}\left({\pi }_{(i,N)}\right)·p\left({\pi }_{(i,N)}\right)d{\pi }_{(i,N)}\hfill \\ & \approx & {\int }_{{\pi }_{(i,i+\kappa )}\in {\mathcal{P}}_F}p\left({\pi }_{(i,i+\kappa )}\right)d{\pi }_{(i,i+\kappa )}.\hfill \end{array}$$

Figure 2.

Look-ahead strategy. Two reactions leading current state ${\mathit{x}}_{i-1}$ to next state ${\mathit{x}}_i$ are shown in dashed lines (blue and pink). All possible look-ahead paths of length κ = 2 for both reactions from two different next states ${\mathit{x}}_i$ are illustrated in solid blue and pink lines. State B is the initial state, and D is the target state. The thickness of lines indicates reaction rates. The color of circles at the end state of each path indicates moving forward (green, ${\mathcal{P}}_F$), backward (red, ${\mathcal{P}}_B$), and non-moving (yellow, ${\mathcal{P}}_N$).

Bias function with κ-step look-ahead

Recall that the probability of a path p(π_{(i, N)}) is computed as

$$p\left({\pi }_{(i,N)}\right)=p\left({\mathit{x}}_i\right)\prod _{l=i}^N{A}_0\left({\mathit{x}}_l\right)e^{-A_0\left({\mathit{x}}_l\right){\tau }_l}d{\tau }_l\frac{A_k\left({\mathit{x}}_l\right)}{A_0\left({\mathit{x}}_l\right)}.$$

To design bias functions that are fast to compute, we consider only the overall probability p_r(π_(i, N)) of reaction choices accumulated along the path, and ignore the rates of reactions for now:

$$p_r\left({\pi }_{(i,N)}\right)=\prod _{l=i}^N\frac{A_k\left({\mathit{x}}_l\right)}{A_0\left({\mathit{x}}_l\right)}.$$

We have

$$p_{\mathsf{F}}\left({\mathit{x}}_i\right)=\sum _{{\pi }_{(i,i+\kappa )}\in {\mathcal{P}}_F}p\left({\pi }_{(i,i+\kappa )}\right)\approx \sum _{{\pi }_{(i,i+\kappa )}\in {\mathcal{P}}_F}\left[\prod _{l=i}^{i+\kappa -1}\frac{A_k({\mathit{x}}_l,{\mathit{x}}_{l+1})}{A_0\left({\mathit{x}}_l\right)}\right]$$ (9)

and

$$p_{\mathsf{B}}\left({\mathit{x}}_i\right)=\sum _{{\pi }_{(i,i+\kappa )}\in {\mathcal{P}}_B}p\left({\pi }_{(i,i+\kappa )}\right)\approx \sum _{{\pi }_{(i,i+\kappa )}\in {\mathcal{P}}_B}\left[\prod _{l=i}^{i+\kappa -1}\frac{A_k({\mathit{x}}_l,{\mathit{x}}_{l+1})}{A_0\left({\mathit{x}}_l\right)}\right].$$ (10)

We can then design a bias function $g_k\left({\mathit{x}}_{i-1}\right)=f(p_{\mathsf{F}}\left({\mathit{x}}_i\right),p_{\mathsf{B}}\left({\mathit{x}}_i\right),A_k({\mathit{x}}_{i-1},{\mathit{x}}_i),A_0\left({\mathit{x}}_{i-1}\right))$ for selecting reaction k, and set $B_k\left({\mathit{x}}_{i-1}\right)=g_k\left({\mathit{x}}_{i-1}\right)A_k\left({\mathit{x}}_{i-1}\right)$ as the biased reaction rate. The general biased probability for each step in the ABSIS sampling path is then:

$$\begin{array}{ccc}& & p_{ABSIS}\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1}\hfill \\ & & =B_0\left({\mathit{x}}_{i-1}\right)e^{-B_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}d{\tau }_{i-1}·\frac{B_k\left({\mathit{x}}_{i-1}\right)}{B_0\left({\mathit{x}}_{i-1}\right)}.\hfill \end{array}$$ (11)

Note that calculating the probability $p_{\mathsf{F}}\left({\mathit{x}}_i\right)$ and $p_{\mathsf{B}}\left({\mathit{x}}_i\right)$ is equivalent to follow a κ-step Markov process $\mathit{p}(i+\kappa )={\mathit{T}}^{\kappa }\mathit{p}\left(i\right)$, where the probability transition matrix is:

$$\mathit{T}={\mathit{A}}^T\mathrm{diag}(1/A_0\left({\mathit{x}}_{i,1}\right),1/A_0\left({\mathit{x}}_{i,2}\right),\cdots ,1/A_0\left({\mathit{x}}_{i,\left|\mathcal{S}\right|}\right))+\mathit{I},$$

in which A is the transition rate matrix of dCME in Eq. 1, $A_0\left({\mathit{x}}_i\right)={\sum }_{k=1}^m{A}_k({\mathit{x}}_i,{\mathit{x}}_{i^{\prime }})$, I is the identity matrix, and $\{{\mathit{x}}_{i,1},\cdots ,{\mathit{x}}_{i,\left|\mathcal{S}\right|}\}$ form the state space S for this κ-step Markov process starting from ${\mathit{x}}_i$. The initial probability distribution $\mathit{p}\left(i\right)$ for this Markov process is such that the probability for the current state ${\mathit{x}}_i$ is 1 and 0 for all other states.

Biasing strategy.

The bias in selecting reaction k that brings state ${\mathit{x}}_{i-1}$ to ${\mathit{x}}_i$ is based on the forward-moving and backward-moving probabilities. We first have

$$g_k\left({\mathit{x}}_{i-1}\right)=\left\{\begin{array}{cc}1-{\lambda }_1{p}_{\mathsf{F}}\left({\mathit{x}}_i\right)\log \left[\frac{A_k({\mathit{x}}_{i-1},{\mathit{x}}_i)}{A_0\left({\mathit{x}}_{i-1}\right)}·p_{\mathsf{F}}\left({\mathit{x}}_i\right)\right],\hfill & \text{if}{p}_{\mathsf{F}}\left({\mathit{x}}_i\right)>p_{\mathsf{B}}\left({\mathit{x}}_i\right)\hfill \\ \min \left\{1.0,-\frac{1}{\lambda 2}(1-p_{\mathsf{B}}\left({\mathit{x}}_i\right))\log \left[\frac{A_k({\mathit{x}}_{i-1},{\mathit{x}}_i)}{A_0\left({\mathit{x}}_{i-1}\right)}·p_{\mathsf{B}}\left({\mathit{x}}_i\right)\right]\right\},\hfill & \text{if}{p}_{\mathsf{F}}\left({\mathit{x}}_i\right)<p_{\mathsf{B}}\left({\mathit{x}}_i\right)\hfill \\ 1,\hfill & \text{if}{p}_{\mathsf{F}}\left({\mathit{x}}_i\right)=p_{\mathsf{B}}\left({\mathit{x}}_i\right),\hfill \end{array}\right.$$ (12)

where $A_0\left({\mathit{x}}_{i-1}\right)={\sum }_{k=1}^m{A}_k({\mathit{x}}_{i-1},{\mathit{x}}_{i^{\prime }})$, i^′ is any reachable state from ${\mathit{x}}_{i-1}$. Here λ₁ ⩾ 0 and λ₂ ⩾ 0 are the parameters for biasing towards forward-moving and against backward-moving reactions, respectively. Overall, there are only these two bias parameters, regardless which reaction k is considered. The surface maps of bias function $g_k\left({\mathit{x}}_{i-1}\right)$ with λ₁ = λ₂ = 0.5 for encouraging and discouraging reaction k at different values of $p_{\mathsf{F}}$, $p_{\mathsf{B}}$, and $\frac{A_k({\mathit{x}}_{i-1},{\mathit{x}}_i)}{A_0\left({\mathit{x}}_{i-1}\right)}$ are shown in Figs. 3a, 3b.

Figure 3.

Surface map of bias function $g_k\left({\mathit{x}}_{i-1}\right)$ in Eq. 12 with λ₁ = λ₂ = 0.5. Bias strengths for encouraging reaction k at different values of forward-moving probability $p_{\mathsf{F}}$ and $\frac{A_k({\mathit{x}}_{i-1},{\mathit{x}}_i)}{A_0\left({\mathit{x}}_{i-1}\right)}$ are shown in (a). Bias strengths for discouraging reaction k at different values of backward-moving probability $p_{\mathsf{B}}$ and $\frac{A_k({\mathit{x}}_{i-1},{\mathit{x}}_i)}{A_0\left({\mathit{x}}_{i-1}\right)}$ are shown in (b).

The construction of $g_k\left({\mathit{x}}_{i-1}\right)$ is based on the following consideration. The rate ratio $\frac{A_k({\mathit{x}}_{i-1},{\mathit{x}}_i)}{A_0\left({\mathit{x}}_{i-1}\right)}$, which is the original probability of choosing the reaction k that reaches ${\mathit{x}}_i$, is now modified by the forward probability $p_{\mathsf{F}}\left({\mathit{x}}_i\right)$ at ${\mathit{x}}_i$, obtained by looking κ-steps ahead. The term $\frac{A_k({\mathit{x}}_{i-1},{\mathit{x}}_i)}{A_0\left({\mathit{x}}_{i-1}\right)}·p_{\mathsf{F}}\left({\mathit{x}}_i\right)$ therefore represents the probability of selecting reaction k and moving forward. To encourage forward-moving reactions with lower reaction rates, we use the term $-\log [\frac{A_k({\mathit{x}}_{i-1},{\mathit{x}}_i)}{A_0\left({\mathit{x}}_{i-1}\right)}·p_{\mathsf{F}}\left({\mathit{x}}_i\right)]$ instead. This is then further modified by $p_{\mathsf{F}}\left({\mathit{x}}_i\right)$ so that reactions with higher forward-moving probability is favored proportionally (Fig. 3a). The bias coefficient λ₁ is used to adjust the bias strength for forward-moving reactions. Larger λ₁ gives stronger encouragement. As $-{\lambda }_1{p}_{\mathsf{F}}\left({\mathit{x}}_i\right)\log [\frac{A_k({\mathit{x}}_{i-1},{\mathit{x}}_i)}{A_0\left({\mathit{x}}_{i-1}\right)}·p_{\mathsf{F}}\left({\mathit{x}}_i\right)]$ falls in the interval [0, +∞), we add the constant 1 so the function $g_k\left({\mathit{x}}_{i-1}\right)$ is now in the interval [1, +∞) when reaction k should be encouraged. Setting bias according to $g_k\left({\mathit{x}}_{i-1}\right)$ will increase the probability for a forward moving reaction to be selected. Overall, if a larger λ₁ value is chosen, a slower reaction with higher probability of moving forward will be encouraged more (Fig. 3a).

Similarly, backward reactions are biased against, with stronger discouragement when using a larger λ₂ value. The discouragement is also stronger for reactions with larger backward probability $p_{\mathsf{B}}\left({\mathit{x}}_i\right)$ and larger rate ratio $\frac{A_k({\mathit{x}}_{i-1},{\mathit{x}}_i)}{A_0\left({\mathit{x}}_{i-1}\right)}$ (Fig. 3b). To ensure $g_k\left({\mathit{x}}_{i-1}\right)$ fall within the interval (0, 1], a “min ” function is used here to add an upper bound for the bias. If ${\mathit{x}}_i$ neither advances nor backtracks the system, no bias is introduced.

Corrections of biases and biased reaction rates.

In principle, both reaction selection probability and the Poisson time scale can be biased. In this study, we focus on effects of directly biasing reaction selection probability alone. The effects of directly biasing specific reaction rates are the subject for future studies. Specifically, we now insist that the overall reaction rate of the system is unchanged, namely,

$$B_0\left({\mathit{x}}_{i-1}\right)=A_0\left({\mathit{x}}_{i-1}\right).$$ (13)

As $B_0\left({\mathit{x}}_{i-1}\right)={\sum }_k{B}_k\left({\mathit{x}}_{i-1}\right)={\sum }_k{g}_k\left({\mathit{x}}_{i-1}\right)A_k\left({\mathit{x}}_{i-1}\right)$ and $A_0\left({\mathit{x}}_{i-1}\right)={\sum }_k{A}_k\left({\mathit{x}}_{i-1}\right),$ we use a normalizing constant α:

$$\alpha =A_0\left({\mathit{x}}_{i-1}\right)/\sum _k{g}_k\left({\mathit{x}}_{i-1}\right)A_k\left({\mathit{x}}_{i-1}\right),$$

and the biased reaction rate is tentatively set to:

$$B_k^{\prime }\left({\mathit{x}}_{i-1}\right)=\alpha g_k\left({\mathit{x}}_{i-1}\right)A_k\left({\mathit{x}}_{i-1}\right).$$ (14)

This ensures Eq. 13 holds.

As there are occasions where the bias changes direction after normalization, namely, from <1.0 to >1.0, or vice versa, we further insist that:

$$\begin{array}{c}\hfill B_k^{\prime }\left({\mathit{x}}_{i-1}\right)\le A_k\left({\mathit{x}}_{i-1}\right),\text{if}{p}_{\mathsf{B}}\left({\mathit{x}}_i\right)\ge 2·p_{\mathsf{F}}\left({\mathit{x}}_i\right),\\ \hfill B_k^{\prime }\left({\mathit{x}}_{i-1}\right)\ge A_k\left({\mathit{x}}_{i-1}\right),\text{if}{p}_{\mathsf{F}}\left({\mathit{x}}_i\right)\ge 2·p_{\mathsf{B}}\left({\mathit{x}}_i\right).\end{array}$$ (15)

To satisfy this requirement, we partition all reactions into two disjoint sets based on whether their corresponding $B_k^{\prime }\left({\mathit{x}}_{i-1}\right)$ satisfy the above inequalities:

$$\begin{array}{cc}\hfill {\mathcal{R}}_S& =\left\{k\right|\text{if}{B}_k^{\prime }\left({\mathit{x}}_{i-1}\right)\text{satisfies}\text{Eq.}\text{(}15\text{)}\},\hfill \\ \hfill {\mathcal{R}}_U& =\left\{k\right|\text{otherwise}\}.\hfill \end{array}$$ (16)

Inequalities in Eq. 15 are maintained by simply assigning no bias for all reactions in ${\mathcal{R}}_U$, and redistribute their surpluses and deficits evenly to all reactions in ${\mathcal{R}}_S$. As a result, the total reaction rate $B_0\left({\mathit{x}}_{i-1}\right)$ is unchanged. We have the final biased reaction rates:

$$\begin{array}{ccc}\hfill B_k\left({\mathit{x}}_{i-1}\right)& =& A_k\left({\mathit{x}}_{i-1}\right),\text{if}k\in {\mathcal{R}}_U,\hfill \\ \\ \hfill B_k\left({\mathit{x}}_{i-1}\right)& =& B_k^{\prime }\left({\mathit{x}}_{i-1}\right)(1+\sum _{l\in {\mathcal{R}}_U}[B_l^{\prime }\left({\mathit{x}}_{i-1}\right)-A_l\left({\mathit{x}}_{i-1}\right)]/\hfill \\ & & \sum _{j\in {\mathcal{R}}_S}\left[B_j^{\prime }\left({\mathit{x}}_{i-1}\right)\right]),\text{if}k\in {\mathcal{R}}_S,\hfill \end{array}$$ (17)

where $B_k^{\prime }\left({\mathit{x}}_{i-1}\right)$ is given by Eq. 14. The final biased probability for each step in an ABSIS sampling path is then calculated as

$$\begin{array}{ccc}& & p_{ABSIS}\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1}\hfill \\ & & =B_0\left({\mathit{x}}_{i-1}\right)e^{-B_0\left({\mathit{x}}_{i-1}\right){\tau }_{i-1}}d{\tau }_{i-1}·\frac{B_k\left({\mathit{x}}_{i-1}\right)}{B_0\left({\mathit{x}}_{i-1}\right)}.\hfill \end{array}$$ (18)

Weights of ABSIS path

The weight for correcting the bias for taking the kth reaction at step i − 1 is obtained by dividing Eq. 3 by Eq. 18:

$$w_{ABSIS}(k,{\mathit{x}}_{i-1})=\frac{A_k\left({\mathit{x}}_{i-1}\right)e^{[B_0\left({\mathit{x}}_{i-1}\right)-A_0\left({\mathit{x}}_{i-1}\right)]{\tau }_{i-1}}}{B_k\left({\mathit{x}}_{i-1}\right)}.$$ (19)

For the special case when the overall reaction rate is unchanged, namely, when $B_0\left({\mathit{x}}_{i-1}\right)=A_0\left({\mathit{x}}_{i-1}\right)$, we have $w_{ABSIS}(k,{\mathit{x}}_{i-1})=\frac{A_k\left({\mathit{x}}_{i-1}\right)}{B_k\left({\mathit{x}}_{i-1}\right)}$. The weight for a full ABSIS path is then:

$$\begin{array}{ccc}\hfill w_{ABSIS}\left({\pi }_{(0,N)}\right)& =& \prod _{i=1}^N{w}_{ABSIS}(k,{\mathit{x}}_{i-1})\hfill \\ & =& \prod _{i=1}^N\frac{A_k\left({\mathit{x}}_{i-1}\right)e^{[B_0\left({\mathit{x}}_{i-1}\right)-A_0\left({\mathit{x}}_{i-1}\right)]{\tau }_{i-1}}}{B_k\left({\mathit{x}}_{i-1}\right)}\hfill \\ & =& \prod _{i=1}^N\frac{A_k\left({\mathit{x}}_{i-1}\right)}{B_k\left({\mathit{x}}_{i-1}\right)}.\hfill \end{array}$$ (20)

The biased probability for a full ABSIS path π_{(0, N)} is

$$p_{ABSIS}\left({\pi }_{(0,N)}\right)=p\left({\mathit{x}}_0\right)\prod _{i=1}^N{p}_{ABSIS}\left({\mathit{x}}_i\right|{\mathit{x}}_{i-1},{\tau }_{i-1})d{\tau }_{i-1}.$$

The true probability of the path π_{(0, N)} can be recovered as

$$p\left({\pi }_{(0,N)}\right)=w_{ABSIS}\left({\pi }_{0,N}\right)·p_{ABSIS}\left({\pi }_{0,N}\right).$$

The ABSIS algorithm

We summarize the ABSIS method in Algorithm 1.

In order to improve computing efficiency, we enumerate the κ-step look-ahead paths for each microstate when encountered. As implementation and data structure greatly affect computing speed, $A_k\left({\mathit{x}}_{i-1}\right)$, $A_0\left({\mathit{x}}_{i-1}\right)$, $B_k\left({\mathit{x}}_{i-1}\right)$, $B_0\left({\mathit{x}}_{i-1}\right)$, and $g_k\left({\mathit{x}}_{i-1}\right)$ are all calculated only once when the microstate ${\mathit{x}}_{i-1}$ is first visited, with their values stored in hash tables using the microstate ${\mathit{x}}_{i-1}$ as the key. All subsequent visits to the microstate ${\mathit{x}}_{i-1}$ need only to retrieve relevant values stored in the hash tables. This leads to dramatically improved time efficiency.

Determining look-ahead step κ and bias parameter λ₁ and λ₂

In ABSIS, we only have one look-ahead steps parameter κ, and two bias parameters λ₁ and λ₂ to be determined, regardless of the number of reactions and the overall network complexity.

The ABSIS

// Input (X, R, ${\mathit{x}}_0$, D, θ, κ, M, λ1, λ2)

Define network $N\leftarrow (\mathcal{X},\mathcal{R})$

Initialize hash table $\mathcal{H}=\{\left({\mathit{x}}_i\right):[A_0\left({\mathit{x}}_i\right),B_0\left({\mathit{x}}_i\right),A_k\left({\mathit{x}}_i\right),B_k\left({\mathit{x}}_i\right),g_k\left({\mathit{x}}_i\right)]\}\leftarrow \varnothing$

j ← 0,   total weight wM ← 0,   weight square vM ← 0,

Number of successful paths Ns ← 0

whilej < Mdo

Path length i ← 1

Initialize path with the initial state ${\mathit{x}}_{i-1}\leftarrow {\mathit{x}}_0$

Time on current path t ← 0,   weight of current path w ← 1

whilet < θ and $d({\mathit{x}}_{i-1},\mathcal{D})>0$do

if${\mathit{x}}_{i-1}\notin \mathcal{H}$then

Calculate reaction rates $A_k\left({\mathit{x}}_{i-1}\right)$ of state ${\mathit{x}}_{i-1}$ for all reactions $R_k\in \mathcal{R}$

$A_0\left({\mathit{x}}_{i-1}\right)\leftarrow {\sum }_{k=1}^m{A}_k\left({\mathit{x}}_{i-1}\right)$

Enumerate all possible κ step paths π(i − 1, i + κ) starting from state ${\mathit{x}}_{i-1}$ using Algorithm of Ref 19.

Calculate $p_{\mathsf{F}}\left({\mathit{x}}_i\right)$ and $p_{\mathsf{B}}\left({\mathit{x}}_i\right)$ for each Rk using Eq. 9, 10

Calculate bias strength $g_k\left({\mathit{x}}_{i-1}\right)$ for each Rk according to Eq. 12

Calculate tentative reaction rate $B_k^{\prime }\left({\mathit{x}}_{i-1}\right)$ for all Rk according to Eq. 14

Calculate final biased reaction rate $B_k\left({\mathit{x}}_{i-1}\right)$ for all Rk according to Eq. 17

Calculate $B_0\left({\mathit{x}}_{i-1}\right)\leftarrow {\sum }_{k=1}^m{B}_k\left({\mathit{x}}_{i-1}\right)$

$\mathcal{H}\leftarrow \mathcal{H}\cup \{\left({\mathit{x}}_{i-1}\right):[A_0\left({\mathit{x}}_{i-1}\right),B_0\left({\mathit{x}}_{i-1}\right),A_k\left({\mathit{x}}_{i-1}\right),B_k\left({\mathit{x}}_{i-1}\right),g_k\left({\mathit{x}}_{i-1}\right)]\}$

end if

Retrieve $B_k\left({\mathit{x}}_{i-1}\right)$, $A_k\left({\mathit{x}}_{i-1}\right)$, $B_0\left({\mathit{x}}_{i-1}\right)$, $A_0\left({\mathit{x}}_{i-1}\right)$ and ${\mathit{g}}_k\left({\mathit{x}}_{i-1}\right)$ from H using key ${\mathit{x}}_{i-1}$

Generate two uniform random numbers ${\mu }_1\sim \mathcal{U}(0,1)$ and ${\mu }_2\sim \mathcal{U}(0,1)$

${\tau }_{i-1}\leftarrow -\ln \left({\mu }_1\right)/B_0\left({\mathit{x}}_{i-1}\right)$

$r\leftarrow \text{smallest}\text{integer}\text{satisfying}{\sum }_{r=1}^{k-1}{B}_r\left({\mathit{x}}_{i-1}\right)<{\mu }_2{B}_0\left({\mathit{x}}_{i-1}\right)\le {\sum }_{r=1}^k{B}_r\left({\mathit{x}}_{i-1}\right)$

t ← t + τi − 1, ${\mathit{x}}_{i-1}\leftarrow {\mathit{x}}_{i-1}+{\mathit{s}}_r$

$w\leftarrow w·\frac{A_k\left({\mathit{x}}_{i-1}\right)e^{(B_0\left({\mathit{x}}_{i-1}\right)-A_0\left({\mathit{x}}_{i-1}\right)){\tau }_{i-1}}}{B_k\left({\mathit{x}}_{i-1}\right)}$

i ← i + 1

end while

ift < θ and $d({\mathit{x}}_{i-1},\mathcal{D})=0$thenwM ← wM + w, vM ← vM + w2, Ns ← Ns + 1

end if

end while

return${\mathit{p}}_{ABSIS}({\mathit{x}}_0,\mathcal{D},\theta )=w_M/M$

return${\sigma }_{ABSIS}^2({\mathit{x}}_0,\mathcal{D},\theta )=(v_M/M)-{(w_M/M)}^2$

return${\sigma }_{ABSIS}({\mathit{x}}_0,\mathcal{D},\theta )=\sqrt{{\sigma }_{ABSIS}^2/M}$

return Success Rate: s = Ns/M

Estimation of look-ahead steps κ and parameter search space l for ABSIS

1: // Input (X, R, ${\mathit{x}}_0$, D, θ)

2: Look-ahead step: κ ← 2

3: Maximum range of parameter search space for determining λ1 and λ2: l ← 1.0

4: Sample size: M ← 1000

5: Success rate: s ← 0

6: while 1 do                      ▷ Determine the optimal κ

7:   s ← s + success rate of ABSIS(X, R, ${\mathit{x}}_0$, D, θ, κ, M, λ1 = 0.0, λ2 = 1.0)

8:   s ← s + success rate of ABSIS(X, R, ${\mathit{x}}_0$, D, θ, κ, M, λ1 = 1.0, λ2 = 0.0)

9:   s ← s + success rate of ABSIS(X, R, ${\mathit{x}}_0$, D, θ, κ, M, λ1 = 1.0, λ2 = 1.0)

10:   s ← s/3

11:   ifs > 0.5 then

12:    break

13:   end if

14:   κ ← κ + 1

15: end while

16: s ← success rate of ABSIS(X, R, ${\mathit{x}}_0$, D, θ, κ, M, λ1 = 0.5, λ2 = 0.5)

17: ifs > 0.8 then       ▷ Determine the maximum range for parameter search: l

18:   l ← 0.5

19: end if

20: return κ

21: returnl

To determine κ, we make the reasonable assumption that longer look-ahead paths lead to better bias parameters. Starting from κ = 2, we test different κ values with an increment of 1 using 10³ ABSIS paths. We take the first value of κ that gives an average success rate s of >0.50 at three different parameters locations of (λ₁, λ₂) = (0.0, 1.0), (1.0, 0.0), and (1.0, 1.0). This is very efficient as it typically only takes 3 × 10³ ABSIS paths to evaluate one κ. This is summarized in Algorithm 2.

To determine the optimal biasing parameters (λ₁, λ₂) ∈ [0.0, 1.0] × [0.0, 1.0], we use a grid search, where 10³ paths are generated at each grid point. The sample variance, success rate, total path weight and total weight square are stored at each grid point. We assume that the success rate s of ABSIS increases monotonically with parameters λ₁ and λ₂, at the cost of reduced diversity among sampled paths. We first evaluate s at (λ₁, λ₂) = (0.5, 0.5) using 10³ ABSIS paths. If s > 0.8, we focus on exploring more diverse paths and restrict our search space to (λ₁, λ₂) ∈ [0.0, 0.5] × [0.0, 0.5]. Otherwise, the search space remains as [0.0, 1.0] × [0.0, 1.0].

We start at (λ₁, λ₂) = (0.0, 0.0), and move first along the direction of λ₂, and then continue at an increased λ₁ value, all with an interval of Δ = 0.1. We stop our search along the λ₂ direction if s > 0.8. If s at a specific point of (λ₁, λ₂) is 0.5 better than its visited neighbors in either the λ₁ or the λ₂ directions, we retrospectively increase the number of grid points in that direction with a finer interval of Δ^′ = 0.02, and carry out searches on these grid points. After the search concludes, grid points with the smallest variance and s ∈ [0.1, 0.8] are taken as candidates.

We repeat this search process starting at (0.5, 0.5) again. The first candidate grid point that is again identified from a second independent search is taken as our final choice. When no candidate grid points are found in two independent searches, we repeat the overall search process, and update the stored sampling variances and success rates with results from new samples, until an optimal parameter pair is found. To further reduce computing costs, we skip grid points with previous s outside the range of [0.1, 0.8] when updating variance and success rates. The procedure for parameter estimation is summarized in Algorithm 3.

BIOLOGICAL EXAMPLES

Below we describe examples of applying ABSIS to four biochemical reaction networks. We show that ABSIS can provide accurate estimation of transition probabilities with efficient computation. Results are then compared with the true answer obtained from the finite buffer dCME method, and those obtained using other methods (the dwSSA,²³ as well as the swSSA method²⁷ and sdwSSA²⁶ method when possible), with differences discussed.

Birth-death process

The birth-death process is a simple chemical reaction system that involves one molecular species and two reactions. Synthesis and degradation are the only reactions, and there is only one molecular species. The network and parameters are specified as follows:

$$\begin{array}{cc}& R_1:\varnothing \stackrel{k_1}{\to }X,k_1=1,\hfill \\ & R_2:X\stackrel{k_2}{\to }\varnothing ,k_2=0.025.\hfill \end{array}$$ (21)

We study the problem of estimating the rare event probability, p(x(t) = 80|x(0) = 40, t ⩽ θ), that the system transits from the initial state x(0) = 40 to the target state x(t) = 80 within the time threshold θ = 100. This same problem was studied in Daigle et al.²³ and Roh et al.²⁷

Estimation of Bias Parameters λ₁, λ₂ for ABSIS.

1: // Input (X, R, ${\mathit{x}}_0$, D, θ, κ, l)

2: Sample size: M ← 1000

3: Bias parameters: λ1 ← 0, λ2 ← 0

4: Grid size: Δ = 0.1 and refined grid size: Δ′ = 0.02

5: Initialize hash tables ${\mathcal{H}}_1\{({\lambda }_1,{\lambda }_2):[{\sigma }_{1({\lambda }_1,{\lambda }_2)}^2,s_{1({\lambda }_1,{\lambda }_2)},M_{1({\lambda }_1,{\lambda }_2)}]\}\leftarrow \varnothing$

6: Initialize hash tables ${\mathcal{H}}_2\{({\lambda }_1,{\lambda }_2):[{\sigma }_{2({\lambda }_1,{\lambda }_2)}^2,s_{2({\lambda }_1,{\lambda }_2)},M_{2({\lambda }_1,{\lambda }_2)}]\}\leftarrow \varnothing$

7: Total sample size for parameter estimation: Mtot ← 0

8: while 1 do

9:   while λ1 ⩽ ldo

10:    while λ2 ⩽ ldo

11:     fori = 1 → 2 do

12:      if (λ1, λ2) $\in {\mathcal{H}}_i$ AND $s_{i({\lambda }_1,{\lambda }_2)}\notin [0.1,0.8]$then

13:       λ2 ← λ2 + Δ

14:       i ← i + 1, and go to next iteration.

15:      end if

16:      [${\sigma }_{i({\lambda }_1,{\lambda }_2)}^2$, $s_{i({\lambda }_1,{\lambda }_2)}$] = ABSIS(X, R, ${\mathit{x}}_0$, D, θ, κ, M, λ1, λ2)

17:      Mtot ← Mtot + M

18:      if$({\lambda }_1,{\lambda }_2)\notin {\mathcal{H}}_i$then

19:       ${\mathcal{H}}_i\leftarrow {\mathcal{H}}_1\cup \{({\lambda }_1,{\lambda }_2):[{\sigma }_{i({\lambda }_1,{\lambda }_2)}^2,s_{i({\lambda }_1,{\lambda }_2)},M]\}$

20:      else

21:       Update $[{\sigma }_{i({\lambda }_1,{\lambda }_2)}^2,s_{i({\lambda }_1,{\lambda }_2)},M_{i({\lambda }_1,{\lambda }_2)}]\in {\mathcal{H}}_i$ using $M_{i({\lambda }_1,{\lambda }_2)}+M$ samples

22:      end if

23:      if$(s_{i({\lambda }_1,{\lambda }_2)}-s_{i({\lambda }_1,{\lambda }_2-\Delta )})>0.5$then

24:       Repeat line 16–22 for refined grids in [λ2 − Δ, λ2] with interval Δ′

25:      end if

26:      if$(s_{i({\lambda }_1,{\lambda }_2)}-s_{i({\lambda }_1-\Delta ,{\lambda }_2)})>0.5$then

27:       Repeat line 16–22 for refined grids in [λ1 − Δ, λ1] with interval Δ′

28:      end if

29:     end for

30:     λ2 ← λ2 + Δ

31:    end while

32:    λ1 ← λ1 + Δ

33:   end while

34:   if$\arg {\min }_{({\lambda }_1,{\lambda }_2)\in {\mathcal{H}}_1,s_{1({\lambda }_1,{\lambda }_2)}\in [0.1,0.8]}\left\{{\sigma }_{1({\lambda }_1,{\lambda }_2)}^2\right\}=\arg {\min }_{({\lambda }_1,{\lambda }_2)\in {\mathcal{H}}_2,s_{2({\lambda }_1,{\lambda }_2)}\in [0.1,0.8]}\left\{{\sigma }_{2({\lambda }_1,{\lambda }_2)}^2\right\}$then Exit

35:   end if

36: end while

37: return λ1 and λ2

38: return total sample size: Mtot.

Exact probability landscape and transition probability

We first enumerate the full state space S of the birth-death model of Eq. 21, starting from the initial state of x(0) = 40 using the finite state buffer dCME method with a buffer size of 200.^{8, 19} There are a total of 241 microstates. To calculate the exact rare event transition probability of p(80|40, t ⩽ θ), the 241 × 241 transition rate matrix A is modified by making the target states x = 80 as an absorbing state, following the approach of Ref. 20. The exact transition probability p(80|40, t ⩽ θ) can then be computed from the modified ${\mathit{A}}_{abs}$:

$$\mathit{p}\left(\theta \right)=\mathit{p}\left(0\right)\exp (-{\mathit{A}}_{abs}\theta ),$$

where the initial state probability landscape $\mathit{p}\left(0\right)$ has $\mathit{p}{\left(0\right)}_{x=40}=1$ and 0 for all other 240 states. p(80|40, t ⩽ θ) is obtained from $\mathit{p}{\left(\theta \right)}_{x=80}$. We use the matrix exponential software EXPOKIT³⁷ to calculate p(80|40, t ⩽ θ) for θ = 100 numerically. The exact transition probability is found to be 2.986 × 10⁻⁷. This indicates that there would be only about 3 successful transition paths observed in 10 million sampled paths if the unmodified original SSA were used.

The calculated exact time-evolving probability landscape of the system is plotted in Fig. 4a. The blue and black curves show the landscapes at time t = 100 and at the steady state, respectively. There is one high probability region centered at x = 40 in both landscapes (green dots in Fig. 4a). The target state x = 80 (red dots in Fig. 4a) is located at a region with very low probability. Transitions from x = 40 to x = 80 is therefore of very low probability, as crossing a large barrier between these two states is necessary.

Figure 4.

The birth-death model of Eq. 21. (a) Its time-evolving probability landscape. The blue and black curves highlight the landscape at t = 100 and at the steady state, respectively. There is one high probability region located at x = 40 in both landscapes (green dots), which is also the initial state. The target state (red dots) is outside the high probability region. (b) and (c) The variance (b) and success rate (c) of pilot ABSIS sampling during parameter search using a total sampling size of M = 9.2 × 10⁴ and look-ahead path length of κ = 2. The yellow dot in (b) shows the location of the optimal parameters. (d) The estimated transition probability and sample convergence using ABSIS. The solid red line indicates the exact probability calculated from dCME. Black bars and heights of the box-plots are the mean and its standard deviations of estimated transition probability calculated from 4 independent ABSIS simulations, each for a different sample size of M = 10⁴, 10⁵, 10⁶, 10⁷, and 10⁸, respectively. (e) Standard deviations of ABSIS (blue line) and dwSSA (red line) at different sample sizes. (f) Sample variances of the ABSIS (blue line) and the dwSSA (red line) method at different sample sizes.

Determination of look-ahead steps and bias parameters

The look-ahead steps for ABSIS is determined to be κ = 2, and the parameter search space is determined to be 0.5 by running Algorithm 2. The Algorithm 3 is then used to determine λ₁ and λ₂ from the search space [0, 0.5] × [0, 0.5]. The optimal parameters are determined to be λ₁ = 0.50 and λ₂ = 0.18, which have a success rate of 0.63. Figures 4b, 4c show the variances of sampling weights and success rates of reaching the target state at different values of λ₁ and λ₂. The optimal parameters λ₁ = 0.50 and λ₂ = 0.18 are located in the lowest variance region of the parameter space (yellow dot in Fig. 4b). The total sample size for parameter search is 9.5 × 10⁴, which is much smaller than the reported sample size of 7 × 10⁵ in dwSSA.²³

Estimated transition probability

The estimated transition probability and variance from four independent simulations are plotted in Fig. 4d for sample size M of 10⁴, 10⁵, 10⁶, 10⁷, and 10⁸ used for each simulation. The estimated rare transition probability with M = 10⁷ is:

$$p_{ABSIS}\left(80\right|40,t\le 100)=2.981\times {10}^{-7}\pm 0.001\times {10}^{-7},$$

which is very close to the exact value of 2.986 × 10⁻⁷ (red line in Fig. 4d). In addition, ABSIS converge rapidly as the sample size increases.

We compare our results with those from the dwSSA method, which was implemented following Ref. 23. We use the exact bias constants of γ₁ = 1.454 and γ₂ = 0.686 as in Daigle et al.²³ The probability estimated from dwSSA is 2.937 × 10⁻⁷ ± 0.017 × 10⁻⁷ using a sample size of M = 10⁷, which is accurate but less so than that of ABSIS. Additionally, the ABSIS method has a higher success rate (0.63) than the dwSSA method (0.59). The comparisons of mean standard deviations between ABSIS and dwSSA calculated from four independent simulations using different sample size are plotted in Fig. 4e. ABSIS has a standard deviation about one order of magnitude smaller than dwSSA. In addition, ABSIS requires much less samples to achieve the same accuracy of dwSSA. For example, 10⁴ samples of ABSIS has a smaller standard deviation than 10⁶ samples of dwSSA. We also compare ABSIS estimation with the results from swSSA as reported in Roh et al.²⁷ The estimation of 95% confidence interval from 10⁵ samples of ABSIS is 2.986 × 10⁻⁷ ± 0.020 × 10⁻⁷, which is comparable to the estimation of swSSA 2.986 × 10⁻⁷ ± 0.019 × 10⁻⁷ using the same sample size.²⁷

The sample variances of the ABSIS method when using different sample sizes are shown in Fig. 4f (blue line), along with variances using dwSSA sampling (red line, Fig. 4f). Overall, ABSIS gives consistently small variance. At M = 10⁷, the variance (1.0 × 10⁻¹³) is two orders of magnitude smaller than the variance of 3.1 × 10⁻¹¹ when using the dwSSA method. We further note that the variance of estimated transition probabilities using dwSSA seems to increase with the sample size.

Overall, our results show that the ABSIS method converges rapidly to the true transition probability when sample size is increased, whereas the dwSSA method converges less rapidly and has larger variance.

Bias mechanism of ABSIS

Examining the forward-moving probability (Figs. 5a, 5b, green lines) and the backward-moving probability (red lines) of both reactions R₁ and R₂ at different states helps to gain insight into how ABSIS works. The synthesis reaction R₁ has a much higher forward-moving than backward-moving probability in majority of the states, and the degradation reaction R₂ has a much higher backward-moving probability in majority of the states. These observations suggest that in most cases, one should bias to encourage reaction R₁ and to discourage reaction R₂.

Figure 5.

Forward and backward moving probabilities. (a) and (b) Probability of moving-forward and moving-backward for two reactions R₁ and R₂ in birth-death model. The x-axis is the system state, i.e., the copy number of molecular species X, and y-axis is the forward-moving (green lines) and backward-moving (red lines) probabilities of reaction R1 and R2 in each state. (c) and (d) The final ABSIS bias strengths for both reactions in birth-death model. Blue lines show the steady state probability landscape of birth-death model. The black lines in (c) and (d) show the curves of bias strengths for reactions R₁ and R₂, respectively. Green and red vertical lines indicate the start and end state.

As the system approaches the target state, the forward-moving probability of R₁ (green line in Fig. 5a) decreases dramatically, while the backward-moving probability of R₂ (red line in Fig. 5b) increases. This is due to the fact that the propensity for backward-moving becomes stronger as the rate of the degradation reaction R₂ increases monotonically with the copy number of X, while the rate of the synthesis reaction R₁ remains constant.

It is clear that constant biases will not work well for this problem, as the rare event transition requires overcoming the steep probability barrier between the two states of x = 40 and x = 80 (Fig. 5c, blue line). The optimal bias strengths will need to depend on the current propensity of forward moving, and should be adaptive.

For this problem, the ABSIS strategy of designing biases based on estimations from look-ahead paths works well. The bias strengths generated by the ABSIS algorithm for both reactions R₁ and R₂ are plotted in Figs. 5c, 5d (black lines), along with the steady state probability landscape (blue lines) as reference. In general, the biases for R₁ are all favorable (Fig. 5c), and the biases for R₂ are all unfavorable (Fig. 5d). However, the strength of the bias is adaptively adjusted following changes in the reaction propensity, as well as the need for overcoming the probability barriers. When approaching the target state, bias is set such that R₁ is much more strongly encouraged to produce more X, whereas R₂ is severely repressed to reduce the degradation of X.

Overall, by utilizing future information from κ = 2 look-ahead paths, ABSIS can identify automatically the reaction to encourage, as well as the reaction to discourage at any given state. The forward and backward-moving probabilities estimated from look-ahead paths can aid in crossing the probability barrier of rare event transitions. By adaptively changing biases according to changes in reaction propensity and future information about the probability barrier, the ABSIS method can provide estimates for the birth-death model with much smaller sampling variance compared to methods using constant biases such as the dwSSA method.²³

Reversible isomerization

We also apply the ABSIS method to the reversible isomerization network taken from the Ref. 26, where the sdwSSA method was applied.²⁶ The reversible isomerization network involves two molecular species and two reactions:

$$\begin{array}{cc}& R_1:A\stackrel{k_1}{\to }B,k_1=0.12,\hfill \\ & R_2:B\stackrel{k_2}{\to }A,k_2=1.\hfill \end{array}$$ (22)

Our goal is to estimate the rare event probability $p(x_B=30|\mathit{x}\left(0\right),t\le 10)$ that the system transitions from an initial state $\mathit{x}\left(0\right)=\left\{\right(100,0\left)\right\}$ to any state with 30 copies of B within the time interval of t ⩽ 10.

Exact probability landscape and transition probability

We first enumerate the full state space S of the reversible isomerization model in Eq. 22, starting from the initial state $\mathit{x}\left(0\right)=(100,0)$ using the dCME method. This reversible isomerization model is a closed system, therefore no buffer is needed. There are a total of 101 microstates in the state space S. The exact transition probability of the rare event $p(x_B=30|\mathit{x}\left(0\right),t\le 10)$ is calculated by solving the matrix exponential problem $\mathit{p}\left(10\right)=\mathit{p}\left(0\right)\exp (-{\mathit{A}}_{abs}·10)$ using the EXPOKIT software,³⁷ where ${\mathit{A}}_{abs}$ is the modified transition rate matrix by making the target states $\left\{\mathit{x}\right|x_B=30\}$ absorbing states, following the approach of Ref. 20. The exact transition probability is found to be 1.1911 × 10⁻⁵.

The time-evolving probability landscape of the system is calculated, and its projection to B is plotted in Fig. 6a. The blue and black curves show the landscape at t = 10, and at the steady state, respectively. There is one high probability region centered at x_B = 10 (red circles in Fig. 6a). The target state x_B = 30 (red solid dots in Fig. 6a) is located in a region with very low probability. Transitions from x_B = 0 (green dots in Fig. 6a) to x_B = 30 therefore has very low probability, as a large barrier between these two states need to be crossed.

Figure 6.

Rare event estimation of the reversible isomerization network model. (a) Its time-evolving probability landscape projected to B. The blue and black curves show the landscape at t = 10, and at the steady state, respectively. There is only one high probability region in both landscapes, which is located at x_B = 10 (red circles). The probability landscape at time t = 10 (blue curve) largely overlaps with that of the steady state (black curve). The initial state x_B = 0 (green dots) is on the tail of the left-side of the probability peak, and the target state x_B = 30 (red dots) is on the far right-side of the low probability region. (b) and (c) The variances (b) and success rates (c) of pilot ABSIS sampling during parameter search using a total sample size of M = 8.8 × 10⁴ and look-ahead steps κ = 2. The yellow dot in (b) shows the location of the optimal parameters. (d) The estimated rare event probability and sampling convergence using ABSIS. The solid red line represents the exact probability calculated from directly solving dCME. The black bars and the box heights are the means and standard deviations calculated from 4 independent ABSIS simulations, for different sample sizes of M10⁴, 10⁵, 10⁶, 10⁷, and 10⁸. (e) Standard deviations of ABSIS (blue line) and dwSSA (red line) at different sample sizes. (f) Sampling variances of ABSIS (blue line) and dwSSA (red line) for different sample sizes.

Determination of look-ahead steps and bias parameters

The look-ahead steps for ABSIS is determined to be κ = 2 and parameter search space to be l = 0.5 after running Algorithm 2. Algorithm 3 is used to determine λ₁ and λ₂ from the search space [0, 0.5] × [0, 0.5]. The optimal parameters are found to be λ₁ = 0.20 and λ₂ = 0.16, which have a success rate of 0.76. Figures 6b, 6c show the variances of sampling weights and success rates of reaching the target state at different values of λ₁ and λ₂. The optimal parameter pair λ₁ = 0.20 and λ₂ = 0.16 are located in the lowest variance region of the parameter space (yellow dot in Fig. 6b). The total sample size for parameter search is 9.1 × 10⁴, which is much smaller than the reported 7 × 10⁵ samples for parameter estimations for dwSSA and sdwSSA in Ref. 26.

Estimated transition probability

The estimated transition probability and standard deviation by averaging four independent simulations using different sample size of M = 10⁴, 10⁵, 10⁶, 10⁷, and 10⁸ are plotted in Fig. 6d. ABSIS simulation provides accurate estimate of 1.1909 × 10⁻⁵ ± 0.0004 × 10⁻⁵ using the sample size of M = 10⁷. In addition, ABSIS converges rapidly to the exact rare event probability computed from the dCME method (red line in Fig. 6d) as the sample size increases. When the same sample size M = 10⁶ as that of Roh et al.²⁶ is used, the ABSIS method gives the estimation of 1.192 × 10⁻⁵ ± 0.001 × 10⁻⁵, with its standard deviation about only one half of the estimation 1.193 × 10⁻⁵ ± 0.002 × 10⁻⁵ from the sdwSSA method.²⁶ When using the same sample size M = 10⁵ as in Roh et al.,²⁷ the ABSIS method gives the 95% confidence interval estimation as 1.191 × 10⁻⁵ ± 0.007 × 10⁻⁵, which has a much smaller 95% confidence interval than the estimation of swSSA 1.190 × 10⁻⁵ ± 0.011 × 10⁻⁵ with the same sample size.²⁷

We also compare our results with those from the dwSSA method. For dwSSA sampling, we use bias constants (γ₁ = 1.301, γ₂ = 0.719) for the two reactions in the network as reported in Roh et al.²⁶ The rare event probability estimated from dwSSA using the sample size of M = 10⁷ is 1.278 × 10⁻⁵ ± 0.060 × 10⁻⁵, and the success rate is only 0.07. The comparisons of mean standard deviations between ABSIS and dwSSA calculated from four independent simulations using different sample size are plotted in Fig. 6e. ABSIS results show 1–2 orders of magnitude smaller standard deviation (Fig. 6e) than dwSSA in estimating the rare event probability. In addition, ABSIS requires much less samples to achieve the same accuracy of dwSSA. In this example, 10⁴ samples of ABSIS has a much smaller standard deviation than dwSSA with 10⁸ samples.

The ABSIS method gives more accurate estimations than dwSSA (1.191 × 10⁻⁵ vs 1.278 × 10⁻⁵ at M = 10⁷, 1.192 × 10⁻⁵ vs 1.201 × 10⁻⁵ at M = 10⁶, and 1.191 × 10⁻⁵ vs 1.075 × 10⁻⁵ at M = 10⁵ compared to the exact value of 1.191 × 10⁻⁵), and has higher success rate (0.76) compared to the dwSSA method (0.07). The ABSIS method also gives estimations with much smaller standard deviations than swSSA (at M = 10⁵, where data are reported in Ref. 27) and sdwSSA (at M = 10⁶, where data are reported in Ref. 26). In addition, it gives consistently smaller sample variances (1.3 × 10⁻¹⁰ at M = 10⁷), which is four orders of magnitude smaller than the variance 4.2 × 10⁻⁶ obtained when using the dwSSA method. The sample variance of ABSIS using different sample size is shown in log-scale in Fig. 6f (blue line), along with variances using dwSSA sampling (red line, Fig. 6f).

Bistable Schlögl model

Schlögl model is a one-dimensional bistable system first proposed in Ref. 38, and extensively studied subsequently.^{39, 40, 41} It is an auto-catalytic network consisting of one molecular species (X) whose concentration can change through four reactions:^{38, 39}

$$\begin{array}{cc}& R_1:A+2X\stackrel{k_1}{\to }3X,k_1=3,\hfill \\ & R_2:3X\stackrel{k_2}{\to }A+2X,k_2=0.6,\hfill \\ & R_3:B\stackrel{k_3}{\to }X,k_3=0.25,\hfill \\ & R_4:X\stackrel{k_4}{\to }B,k_4=2.95,\hfill \end{array}$$ (23)

where A and B are species with constant concentrations (set to a = 1 and b = 2, respectively). Values of reaction rate constants k₁, k₂, k₃, and k₄ are taken from Vellela and Qian.³⁹ The volume of the system is fixed as V = 25. Following Vellela and Qian,³⁹ the reaction rates are calculated using formulas $A_1\left(x\right)=\frac{a{k}_{1}x(x-1)}{V}$, $A_2\left(x\right)=\frac{k_{2}x(x-1)(x-2)}{V^2}$, A₃(x) = bk₃V, and A₄(x) = k₄x, respectively. Our task is to estimate the probability p(92|0, t ⩽ θ) that the Schlögl system transitions from an initial state x = 0 to the target state x = 92 within a given time threshold of θ = 2.

Exact probability landscape and transition probability

We first enumerate the full state space S of the Schlögl model of Eq. 23, starting from the initial state of x = 0 using the dCME method with a buffer size of 1,000. There are 1,001 microstates in the state space S. The exact transition probability of the rare event p(92|0, t ⩽ 2) is calculated by solving the matrix exponential problem $\mathit{p}\left(\theta \right)=\mathit{p}\left(0\right)\exp (-{\mathit{A}}_{abs}·2)$ using the EXPOKIT software,³⁷ where ${\mathit{A}}_{abs}$ is the modified transition rate matrix by making the target states x = 92 an absorbing state, following the practice of Ref. 20. The calculated exact transition probability is 5.419 × 10⁻⁵. That is, if 10⁵ paths are sampled using the original SSA method, there will only be about 5 successful transition paths.

The calculated exact time-evolving probability landscape of the system is plotted in Fig. 7a. The blue and black curves show the landscape at time t = 2 and at the steady state, respectively (Fig. 7a). There are two high probability regions centered at x = 4 (red circle on black curve) and x = 92 (red solid dot on black curve), respectively, on the steady state probability landscape (black curve). They are separated by a low probability barrier. The probability landscape at time t = 2 (blue curve) shows a much sharper peak centered at x = 3 (red circle on blue curve). It is clear that transition paths from x = 0 to x = 92 within t = 2 have a steep barrier to cross.

Figure 7.

The Schlögl model. (a) Its time-evolving probability landscape. The blue and black curves show the landscape at t = 2 and at the steady state, respectively. There are two high probability regions at steady state (black curve) located at x = 4 (red circle on black curve) and x = 92 (red dot on black curve), respectively. The initial state x = 0 (green dot) is near the first peak, and the target state (red dot) is at the center of the second peak. (b) and (c) Variance and success rate of pilot ABSIS sampling during parameter search using a total sampling size of M = 3.28 × 10⁵ and look-ahead steps κ = 2. The yellow dot in (b) shows the location of the optimal parameters. (d) The estimated transition probability and convergence behavior using ABSIS. The solid red line indicates the exact probability calculated from dCME. The black bars and the heights of boxes in the box-plots are the average means and standard deviations calculated from 4 independent ABSIS simulations, for a different sample size of M = 10⁴, 10⁵, 10⁶, 10⁷ and 10⁸. (e) Standard deviations of ABSIS (blue line) and dwSSA (red line) at different sample sizes. (f) Sampling variances of ABSIS (blue line) and dwSSA (red line) for different sample sizes.

Determination of look-ahead steps and bias parameters

The look-ahead steps for ABSIS in Schlögl model is determined to be κ = 2, and the parameter search space is determined to be 0.5 after running Algorithm 2. Algorithm 3 is then used to determine λ₁ and λ₂ from the search space [0, 0.5] × [0, 0.5]. The optimal parameters are found to be λ₁ = 0.10 and λ₂ = 0.40, which have a success rate of 0.15. Figures 7b, 7c show the variances of sampling weights and success rates of reaching the target state at different values of λ₁ and λ₂. The optimal parameter pair λ₁ = 0.10 and λ₂ = 0.40 is located in the lowest variance region of the parameter space (yellow dot in Fig. 7b). The total sample size for parameter search is 3.28 × 10⁵, which is much smaller than the typical sample size of 7 × 10⁵ reported in dwSSA.²³

Estimated transition probability

The estimated transition probability and variance by averaging four independent simulations using different sample size of M = 10⁴, 10⁵, 10⁶, 10⁷, and 10⁸ are plotted in Fig. 7d. With the sample size M of 10⁷, ABSIS simulation provides an accurate estimate of p_ABSIS(92|0, t ⩽ 2) = 5.394 × 10⁻⁵ ± 0.009 × 10⁻⁵, which is very close to the exact value of 5.419 × 10⁻⁵. In addition, ABSIS converges rapidly (Fig. 7e) as the sample size increases.

We also compare our results with those obtained using the dwSSA method. For dwSSA sampling, we followed the original authors' recommendation of choosing parameters such that the minimum fraction ρ of trajectories reaching the target states is 0.02.²³ This gives the bias constants of γ₁ = 1.115, γ₂ = 0.967, γ₃ = 1.171, and γ₄ = 0.872 for reaction 1–4 in the network, respectively. The rare event probability estimated from dwSSA is 5.976 × 10⁻⁵ ± 0.342 × 10⁻⁵ using a sample size M = 10⁷, which is less accurate than that of ABSIS (5.394 × 10⁻⁵ ± 0.009 × 10⁻⁵ vs. the exact value of 5.419 × 10⁻⁵). It also has a lower success rate of 0.02 compared to ABSIS (0.15). The comparisons of mean standard deviations between ABSIS and dwSSA calculated from four independent simulations using different sample sizes are plotted in Fig. 7e. ABSIS results show about one order of magnitude smaller standard deviation (Fig. 7e) than dwSSA. In terms of computing efficiency, ABSIS sampling is able to achieve better accuracy than dwSSA with 1/10 of samples.

The sample variance of ABSIS using different sample size is shown in Fig. 7f (blue line), along with variances using dwSSA sampling (red line). Overall, ABSIS sampling gives consistently small sample variances (8.712 × 10⁻⁸ at M = 10⁷), which is roughly four orders of magnitude smaller than the variance 3.233 × 10⁻⁴ when using the dwSSA method.

Bias Mechanism of ABSIS

By examining the forward-moving probability (green lines in Figs. 8a, 8b, 8c, 8d) and the backward-moving probability (red lines) of all four reactions at different states, we found that the synthesis reactions R₁ and R₃ have much higher forward-moving than backward-moving probability in majority of the states, and the degradation reactions R₂ and R₄ have much higher backward-moving probability in majority of the states. These observations suggest that reactions R₁ and R₃ should be encouraged and reactions R₂ and R₄ should be discouraged.

Figure 8.

Forward and backward-moving probabilities and ABSIS biases for each reactions. (a)–(d) Probability of moving-forward and moving-backward for four reactions in Schlögl model. The x-axis is the system state, i.e., the copy number of molecular species X, and y-axis is the forward-moving (green lines) and backward-moving (red lines) probabilities of reaction R1–R4 in each state. (e)–(h) The final ABSIS bias strengths for reactions in Schlögl model. Blue lines show the steady state probability landscape of birth-death model. The black lines show the curves of bias strengths for four reactions, respectively.

Obviously, constant biases will not work well for this problem because of the steep barrier crossing region between the initial and the target state (blue curve in Fig. 8e). The optimal bias strengths should be adaptive and should be determined by the complex probability landscape of the system.

For the Schlögl model, the ABSIS strategy works well. The bias strengths calculated by the ABSIS algorithm for all four reactions are plotted in Figs. 8e, 8f, 8g, 8h (black curves), along with the steady state probability landscape as a reference (blue curves). The biases for R₁ and R₃ are all favorable (Figs. 8e, 8g), and the biases for R₂ and R₄ are all unfavorable (Figs. 5f, 5h). Interestingly, the strongest biased region of R₁, R₃, and R₄ overlapped with the steepest barrier crossing region in the landscape (Figs. 8e, 8g, 8h), which shows that ABSIS can capture the urgent need for overcoming the probability barriers at the time. The insignificant bias of R₂ is due to its smaller reaction rates, although it has a similar backward-moving probability as R₄ (Fig. 8e).

Estimating rare event probability for Schlögl model is a difficult task for methods with constant biases, as reported in Ref. 34. However, by utilizing future information from 3-step look-ahead paths, ABSIS successfully estimated the probability of rare event transition in the bistable Schlögl model with accuracy and small sampling variance, and compares favorably to the constant biased dwSSA method.²³

Enzymatic futile cycle

Enzymatic futile cycle is a ubiquitous network motif consisting of six different molecular species and six reactions. Samoilov et al. studied this network in detail.⁴² The molecular species, reactions, and corresponding reaction rate constants of the enzymatic futile cycle system are as follows:

$$\begin{array}{cc}& R_1:X_1+X_2\stackrel{k_1}{\to }{X}_3,k_1=1,\hfill \\ & R_2:X_3\stackrel{k_2}{\to }{X}_1+X_2,k_2=1,\hfill \\ & R_3:X_3\stackrel{k_3}{\to }{X}_1+X_5,k_3=0.1,\hfill \\ & R_4:X_4+X_5\stackrel{k_4}{\to }{X}_6,k_4=1,\hfill \\ & R_5:X_6\stackrel{k_5}{\to }{X}_4+X_5,k_5=1,\hfill \\ & R_6:X_6\stackrel{k_6}{\to }{X}_4+X_2,k_6=0.1.\hfill \end{array}$$ (24)

Estimating rare event probability of the enzymatic futile cycle system has been the subject of recent studies using the wSSA method²² and the dwSSA method.²³ Here the goal is to estimate the probability $p(x_5=25|\mathit{x}\left(0\right),t<\theta )$ that the system starts from the initial state $\mathit{x}\left(0\right)=\left\{\right(1,50,0,1,50,0\left)\right\}$ to any other states with exactly 25 copies of X₅ within the time-threshold of θ = 100. The same task was also studied in Daigle et al. using dwSSA.²³

Exact probability landscape and transition probability

We first enumerate the full state space S of the futile cycle model of Eq. 24, starting from the initial state $\mathit{x}\left(0\right)=(1,50,0,1,50,0)$ using the dCME method. As the futile cycle model is a closed system, no buffer is needed for the dCME method. There are a total of 400 microstates in the state space S. The exact transition probability of the rare event p(x₅ = 25|(1, 50, 0, 1, 50, 0), t < 100) is calculated by solving the matrix exponential problem $\mathit{p}\left(100\right)=\mathit{p}\left(0\right)\exp (-{\mathit{A}}_{abs}·100)$ using the EXPOKIT software,³⁷ where ${\mathit{A}}_{abs}$ is the modified transition rate matrix by making the target states $\left\{\mathit{x}\right|x_5=25\}$ absorbing states. The exact transition probability is calculated to be 1.738 × 10⁻⁷. That is, if we use the original SSA method, there will be only about 2 successful transition paths sampled in 10 million different sampled trajectories.

The time-evolving probability landscape of the system is calculated, and its projection to X₅ is plotted in Fig. 9a. The inset figure in Fig. 9a shows the time-evolving landscape from time t = 1 to t = 100, and the main figure shows the time frame from t = 100 to t = 10⁴. The blue and black curves show the landscape at t = 100, and at steady state, respectively. There is only one high probability region in the projected steady state probability landscape (black curve), which is centered at x₅ = 50 (green dots). The probability landscape at time θ = 100 (blue curve) shows a much sharper peak centered at the same location x₅ = 50. It is clear that transition paths from x₅ = 50 to x₅ = 25 within t ⩽ 100 have a steep barrier to cross, although there is no such barrier if sampling time is not restricted.

Figure 9.

The enzymatic futile cycle model. (a) Its time-evolving probability landscape projected to X₅. The inset figure shows the time-evolving landscape from time t = 1 to t = 100, and the main figure shows the time frame from t = 100 to t = 10⁴. The blue curves show the landscape at t = 100, and the black curve shows the landscape at steady state. There is only one high probability region, which is located at x₅ = 50 (green dots). The probability landscape at time t = 100 (blue curve) shows a much sharper peak also centered at x₅ = 50. The initial state with x₅ = 50 (green dots) is at the height of the probability peak, and the target state at x₅ = 25 (red dots) is in the low probability region to the left of the peak. (b) and (c) shows the variance (b) and success rate (c) of pilot ABSIS sampling during parameter search using a total sampling size of M = 2.11 × 10⁵ and look-ahead steps of κ = 3. The yellow dot in (b) shows the location of the optimal parameters. (d) The estimated transition probability and convergence behavior using ABSIS. The solid red line represents the exact probability calculated from dCME. The black bars and the box-plots are the means and standard deviations calculated from 4 independent ABSIS simulations using a sample size of M of 10⁴, 10⁵, 10⁶, 10⁷, and 10⁸. (e) Standard deviations of ABSIS (blue line) and dwSSA (red line) at different sample sizes. (f) Sample variances of ABSIS (blue line) and dwSSA (red line) at different sample sizes.

Determination of look-ahead steps and bias parameters

The look-ahead steps for ABSIS in the futile cycle model is determined to be κ = 3, and the parameter search space is determined to be l = 1.0 after running Algorithm 2. Algorithm 3 is then used to determine λ₁ and λ₂ from the search space [0, 1.0] × [0, 1.0]. The optimal parameters are determined to be λ₁ = 0.60 and λ₂ = 0.40, which have a success rate of 0.41. Figures 9b, 9c show the variances of sampling weights and success rates of reaching the target state at different values of λ₁ and λ₂. The optimal parameter pair λ₁ = 0.60 and λ₂ = 0.40 is located in the lowest variance region of the parameter space (yellow dot in Fig. 9b). The total sample size for parameter search is 2.11 × 10⁵, which is much smaller than the reported sample size of 7 × 10⁵ using dwSSA.²³

Estimated transition probability

The estimated transition probability and standard deviation by averaging four independent simulations using different sample size of M = 10⁴, 10⁵, 10⁶, 10⁷, and 10⁸ are plotted in Fig. 9d. ABSIS simulation provides an accurate estimate of 1.730 × 10⁻⁷ ± 0.001 × 10⁻⁷ with sample size M = 10⁷, which is very close to the exact value of 1.738 × 10⁻⁷. The success rate is 0.41. In addition, ABSIS converges rapidly to the exact rare event probability (red line in Fig. 9d) as the sample size increases.

We also compared our results with those obtained using the dwSSA method. For dwSSA sampling, we use bias constants (γ₁ = 1.000, γ₂ = 1.003, γ₃ = 0.320, γ₄ = 1.003, γ₅ = 0.993, and γ₆ = 3.008) taken from Daigle et al.²³ for the six reactions in the network. The rare event probability estimated from dwSSA is 1.741 × 10⁻⁷ ± 0.001 × 10⁻⁷ using a sample size M = 10⁷, which is slightly better than the estimate from ABSIS, with a higher success rate of 0.67. The comparisons of mean standard deviations between ABSIS and dwSSA calculated from four independent simulations using different sample size are plotted in Fig. 9e. ABSIS results show about 1.5 times larger standard deviations (Fig. 9e) than dwSSA in estimating the rare event probability in futile cycle model. In terms of computing efficiency, ABSIS sampling needs about 1.5 times more samples to achieve the same accuracy as dwSSA with.

The sample variance of ABSIS using different sample size are shown in Fig. 9f (blue line), along with variances using dwSSA sampling (red line, Fig. 9f). ABSIS sampling gives consistently small sample variances (1.708 × 10⁻¹³ at M = 10⁷), although it is about twice as large as the variance of 7.901 × 10⁻¹⁴ when using the dwSSA method. It has also a lower success rate of 0.41.

Bias mechanism of ABSIS

The enzymatic futile cycle network has different characteristics from the other networks studied here. This network includes two enzymes, and each has its active and inactive forms, which amount to a total of four enzyme molecular species, namely the first enzyme X₁ and its inactive form X₃, the second enzyme X₄ and its inactive form X₆. However, as each enzyme has only one copy in the system (X₁ + X₃ = 1, and X₄ + X₆ = 1), the occurrence of reactions is highly restricted by the availability of the enzymes. To study the biasing mechanism of the reactions, we project the forward-moving probability and the backward-moving probability of each reaction to the space of species X₅ and fixed combinations of X₁ and X₄ (Figs. 10a, 10b, 10c, 10d, 10e, 10f for forward-moving probabilities and Figs. 10g, 10h, 10i, 10j, 10k, 10l for backward-moving probabilities). We found that the surfaces of forward and backward-moving probabilities are rather rugged. The forward and backward-moving probabilities for the same reaction can be very different for microstates with only one copy difference in X₁ or X₄. For example, the $p_{\mathsf{F}}$ of R₁ at X₁ = 1, X₄ = 0 (red histograms in Fig. 10a) is close to 1, but with only one copy difference in X₄, the $p_{\mathsf{F}}$ of R₁ at X₁ = 1, X₄ = 1 is very close 0 (yellow histograms in Fig. 10a). This ruggedness is due to the fact that neighboring microstates have different available enzymes, therefore very different reactions occur according to Eq. 24. In fact, no microstates can have all six reactions occurring simultaneously. The ruggedness of the surfaces of forward and backward-moving probabilities requires biases with large fluctuations (as shown in Figs. 11c, 11f). Our bias scheme seems to offer no improvement for reducing sampling variance compared to dwSSA.

Figure 10.

ABSIS forward and backward-moving probabilities for each reactions in the enzymatic futile cycle model. (a)–(f) Projected forward-moving probabilities of all six reactions on the space of X₅ and four different combinations of X₁ and X₄ shown in different colors. (g)–(l) Projected backward-moving probabilities of all six reactions on the same space of X₅ and four different combinations of X₁ and X₄.

Figure 11.

ABSIS biases for each reaction in futile cycle model. (a)–(f) The final ABSIS bias strengths for each reaction in enzymatic futile cycle model.

Although reactions R₁, R₂, R₄, and R₆ have overall larger forward-moving probabilities (Fig. 10) and should be encouraged, and reactions R₃ and R₅ have overall larger backward-moving probabilities and should be discouraged, the ABSIS biases for reactions R₁, R₂, R₄, and R₅ are all very close to 1, and only reactions R₃ and R₆ are significantly biased (Fig. 11), as they have the slowest rates among all six reactions and thus biased. Biases for R₃ are clustered into two nearly flat biases with different mean 0.3364 and 0.4651, respectively (Fig. 11c). There are also two different nearly flat biases for R₆ around the mean 2.6330 and 3.5129, respectively (Fig. 11f). In general, the current bias scheme used in ABSIS for the futile cycle network are not very different from the bias constants of dwSSA used in Daigle et al.²³

Overall the ABSIS method is comparable to the dwSSA method for studying the futile cycle network. It can provide accurate estimates of the rare event probability. The sampling variance is constantly small at around 1.7 × 10⁻¹³, regardless of sample sizes. ABSIS also correctly identifies reactions that need to be encouraged and to be discouraged, although the sampling variance of ABSIS is about twice as large as that of dwSSA due to the ruggedness of the surfaces of forward and backward-moving probabilities.

DISCUSSIONS AND CONCLUSIONS

Sampling rare events is an important task for studying key events important for biological processes. In this work, we described a general theoretical framework for obtaining optimized bias in sampling individual reactions to estimate probabilities of rare events. We further developed a practical algorithm named ABSIS for efficient estimation of probabilities of rare events. By adopting a look-ahead strategy and by examining κ-step look-ahead paths following each reaction from the current microstate, we can estimate the reaction-specific and state-dependent forward-moving and backward-moving probabilities of the system. These probabilities are then used to adaptively adjust biases towards selecting each reaction. Overall, ABSIS is well suited for studying rare events in networks with complex probability landscape and steep probability barrier.

Our method addresses a major challenge in estimating rare event probability in biological networks, namely, the need to cross barriers on the probability landscape. As reactions in a network proceeds, the local neighborhood of the probability landscape changes, and different biases are often necessary for barrier-crossing. Unlike previous importance sampling methods such as sdwSSA²⁶ and swSSA,²⁷ in which biases are only based on reaction rates in the current state with no consideration of future information, the ABSIS method can detect barrier-crossing region in the probability landscape by incorporating future information. The bias introduced by the ABSIS method not only depends on the current state, but also depends on the need to cross the probability barrier, which is detected by the κ-step look-ahead strategy. The calculation of κ-step forward-moving and backward-moving probabilities is equivalent to solving a small local version of a chemical master equation of κ-steps.¹⁹

Our method also addresses the issue of proliferation of parameters and associated computational costs. Regardless of the number of reactions in the system and the complexity of the network, bias strengths for all reactions in ABSIS are adjusted using only two general parameters: λ₁ for promoting forward-moving reactions, and λ₂ for repressing backward-moving reactions. The biasing scheme is designed such that forward-moving reactions with lower reaction rates are encouraged, and backward-moving reactions with higher reaction rates are repressed. As κ is small, bias strengths can be determined without lengthy simulations.

We have applied the ABSIS method to four biological networks: the birth-death process, the reversible isomerization, the bistable Schlögl model, and the enzymatic futile cycle model. ABSIS can accurately and efficiently estimate rare event probabilities for all examples. For the birth-death process and Schlögl model, the rare event probabilities can be estimated by ABSIS with a variance of about 1/100 of that of the dwSSA method.²³ For the reversible isomerization model, sampling variance of ABSIS is only about 1/10, 000 of that of the dwSSA. ABSIS also shows significant improvements in standard deviations in comparisons to dwSSA. For the reversible isomerization, ABSIS estimates the rare event probability with only about 1/100 standard deviation of that of dwSSA. For the birth-death model and bistable Schlögl model, the standard deviation of ABSIS sampling is less than 1/10 than that of dwSSA. In terms of computing efficiency, smaller standard deviation indicates ABSIS can achieve the same accuracy as dwSSA with only a small fraction of sample size that dwSSA needs.

Although ABSIS has no significant advantages over constant biasing methods such as dwSSA in studying the futile cycle model, as the current bias scheme in ABSIS gives nearly constant biases, the sampling variances of ABSIS are comparable to those of dwSSA. Future work includes designing more sophisticated bias functions that captures the ruggedness of the probability landscape, which may provide better solutions to problems such as the model of enzymatic futile cycle. In addition, replacing enumeration of κ-steps of the reactions with longer term look-ahead path sampling of comparable computational costs may help to explore potential for barrier-crossing at a longer time scale. Computational costs can be further reduced for larger networks if such long look-ahead sampling is strategically biased. It may also be possible to classify reaction networks based on their topology and rate constants and design different sub-schemes of bias.