Analysing ill-conditioned Markov chains

Abstract

Discrete state Markov chains in discrete or continuous time are widely used to model phenomena in the social, physical and life sciences. In many cases, the model can feature a large state space, with extreme differences between the fastest and slowest transition timescales. Analysis of such ill-conditioned models is often intractable with finite precision linear algebra techniques. In this contribution, we propose a solution to this problem, namely partial graph transformation, to iteratively eliminate and renormalize states, producing a low-rank Markov chain from an ill-conditioned initial model. We show that the error induced by this procedure can be minimized by retaining both the renormalized nodes that represent metastable superbasins, and those through which reactive pathways concentrate, i.e. the dividing surface in the discrete state space. This procedure typically returns a much lower rank model, where trajectories can be efficiently generated with kinetic path sampling. We apply this approach to an ill-conditioned Markov chain for a model multi-community system, measuring the accuracy by direct comparison with trajectories and transition statistics.

This article is part of a discussion meeting issue ‘Supercomputing simulations of advanced materials’.

1. Introduction

Complex stochastic network models are important in a wide range of fields, including socio-economics [1], epidemiology [2–4], biochemistry [5–7], systems biology [8–10], biophysics [11–15] and condensed matter [16–18]. The networks can be represented as a graph, $\mathcal{G}(V,E)$, where the $V$ nodes constitute discrete microstates of the system, and the $E$ edges are parameterized by transition rates, in the case of continuous time Markov chains (CTMCs), or by transition probabilities, in the case of discrete time Markov chains (DTMCs). In both cases, the dynamics on the network are assumed to be memoryless. The mathematical analysis of such models reveals a rich and subtle interplay between network topology and dynamics [17–23].

We are often interested in studying the transition path ensemble (TPE), namely the set of pathways from a set of initial nodes $\mathcal{B}$ to a set of absorbing nodes $\mathcal{A}$ [7,24]. However, many networks exhibit a range of dynamical time scales and when this leads to a large separation of the characteristic timescales, numerical problems become pervasive in the analysis of the corresponding Markov chains [25–29]. Because the transition matrix is ill-conditioned in a metastable Markov chain, linear algebra operations, such as eigendecomposition, for the computation of the $\mathcal{A}\leftarrow \mathcal{B}$ first passage time (FPT) distribution, break down due to finite precision [30]. Moreover, sampling of $\mathcal{A}\leftarrow \mathcal{B}$ pathways via the standard kinetic Monte Carlo (kMC) algorithm becomes unfeasible due to the ‘flickering’ of trajectories within metastable groups of nodes, which act as kinetic traps [31–34]. Dimensionality reduction aims to alleviate these problems by focusing on the slowest dynamical processes between key macrostates of interest. The resulting coarse-grained model is also easier to interpret because details of the faster processes become implicit.

A common dimensionality reduction approach is to partition the Markovian network model into communities of nodes and then estimate the rates for inter-community transitions. However, analytical methods to compute the optimal coarse-grained rate matrix for a given community structure do not scale well [35]. Simpler approximations, including the local equilibrium approximation [36], may introduce significant errors in the inter-community dynamics [35].

A time-independent rate can strictly be defined only for an inter-community transition if the FPT dynamics are ‘ideally’ metastable [30]. Here, the communities must be chosen such that the distribution of escape times from each metastable macrostate follows a single-exponential decay. However, in realistic models, it may not be possible to find such an ideal community structure, and assuming Markovianity in the reduced state space may then introduce significant errors for the inter-community dynamics [37,38]. In particular, it is especially difficult to preserve higher moments of the FPT distribution beyond the mean first passage time (MFPT). A more faithful representation of inter-community transitions is provided by the full FPT distribution from an initial macrostate $\mathcal{B}$ (reactants) to an absorbing macrostate $\mathcal{A}$ (products). However, the required eigendecomposition breaks down because of finite precision due to the inherent metastability [30]. For systems with sufficiently large spectral gaps, it is also possible to develop a formalism based on perturbation theory to describe the slower dynamical time scales [39].

An alternative approach is to employ enhanced kMC methods that leverage knowledge of the metastable macrostates to simulate non-Markovian trajectories between communities. One class of methods is based on formulating the escape of a trajectory from a metastable basin as an absorbing Markov chain, as in the Monte Carlo with absorbing Markov chains (MCAMC) [40,41] method. The key idea behind these methods is to solve the master equation for the limited number of states in a trapping basin, so that a simulated trajectory escapes the basin in a single iteration [42]. However, if the trapping basins are too large, the overhead associated with solving for the exit probabilities and FPTs to particular absorbing microstates severely limits the efficiency of the method [32,43]. A related approach is kinetic path sampling (kPS) [33,43,44], which employs graph transformation [45] (GT), a node reduction algorithm that preserves mean FPTs, and an iterative reverse randomisation procedure, to sample a stochastic escape path to the absorbing boundary of a trapping basin along with an associated waiting time. As for MCAMC, the efficiency of kPS is severely limited by the number of nodes in each metastable basin [43]. Moreover, if the communities do not appropriately characterize the metastable macrostates, simulations may become unfeasibly slow due to trajectories flickering between the community boundaries.

A promising solution to the problem of overly large communities is to eliminate ‘fast’, i.e. rapidly evolving, states that are insignificant for global dynamics. States that contribute fast dynamical eigenmodes, corresponding to unproductive fluctuations within metastable basins, stiffen the Markov chain and hamper numerical analysis [46]. If these states are removed, the fast eigenmodes are no longer present in the reduced Markov chain, thus circumventing the flickering problem that precludes the use of the standard kMC algorithm. Moreover, if the transition rate matrix of the reduced Markov chain is less ill-conditioned, linear algebra operations, such as eigendecomposition, might retain sufficient precision to compute the full FPT distribution between two endpoint macrostates in the reduced model.

The elimination of states from a Markov chain, known as state reduction [47], has previously been employed to compute dynamical quantities such as the stationary distribution [48,49], fundamental matrix [50], and MFPTs [51,52]. Similar node-elimination schemes have been used to project the master equation onto a system of slow eigenmodes [46,53–55], which has inspired several stochastic simulation algorithms for sampling rare events in stiff Markov chains [54,56,57]. These methods use state reduction as a numerical tool for extracting properties from the original Markov chain.

In this contribution, we employ GT [45] as the state reduction technique, using a recent implementation (block GT) that enables the removal of multiple states [27,34,45,58–61]. Our focus is on numerically challenging Markov chains, with predefined community structure, which cannot be analysed with numerical linear algebra techniques due to loss of numerical precision. Our main result concerns the optimal selection of states so that the reduced Markov chain retains high accuracy, as measured by the generation of statewise trajectories. We find that accurate reduced Markov chains can be obtained if we retain, not only a node to represent the minimum free energy state of each metastable community, but also ‘boundary states’ that mediate inter-community connections, and thus have maximum participation in reactive pathways. We apply our approach to a numerically challenging multi-community system, directly comparing the trajectories and transition statistics. Reduction of the network produces a lower dimension Markov chain, which can be analysed by kPS in the metastable regime.

2. Discrete state Markov chains

We first briefly recall some relevant properties of discrete state Markov Chains. In a discrete state space $\Omega$, a Markov chain is completely described by a stochastic transition (branching) probability matrix $\mathbf{B}$, which in continuous time, has elements $B_{ij}=K_{ij}{\tau }_j$, where $K_{ij}$ is the transition rate from $j$ to $i$, and ${\tau }_j=1/\sum _{\gamma \ne j}{K}_{\gamma j}$ is the mean waiting time associated with node $j$. In matrix notation, $\mathbf{B}=\mathbf{K}{\mathbf{D}}^{-1}$, where $\mathbf{\text{D}}$ is a diagonal matrix with entries $D_{ij}={\delta }_{ij}/{\tau }_j$. The corresponding Markov transition matrix is $\mathbf{Q}=\mathbf{K}-\mathbf{D}$, with the occupation probability vector $\mathbf{P}(t)$ for the states in $\Omega$ evolving as

$$\frac{\mathrm{d}\mathbf{P}(t)}{\mathrm{d}t}=\mathbf{Q}\mathbf{P}(t).$$ 2.1

For ergodic systems (all states mutually accessible), $\mathbf{Q}$ has a unique equilibrium state with eigenvalue zero, right eigenvector $\mathit{\pi }$ with components corresponding to equilibrium occupation probabilities ${\pi }_i$, and left eigenvector $\mathbf{1}$, a row vector of ones, while all the other eigenvalues are negative. It is straightforward to show that this formulation satisfies conservation of the total occupation probability, $\mathbf{1}\cdot \mathrm{d}\mathbf{P}(t)/\mathrm{d}t=0$. The condition $\mathbf{Q}\mathit{\pi }=\mathbf{0}$ required for existence of a steady state is known as global balance, where $\mathit{\pi }$ is the vector of equilibrium occupation probabilities. In the present work, we assume that the transition rates satisfy the stronger detailed balance condition, $K_{ji}{\pi }_i=K_{ij}{\pi }_j$, which maintains equilibrium at the level of individual transitions [62].

(a) . First passage time distributions

The key observable for which we will compare and evaluate the renormalized Markov chains is the FPT distribution into target states $\mathcal{A}\in \Omega$ from source states $\mathcal{B}\in \Omega$. Consider partitioning the total space $\Omega$ into two regions $\Omega =\mathcal{A}\cup \mathcal{S}$, such that $\mathcal{B}\in \mathcal{S}$ and $\mathcal{A}\cap \mathcal{S}=0$, i.e. $\mathcal{A}$ and $\mathcal{S}$ do not overlap. Let ${\mathbf{Q}}_{\mathcal{S}}$ be the subset of the full transition matrix $\mathbf{Q}$ containing the inter-state transition rates within $\mathcal{S}$, and let ${\mathbf{D}}_{\mathcal{S}}$ be the corresponding subset of the diagonal matrix of total rates $\mathbf{D}$ including the escape rates to $\mathcal{A}$. As a result, the system evolving as $\mathrm{d}{\mathbf{P}}_{\mathcal{S}}(t)/\mathrm{d}t={\mathbf{Q}}_{\mathcal{S}}{\mathbf{P}}_{\mathcal{S}}(t)$ will eventually decay completely, ${\mathbf{P}}_{\mathcal{S}}(t)\to \mathbf{0}$, because all trajectories are eventually absorbed in $\mathcal{A}$ for a connected network. When the detailed balance condition is satisfied, the probability distribution (density function) for $\mathcal{A}\leftarrow \mathcal{B}$ FPTs can be written as the sum [30]

$$p_{\mathcal{A}\leftarrow \mathcal{B}}(t)=\sum _{\ell =1}^{|\mathcal{S}|}{\lambda }_{\ell ,\mathcal{S}}\hspace{0.17em}{\mathrm{e}}^{-{\lambda }_{\ell ,\mathcal{S}}t}{\mathbf{1}}_{\mathcal{S}}({\mathbf{w}}_{\ell ,\mathcal{S}}^R\otimes {\mathbf{w}}_{\ell ,\mathcal{S}}^L){\mathbf{P}}_{\mathcal{S}}(0),$$ 2.2

with the corresponding probability distribution for $y=\ln t$,

$${\mathcal{P}}_{\mathcal{A}\leftarrow \mathcal{B}}(y)=\sum _{\ell =1}^{|\mathcal{S}|}{\lambda }_{\ell ,\mathcal{S}}\hspace{0.17em}{\mathrm{e}}^{y-{\lambda }_{\ell ,\mathcal{S}}{e}^y}{\mathbf{1}}_{\mathcal{S}}({\mathbf{w}}_{\ell ,\mathcal{S}}^R\otimes {\mathbf{w}}_{\ell ,\mathcal{S}}^L){\mathbf{P}}_{\mathcal{S}}(0),$$ 2.3

where $\{-{\lambda }_{\ell ,\mathcal{S}}\}$ are the eigenvalues of ${\mathbf{Q}}_{\mathcal{S}}$, so that ${\lambda }_{\ell ,\mathcal{S}}>0$, and ${\mathbf{w}}_{\ell ,\mathcal{S}}^R$ and ${\mathbf{w}}_{\ell ,\mathcal{S}}^L$ are the corresponding right (column) and left (row) eigenvectors. The logarithmic time distribution clearly shows distinct peaks for competing pathways [30,33,63]. The probability vector ${\mathbf{P}}_{\mathcal{S}}(0)$ has dimension $|\mathcal{S}|$ but the elements are non-zero only for source states in $\mathcal{B}$. As $p_{\mathcal{A}\leftarrow \mathcal{B}}(t)$ can generate all moments of the FPT, it is an ideal observable for comparing the complete and reduced/renormalized Markov chains we treat in this paper. It will also be useful to define the fundamental matrix [64,65]

$${\mathbf{G}}_{\mathcal{S}}=-{\mathbf{D}}_{\mathcal{S}}{\mathbf{Q}}_{\mathcal{S}}^{-1}={[{\mathbb{I}}_{\mathcal{S}}-{\mathbf{B}}_{\mathcal{S}\mathcal{S}}]}^{-1},$$ 2.4

where ${\mathbb{I}}_{\mathcal{S}}$ is the identity matrix of dimension $|\mathcal{S}|$ and ${\mathbf{B}}_{\mathcal{S}\mathcal{S}}$ is the branching matrix between all pairs of states in $\mathcal{S}$. The elements ${[{\mathbf{G}}_{\mathcal{S}}]}_{s{s}^{\prime }}$ of the fundamental, or Green’s, matrix are equal to the expected number of visits to a state $s$ along a trajectory starting at $s^{\prime }$ and terminating anywhere at the absorbing boundary of $\mathcal{S}$ [62].

(b) . Metastability

This study concentrates on systems with highly metastable communities, where the expected escape time from a community is much greater than the time required to reach a local steady state. As the temperature falls, trapping basins become more stable, leading to greater metastability.

Metastability can be more precisely defined using the spectral approach introduced above. For a given set of nodes $\mathcal{X}$ let ${\mathbf{Q}}_{\mathcal{X}}$ be the elements of the full transition matrix $\mathbf{Q}$ that connect the states in $\mathcal{X}$, where the diagonal matrix of total rates ${\mathbf{D}}_{\mathcal{X}}$ includes all escape rates out of $\mathcal{X}$. Ordering the reversed sign eigenvalues as $0<{\lambda }_{0,\mathcal{X}}\le {\lambda }_{1,\mathcal{X}}\dots$ of ${\mathbf{Q}}_{\mathcal{X}}$, the degree of metastability is defined by the mixing time

$${\tau }_m=\frac{1}{({\lambda }_{1,\mathcal{X}}-{\lambda }_{0,\mathcal{X}})}>0.$$ 2.5

For ${\lambda }_{0,\mathcal{X}}{\tau }_m\ll 1$, an initial probability density confined to $\mathcal{X}$ will decay to the corresponding right eigenvector ${\mathbf{w}}_{0,\mathcal{X}}^R$ on a timescale ${\tau }_m\ll 1/{\lambda }_{0,\mathcal{X}}$, from which point the escape time distribution from $\mathcal{X}$ will be a simple exponential decay with an expected escape time of $1/{\lambda }_{0,\mathcal{X}}$.

The distribution ${\mathbf{w}}_{0,\mathcal{X}}^R$ is known as the quasi-stationary distribution [66]; in the limit ${\lambda }_{0,\mathcal{X}}{\tau }_m\to 0$, the quasi-stationary distribution tends to local equilibrium in $\mathcal{X}$, i.e. ${\mathbf{w}}_{0,\mathcal{X}}^R\to {\mathit{\pi }}_{\mathcal{X}}$, while the left eigenvector ${\mathbf{w}}_{0,\mathcal{X}}^L\to {\mathbf{1}}_{\mathcal{X}}$, i.e. a row vector of ones, as discussed in previous work [61].

A consequence of metastability is that any initial distribution in $\mathcal{X}$ that gives negligible probability leakage from $\mathcal{X}$ over times $t<{\tau }_m$ will produce essentially identical FPT distributions to other communities.¹ This relative insensitivity with respect to initial conditions will be invoked when justifying which nodes to remove in later sections.

3. Partial graph transformation

The GT algorithm removes nodes individually, renormalizing all retained nodes such that $\mathcal{A}\leftarrow \mathcal{B}$ branching probabilities and mean FPTs are unchanged [34,45,59]. This procedure was made numerically robust for arbitrarily ill-conditioned systems, by ensuring only floating point numbers of a similar magnitude are compared, making the procedure applicable for arbitrarily ill-conditioned systems [27]. The GT algorithm has recently [61] been extended for removal of multiple states simultaneously, making GT computations possible in fewer iterations [58,60,61]. When the block matrices containing all states to be removed in a single iteration are better conditioned than the entire system it may be possible to employ linear algebra methods to solve for the local dynamics. We use condition number estimations from LAPACK to revert to the numerically robust state-by-state GT algorithm when block matrices are too ill-conditioned, on-the-fly [61]. We have normally employed GT procedures to remove all states aside from sources and sinks, to compute mean FPTs. However, the FPT distribution is not available within this framework. Here, we introduce our partial GT algorithm, which retains all the boundary states between communities and the global community minima. We show that this approach can reduce the dimensionality of the network, while preserving the FPT distribution.

(a) . The partial GT procedure

We begin by partitioning the full state space $\Omega$ into a set of nodes to be eliminated $\mathcal{Z}$ and the set of nodes to be retained, ${\Omega }^{\mathcal{Z}}$, such that $\mathcal{Z}\cup {\Omega }^{\mathcal{Z}}\equiv \Omega$. The branching probability matrix can then be written

$$\mathbf{B}=\left[\begin{array}{cc}{\mathbf{B}}_{\mathcal{Z}\mathcal{Z}}& {\mathbf{B}}_{\mathcal{Z}{\Omega }^{\mathcal{Z}}}\\ {\mathbf{B}}_{{\Omega }^{\mathcal{Z}}\mathcal{Z}}& {\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}\end{array}\right].$$ 3.1

The renormalized branching probabilities and waiting times after removing $\mathcal{Z}$ with GT are given by Swinburne & Wales [61]

$${\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}={\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}+{\mathbf{B}}_{{\Omega }^{\mathcal{Z}}\mathcal{Z}}{\mathbf{G}}_{\mathcal{Z}}{\mathbf{B}}_{\mathcal{Z}{\Omega }^{\mathcal{Z}}}$$ 3.2

and

$${\mathit{\tau }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}={\mathit{\tau }}_{{\Omega }^{\mathcal{Z}}}+{\mathit{\tau }}_{\mathcal{Z}}{\mathbf{G}}_{\mathcal{Z}}{\mathbf{B}}_{\mathcal{Z}{\Omega }^{\mathcal{Z}}},$$ 3.3

where the superscript indicates the set of eliminated nodes, the subscript indicates the set of retained nodes, ${\mathbf{G}}_{\mathcal{Z}}={[{\mathbb{I}}_{\mathcal{Z}}-{\mathbf{B}}_{\mathcal{Z}\mathcal{Z}}]}^{-1}$ is Green’s matrix for $\mathcal{Z}$ and ${\mathit{\tau }}_{\mathcal{X}}={\mathbf{1}}_{\mathcal{X}}{\mathbf{\text{D}}}_{\mathcal{X}}^{-1}$ is a row vector of waiting times for nodes in region $\mathcal{X}$.

(b) . Properties of the renormalized Markov chain

The matrix ${\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}$ describes transitions in a reduced Markov chain associated with a transition rate matrix

$${\mathbf{\text{Q}}}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}=[{\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}-{\mathbb{I}}_{{\Omega }^{\mathcal{Z}}}]{\mathbf{\text{D}}}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}},$$ 3.4

where ${[{\mathbf{\text{D}}}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{ij}={\delta }_{ij}/{[{\mathit{\tau }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_i$. The matrix ${\mathbf{\text{Q}}}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}$ is less sparse than the original transition rate matrix $\mathbf{\text{Q}}$ because the GT procedure adds edges to the network to account for transitions via eliminated nodes (equation (3.2)).

(c) . Stationary distribution of a GT-renormalized Markov chain

For a steady state $\mathit{\pi }$, the global balance condition (existence of a steady state) requires that the total probability flux into each state is exactly balanced by the total probability flux leaving each state, leading to the equality

$$\mathbf{B}\mathbf{D}\mathit{\pi }=\mathbf{D}\mathit{\pi }.$$ 3.5

Global balance is more general than the detailed balance condition, which imposes a flux balance for each individual state-to-state transition, giving $B_{ij}{[\mathbf{D}\mathit{\pi }]}_j=B_{ji}{[\mathbf{D}\mathit{\pi }]}_i$, as discussed in §2.

Consider removing a single state $z$ through GT, giving a renormalized branching probability matrix and waiting times for the remaining states in ${\Omega }^{\mathcal{Z}}$

$${[{\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{ij}=B_{ij}+B_{iz}{B}_{zj}/(1-B_{zz})$$ 3.6

and

$${[{\mathit{\tau }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_i={\tau }_i+{\tau }_z{B}_{zi}/(1-B_{zz}).$$ 3.7

It is straightforward to show that the vector ${[\mathbf{f}]}_i={[\mathbf{D}\mathit{\pi }]}_i$, with $i\ne z$, satisfies the new global balance condition

$$\begin{array}{rl}\sum _{\omega \in {\Omega }^{\mathcal{Z}}}{[{\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{i\omega }{f}_{\omega }& \hspace{0.17em}={[\mathbf{B}\mathbf{D}\mathit{\pi }]}_i-B_{iz}{[\mathbf{D}\mathit{\pi }]}_z+\frac{B_{iz}}{1-B_{zz}}({[\mathbf{B}\mathbf{D}\mathit{\pi }]}_z-B_{zz}{[\mathbf{D}\mathit{\pi }]}_z)\\ & \hspace{0.17em}=f_i+\frac{B_{iz}}{1-B_{zz}}({[\mathbf{D}\mathit{\pi }]}_z-B_{zz}{[\mathbf{D}\mathit{\pi }]}_z-{[\mathbf{D}\mathit{\pi }]}_z+B_{zz}{[\mathbf{D}\mathit{\pi }]}_z)=f_i.\end{array}$$

We require the new steady-state distribution ${\mathit{\pi }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}$ to satisfy ${\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}{\mathbf{D}}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}{\mathit{\pi }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}={\mathbf{D}}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}{\mathit{\pi }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}$. Since ${\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}\mathbf{f}=\mathbf{f}$ it follows that ${[{\mathbf{D}}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}{\mathit{\pi }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_i\propto {[\mathbf{D}\mathit{\pi }]}_i$ for $i\in {\Omega }^{\mathcal{Z}}$. Normalization of $\mathit{\pi }$ over $\Omega$ and ${\mathit{\pi }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}$ over ${\Omega }^{\mathcal{Z}}$ shows that the constant of proportionality is unity. Hence

$$\frac{{[{\mathit{\pi }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_i}{{[{\mathit{\tau }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_i}=\frac{{\pi }_i}{{\tau }_i}\hspace{0.17em}\mathrm{or}\hspace{0.17em}\frac{{\pi }_i^{\mathcal{Z}}}{{\pi }_i}=\frac{{\tau }_i^{\mathcal{Z}}}{{\tau }_i}\hspace{0.17em}\mathrm{for}\hspace{0.17em}i\in {\Omega }^{\mathcal{Z}}\mathrm{with}{\pi }_i^{\mathcal{Z}}\equiv {[{\mathit{\pi }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_i,\hspace{0.17em}{\tau }_i^{\mathcal{Z}}\equiv {[{\mathit{\tau }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_i.$$ 3.8

As this result generalizes to the GT removal of multiple states, we see that for each individual state $i\in {\Omega }^{\mathcal{Z}}$ the renormalized steady-state occupation probabilities are not proportional to the original equilibrium occupation probabilities of the retained nodes, but reweights each retained state $i\in {\Omega }^{\mathcal{Z}}$ by the factor ${\tau }_i^{\mathcal{Z}}/{\tau }_i$ involving the renormalized and original waiting times.

In summary: if the original rate matrix satisfies detailed balance with distribution $\mathit{\pi }$, the renormalized rate matrix ${\mathbf{\text{Q}}}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}$ satisfies detailed balance with distribution ${\mathit{\pi }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}$.

4. Dimensionality reduction of networks with partial GT

In this section, we use the partial GT procedure outlined above to coarse-grain a general Markov chain with a predefined community structure, which maps any state $i\in \Omega$ to non-overlapping communities $\{\mathcal{A},\mathcal{B},\dots \}$ through some participation function

$$C(i)\in \{\mathcal{A},\mathcal{B},\dots \},i\in \Omega .$$ 4.1

Any coarse-graining procedure inevitably leads to information loss, incurring error. In the present case, this error can be precisely defined in certain limits, as GT exactly preserves the branching probabilities, $B_{ij}$, and MFPT between retained nodes. This result also applies to inter-community MFPTs, using (2.2),

$${\mathcal{T}}_{\mathcal{A}\mathcal{B}}\equiv {\int }_0^{\mathrm{\infty }}t\hspace{0.17em}{p}_{\mathcal{A}\leftarrow \mathcal{B}}(t)\hspace{0.17em}\mathrm{d}t,$$ 4.2

which additionally requires specification of some initial distribution for the first passage process. Providing the initial distribution in $\mathcal{B}$ is only over retained nodes, GT also preserves ${\mathcal{T}}_{\mathcal{A}\mathcal{B}}$ [61]. For metastable communities, a plausible choice is the local equilibrium distribution ${\mathit{\pi }}_{\mathcal{B}}$ of the initial community.

(a) . Boundary nodes

When considering which nodes to remove with GT, giving reduced communities $\{{\mathcal{A}}^{\mathcal{Z}},{\mathcal{B}}^{\mathcal{Z}},\dots \}$, it is important to note that ${\mathcal{T}}_{\mathcal{A}\mathcal{B}}$ measures the average time to reach the boundary nodes $\mathrm{\partial }\mathcal{A}$ of the target community $\mathcal{A}$. The boundary nodes are defined as the nodes that have direct connections to multiple communities. This observation allows us to write

$${\mathcal{T}}_{\mathcal{A}\mathcal{B}}\equiv {\mathcal{T}}_{\mathrm{\partial }\mathcal{A}\mathcal{B}},$$ 4.3

where the initial distribution in $\mathcal{B}$ is still unspecified. When building a reduced Markov model, we must consider $\mathcal{A}\leftarrow \mathcal{B}$ and $\mathcal{B}\leftarrow \mathcal{A}$ processes. To preserve MFPTs between reduced communities, and therefore maintain accuracy in the FPT distributions, we retain the boundary nodes of all communities, i.e. we require $\mathrm{\partial }\mathcal{X}=\mathrm{\partial }{\mathcal{X}}^{\mathcal{Z}},\mathcal{X}=\mathcal{A},\mathcal{B},\dots$.

(b) . Initial distributions under metastability

Section 2(b) discussed the metastable limit, where the mixing timescale ${\tau }_m$ in equation (2.5) of some community is much smaller than the characteristic escape time from that community. It is important to note that for systems with more than two communities, the escape and first passage times are quite distinct quantities. In the metastable limit, any initial distribution that has negligible probability of escape over times $t<{\tau }_m$ will have essentially identical first passage statistics.

A plausible choice of initial distribution for the first passage calculation (4.2) is the local equilibrium distribution ${\mathit{\pi }}_{\mathcal{B}}$, which for metastable communities will typically have negligible probability of escape over times $t<{\tau }_m$. However, the insensitivity to initial conditions allows us to instead take an initial distribution concentrated entirely on some interior community state, such as the one with the minimal free energy.

This choice motivates a reduced state space for a community $\mathcal{B}$ of

$${\mathcal{B}}^{\mathcal{Z}}\equiv \mathrm{\partial }\mathcal{B}\cup b_0,b_0=\arg \underset{b\in \mathcal{B}}{max}{\pi }_b,$$ 4.4

such that the boundary nodes remain, i.e. $\mathrm{\partial }{\mathcal{B}}^{\mathcal{Z}}=\mathrm{\partial }\mathcal{B}$, which ensures that the reduced state space also preserves the reverse MFPT, ${\mathcal{T}}_{\mathcal{A}\mathcal{B}}$.

By taking the same initial distribution, concentrated on $b_0$, for the first passage calculation (4.2) in the full and reduced network, we can ensure that the MFPT between communities is exactly preserved. This property can be written as ${\mathcal{T}}_{{\mathcal{A}}^{\mathcal{Z}}{\mathcal{B}}^{\mathcal{Z}}}={\mathcal{T}}_{{\mathcal{A}}^{\mathcal{Z}}{\mathcal{B}}^{\mathcal{Z}}}^{\mathcal{Z}}$, i.e. the MFPT from ${\mathcal{B}}^{\mathcal{Z}}$ to ${\mathcal{A}}^{\mathcal{Z}}$ is unchanged whether calculated for the full or GT reduced model.

Although we find that retaining boundary nodes gives an accurate and much lower rank model in the metastable limit, the model is not necessarily better conditioned since both fast and slow rates are retained. Nevertheless, the reduced rank significantly aids interpretation and renders kPS methods [33,43,44], which employ GT to overcome ill-conditioning problems, highly efficient.

5. Application to a model nine-community network

We now apply partial GT to reduce the number of nodes within each metastable community in a transition network, resulting in a coarse-grained model that can be more easily analysed with enhanced kMC methods, such as kPS, which scale poorly with community size.

We consider a model kinetic transition network with 994 nodes and 4320 bidirectional edges embedded in a two-dimensional potential energy surface with nine principal communities, illustrated in figure 1. The network is designed to mimic the energy landscape of a molecular system, with a Poissonian node degree distribution [68] and multi-pathway kinetics for transitions between competing communities [69]. The community participation is determined using the Bayesian agglomerative clustering engine (BACE) [70], as implemented in the PyEmma [71] software package. The time scale for inter-community MFPTs ranges from ${10}^3$ at temperature $T=10$, to ${10}^{12}$ at $T=1$. It is useful to compare these values with the typical mean waiting times of ${10}^0$ at $T=10$ and ${10}^2$ at $T=1$, and mixing times ${\tau }_m$, of around ${10}^1$ at $T=10$ and ${10}^3$ at $T=1$. Thus, inter-community transitions are ‘rare events’ at lower temperatures, although the entire model remains sufficiently well-conditioned at $T=1$ to be treatable with standard linear algebra routines, allowing us to generate essentially exact reference data.

Figure 1.

Dimensionality reduction pipeline for a model network (visualized using Gephi [67]) consisting of 994 nodes and 4320 bidirectional edges, parameterized by Arrhenius transition rates. (a) The network is embedded in a two-dimensional nine-well potential, which is clear in the disconnectivity graph shown in figure 2a. Larger darker blue nodes are associated with lower energies (higher equilibrium probabilities) and smaller darker red nodes are associated with higher energies (lower stationary probabilities). (b) First, the network is partitioned into nine metastable communities (communities of nodes), corresponding to the nine potential energy basins, using the Bayesian agglomerative clustering engine (BACE). (c) Partial GT is used to iteratively eliminate all but the lowest-energy node internal to each community, and the boundary nodes that connect communities, resulting in a reduced network with 215 nodes and 2217 edges. This reduction ensures all dominant basin escape paths from each community are preserved in a renormalized form. (Online version in colour.)

The organization of the landscape is clearly visible in the disconnectivity graph [72,73] shown in figure 2a. A further disconnectivity graph is generated by exploiting the concept of monotonic sequences, where the energy of the local minima decreases at every step [75–78]. If we include only the termini of such sequences then we restrict the disconnectivity graph to minima that have no directly connected minimum of lower energy [74]. The resulting graph highlights the monotonic sequence basins, as shown in figure 2b.

Figure 2.

Disconnectivity graphs [72,73] for the nine-community model network shown in figure 1a. (a) The landscape for the complete network with 994 local minima. (b) The landscape including only minima that correspond to monotonic sequence basins, defined by minima that are not directly connected to any lower energy neighbour [74–78]. The colour scheme highlights the different communities, as labelled in figure 1. (Online version in colour.)

Following the procedure detailed in §4, a reduced network was then produced with partial GT, at $T=1$ and $T=10$, retaining boundary nodes and the minimum free energy node for each community. The resulting renormalized network, visualized in figure 1c, contains only 215 nodes and 2217 edges, representing an 80% reduction in the state space. The 215-node network cannot be reduced further without eliminating the boundary nodes and therefore represents the maximum level of dimensionality reduction possible without incurring significant error in the inter-community MFPTs.

The renormalized equilibrium probabilities, ${\mathit{\pi }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}$, and branching probabilities, ${\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}$, can be used to construct a disconnectivity graph for the reduced network produced using the partial GT procedure (figure 3). We define effective free energies for the retained minima, $f_{\omega }(T)$, and the transition states that connect them, $f_{\omega {\omega }^{\prime }}^{†}(T)$, using

$$f_{\omega }(T)=-k_{B}T\ln ({[{\mathit{\pi }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{\omega })$$

and

$$\begin{array}{rl}{f}_{\omega {\omega }^{\prime }}^{†}(T)& \hspace{0.17em}=f_{{\omega }^{\prime }}(T)-k_{B}T\ln (\frac{{[{\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{\omega {\omega }^{\prime }}}{{[{\mathit{\tau }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{{\omega }^{\prime }}})+k_{B}T\ln \left(\frac{k_{B}T}{h}\right),\\ & \hspace{0.17em}=f_{\omega }(T)-k_{B}T\ln (\frac{{[{\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{{\omega }^{\prime }\omega }}{{[{\mathit{\tau }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{\omega }})+k_{B}T\ln \left(\frac{k_{B}T}{h}\right),\end{array}$$

where the Boltzmann and Planck constants, $k_B$ and $h$, are set to unity in the reduced unit system. The ratio ${[{\mathbf{B}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{{\omega }^{\prime }\omega }/{[{\mathit{\tau }}_{{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{\omega }$ corresponds to the rate constant ${[{\mathbf{K}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{{\omega }^{\prime }\omega }$ in the new network. Hence, the free energy of the transition state is defined to reproduce the rate as

$${[{\mathbf{K}}_{{\Omega }^{\mathcal{Z}}{\Omega }^{\mathcal{Z}}}^{\mathcal{Z}}]}_{{\omega }^{\prime }\omega }=\frac{k_{B}T}{h}\exp [-\frac{(f_{\omega {\omega }^{\prime }}^{†}(T)-f_{\omega }(T))}{k_{B}T}].$$ 5.1

Figure 3.

Disconnectivity graph [72,73] for the network in figure 1c, where 215 states are retained using the partial GT approach. This graph is based on effective free energies at $T=1$, as described in the text. (Online version in colour.)

(a) . First escape time distributions

First escape time distributions were calculated for each community, with the system initialized in different starting configurations, using eigendecomposition of the probability matrix, for the full network. We look at four different starting distributions for the full network: (a) Boltz: probability is distributed according to the local Boltzmann distribution within the starting community; (b) Min: probability is localized in the global minimum free energy state of the starting community; (c) Uni: probability is uniform over all states in the community; and (d) Mix: probability is initially uniform, then conditioned to remain within the starting community for a mixing time ${\tau }_m$.

Cases (a) and (b) are plausible initial distributions for first escape and first passage problems, while the uniform distributions (c) and (d) are designed to test the sensitivity to initial conditions, rather than representing any physical scenario.

The distribution in (d) can be produced using the eigenvalues and eigenvectors of the rate matrix for some starting community $\mathcal{X}$, as discussed in (b). The initially uniform density in $\mathcal{X}$ is first evolved to time ${\tau }_m$ via

$${\mathbf{p}}_{\mathcal{X}}({\tau }_m)=\sum _l\exp (-{\tau }_m{\lambda }_{l,\mathcal{X}}){\mathbf{w}}_{l,\mathcal{X}}^R{\mathbf{w}}_{l,\mathcal{X}}^L{\mathbf{p}}_{\mathcal{X}}(0).$$ 5.2

The resulting distribution is then normalized and used as the initial distribution for (2.2), giving a first escape time distribution shifted by ${\tau }_m$. Access to the full eigenspectrum of a local community requires stability of linear algebra routines, which limits the range of temperatures at which this artificial test distribution can be evaluated.

The resulting distributions for $T=1$ and $T=10$, for the full network (before partial GT), are shown in figures 4 and 5. As discussed in the previous section, any normalized starting distribution in the metastable limit will quickly decay to the quasi-stationary distribution, which itself will be close to local equilibrium ${\mathit{\pi }}_{\mathcal{B}}$ or ${\mathit{\pi }}_{\mathcal{B}}^{\mathcal{Z}}$. This result implies that the escape statistics will be essentially invariant to changes in the initial distribution for times longer than some short mixing time.

Figure 4.

First escape time distributions for the nine-community model, before partial GT at $T=1$. The system is initialized in four different starting configurations, Boltz, Min, Uni and Mix, within each community, as labelled in the top right corner of each plot. An additional small peak is seen at small time for Uni, due to significant starting probability in boundary nodes causing rapid escape. At this low temperature, most of the probability in the initial Boltzmann distribution is localized in or near the global minimum of the community, which results in almost identical escape distributions for Boltz and Min. For longer times, Mix tends towards the Boltzmann distribution. (Online version in colour.)

Figure 5.

First escape time distributions for the nine-community model, before partial GT at $T=10$. The system is initialized in four different starting configurations, Boltz, Min, Uni and Mix, within each community, as labelled in the top right corner of each plot. Both the Boltz and Uni escape distributions have larger escape probabilities at small time compared with Min, due to significant starting probabilities in boundary nodes. Mix shows a peak centred on the Min peak, but of different height, as both curves are normalized to have unit area. (Online version in colour.)

At low temperature, we see that (a), (b) and (d) all give very similar first passage distributions. For the local Boltzmann distribution at $T=1$, the majority of the probability (25–99%) is localized in the lowest minimum of the starting community, while at $T=10$ this probability drops to around $3\mathrm{\%}$. For $T=1$, we see that the (a) and (b) escape distributions are indistinguishable.

Case (c) causes some initial probability to quickly leak to neighbouring communities due to significant starting probabilities on boundary nodes. At $T=1$, this effect causes a distinct peak at short time, while at $T=10$, the short time and long time peaks overlap, producing one strongly asymmetrical peak, with larger probability at short time. However, when we restrict to a uniform initial configuration, conditional on not leaving the starting community on timescale $t<{\tau }_m$, i.e. case (d), the resulting escape distribution moves towards the local Boltzmann distribution, illustrating the limit on invariance to initial conditions.

(b) . First passage time distributions

FPT distributions were also calculated for every community pairing for the full network, before partial GT, with the system initialized in the same four starting configurations: (a) Boltz, (b) Min, (c) Uni and (d) Mix.

A representative selection of FPT distributions calculated using eigendecomposition is shown in figure 6. The distribution shape is not the single peak expected for a two state network, because reactive trajectories spend significant amounts of time in communities other than the reactant and product. These inter-community dynamics are controlled by the boundary nodes, which establishes the importance of these states in the retained network.

Figure 6.

Representative first passage time (FPT) distributions for the full nine-community model before partial-GT at (a) $T=1$ and (b) $T=10$. The system is initialized in four different starting configurations, within each community: Boltz, Min, Uni and Mix. The FPT results are rather similar for the different initial conditions. Starting in a uniform distribution can produce additional peaks in the FPT distribution, but these peaks become less prominent for Mix. (Online version in colour.)

As was the case for the escape time distributions, the FPT distributions are very similar for different initial conditions. When the system is initialized in a uniform distribution, the FPT distribution can include distinct additional lower time peaks at $T=1$, as seen for the $2\leftarrow 1$ transition. Additional peaks occur at very short times when the source and sink communities are neighbours. Even when communities are not neighbours, additional peaks can be seen if significant probability quickly transfers to another community (i.e. on a time scale less than the mixing time), as the resulting FPT distribution will also pick up the FPT from this community to the sink. This additional peak may be at a longer or shorter time than the main peak. Once again, initial condition (d) tends towards initial condition (a), confirming the insensitivity to initial distributions.

(c) . Validation of the partial GT reduced network

To benchmark the effectiveness of partial GT reduction, we compare first escape time distributions, FPT distributions and simulation trajectories between the full and GT-reduced networks.

Representative first escape and passage time distributions comparing the full and partial GT reduced network are shown in figures 7–10. At $T=1$, the reduced and full networks have identical distributions when initialized in Boltz or Min, showing that partial GT can preserve higher order moments of the FPT. Starting from a uniform distribution within the reduced network more strongly perturbs the distributions compared with the full network, due to the significant proportion of probability in the boundary nodes. However, if the system is initialized such that no probability leaves on a time scale less than the mixing time, the full and reduced distributions match quite well. Since the system becomes less metastable as the temperature increases, the matching between full and reduced networks decreases. At these higher temperatures, the Boltzmann distribution becomes more uniform, which results in better agreement across all initial conditions. Overall, the accuracy of the GT reduced network is clear. Only at the higher temperature $T=10$, where metastability is reduced, is some appreciable disagreement seen between full and GT reduced, for initial conditions, Boltz and Min.

Figure 8.

First escape time distributions for transitions between different communities in the nine-community model at $T=10$ for both the full and graph transformed (GT) networks, calculated using eigendecomposition. The system is initialized in different local distributions, (a) Boltz and Min, (b) Uni and Mix, within the starting communities, which are labelled in the top right corner of each panel. (Online version in colour.)

Figure 9.

First passage time (FPT) distributions for transitions between different communities in the nine-community model at $T=1$ for both the full and graph transformed (GT) networks, calculated using eigendecomposition. The system is initialized in different local distributions within the starting communities: (a) Boltz and Min, (b) Uni and Mix. (Online version in colour.)

Figure 7.

First escape time distributions for transitions between different communities in the nine-community model at $T=1$ for both the full and graph transformed (GT) networks, calculated using eigendecomposition. The system is initialized in different local distributions, (a) Boltz and Min, (b) Uni and Mix, within the starting communities, which are labelled in the top right corner of each panel. (Online version in colour.)

Figure 10.

First passage time distributions for transitions between different communities in the nine-community model at $T=10$ for both the full and graph transformed (GT) networks, calculated using eigendecomposition. The system is initialized in different local distributions within the starting communities: (a) Boltz and Min, (b) Uni and Mix. (Online version in colour.)

As a final benchmark, we simulated 1000 non-equilibrium trajectories using kPS, initialized according to the local Boltzmann distribution of each of the nine communities of the original network, for a fixed trajectory time, at $T=1$. We also simulated another set of trajectories, for the same elapsed time, on the maximally GT reduced 215-node network. Three repeats were run for each system.

Figure 11 shows the time-dependent community occupation probabilities obtained from three sets of 1000 trajectories initialized in community 4 of the network in figure 1b. These community occupation probabilities closely follow those derived from the 1000 trajectories initialized in community 4 of the reduced network in figure 1c. Thus, trajectories simulated on GT-reduced networks successfully reproduce the dynamical properties of the original model.

Figure 11.

Community occupation probability, defined as the fraction of simulated trajectories assigned to a given community, as a function of simulation time. A set of 1000 trajectories initialized according to the local equilibrium distribution in community 4 of the nine-community model network (as shown in figure 1b) were simulated using kPS at a temperature $T=1$. The time-dependent occupation probabilities derived from trajectories on the original network closely match those derived from simulated trajectories on the GT-reduced 215-node network (figure 1c), which only retains the boundary nodes and the internal node with the largest stationary probability in each community. Since community 4 is metastable, the evolution of its occupation probability closely matches a simple exponential decay. Over time, the probability flow leaks into neighbouring communities, as shown by the gradual increase in the occupation probabilities of the neighbouring communities 3 and 1. The dynamics are accurately represented even at longer time scales, i.e. at 10 times the mean escape time from community 4. (Online version in colour.)

Although partial GT significantly reduces the number of nodes in each community, it also decreases the sparsity of the network, which can hinder the efficiency of kPS [43]. Nonetheless, the CPU time required to simulate 9000 trajectories on the reduced network represents a significant speedup compared with the CPU time required for simulations on the original network (table 1).

Table 1.

Wall clock time (ran on all 16 processors of an eight dual-core Intel(R) Core(TM) i7-11700 2.50GHz) for simulating nine sets of 1000 trajectories initialized from each of the nine communities of the original and reduced versions of the model network in figure 1.

wall clock time in original network	wall clock time in reduced network
729 min	21 min

The trade-off between dimensionality and sparsity does not affect the time complexity of MCAMC, which relies on eigendecomposition of the transition matrix for each community subnetwork to compute basin escape statistics [40,41]. Therefore, dimensionality reduction with partial GT should provide an even greater simulation speedup for MCAMC.

(d) . First passage time distributions in the metastable regime

Although inter-community FPT distributions are well preserved in GT-reduced networks, calculation of these distributions using linear algebra routines fails for ill-conditioned problems. Although we can always extract MFPTs through application of GT, trajectory and FPT distribution information provides more insight into non-equilibrium dynamics [43]. These distributions can be obtained efficiently using enhanced kMC methods that rely on a partitioning of state space, such as kPS [33,43,44] and MCAMC [40,41].

In practice, the cubic scaling of MCAMC and kPS methods with community size presents problems for application to realistic systems [33,40,43]. This issue motivates the use of partial GT to subsume fast internal nodes, to improve scalability without introducing significant errors.

First, we compare FPT distributions computed using kPS and linear algebra techniques, in figure 12, which shows excellent agreement. Just below $T=0.8$ and $T=0.7$, for the full and reduced system, respectively, the system becomes too ill-conditioned for eigendecomposition to succeed, and we must reply on methods which are immune to high metastability, such as kPS. At the lower temperature of $T=0.5$, kPS on the full network is unfeasible due to a large computational time. However, on the reduced network, kPS remains viable and FPTs can be computed, as in figure 13.

Figure 12.

First passage time (FPT) distributions for transitions between different communities in the nine-community model at $T=1$. Distributions were computed using linear algebra eigendecomposition methods (LA) and kinetic path sampling (kPS) on the partial-GT reduced network. Excellent agreement is obtained between the two methods, for the different initial starting distributions of (a) stationary, (b) uniform across all states. (Online version in colour.)

Figure 13.

Low temperature first passage time (FPT) distributions for transitions between different communities in the nine-community model at $T=0.5$, computed using kinetic path sampling (kPS) on the reduced network. At this temperature, eigendecomposition is unable to produce the FPTs due to the increase of metastability causing loss of precision. However, kinetic path sampling remains functional, and due to the lower dimension of the reduced partial-GT network, is computationally feasible. The system is initialized according to the local equilibrium distribution within the starting community. At this low temperature, the efficiency of kPS varies between pairings, with some of the more difficult pairings being the passage times to community 2. (Online version in colour.)

6. Conclusion

In this paper, we have investigated computationally stable and efficient techniques to study the flow of probability in numerically challenging Markov chains, where standard linear algebra techniques fail due to loss of numerical precision. Our main result is that the state reduction technique GT [45], in particular, a recent extension that enables the removal of multiple states [27,34,45,58–61], can produce a reduced network of much smaller size, while retaining highly accurate kinetics compared with the full network, as measured through FPT distributions. The optimal node removal strategy retains only the boundary states connecting multiple communities and the minimum free energy state within each community. This approach can be justified theoretically by considering the kinetics of first passage processes and their relative insensitivity to initial conditions in the metastable regime.

While the reduced network is much smaller, the retention of boundary states, essential for kinetic accuracy, means ill-conditioning issues still remain. However, the small network size renders numerically stable kPS techniques more efficient, providing access to the full FPT distribution.

The FPT distribution gives a much richer insight into system kinetics than MFPT calculations [63], which obscure the presence of multiple distinct relaxation time scales. The present results highlight the importance of such effects for complex, multi-community systems, which likely play a key role in multi-functional systems with multi-funnel energy landscapes [11,63]. The present contribution shows that these dynamical signatures can be captured even for numerically challenging systems, which cannot be treated with standard approaches.

Data accessibility

The graph transformation and dimensionality reduction analysis used the freely available PyGT python package [79], with additional calculations using PATHSAMPLE [80], while the kinetic path sampling simulations presented were performed with DISCOTRESS simulation Suite [81]. The nine-community dataset used in this study is freely available online [82].

Authors' contributions

E.J.W.: investigation, writing—original draft, writing—review and editing; D.K.: conceptualization, formal analysis, investigation, methodology, software, writing—original draft; D.J.S.: formal analysis, investigation, methodology, software, writing—review and editing; T.D.S.: conceptualization, formal analysis, investigation, methodology, software, validation, writing—original draft, writing—review and editing; D.J.W.: investigation, methodology, project administration, resources, software, supervision, writing—review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

No funding has been received for this article.