Array-representation Integration Factor Method for High-dimensional Systems

Abstract

High order spatial derivatives and stiff reactions often introduce severe temporal stability constraints on the time step in numerical methods. Implicit integration method (IIF) method, which treats diffusion exactly and reaction implicitly, provides excellent stability properties with good efficiency by decoupling the treatment of reactions and diffusions. One major challenge for IIF is storage and calculation of the potential dense exponential matrices of the sparse discretization matrices resulted from the linear differential operators. Motivated by a compact representation for IIF (cIIF) for Laplacian operators in two and three dimensions, we introduce an array-representation technique for efficient handling of exponential matrices from a general linear differential operator that may include cross-derivatives and non-constant diffusion coefficients. In this approach, exponentials are only needed for matrices of small size that depend only on the order of derivatives and number of discretization points, independent of the size of spatial dimensions. This method is particularly advantageous for high dimensional systems, and it can be easily incorporated with IIF to preserve the excellent stability of IIF. Implementation and direct simulations of the array-representation compact IIF (AcIIF) on systems, such as Fokker-Planck equations in three and four dimensions and chemical master equations, in addition to reaction-diffusion equations, show efficiency, accuracy, and robustness of the new method. Such array-presentation based on methods may have broad applications for simulating other complex systems involving high-dimensional data.

1. Introduction

Consider a general reaction-diffusion equation of the form

$${\partial }_{t}u(\mathbf{x},t)=\sum _{i,j=1}^n{\partial }_{x_i}(D_{ij}(u,\mathbf{x}){\partial }_{x_j}u(\mathbf{x},t))+F(u,\mathbf{x}),$$ (1)

where n is the spatial dimension and x = {x₁, …, x_n}. A non-linear term F(u, x) is often interpreted as a reaction term. Coefficients D_ij can be either constants or functions of u and x. The function u(x, t) usually represents concentrations of physical or biological species with reactions among them, for which one usually has n = 2 or n = 3. However, u(x, t) can also be considered as a probability density functions for stochastic dynamics described by Fokker-Planck equations [1], for which the dimension n represents the number of biochemical species.

Spatial discretizaton for differential equations of higher spatial dimensions (even for n = 3) often requires large, sometimes prohibitive, data storage and management as well as expensive CPU time at a fixed time point. In addition, temporal discretization, which strongly depends on the stiffness of reactions and treatment of the high order derivatives (e.g. the diffusion term), may lead to severe stability conditions that require very small time steps, resulting in excessive computational cost.

Integration factor (IF) or exponential time differencing (ETD) methods are effective approaches to deal with temporal stability constraints associated with high order derivatives [2, 3, 4]. By treating linear operators of the highest order derivative exactly, IF or ETD methods are able to achieve excellent temporal stability [2, 5, 6]. To deal with additional stability constraints from stiff reactions, a class of semi-implicit integration factor (IIF) methods [7] were developed for implicit treatment of the stiff reactions. In the IIF approach, the diffusion term is solved exactly like the IF method while the nonlinear equations resulted from the implicit treatment of reactions is decoupled from the diffusion term to avoid solving large nonlinear systems involving both diffusions and reactions, such as in a standard implicit method for reaction-diffusion equations. IIF methods have a great stability property with its second order scheme being linearly unconditionally stable.

In IF or ETD type of methods, the dominant computational cost arises from the storage and calculation of exponentials of matrices resulting from discretization of the linear differential operators in the PDEs. To deal with this difficulty, compact representation of the discretization matrices was introduced in the context of IIF method [8]. In compact implicit integration factor method (cIIF), the discretized solutions are represented in a matrix form rather than a vector while the discretized diffusion operator are represented in matrices of much smaller size than the standard matrices for IIF while preserving the stability property of the IIF. For two or three dimensions, cIIF is significantly more efficient in both storage and CPU cost. In addition, cIIF method is robust in its implementation and integration with other spatial and temporal algorithms. It can handle general curvilinear coordinates as well as combine with adaptive mesh refinements in a straightforward fashion [9]. One can also apply cIIF to stiff reactions and diffusions while using other specialized hyperbolic solvers (e.g WENO methods [10, 11]) for convection terms to solve reaction-diffusion-convection equations efficiently [12]).

One alternative approach for IF (or ETD) methods to avoid storage of the exponentials of large matrices is to use Krylov subspace method to compute the multiplication between the vector and the exponentials of matrices without explicitly forming the matrices [13, 14]. The advantage of applying Krylov subspace method is that it can handle complicated diffusion operators, e.g. diffusion coefficients are spatial functions or elliptic operators contains cross derivatives, while cIIF in previous studies [8] can only handle systems of constant diffusion coefficients and Laplacian operators restricted to two and three dimensions. In contrast to cIIF, in which exponentials of matrices are pre-calculated only once and stored for repeated usages at each time step of the temporal updating, the Krylov subspace method needs to be carried out at each time step, leading to a significant increase in CPU time.

In this paper, we introduce an array representation for the linear differential operators that may contain non-constant diffusion coefficients as well as cross-derivatives in two, three or higher dimensions. This array-representation approach is based on the idea of compact Implicit Integration Factor (cIIF), that is, when discretizing the terms with partial derivatives, regard the unknown solution as a vector with index connected to corresponding variables, while keeping other indexes fixed with unrelated variables. This new approach yields several discretization matrices of a small size that depend only on the number of derivatives in the continuous operators and the number of spatial discretization points in the direction of each derivative, in contrast to IF (or ETD) that requires exponentials of matrices whose size depend on the number of dimensions. In particular, the array representation can be incorporated into IIF to maintain the nice stability property of IIF as well as the implicit local treatment of the reactions decoupled from the diffusions. Like IIF, the second order array-representation (compact) implicit integration method (AcIIF) is A-stable. An operator splitting technique is incorporated into AcIIF for certain differential operators, resulting in non-commutable operations between discretization matrices. The AcIIF method is an extension of cIIF method that is able to deal with cross derivatives and non-constant diffusion coefficients in addition to other applications.

To study the accuracy and efficiency of AcIIF, we implement AcIIF methods and compare it with several other existing methods for both two and three dimensional reaction-diffusion equations. In addition, we apply AcIIF to solve Fokker-Planck equations in three and four dimensions. To demonstrate other applications of the array representation, we also use this approach to directly solve chemical master equations. In CMEs, the structure of the rate matrix for a reaction containing k species of molecules is very similar to the discretization matrix for a k-th order partial differential equation with cross derivatives. The overall direct simulations show the excellent properties of AcIIF and its distinct advantage in high spatial dimensions.

2. Array-representation (compact) Implicit Integration Factor Method (AcIIF)

2.1. Array representation for reaction-diffusion systems in three dimension without cross derivatives

To illustrate the array-representation approach, we first consider three-dimensional reaction-diffusion equations without cross-derivatives and with constant diffusion coefficients and periodic boundary conditions:

$$u_t(x,y,z,t)=D\mathrm{\Delta }u(x,y,z,t)+f(u(x,y,z,t)),$$ (2)

where (x, y, z) ∈ Ω = {0 < x, y, z < l}. Let N_x, N_y, N_z be the number of spatial grid points in each spatial direction and h_x, h_y, h_z be the grid size, respectively. Denote U_k1,k2,k3 as the approximated solution of u at the grid point (k₁h_x, k₂h_y, k₃h_z). The approximation of D∂²/∂x² using the second order central difference discretization can be written in terms of multi-dimensional arrays, U = (U_k1,k2,k3), through a linear mapping ,

$${({ℒ}_{x}U)}_{k_1,k_2,k_3}:=\frac{D}{h_x^2}(U_{k_1+1,k_2,k_3}-2U_{k_1,k_2,k_3}+U_{k_1-1,k_2,k_3})$$ (3)

where 1 ≤ k₁ ≤ N_x, 1 ≤ k₂ ≤ N_y, and 1 ≤ k₃ ≤ N_z. Similarly, using and to represent the approximations D∂²/∂y² and D∂²/∂z², respectively, Eq. (2) is approximated by

$$\frac{dU}{dt}={ℒ}_{x}U+{ℒ}_{y}U+{ℒ}_{z}U+f(U).$$ (4)

Multiplying the integration factor, e^{( + + )t}, to both sides and integrating from t_n to t_n+1, two adjacent discretized temporal points, we derive a class of semi-implicit integration factor methods (IIF) after approximating the integral [7]. For example, the second order IIF takes the form

$$U^{n+1}-\frac{\mathrm{\Delta }t}{2}f(U^{n+1})=e^{({ℒ}_x+{ℒ}_y+{ℒ}_z)\mathrm{\Delta }t}(U^n+\frac{\mathrm{\Delta }t}{2}f(U^n))$$ (5)

where Uⁿ ≈ U at time point t_n.

In a typical representation of the linear differential operator, the matrix ( + + )Δt has a size of N_xN_yN_z × N_xN_yN_z. Although the matrix itself is sparse, its exponential is usually not, leading to prohibitive storage and computing cost for any fine spatial meshes. Next, we decompose this matrix into small matrices based on an array representation.

If one defines a vector by fixing the last two indices, k₂, k₃, of the the three-dimensional array U,

$$U(:,k_2,k_3)={(U_{1,k_2,k_3},U_{2,k_2,k_3},\dots ,U_{N_x,k_2,k_3})}^T,$$ (6)

Then the three dimensional array U, can be treated as the collection of all such one dimensional vector on a two-dimensional array, with all k₂, k₃ going through from 1 to N_y and from 1 to N_z, respectively. We present this collection using symbol ⊗, with the super index indicates that this collection is along x_i axis, then we have:

$$U=\underset{\begin{array}{l}1\le k_2\le N_y\\ 1\le k_3\le N_z\end{array}}{\otimes }U(:,k_2,k_3).$$ (7)

Next, we define a N_x × N_x matrix $M_x=D/h_x^2{W}_{N_x\times N_x}$, where

$$W_{N\times N}={\left(\begin{array}{ccccc}-2& 1& 0& \dots & 1\\ 1& -2& 1& \dots & 0\\ 0& 1& -2& 1& \dots \\ \dots & \dots & \dots & \dots & \dots \\ 1& 0& \dots & 1& -2\end{array}\right)}_{N\times N}.$$ (8)

Then, M_xU(:, k₂, k₃) represents the vector and matrix multiplication for any fixed pair of k₂, k₃. Using this approach, the linear mapping in the array representation becomes,

$${ℒ}_{x}U=\underset{\begin{array}{l}1\le k_2\le N_y\\ 1\le k_3\le N_z\end{array}}{\otimes }{M}_{x}U(:,k_2,k_3).$$ (9)

Consequently, the exponential of in the array representation takes the following form,

$$e^{{ℒ}_x}U=\underset{\begin{array}{l}1\le k_2\le N_y\\ 1\le k_3\le N_z\end{array}}{\otimes }{e}^{M_x}U(:,k_2,k_3),$$ (10)

as induced from the relation,

$${({ℒ}_x)}^{m}U=\underset{\begin{array}{l}1\le k_2\le N_y\\ 1\le k_3\le N_z\end{array}}{\otimes }{(M_x)}^{m}U(:,k_2,k_3),\forall m\in {\mathbb{N}}^{+}.$$ (11)

Applying the definition of linear mapping exponential yields Eq. (10).

Clearly, and have similar array representations,

$${ℒ}_{y}U=\underset{\begin{array}{l}1\le k_1\le N_x\\ 1\le k_3\le N_z\end{array}}{\otimes }{M}_{y}U(k_1,:,k_3),{ℒ}_{z}U=\underset{\begin{array}{l}1\le k_1\le N_x\\ 1\le k_2\le N_y\end{array}}{\otimes }{M}_{z}U(k_1,k_2,:),$$ (12)

where $M_y=D/h_y^2{W}_{N_y\times N_y}$ and $M_z=D/h_z^2{W}_{N_z\times N_z}$.

Using the array representations, one can easily show that the three linear mappings , and commute with each other, i.e.,

$${ℒ}_a{ℒ}_{b}U={ℒ}_b{ℒ}_{a}U,\text{for}a,b\in \{x,y,z\}.$$ (13)

This commuting property results in

$$e^{{\mathcal{L}}_x+{\mathcal{L}}_y+{\mathcal{L}}_z}U=e^{{\mathcal{L}}_x}{e}^{{\mathcal{L}}_y}{e}^{{\mathcal{L}}_z}U.$$ (14)

Direct application of Eq. (14) to Eq. (5) results in the following second order array-representation Implicit Integration Factor (AcIIF) method:

Algorithm 1. Second order AcIIF (AcIIF2)

$$U^{n+1}-\frac{\mathrm{\Delta }t}{2}f(U^{n+1})=\underset{\begin{array}{l}1k_2\le N_y\\ 1\le k_3\le N_z\end{array}}{\otimes }{e}^{M_x\mathrm{\Delta }t}\left(\underset{\begin{array}{l}1\le k_1\le N_x\\ 1\le k_3\le N_z\end{array}}{\otimes }{e}^{M_y\mathrm{\Delta }t}\left(\underset{\begin{array}{l}1\le k_1\le N_x\\ 1\le k_2\le N_y\end{array}}{\otimes }{e}^{M_z\mathrm{\Delta }t}V(k_1,k_2,:)\right)(k_1,:,k_3)\right)(:,k_2,k_3),$$ (15)

where V = Uⁿ + Δt/2f(Uⁿ).

Previously, a compact IIF (cIIF) was derived in a different fashion in two spatial dimensional systems by treating unknowns as a matrix, then the action of is like a left product to the matrix and is as its right product. And in three spatial dimensional cases, in addition to the left and right multiplications, a middle multiplication represents [8]. One major advantage of both cIIF and AcIIF methods that one only needs to compute the exponentials of M_x, M_y and M_z, which are much smaller matrices (only about N × N), in comparison to standard IF or ETD methods [15, 16], for which the exponential e^{( + + )Δt} of dimension of N_xN_yN_z × N_xN_yN_z are needed. Clearly, cIIF and AcIIF have significant savings in both CPU cost and storage, in particular, for equations in three or higher dimensions.

For the systems without cross-derivatives (e.g. Eq. (15)), the second order AcIIF (2) is equivalent to the second order cIIF method [8]. As it will be shown next, the advantage of AcIIF lies in its potential applications to reaction-diffusion systems with cross derivatives and non-constant diffusion coefficients for which cIIF is unable to achieve.

2.2. AcIIF method for three-dimensional reaction-diffusion systems with cross derivatives

Consider the reaction-diffusion equations with second order cross derivatives:

$$u_t=\left(a_1\frac{{\partial }^2}{\partial x^2}+2b_1\frac{{\partial }^2}{\partial x\partial y}+c_1\frac{{\partial }^2}{\partial y^2}\right)u+\left(a_2\frac{{\partial }^2}{\partial x^2}+2b_2\frac{{\partial }^2}{\partial x\partial z}+c_2\frac{{\partial }^2}{\partial z^2}\right)u+\left(a_3\frac{{\partial }^2}{\partial y^2}+2b_3\frac{{\partial }^2}{\partial y\partial z}+c_3\frac{{\partial }^2}{\partial z^2}\right)u+f(u),$$ (16)

in a cube, {(x, y, z) : 0 < x, y, z < l}, with periodic boundary conditions, satisfying the conditions $a_i{c}_i>b_i^2$, for i = 1, 2, 3. Applying a standard second order central difference approximation to $a_1\frac{{\partial }^2}{\partial x^2}+2b_1\frac{{\partial }^2}{\partial x\partial y}+c_1\frac{{\partial }^2}{\partial y^2}$, one obtains its approximation, denoted by , as the following

$$\begin{array}{l}{({\mathcal{L}}_{xy}U)}_{k_1,k_2,k_3}=\frac{a_1}{h_x^2}(U_{k_1+1,k_2,k_3}-2U_{k_1,k_2,k_3}+U_{k_1-1,k_2,k_3})\\ \frac{2b_1}{4h_x{h}_y}(U_{k_1+1,k_2+1,k_3}+U_{k_1-1,k_2-1,k_3}-U_{k_1+1,k_2-1,k_3}-U_{k_1-1,k_2+1,k_3})\\ +\frac{c_1}{h_y^2}(U_{k_1,k_2+1,k_3}-2U_{k_1,k_2,k_3}+U_{k_1,k_2-1,k_3}).\end{array}$$ (17)

Using similar definitions for and , the spatial approximation of Eq. (16) becomes

$$\frac{dU}{dt}=({\mathcal{L}}_{xz}+{\mathcal{L}}_{xy}+{\mathcal{L}}_{yz})U+f(U).$$ (18)

To derive the array representation of the operator , we first fix k₃ in U(:, :, k₃) that represents a N_xN_y × N_xN_y matrix. Collect all these two dimensional matrices along a vector leads to:

$$U=\underset{1\le k_3\le N_z}{\otimes }U(:,:,k_3).$$ (19)

Define a linear mapping, , from a matrix space consisting of all N_x × N_y matrices to itself as follows:

$${({\mathcal{A}}_{xy}M)}_{i,j}=\frac{2b_1}{4h_x{h}_y}(M_{i+1,j+1}+M_{i-1,j-1}-M_{i-1,j+1}-M_{i+1,j-1})+\frac{a_1}{h_x^2}(M_{i+1,j}-2M_{i,j}+M_{i-1,j})+\frac{c_1}{h_y^2}(M_{i,j+1}-2M_{i,j}+M_{i,j-1}).$$ (20)

Then, the array representation of , in terms of , and its exponential become

$${\mathcal{L}}_{xy}U=\underset{1\le k_3\le N_z}{\otimes }{\mathcal{A}}_{xy}U(:,:,k_3),e^{{\mathcal{L}}_{xy}}U=\underset{1\le k_3\le N_z}{\otimes }{e}^{{\mathcal{A}}_{xy}}U(:,:,k_3).$$ (21)

Similarly, the array representation for and may be written in terms of and , respectively.

As long as , and commute with each other, applying Eq. (5) using the array representation to Eq. (18) leads to the following algorithm:

Algorithm 2. AcIIF2 for reaction-diffusion systems with cross derivatives

$$U^{n+1}-\frac{\mathrm{\Delta }t}{2}f(U^{n+1})=\underset{1\le k_1\le N_x}{\otimes }{e}^{{\mathcal{A}}_{yz}\mathrm{\Delta }t}\left(\underset{1\le k_2\le N_y}{\otimes }{e}^{{\mathcal{A}}_{xz}\mathrm{\Delta }t}\left(\underset{1\le k_3\le N_z}{\otimes }{e}^{{\mathcal{A}}_{xy}\mathrm{\Delta }t}V(:,:,k_3)\right)(:,k_2,:)\right)(k_1,:,:),$$ (22)

where V = Uⁿ + Δt/2f(Uⁿ).

In this algorithm, the exponential of is a N_xN_y × N_xN_y matrix, in comparison to that is a N_xN_yN_z × N_xN_yN_z matrix. Thus applying array representation leads to significant saving.

Cross derivatives may affect the commutable property of the discretized operators, resulting in the questions: under what conditions, , and can commute with each other and what to do with the algorithms if the commuting property doesn’t hold. In Section 3, we will give a sufficient condition for the commuting property. Alternatively, we next introduce a splitting technique to deal with the cases without such commuting property.

2.3. AcIIF method for reaction-diffusion systems with non-constant diffusion coefficients

When the diffusion coefficients in Eq. (16) are functions in space, minor modification is needed for the three discretization operators , , in array representations. Because each of the three operators depends on the other spatial dimension, we introduce a super index for to represent the operator at difference value of z = k₃h_z for the array representation of :

$${\mathcal{L}}_{xy}U=\underset{1\le k_3\le N_z}{\otimes }{\mathcal{A}}_{xy}^{k_3}U(:,:,k_3)$$ (23)

Similarly, the array representation of and can be written in terms of ${\mathcal{A}}_{yz}^{k_1}$ and ${\mathcal{A}}_{xz}^{k_2}$.

As a result, the three operators can no longer commute with each other. Notice that

$$e^{({\mathcal{L}}_{xy}+{\mathcal{L}}_{yz}+{\mathcal{L}}_{xz})\mathrm{\Delta }t}=e^{{\mathcal{L}}_{xy}\mathrm{\Delta }t}{e}^{{\mathcal{L}}_{xz}\mathrm{\Delta }t}{e}^{{\mathcal{L}}_{yz}\mathrm{\Delta }t}+O(\mathrm{\Delta }{t}^2),$$ (24)

the algorithm for Eq. (22) for this general case becomes only first order in time.

The order of accuracy can be improved by using a Strang splitting scheme to approximate e^{( + + )Δt}. In the Strang splitting method, two linear operators and defined on the same linear space have the following property [17]:

$$e^{({\mathcal{L}}_1+{\mathcal{L}}_2)\mathrm{\Delta }t}=e^{{\mathcal{L}}_1\mathrm{\Delta }t/2}{e}^{{\mathcal{L}}_2\mathrm{\Delta }t}{e}^{{\mathcal{L}}_1\mathrm{\Delta }t/2}+R(\mathrm{\Delta }t),$$ (25)

where

$$R(\mathrm{\Delta }t)=({\mathcal{L}}_2{\mathcal{L}}_1^{2}d+{\mathcal{L}}_1^2{\mathcal{L}}_2+4{\mathcal{L}}_2{\mathcal{L}}_1{\mathcal{L}}_2-2{\mathcal{L}}_1{\mathcal{L}}_2{\mathcal{L}}_1-2{\mathcal{L}}_1{\mathcal{L}}_2^2-2{\mathcal{L}}_2^2{\mathcal{L}}_1)\mathrm{\Delta }{t}^3/24.$$ (26)

For multiple linear operators, , , … , the Strang splitting method can be extended to the following by induction:

$$e^{{\sum }_{i=1}^m{\mathcal{L}}_i\mathrm{\Delta }t}=\prod _{i=1}^m{e}^{{\mathcal{L}}_i\mathrm{\Delta }t/2}\prod _{i=m}^1{e}^{{\mathcal{L}}_i\mathrm{\Delta }t/2}+O(\mathrm{\Delta }{t}^3).$$ (27)

Now, applying Strang splitting to Eq. (5) leads to a second order method in both time and space:

$$U^{n+1}-\frac{\mathrm{\Delta }t}{2}F(U^{n+1})=e^{\frac{\mathrm{\Delta }t}{2}{\mathcal{L}}_{xy}}{e}^{\frac{\mathrm{\Delta }t}{2}{\mathcal{L}}_{xz}}{e}^{\mathrm{\Delta }t{\mathcal{L}}_{yz}}{e}^{\frac{\mathrm{\Delta }t}{2}{\mathcal{L}}_{xz}}{e}^{\frac{\mathrm{\Delta }t}{2}{\mathcal{L}}_{xy}}\left(U^n+\frac{\mathrm{\Delta }t}{2}F(U^n)\right).$$ (28)

Consequently, the array representation for solving Eq. (16) with non-constant diffusion coefficients leads to

Algorithm 3. AcIIF2 for system (16)

$$\begin{array}{l}V=U^n+\frac{\mathrm{\Delta }t}{2}f(U^n),\\ V^{\ast }=\underset{1\le k_1\le N_x}{\otimes }{e}^{{\mathcal{A}}_{yz}^{k_1}\mathrm{\Delta }t}\left(\underset{1\le k_2\le N_y}{\otimes }{e}^{{\mathcal{A}}_{xz}^{k_2}\mathrm{\Delta }t/2}\left(\underset{1\le k_3\le N_z}{\otimes }{e}^{{\mathcal{A}}_{xy}^{k_3}\mathrm{\Delta }t/2}V(:,:,k_3)\right)(:,k_2,:)\right)(k_1,;,:),\\ U^{n+1}-f(U^{n+1})=\underset{1\le k_3\le N_z}{\otimes }{e}^{{\mathcal{A}}_{xy}^{k_3}\mathrm{\Delta }t/2}\left(\underset{1\le k_2\le N_y}{\otimes }{e}^{{\mathcal{A}}_{xz}^{k_2}\mathrm{\Delta }t/2}{V}^{\ast }(:,k_2,:)\right)(:,:,k_3).\end{array}$$ (29)

Operator splitting leads to twice as many exponential-matrix and vector multiplication compared to the non-splitting case in Algorithm 2. Therefore, it is important to use appropriate order of splitting if a subset of operators can commute with each other to improve computational efficiency. For instance, for three operators , i = 1, 2, 3 where and can commute, however, cannot, one may have two different kinds of splittings:

$$e^{({\mathcal{L}}_1+{\mathcal{L}}_2+{\mathcal{L}}_3)\mathrm{\Delta }t}=e^{{\mathcal{L}}_3\mathrm{\Delta }t/2}{e}^{{\mathcal{L}}_2\mathrm{\Delta }t}{e}^{{\mathcal{L}}_1\mathrm{\Delta }t}{e}^{{\mathcal{L}}_3\mathrm{\Delta }t/2}+O({\mathrm{\Delta }}^{3}t)$$ (30)

and

$$e^{({\mathcal{L}}_1+{\mathcal{L}}_2+{\mathcal{L}}_3)\mathrm{\Delta }t}=e^{{\mathcal{L}}_1\mathrm{\Delta }t/2}{e}^{{\mathcal{L}}_2\mathrm{\Delta }t/2}{e}^{{\mathcal{L}}_3\mathrm{\Delta }t}{e}^{{\mathcal{L}}_2\mathrm{\Delta }t/2}{e}^{{\mathcal{L}}_1\mathrm{\Delta }t/2}+O({\mathrm{\Delta }}^{3}t).$$ (31)

Clearly, the splitting in Eq. (30) computes one fewer exponential matrix and vector multiplication than the splitting in Eq. (31).

2.4. AcIIF method for high dimensional reaction-diffusion systems

We next extend AcIIF to the reaction-diffusion equation in d spatial dimensions with d ≥ 3:

$$u_t=\sum _{1\le i<j\le d}{L}_{x_1{x}_j}u+f(u)0<x_i<1$$ (32)

where

$$L_{x_i{x}_j}:=a_{ij}\frac{{\partial }^2}{\partial x_i^2}+2b_{ij}\frac{{\partial }^2}{\partial x_i\partial x_j}+c_{ij}\frac{{\partial }^2}{\partial x_i^2},1\le i<j\le d,$$ (33)

and we assume that diffusion coefficients, a_ij, b_ij and c_ij are spatial functions that satisfy the elliptical conditions:

$$a_{ij}>0,c_{ij}>0,a_{ij}{c}_{ij}>b_{ij}^2.$$ (34)

We also assume that the boundary conditions for the system are periodic.

Similar to the three dimensional case, in each direction x_i, there are N_xi grid points with the grid size of h_xi. We use a N_x1 × N_x2 × … N_xd d-dimensional array U = (U_k1,k2,…,kd), 1 ≤ k_i ≤ N_xi, i = 1, 2, …, d to represent the solution, and to represent the discretized operator of L_xixj, 1 ≤ i < j ≤ d. Next, we denote $U{\mid }_{x_i,x_j}^{(k_r),r\ne i,j}$ as the matrix derived from U by fixing the dimensional index k_r, r ≠ i, j. Thus, the array representation of becomes

$${\mathcal{L}}_{x_i{x}_j}U=\underset{\begin{array}{c}1\le k_r\le N_{x_r}\\ r\ne i,j\end{array}}{\otimes }{\mathcal{A}}_{x_i{x}_j}^{(k_r),r\ne i,j}U{\mid }_{x_i,x_j}^{(k_r),r\ne i,j}$$ (35)

where ${\mathcal{A}}_{x_i{x}_j}^{(k_r),r\ne i,j}$ are linear mappings from the matrix space with all N_xi × N_xj matrices to itself and is similarly defined in the three-dimensional case.

If commute with each other, we are able to directly apply array representation to the IIF2 method to obtain a second order AcIIF method for solving Eq. (32):

$$U^{n+1}-\frac{\mathrm{\Delta }t}{2}F(U^{n+1})=\underset{\begin{array}{c}1\le k_r\le N_{x_r}\\ r\ne 1,2\end{array}}{\otimes }{e}^{{\mathcal{A}}_{x_1{x}_2}^{(k_r),r\ne 1,2}\mathrm{\Delta }t}\underset{\begin{array}{c}1\le k_r\le N_{x_r}\\ r\ne 1,3\end{array}}{\otimes }{e}^{{\mathcal{A}}_{x_1{x}_3}^{(k_r),r\ne 1,3}\mathrm{\Delta }t}\cdots \underset{\begin{array}{c}1\le k_r\le N_{x_r}\\ r\ne d-1,d\end{array}}{\otimes }{e}^{{\mathcal{A}}_{x_{d-1}{x}_d}^{(k_r),r\ne d-1,d}\mathrm{\Delta }t}V.$$ (36)

If are not commutable, we apply Strang splitting and array representation to obtain the second order AcIIF method:

$$\begin{array}{r}{U}^{n+1}-\frac{\mathrm{\Delta }t}{2}F(U^{n+1})=\underset{\begin{array}{c}1\le k_r\le N_{x_r}\\ r\ne 1,2\end{array}}{\otimes }{e}^{{\mathcal{A}}_{x_1{x}_2}^{(k_r),r\ne 1,2}\mathrm{\Delta }t/2}\cdots \underset{\begin{array}{c}1\le k_r\le N_{x_r}\\ r\ne d-1,d\end{array}}{\otimes }{e}^{{\mathcal{A}}_{x_{d-1}{x}_d}^{(k_r),r\ne d-1,d}\mathrm{\Delta }t/2}\\ \underset{\begin{array}{c}1\le k_r\le N_{x_r}\\ r\ne d-1,d\end{array}}{\otimes }{e}^{{\mathcal{A}}_{x_{d-1}{x}_d}^{(k_r),r\ne d-1,d}\mathrm{\Delta }t/2}\cdots \underset{\begin{array}{c}1\le k_r\le N_{x_r}\\ r\ne 1,2\end{array}}{\otimes }{e}^{{\mathcal{A}}_{x_1{x}_2}^{(k_r),r\ne 1,2}\mathrm{\Delta }t/2}V,\end{array}$$ (37)

where V = Uⁿ + Δt/2F(Uⁿ) and 1 ≤ k_r ≤ N_xr, r = 1, 2, …, d.

2.5. A sufficient condition for operator commuting

As evident in Strang splitting, proper choice of the order of operators Eq. (29) will decrease the computational cost, which can be improved if commuting operators can be found. Now, we give a sufficient condition for commutable operators.

Proposition 1

All linear operators , 1 ≤ i < j ≤ d commute with each other if the system in Eq. (32) satisfies:

1. the diffusion coefficients are constant,

2. the boundary conditions are periodic along each direction.

PROOF

With a given basis, the linear operator for the central difference discretization are N₁N₂…N_d × N₁N₂…N_d matrices in the following form,

$$\left(\begin{array}{ccccc}{m}_1& m_2& \dots & m_{N-1}& m_N\\ m_N& m_1& m_2& \dots & m_{N-1}\\ m_{N-1}& m_N& m_1& \dots & m_{N-2}\\ \dots & \dots & \dots & \dots & \dots \\ m_2& m_3& \dots & m_N& m_1\end{array}\right)$$ (38)

where N = N₁N₂…N_d and m_i, i = 1, 2, …, N are real. Let two matrices A = (a_i)_N×N and B = (b_i)_N×N both take the form of Eq. (38). One can show directly that

$${(AB)}_{ij}=\sum _{k=1}^N{a}_{k-i+2}{b}_{j-k+1}=\sum _{s=1}^N{b}_{s-i+2}{a}_{j-s+1}={(BA)}_{ij},\forall i,j,$$ (39)

because a_i±N = a_i, b_j±N = b_j for ∀i, j where s = j + i − k − 1.

This shows that Strang splitting is unnecessary for constant diffusion coefficients in high spatial dimension and Algorithm 2 is applicable for such reaction-diffusion equations.

3. Stability analysis, higher-order methods, and computational costs

Next, we study the linear stability of second order AcIIF methods, derive a third method, and discuss the computational costs of the methods.

3.1. Stability analysis

Based on linear stability analyses in [7] and [8], we claim that the second order AcIIF methods, Eq. (36) and Eq. (37), are asymptotically stable for the case of F(U) = dU and L_xixju = −cu, where d < 0 and c > 0 correspond to stable reactions and elliptic operators. For such a linear case, one has

$$u^{n+1}=e^{-c\mathrm{\Delta }t}\left(u^n+\frac{d\mathrm{\Delta }t}{2}{u}^n\right)+\frac{d\mathrm{\Delta }t}{2}{u}^{n+1}.$$ (40)

Assuming uⁿ = e^inθ, we obtain

$$e^{i\theta }=e^{-c\mathrm{\Delta }t}\left(1+\frac{1}{2}\lambda \right)+\frac{1}{2}\lambda e^{i\theta },$$ (41)

where λ = dΔt has a real part λ_r and imaginary part λ_i, leading to

$$\begin{array}{l}{\lambda }_r=\frac{2(1-e^{-2c\mathrm{\Delta }t})}{{(1-e^{-c\mathrm{\Delta }t})}^2+2(1+cos\theta )e^{-c\mathrm{\Delta }t}},\\ {\lambda }_i=\frac{4(sin\theta ){ce}^{-c\mathrm{\Delta }t}}{{(1-e^{-c\mathrm{\Delta }t})}^2+2(1+cos\theta )e^{-c\mathrm{\Delta }t}},\end{array}$$ (42)

since c > 0 and λ_r > 0 for 0 ≤ θ ≤ 2π. Then, the second order AcIIF is A-stable since the stability region includes the complex plane for all λ with λ_r < 0.

If we apply AcIIF methods Eq. (36) and Eq. (37) to Fokker-Planck equations or chemical master equations, where in each the operator defines a Markov process,

$$\frac{dU}{dt}={\mathcal{L}}_{x_i{x}_j}U,$$ (43)

then we claim that AcIIF methods Eq. (36) and Eq. (37) are still A-stable.

In order to show the A-stability, we prove that, under certain norm, ||e^Δt|| ≤ 1 holds for any Δt > 0 for such . Then, each can be treated as elliptic operators and the remaining proof goes through Eq. (40) to Eq. (42). Since Eq. (43) defines a Markov process, the total probability of all states, ΣU(Δt) = Σe^ΔtU(0), maintains a value of 1 for any time step, when U(0) is a proper probability distribution, i.e. U(0) > 0 for each compartment and ΣU(0) = 1. Using the maximum norm ||.||₁ and defining the corresponding linear operator norm, we first prove that for any U with U > 0,

$${\Vert e^{{\mathcal{L}}_{x_i{x}_j}\mathrm{\Delta }t}U\Vert }_1=\sum \left|e^{{\mathcal{L}}_{x_i{x}_j}\mathrm{\Delta }t}\left(\frac{U}{\sum U}\right)\right|\sum U={\Vert U\Vert }_1.$$ (44)

Then, for U = U⁺ − U⁻ where U⁺ is with all positive compartments of U,

$$\begin{array}{l}\Vert e^{{\mathcal{L}}_{x_i{x}_j}\mathrm{\Delta }t}\Vert =\underset{U\ne 0}{max}\frac{{\Vert e^{{\mathcal{L}}_{x_i{x}_j}\mathrm{\Delta }t}U\Vert }_1}{{\Vert U\Vert }_1}=\underset{U\ne 0}{max}\frac{\Vert e^{{\mathcal{L}}_{x_i{x}_j}\mathrm{\Delta }t}{U}^{+}-e^{{\mathcal{L}}_{x_i{x}_j}\mathrm{\Delta }t}{U}^{-}\Vert }{\sum \mid U\mid }\\ \le \underset{U\ne 0}{max}\left(\frac{{\Vert e^{{\mathcal{L}}_{x_i{x}_j}\mathrm{\Delta }t}{U}^{+}\Vert }_1}{\sum \mid U\mid }+\frac{{\Vert e^{{\mathcal{L}}_{x_i{x}_j}\mathrm{\Delta }t}{U}^{-}\Vert }_1}{\sum \mid U\mid }\right)=1.\end{array}$$ (45)

Next, we can replace each in the Fokker-Planck equation or the chemical master equation by a negative scalar and the proof of A-stability for Eq. (36) and Eq. (37) for these two cases is done.

3.2. High order AcIIF method

If discretization operators are commutable, higher order (in time) AcIIF methods can be derived from the IIF method in a similar manner. For example, the third order IIF scheme [7] has the form:

$$U^{n+1}=e^{\mathcal{L}\mathrm{\Delta }t}{U}^n+\mathrm{\Delta }t\left(\frac{5}{12}F(U^{n+1})+\frac{2}{3}{e}^{\mathcal{L}\mathrm{\Delta }t}F(U^n)-\frac{1}{12}{e}^{2\mathcal{L}\mathrm{\Delta }t}F(U^{n-1})\right)$$ (46)

where = Σ_i,j . If all , ∀i, j commute with each other, we then obtain

$$e^{\mathcal{L}}U=\prod _{1\le i<j\le d}{e}^{{\mathcal{L}}_{x_i{x}_j}}U.$$ (47)

If all discretized operators commute with each other, applying the array representation leads to the third order AcIIF method:

Algorithm 4. (Third order AcIIF method)

$$\begin{array}{l}{U}^{n+1}-\frac{5\mathrm{\Delta }t}{12}F(U^{n+1})=\underset{(k_r),r\ne 1,2}{\otimes }{e}^{A_{x_1{x}_2}\mathrm{\Delta }t}\cdots \underset{(k_r),r\ne d-1,d}{\otimes }{e}^{A_{x_{d-1}{x}_d}\mathrm{\Delta }t}{V}_1\\ -\underset{(k_r),r\ne 1,2}{\otimes }{e}^{2A_{x_1,x_2}\mathrm{\Delta }t}\cdots \underset{(k_r),r\ne d-1,d}{\otimes }{e}^{2A_{x_{d-1}{x}_d}\mathrm{\Delta }t}{V}_2\end{array}$$ (48)

$$V_1=U^n+\frac{2\mathrm{\Delta }t}{3}F(U^n),V_2=\frac{\mathrm{\Delta }t}{12}F(U^{n-1})$$ (49)

where 1 ≤ k_r ≤ N_r, r = 1, 2, …, d.

In the case that some are not commutable, the splitting techniques may be applied to this subset of operators to achieve high order accuracy. However, since the formulation becomes much more tedious and complicated, we omit them here.

Remarks on higher order derivatives

The array representation can also be extended for equations with operators that contain high order and cross derivatives, such as those in the following form

$$\frac{{\partial }^m}{\partial x_{i_1}\partial x_{i_2}\dots \partial x_{i_m}}.$$ (50)

Similarly, the second order central difference approximation in the multi-dimensional array U representation results in the discretization linear operator of the following compact form,

$${\mathcal{L}}_{x_{i_1},x_{i_2},\dots ,x_{i_m}}U=\underset{1\le k_r\le N_r,r\ne i_1,i_2,\dots ,i_m}{\otimes }{\mathcal{A}}_{x_{i_1}{x}_{i_2}\dots x_{i_m}}U{\mid }_{x_{i_1},x_{i_2},\dots ,x_{i_m}}^{(k_r),r\ne i_1,i_2,\dots ,i_m}$$ (51)

where $U{\mid }_{x_{i_1},x_{i_2},\dots ,x_{i_m}}^{(k_r),r\ne i_1,i_2,\dots ,i_m}$ is a m-dimensional array by fixing the index k_r, r ≠ i₁, i₂, …, i_m of U and , resulting from the central difference approximation, represents a linear mapping from m-dimensional array linear space to itself.

3.3. Computational cost

For stiff reaction-diffusion equations, the size of the time step usually dictates the overall cost of the temporal updating method. For an A-stable method such as AcIIF and IIF, the cost mainly results from the formation of the exponential-matrix and the corresponding vector-matrix multiplication during each time step. In array representation, small matrices of size in N_i × N_i for the Laplacian operator or N_iN_j × N_iN_j when the second order cross derivatives are presented in contrast to IIF in which the exponential of a N₁N₂…N_d × N₁N₂…N_d matrix is required. The advantage of AcIIF becomes more prominent for three or higher dimensional systems.

For a d-spatial dimensional case (d ≥ 3) with second order cross derivatives, the computational cost for manipulating the exponential matrices in IIF is O((N₁N₂…N_d)²), or O(N^2d) for N_i = N, i = 1, 2, …, d, while the corresponding cost for AcIIF is

$$\sum _{1\le i<j\le d}O({(N_i{N}_j)}^2)\frac{N_1{N}_2\dots N_d}{N_i{N}_j}.$$ (52)

For the case of non-constant coefficients in diffusion, it is O(d²N^d+2) when N_i = N, i = 1, 2, …, d. For example, a six-dimensional system requires calculating an exponential of matrix with an approximated size of 10⁸ × 10⁸ when N = 20, in contrast to the AcIIF method that only needs exponentials of matrices of a size of 400 × 400.

Because the exponential-matrices are small in AcIIF, one may pre-calculate the exponential matrices once and store them during the calculations. An alternative approach, which is particularly useful for matrices with sizes exceeding the memory size, is to compute the exponential-matrix vector multiplication without explicit formation of the matrices through, for example, the Krylov subspace method [13, 18, 19]. In the direct simulations shown in the next section, we implement Padé approximation, which has a computational cost of O(N²) (both in storage and time) to compute a matrix exponential of N × N matrix [20], for reaction-diffusion equations with or without cross-derivatives in three dimensions, and we use Krylov subspace method for the Fokker-Planck equations in three or four dimensions and chemical master equations.

3.4. Array representation for Chemical Master Equations

Chemical master equations (CME) is a system of first-order ordinary differential equations for stochastic description of the time evolution of a network of biochemical reactions [21]. The solution of the system yields the probability density vector at discrete states of the bio-chemical network in time. The system is typically stiff and it can have many components and states, presenting difficulties for numerical methods and simulations [22, 23]. As seen below, the array representation provides a convenient approach to decompose the large rate matrix into smaller matrices for efficient usage of integration factor methods that can best deal with stiffness in the system.

If a given chemical reaction system consists of of d molecular species, namely X₁, X₂, …, X_d, with maximal copy numbers of N₁, N₂, …, N_d, respectively, then the system has N_tot = N₁N₂…N_d possible states, for which a vector x = (x₁, x₂, …, x_d) denotes each state. The R – th reaction takes the form

$$R_r:w_1^r{X}_1+w_2^r{X}_2+\dots +w_d^r{X}_d\to v_1^r{X}_1+v_2^r{X}_2+\dots +v_d^r{X}_d,$$ (53)

where $v_i^r$ and $w_i^r$, for every i and r, are non-negative integers and the system contains R number of reactions. The reaction rate at state x is a_r(x) = a_r(x₁, x₂, …, x_d) and the vector

$${\alpha }^r=(v_1^r-w_1^r,v_2^r-w_2^r,\dots ,v_d^r-w_d^r)$$ (54)

denotes the change of copy number of the molecular species after the r – th reaction occurs once.

In array representation, one can use an N₁ × N₂… × N_d d-dimensional array U to denote the probability density function, and each component of U, U_x, as the probability density at state x, which can be written as

$$U_x(t)=\text{Prob}[X_1=x_1,X_2=x_2,\dots ,X_d=x_d,\text{at}\text{time}t].$$ (55)

Define the linear mapping ,

$${({\mathcal{L}}_{r}U)}_x=a_r(x-{\alpha }^r)U_{x-{\alpha }^r}(t)-a_r(x)U_x.$$ (56)

The CME for the probability density functions becomes

$$\stackrel{.}{U}(t)=\sum _{r=1}^R{\mathcal{L}}_{r}U(t).$$ (57)

To introduce the array representation for , we let $i_1^r,i_2^r,\dots ,i_{m_r}^r$ denote the indices of non-zero entries in α^r. Using the same notation as in Eq. (51), $U{\mid }_{X_{i_1^r},X_{i_2^r},\dots ,X_{i_{m_r}^r}}^{(x_j),j\ne i_1^r,i_2^r,\dots ,i_{m_r}^r}$ denotes a m_r-dimensional array by fixing indexes x_j, $j\ne i_1^r,i_2^r,\dots ,i_{m_r}^r$. Then, one obtains

$$U=\underset{\begin{array}{l}1\le x_j\le N_j\\ j\ne i_1^r,\dots ,i_{m_r}^r\end{array}}{\otimes }U{\mid }_{X_{i_1^r},X_{i_2^r},\dots ,X_{i_{m_r}^r}}^{(x_j),j\ne i_1^r,i_2^r,\dots ,i_{m_r}^r}.$$ (58)

Define the linear mapping ${\mathcal{A}}_{X_{i_1^r},X_{i_2^r},\dots ,X_{i_{m_r}^r}}^{(x_j),,j\ne i_1^r,i_2^r,\dots ,i_{m_r}^r}$ on m_r-dimensional array V as

$${({\mathcal{A}}_{X_{i_1^r},X_{i_2^r},\dots ,X_{i_{m_r}^r}}^{(x_j),,j\ne i_1^r,i_2^r,\dots ,i_{m_r}^r}V)}_{(x_i),i\in i_1^r,i_2^r,\dots ,i_{m_r}^r}=a_r(x-{\alpha }^r)V_{(x_i-{\alpha }_i^r),i\in i_1^r,i_2^r,\dots ,i_{m_r}^r}-a_r(x)V_{(x_i),i\in i_1^r,i_2^r,\dots ,i_{m_r}^r}.$$ (59)

Then the array representation of becomes

$${\mathcal{L}}_{r}U=\underset{\begin{array}{c}1\le x_j\le N_j\\ j\ne i_1^r,i_2^r,\dots ,i_{m_r}^r\end{array}}{\otimes }{\mathcal{A}}_{X_{i_1},X_{i_2},\dots ,X_{i_{m_r}}}^{(x_j),j\ne i_1^r,i_2^r,\dots ,i_{m_r}^r}U{\mid }_{X_{i_1},\dots ,X_{i_{m_r}}}^{(x_j),j\ne i_1^r,i_2^r,\dots ,i_{m_r}^r}.$$ (60)

For typical systems, each , r = 1, 2, −, R cannot commute with one another, thus the Strang splitting method is applied to approximate the solution, resulting in a second order integration factor method (AcIIF2) for CME Eq. (57):

$$U^{n+1}-\prod _{r=1}^R{e}^{{\mathcal{L}}_r\mathrm{\Delta }t/2}\prod _{r=R}^1{e}^{{\mathcal{L}}_r\mathrm{\Delta }t/2}{U}^n,$$ (61)

where Uⁿ denotes the probability density functions at time t_n = nΔt.

The exponential of can be written in terms of the exponentials of ${\mathcal{A}}_{X_{i_1},X_{i_2},\dots ,X_{i_{m_r}}}^{(x_j),j\ne i_1^r,i_2^r,\dots ,i_{m_r}^r}$. If the reaction R_r only affects copy numbers of a few species, implying m_r is small, the calculation of the latter exponential is much more efficient than computing the original one. In other words, the array representation saves storage and CPU time for the system containing many molecular species while each reaction only affects the copy number of a small portion of species.

4. Numerical simulations

To explore various applications of the AcIIF methods (Eq. (36) and Eq. (37)), we apply the second order AcIIF methods to five different systems: three-dimensional reaction-diffusion equations with constant diffusion coefficients or spatially-dependent diffusion constants; three-and four-dimensional Fokker-Planck equations; and chemical master equations arising from a biological application.

4.1. Three-dimensional reaction-diffusion equation with constant diffusion coefficients

We first apply AcIIF method Eq. (36) to the following reaction-diffusion equation

$$u_t=(0.1u_{xx}-0.15u_{xy}+0.1u_{yy})+(0.1u_{xx}+0.2u_{xx}+0.2u_{zz})+(0.2u_{yy}+0.15u_{yz}+0.1u_{zz})+0.8u,$$ (62)

where x, y, z ∈ (0, 2π) with periodic boundary conditions. With the initial condition u(x, y, z, 0) = sin(x + y + z), the equation has the exact solution u(x, y, z, t) = e^−0.2t sin(x + y + z).

Based on the result from Section 2.5, for this case, the corresponding linear operators , and can commute with each other. Thus, Eq. (36) is a second order scheme in both time and space. We first compare the second order array-representation compact IIF with the standard IIF, both in second order. Because both methods are A-stable, we choose Δt = 1/N = h_x/2π where N_x = N_y = N_z = N and Δt = 1/N = h_x/2π, and simulation results are evaluated at t = 1. As seen in Table 1, both methods clearly show second order accuracy with similar sizes of errors as N increases, as one may expect from the analysis of both methods. On the other hand, we observe the CPU time for both methods to achieve the same accuracy is much larger in IIF than in AcIIF, because the exponential matrices in IIF have much larger size than AcIIF. When N becomes 32, IIF fails to compute as the size for matrix exponential becomes exceedingly large, leading to a lack of sufficient memory in a Matlab implementation on typical personal computers (4GB). Even in a cluster where computing a 32³ × 32³ matrix exponential is possible, the CPU time needed will be about 2 hours, and computation of a 64³ × 64³ matrix exponential takes more than a day. On the other hand, AcIIF runs normally with good accuracy, showing clear advantages in handling larger grid numbers for convergence of solutions.

Table 1.

A comparison between the second order array-representation compact IIF method (AcIIF2 in Eq. (36)) and IIF2 method.

	Error in L∞		Accuracy order		CPU time
N	AcIIF2	IIF2	AcIIF2	IIF2	AcIIF2	IIF2
8	0.0672	0.0672	-	-	0.05s	0.34s
16	0.0169	0.0169	1.99	1.99	0.15s	30.5s
32	0.0042	-	2.01	-	5.13s	-
64	0.0011	-	1.93	-	246s	-

The symbol “-” denotes insufficient memory in calculation of the exponential matrix.

4.2. Three-dimensional diffusion reaction system with non-constant diffusion coefficients

To test the case with non-commutable differential operators, we consider the following reaction-diffusion equations with non-constant coefficients:

$$\begin{array}{l}{u}_t=(0.5u_{xx}-0.5sin(x+y)u_{xy}+0.5u_{yy})+(0.5u_{xx}-1/3cos{yu}_{xz}+1/3u_{zz})\\ +(1+cosx)(0.5u_{yy}-0.5u_{yz}+1/3u_{zz})+f(x,y,z,u),\end{array}$$ (63)

where x, y, z ∈ (0, 2π) with periodic boundary conditions. With the initial condition u(x, y, z, 0) = sin(x + y + z), the equation has the exact solution u(x, y, z, t) = e^−0.2t sin(x + y + z). Similar to the previous case, we choose N_x = N_y = N_z = N, grid size h_x = h_y = h_z = 2π/N, Δt = 1/N, and t = 1 as the temporal point for evaluating the method.

In this non-commutable case, we need to compute N number of array-representation operators for each of , and . For example, to compute e, we need the following calculations

$$\begin{array}{l}{\mathcal{A}}_{yz}^{k_x}~(1+cos((k_x-1)h_x)((0.5{(·)}_{yy}-0.5{(·)}_{yz}+1/3{(·)}_{zz}),\\ e^{{\mathcal{L}}_{yz}}=\underset{k_x}{\otimes }{\mathcal{A}}_{yz}^{k_x}U{\mid }_{yz}^{k_x}\end{array}$$ (64)

for k_x = 1, 2, …, N. In the commutable case, we only compute and save three of the exponential matrices of size N² × N², in contrast to 3N of exponential matrices of the same size. As a result, the non-commutable case takes significantly more CPU time than the commutable case as seen in Table 1 and Table 2. However, compared to the standard IIF method, AcIIF is still significantly much faster.

Table 2.

A comparison between AcIIF2 method (Eq. (37)) and IIF2 for the case of non-constant diffusion coefficients.

	Error in L∞		Accuracy order		CPU time
N	AcIIF2	IIF2	AcIIF2	IIF2	AcIIF2	IIF2
8	0.2744	0.2754	-	-	0.29s	1.35s
16	0.0675	0.0678	2.02	2.02	2.2s	155s
32	0.0169	-	2.00	-	133s	-

The order of accuracy for both AcIIF2 and IIF2 remain second order, as seen in Table (2). However, the error for the non-commutable case is larger than the commutable case at the same spatial and temporal resolutions, which is likely due to the splitting error in time. Similar to the constant diffusion case, IIF2 fails to run due to the memory problem for relatively larger N.

4.3. Three- and four- dimensional Fokker-Planck equations

The Fokker-Planck equation (FPE) describes the time evolution of the probability density function of stochastic systems [1]. The generalized FPE usually takes the following form

$$\frac{\partial p(x,t)}{\partial t}=-\sum _{r=1}^R\{\sum _i^N{n}_{ri}\frac{\partial }{\partial x_i}\left(q_r(x,t)-\frac{1}{2}\sum _{j=1}^N{n}_{rj}\frac{\partial q_r(x,t)}{\partial x_j}\right)\}.$$ (65)

Here, in the case of bio-chemical reactions, R denotes the total number of chemical reactions involved in the system, N denotes the total number of different species participating the reactions, x_j denotes the copy number of j-th reactant, and n_ri denotes the change of copy number of reactant i when the r-th reaction occurs once. p(x, t) represents the probability density of the system at the state x = (x₁, x₂, …, x_N)(x ∈ R^N+) and time t. In addition, we define

$$q_r(x,t)=w_r(x,t)p(x,t),$$ (66)

where w_r(x, t) is the reaction propensity function for r-th reaction at state x. For example, for the following bio-chemical reactions,

$$X\stackrel{k_1[X]}{\to }Y,X+Y\stackrel{k_2[X][Y]}{\to }Z,\varnothing \stackrel{k_3}{\to }Z,$$ (67)

we have n₁ = (−1, 1, 0), n₂ = (−1, −1, 1), n₃ = (0, 0, 1) and

$$w_1(x,y,z,t)=k_{1}x,w_2(x,y,z,t)=k_{2}xy,w_3(x,y,z,t)=k_{3}z.$$ (68)

In general, FPE is a N-dimensional convection-diffusion equations with non-constant diffusive coefficients and second order cross derivatives. Because the system may be stiff, implicit temporal methods, such as Crank-Nicolson method [24], which requires solving nonlinear systems of large size at each time step, are often needed. While directly apply IIF method, the calculation of the huge matrix exponential is unaffordable in the high dimensional case. AcIIF, which has the good stability like Crank-Nicolson, is a better choice in solving FPE than IIF method as it divides the entire discretization matrix into multiple small pieces by the array-representation technique.

To apply AcIIF to FPE, we first study a three-dimensional case in which there are two metabolites and one enzyme, which is also studied in [25]. The reactions are:

$$\begin{array}{c}\varnothing \stackrel{k_1}{\to }A\varnothing \stackrel{k_2}{\to }B\\ A+B\stackrel{k_3}{\to }\varnothing \\ A\stackrel{k_4}{\to }\varnothing B\stackrel{k_5}{\to }\varnothing \\ \varnothing \stackrel{k_6}{\to }{E}_A{E}_A\stackrel{k_7}{\to }\varnothing \end{array}$$ (69)

The corresponding propensity rates are given as

$$\begin{array}{c}{k}_1=\frac{k_A[E_A]}{1+[A]/K_I}{k}_2=k_B{k}_3=k[A][B]\\ k_4=\mu [A]k_5=\mu [B]k_6=\frac{k_{E_A}}{1+[A]/K_R}{k}_7=\mu [E_A]\end{array}$$ (70)

where k_A = 0.3s⁻¹, k_B = 2s⁻¹, K_I = 30, k = 0.001s⁻¹, μ = 0.004s⁻¹, K_R = 30 and k_EA= 1s⁻¹ [25].

The computational domain for this system is chosen to be Ω_h = [0, 100] × [0, 100] × [0, 45], which is large enough such that the probability of [A] > 100, [B] > 100, [E_A] > 45 is sufficiently small, implying that the domain covers nearly all the possible states of the chemical reactions. After discretizing the FPE using second order central differences, we represent the density function by a three-dimensional array U(t) to represent the density function. Each component U_i1,i2,i3 (t) denotes the probability density for system at time t and state [A] = i₁, [B] = i₂, [E_A] = i₃. There are seven reactions, thus, in FPE Eq. (65), corresponding to R = 7. For r–th reaction, the corresponding discretized operator, denoted by , becomes

$${\mathcal{L}}_r=\sum _i^N{n}_{ri}\frac{\partial }{\partial x_i}\left(q_r(x,t)+\frac{1}{2}\sum _{j=1}^N{n}_{rj}\frac{\partial q_r(x,t)}{\partial x_j}\right).$$ (71)

Because contains no cross derivatives for r ≠ 3, we can use the array representation presented in Section 2.1. On the other hand, contains a cross derivative ∂²/∂[A] ∂[B], we use the array representation presented in Section 2.3. By direct application of AcIIF (Eq (37) based on splitting technique), we obtain an overall second order method. In particular, some of the reactions can be grouped into one matrix to reduce the number of splittings and number of calculations of exponential matrices, such as and , which both have ∂²/∂[A]² and ∂/∂[A] in Eq. (71).

To study the performance of AcIIF2, we also implement the second order Runge Kutta (RK2) method for a comparison. The error of solution in the maximal norm is based on a simulation result from the finest “spatial” grid (N_A = N_B = 200, N_EA = 120) and finest time step (Δt = 5 × 10⁻³). The initial condition for each simulation is a Gaussian distribution centered at point (30, 40, 20) with standard derivation $\sqrt{30}$.

First, we observe in Table 3 that a much smaller Δt is required for RK2 to converge compared to AcIIF2 due to the fact that the reactions are stiff, requiring small Δt, for non A-stable methods such as RK2. Interestingly, AcIIF2 can reach the same overall error level as RK2 using a much larger time step for the same-sized “spatial” mesh, indicating that the numerical error for solving this FPE is likely dominated by spatial discretization. Thus, a large time step is sufficient for A-stable methods, such as IIF, while small time steps are still required for RK2 due to its stability constraints. In each time step RK2 is more efficient than AcIIF2; however, AcIIF2, which requires fewer time steps for a given t, still outperforms RK2 significantly in this case. We also plot the numerical results in Figure 1, where a grid with N_A = N_B = 60 and N_EA = 30 and time step Δt = 1s are used.

Table 3.

A comparison between the second order AcIIF and Runge-Kutta (RK2) for the three-dimensional FPE (69) at t = 30.

		AcIIF2/RK2
Grids (NA, NB, NEA)	Δt	Error in L∞	CPU time	RK2 unstable when
(25,25,15)	5/0.2	2.6 × 10−4/2.6 × 10−4	17.6s/71.2s	Δt ≥ 0.3
(40,40,24)	5/0.15	1.3 × 10−4/1.4 × 10−4	35.3s/182.5s	Δt ≥ 0.2
(50,50,30)	5/0.1	8.6 × 10−5/9.1 × 10−5	65.5s/470.2s	Δt ≥ 0.15

Figure 1.

Numerical solution of system (69) using AcIIF2. Temporal discretization is set by the time step Δt = 1s, and the simulation is ran up to time t = 50s. (a) Shows the initial distribution of molecular species A and B, which are Gaussian distributions centered at (A, B) = (30, 40). (b) The distribution of molecular species A and B at t = 50s. (c) The contour plot of initial and final distributions. The dotted black line connects the centers of the solutions of the rate equations of system (69).

Next, we add another enzyme E_B that synthesizes metabolite B in the same way that E_A synthesizes A in the three-dimensional system Eq. (69). This extension leads to a four-dimensional FPE of four molecular species [A], [B], [E_A] and [E_B] [25],

$$\begin{array}{c}\varnothing \stackrel{\frac{k_A·e_A}{1+a/K_I}}{\to }A\varnothing \stackrel{\frac{k_B·e_B}{1+b/K_I}}{\to }B\\ A+B\stackrel{k_2·a.b}{\to }\varnothing \\ A\stackrel{\mu ·a}{\to }\varnothing B\stackrel{\mu ·b}{\to }\varnothing \\ \varnothing \stackrel{\frac{k_{E_A}}{1+a/K_R}}{\to }{E}_A{E}_A\stackrel{\mu ·e_A}{\to }\varnothing \\ \varnothing \stackrel{\frac{k_{E_B}}{1+b.K_R}}{\to }{E}_B{E}_B\stackrel{\mu ·e_B}{\to }\varnothing \end{array}$$ (72)

where k_A = k_B = 0.3s⁻¹, k₂ = 0.001s⁻¹, K_I = 60, μ = 0.002s⁻¹, k_EA= k_EB = 0.02s⁻¹ and K_R = 30.

The computational domain is chosen to be [0, 80] × [0, 80] × [0, 30] × [0, 30] that contains nearly all possible states of the system. We choose zero Dirichlet boundary conditions with the initial condition as a Gaussian distribution centering at (30, 40, 15, 12) with a standard deviation $\sqrt{40}$. There are nine reactions in the system, corresponding to nine array-representation operators. Based on commutability of the operators, we group some of them similar to the three dimensional case to increase the overall computational efficiency. One interesting observation is that that as the fourth dimension grid number N_EB increases, the CPU time for increases only linearly, as seen in Table 4. For this set of simulations, we fix the other three grid numbers: N_A = N_B = N_EA = 10, and keep doubling N_EB from 4 to 32. IIF method computes the entire matrix exponential, thus its CPU time will increase by a fourth folder. While AcIIF only compute small matrix exponential, its CPU time will linearly depends on N_EB. Finally, we plot the numerical results for N_A = N_B = 40, N_EA = N_EB= 20 and time step Δt = 1.

Table 4.

The CPU time for different grid numbers of the fourth dimension of the FPE.

NEA	4	8	16	32
CPU time(s)	5.6	8.3	14.5	32.6

4.4. An application to Chemical Master Equations

The Chemical Master Equation (CME) describes the time evolution of the probability density function. In CME, each reaction on the probability density evolution may be considered as diffusion-like operators with cross derivatives. Thus, AcIIF can be applied to solve such equations. We consider a family of proteins X with different conformational types X₁, X₂, …, X_d. Two conformational types X_i and X_i+1 can conform to each other through an enzyme E. Suppose that during reactions, no new protein is created; the enzyme is abundant so that one can treat the quantity of the enzyme as a constant; and intermediate products are extremely unstable. As a result, the entire system consists of the following bio-chemical reactions:

$$\begin{array}{l}{X}_1\stackrel{k_1^{+}}{\to }{X}_2\stackrel{k_2^{+}}{\to }{X}_{3\cdots }\stackrel{k_{d-1}^{+}}{\to }{X}_d\\ X_1\stackrel{k_1^{-}}{\leftarrow }{X}_2\stackrel{k_2^{-}}{\leftarrow }{X}_{3\cdots }\stackrel{k_{d-1}^{-}}{\leftarrow }{X}_d\end{array}$$ (73)

for which the total copy number of the protein X is a constant,

$$X_1+X_2+\dots +X_d=N$$ (74)

For simplicity, x = (x₁, x₂, …, x_d) denotes each state where 0 ≤ x_i ≤ N, i = 1, 2, …, d (although some of the states cannot be reached). In particular, the reaction

$$X_i\stackrel{k_i^{+}}{\to }{X}_{i+1}$$ (75)

defines a linear mapping on probability density function in CMEs, with the following array representation,

$${\mathcal{A}}_{X_i,X_{i+1}}^{(x_j),j\ne i,i+1}{M}_{m,n}=-k_1^{+}{mM}_{m,n}+k_1^{+}(m+1)M_{m+1,n-1}$$ (76)

where M is a 2-dimensional array. Other reactions can be treated in a similar way.

For a protein family with d conformational types and N total number of copies, the direct calculation of exponential of the linear mapping requires the exponentiation of a N^d × N^d matrix. However, in the array representation, only N² × N² matrices’ exponentials are required to be calculated. More saving in both storage and CPU time result in using the array representation when the number of species d gets larger.

To demonstrate this through direct simulations, we implement a second order array-representation integration factor method as well as the second order Runge-Kutta (a standard temporal integrator for CMEs) for the case of N = 30 and d = 3. The initial distribution of the molecules is set to be P(X₁ = 30) = 1, that is, initially all molecules take the conformational type X₁. We choose rate coefficients $k_1^{+}=k_2^{-}=1,k_1^{-}=2,k_2^{+}=3$ and we compute the solution up to t = 3 using different Δt. The maximal error of the solution is estimated based on an “exact” solution computed using a very small time step by RK2.

First, the second order accuracy of AcIIF2 method is clearly observed in Table 5. As expected, RK2 requires a very small time step due to its stability constraint in contrast to AcIIF’s stable and good accuracy, even at a time step as large as Δt = 1/4. As Δt decreases significantly (e.g. Δt ≤ 1/128), RK2 becomes stable and converges as seen in Table 5. At the same size of time step, we observe AcIIF2 and RK2 has similar size of errors. Of course, using the same size of time step, RK2 takes less CPU time and storage than AIF2, with both achieving similar accuracy. However, if moderately high accuracy (e.g. 10⁻⁴ for this particular system) is sufficient, AIF2 shows its advantage. In particular, as the number of species increases or the rate constants become more stiff, a combination of the array representation and the integration factor method becomes even more attractive in achieving both efficiency and accuracy.

Table 5.

A comparison between the second order array-representation compact integration factor method (AcIIF2) and Runge-Kutta (RK2) methods for simulating CMEs.

	AcIIF2		RK2
Δt	error	order of acc	error	order of acc
1/4	7.95 × 10−4	-	unstable
1/8	1.96 × 10−4	2.02	unstable
1/16	4.89 × 10−5	2.00	unstable
1/32	1.22 × 10−5	2.00	unstable
1/64	3.06 × 10−6	2.00	unstable
1/128	7.64 × 10−7	2.00	3.74 × 10−7	-
1/256	1.91 × 10−7	2.00	9.30 × 10−7	2.01

5. Discussions and Conclusions

Higher order spatial derivatives and reactions of drastically different time scales demand temporal schemes of the generous stability constraint. Implicit integration factor methods, which solve exactly the linear operator of higher order spatial derivatives along with an implicit treatment of the stiff reactions, are effective approaches for such types of different equations. One unique computational challenge associated with such methods is the handling of exponentials of matrices. Here, we have introduced a new array representation for the discretization matrices of the linear differential operators. Because of such representation, computing exponentials of large matrices is reduced to the calculation of exponentials of matrices of significantly smaller sizes. The saving and advantages for array representations in both storage and CPU time escalate as the dimension of the system increases. In addition, this approach can be directly combined with the implicit integration method for an overall efficient method (termed as AcIIF) of excellent temporal stability.

Due to its advantage for high dimensions and stiff reactions, such an approach is particularly appropriate for solving reaction-diffusion equations and other diffusion-like equations, such as Fokker-Planck equations. Our direct implementation and testing of the second order AcIIF, which is linearly absolute stable, has demonstrated its advantages compared to some existing approaches. Interestingly, such array representation can also be applied to chemical master equations, ODE systems of large size that often is stiff. The computational efficiency for such applications become most evident for biochemical networks of a large number of species with each reaction in the system affecting only few species.

Although the array representation has been presented only in the context of compact implicit integration factor methods, the approach can easily be applied to other integration factor or exponential difference methods. Other type of equations of higher order derivatives, (e.g. Cahn-Hilliard equations [26] of fourth order derivatives) in addition to reaction-diffusion equations and Fokker-Planck equations may also be handled using the array representation for better efficiency. To better deal with high spatial dimensions, one can incorporate the sparse grid [27] into the array-representation technique. The flexibility of such representation allows either direct calculation of the exponentials of matrices or using Krylov subspace for computing their exponential matrix-vector multiplications for saving in storages. Overall, the array representation along with integration factor methods provides an efficient approach for solving a wide range of problems arising from biological and physical applications.

Figure 2.

Numerical solution of system (72) using AcIIF2. Temporal discretization is set by the time step Δt = 1s, and the simulation is ran up to time t = 35s. (a) The distribution of molecular species A and B at t = 35s. (b) The contour plot of initial and final distributions of molecular species A and B. The dotted black line connects the centers of the solutions of the rate equations of system (72). (c) The distribution of molecular species E_A and E_B at t = 35s. (d) The contour plot of initial and final distributions of molecular species E_A and E_B. The dotted black line connects the centers of the solutions of the rate equations of system (72).