An Integration Factor Method for Stochastic and Stiff Reaction-Diffusion Systems

Abstract

Stochastic effects are often present in the biochemical systems involving reactions and diffusions. When the reactions are stiff, existing numerical methods for stochastic reaction diffusion equations require either very small time steps for any explicit schemes or solving large nonlinear systems at each time step for the implicit schemes. Here we present a class of semi-implicit integration factor methods that treat the diffusion term exactly and reaction implicitly for a system of stochastic reaction-diffusion equations. Our linear stability analysis shows the advantage of such methods for both small and large amplitudes of noise. Direct use of the method to solving several linear and nonlinear stochastic reaction-diffusion equations demonstrates good accuracy, efficiency, and stability properties. This new class of methods, which are easy to implement, will have broader applications in solving stochastic reaction-diffusion equations arising from models in biology and physical sciences.

1. Introduction

Complex patterns can be extensively found in nature, from the skin of zebrafish to the disposition of feather buds in chicks and hair follicles in mice. Often, those patterns are created by biochemical reactions along with diffusions of the molecules in a cellular or multi-cellular systems [1]. Such biological systems, which may be described in reaction-diffusion equations, are constantly subjected to stochastic effects such as noises and environmental perturbations. The stochastic effects on the biochemical reactions at the single-cell level can result in heterogeneous responses of cellular populations and influence their behaviors [2]. Previous studies on stochasticity reveal the adaptation of biological systems to noise, which can be characterized by the systems’ strategies to combat noise, whether by attenuating or exploiting it [2] [3]. For example, spatial stochastic effects help to either prompt the tight localization of proteins or enhance the response to the directional change of a moving pheromone input, resulting in a more robust cell polarization [4]; and the boundary of gene expression domains is sharpened as a result of gene-switching prompted by intracellular noise [5]. It has become increasingly important to incorporate these stochastic effects into the reaction-diffusion equations for better understanding of biological systems.

One can describe a biological system in terms of the following stochastic reaction-diffusion equations:

$$\frac{\partial U}{\partial t}=a\frac{{\partial }^{2}U}{\partial x^2}+f(U)+g(U)Ẇ(x,t)$$ (1)

where Ẇ (x, t) is a standard two-dimensional Wiener process.

One typical way of solving Eq.(1) numerically is to apply central difference first to the diffusion and then use the temporal explicit scheme to solve the subsequent system [6], such as using the explicit Euler method [7] or the two, three, and four-stage explicit Runge Kutta schemes for the system containing additive noise and one-dimensional Wiener process in [8]. Another common approach is using the Galerkin projection of the stochastic partial differential equation (SPDE) and then applying the numerical scheme to a finite-dimensional version of the SPDE. For example, Exponential Time Differencing (ETD) scheme may be applied to the Galerkin projection of the SPDE [9] [10]. One other such example is the Lord and Rougemont scheme, which is derived through Galerkin projection and an integrating factor approach [11]. This scheme is most effective for SPDE with Gevrey regularity, and more improvement may be made on such scheme by taking advantage of the Itô-Taylor expansion [11].

While explicit temporal schemes may be directly implemented for various spatial discretization, including finite element and Galerkin methods [12], to deal with the stability constraint associated with the diffusions, one can treat the diffusion term implicitly, while treating other terms explicitly [6] such as implicit Euler and Crank-Nicolson schemes [7]. Higher order methods [13] can be achieved using Galerkin projection and the linear-implicit versions of strong Taylor schemes [10]. Non-uniform time discretization on Brownian motion can also be obtained for implicit Euler scheme [14].

Stochastic stiffness arises from large differences in the magnitudes of Lyapunov exponents [15], resulting in the presence of different time scales. As in the deterministic case, explicit methods face step-size constraint when used to solve stiff stochastic systems [16]. The time-step constraint can be improved with the modification of the stochastic term by adding more terms from the Itô-Taylor expansion for higher order of accuracy and stability. One of the most well-known schemes stemmed from this construction is the Milstein scheme [15]. Treating the stochastic term implicitly is also one of the popular approaches [17] [18] [19] [20], albeit computationally expensive. Hence, a class of explicit methods known as Chebyshev methods are derived, which have better mean-square stability than explicit Euler method and are not as computationally expensive as implicit methods [21]. A combination of different numerical schemes into one method can also be seen, such as the case of the Composite Euler scheme [22]. For this scheme, at each temporal step, the stochastic differential equation is either solved by implicit Euler method or semi-implicit Euler method. The Composite method has similar order of convergence 1/2 to the Euler Maruyama method but better stability.

Most of the methods mentioned so far are derived to combat the stochastic stiffness through the improvement of the stochastic term, which can be costly and not as effective if the stiffness only occurs in the deterministic term. In such case, methods that treat the deterministic term implicitly while keeping an explicit treatment of the stochastic term are preferred [18]. Here, we propose a new approach to the problem of stiffness caused by the deterministic term, more specifically the reaction term in Eq.(1). The approach is based on the semi-Implicit Integration Factor (IIF) methods [23] [24] [25] [26], which has been found to be effective for the stiff reaction-diffusion equations with better stability constraints imposed on the time steps associated with both reaction and diffusion. In this approach, the time-step constraint for the diffusion term arising due to the inverse of the eigenvalues of the diffusion matrix, which can be large in magnitude, is resolved by treating the linear diffusion term exactly using Integration Factor (IF) methods. Such treatment results in an exponential function of the diffusion term and an integral of the nonlinear reaction term, which is then treated using implicit approach through the Lagrange interpolation to deal with its stiffness. Appropriate choices of approximation schemes lead to decoupling on the treatment for the diffusions and reactions such that one only needs to solve nonlinear systems with the size of the original PDEs. The IIF methods also have exceptional stability properties and its second-order version is absolutely stable. For higher-dimensional problems, the compact IIF (cIIF) method [24] is a great improvement on computational efficiency without altering the stability properties of the IIF methods [23].

In this paper, we exploit the simple structure of the IIF methods as well as their desirable stability properties and efficiency for solving the system in Eq.(1). Because of the nice decoupling properties in the IIF method, we will treat the deterministic diffusion and reaction terms in a similar fashion [23], while dealing with the stochastic term explicitly as in the Euler Maruyama method [27]. We compare this approach with similarly constructed schemes whose main difference is in the treatment of the deterministic part of the equation, which can be approximated using ETD, Crank-Nicolson, or Implicit Euler methods. When all of the properties such as order of accuracy, mean errors, and stability region are taken into consideration, the new approach shows many advantages. We also take advantage of the low computational cost of the cIIF methods to similarly construct a stochastic method that can be applied to higher-dimensional problems. The paper is organized as followed. We first present the construction of the method for systems with additive noise or multiplicative noise, along with linear stability analysis and their comparisons with several other methods. Next, we compare the new method with other methods on linear SODEs and SPDEs on their orders of accuracy and stability constraints. Then, we use this approach to study a nonlinear activator-substrate system of two diffusion species and lastly, make our conclusion.

2. Implicit Integration Factor methods

2.1. Construction of General Method

We consider the stiff reaction-diffusion equation with spatial white noise below:

$$\frac{\partial U}{\partial t}=a\frac{{\partial }^{2}U}{\partial x^2}+f(U)+g(U)\frac{{\partial }^{2}W}{\partial x\partial t}$$ (2)

where a∂²U/∂x² is the diffusion term and a is a nonnegative constant, f(U) is the reaction term, and g(U)(∂²W/∂x∂t) is the noise term of two possible forms: g(U)(∂²W/∂x∂t) = σ(∂²W/∂x∂t) for additive noise or g(U)(∂²W/∂x∂t) = σU(∂²W/∂x∂t) for multiplicative noise. Here, σ is a constant to describe the level of noise. Also, ∂²W/∂x∂t denotes the mixed second-order derivative of the Brownian sheet. A one-dimensional Brownian sheet is a 2-parameter, centered Gaussian process B = B(s, t); s, t > 0 whose covariance is given by:

$$𝔼(B(s,t)B(s\prime ,t\prime ))=\text{min}(s,s\prime )\times \text{min}(t,t\prime ),\forall s,s\prime ,t,t\prime \ge 0.$$ (3)

Before discussing the derivation of the numerical methods to solve Eq.(2), we briefly review the Implicit Integration Factor methods (IIF) discussed in [23], which is crucial to the construction of the IIF methods for a stochastic system. Using the semi-discretized form dU/dt = aU + f(U) that is obtained after the discretization of the diffusion operator in space, we multiply both sides of the equation by the integrating factor e^−at and integrating the equation over one time step from t_n to t_n+1 = t_n + Δt to get

$$U(t_{n+1})=U(t_n)e^{a\mathrm{\Delta }t}+e^{\mathrm{a}\mathrm{\Delta }t}\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-a\tau }f(U(t_n+\tau ))d\tau$$ (4)

Using appropriate approximation of the integrands in $\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-a\tau }f(U(t_n+\tau ))d\tau$ one derives r^th-order IIF scheme [23]:

$$U_{n+1}=e^{a\mathrm{\Delta }t}{U}_n+\mathrm{\Delta }t({\alpha }_{n+1}f(U_{n+1})+\sum _{i=0}^{r-2}{\alpha }_{n-i}f(U_{n-1})),$$ (5)

with α_n+1,α_n, α_n−1,…,α_n−r+2 defined as

$${\alpha }_{n-i}=\frac{e^{(i+1)a\mathrm{\Delta }t}}{\mathrm{\Delta }t}{\int }_0^{\mathrm{\Delta }t}\prod _{j=-1,j\ne i}^{r-2}\frac{\tau +j\mathrm{\Delta }t}{(j-i)\mathrm{\Delta }t}d\tau ,-1\le i\le r-2.$$ (6)

Similarly for the stochastic reaction-diffusion systems, we first discretize the space using m points with the spatial interval Δx. Let U_t be a vector whose i^th-entry is the value of the solution to Eq.(2) at the i^th spatial point. A second-order central difference approximation of ∂²U/∂x² in Eq.(2) with periodic boundary condition U(x₀, t) = U(x_f, t) on the SPDE as in [28], where x₀ and x_f indicate the endpoints of the spatial interval, leads to

$$d{U}_t=aM{U}_{t}dt+f(U_t)dt+g(U_t)\frac{d{W}_t}{\sqrt{\mathrm{\Delta }x}}$$ (7)

where

$$M=\frac{1}{{(\mathrm{\Delta }x)}^2}\left(\begin{array}{cccccc}\hfill -2\hfill & \hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill \cdots \hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill -2\hfill & \hfill 1\hfill & \hfill 0\hfill & \hfill \cdots \hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill -2\hfill & \hfill 0\hfill & \hfill \cdots \hfill & \hfill 0\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill \cdots \hfill & \hfill 1\hfill & \hfill -2\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 0\hfill & \hfill \cdots \hfill & \hfill 0\hfill & \hfill 1\hfill & \hfill -2\hfill \end{array}\right).$$ (8)

Let $ḡ(U_t)=g(U_t)/\sqrt{\mathrm{\Delta }x}$ and multiply both sides of this Eq.(7) by the integrating factor e^−aMs. We then have

$$e^{-aMs}d{U}_{t_n+s}=aM{e}^{-aMs}{U}_{t_n+s}ds+e^{-aMs}f(U_{t_n+s})ds+e^{-aMs}ḡ(U_{t_n+s})d{W}_s$$ (9)

$$e^{-aMs}d{U}_{t_n+s}-aM{e}^{-aMs}{U}_{t_n+s}ds=e^{-aMs}f(U_{t_n+s})ds+e^{-aMs}ḡ(U_{t_n+s})d{W}_s$$ (10)

$$\underset{0}{\overset{\mathrm{\Delta }t}{\int }}[e^{-aMs}d{U}_{t_n+s}-aM{e}^{-aMs}{U}_{t_n+s}ds]=\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-aMs}f(U_{t_n+s})ds+\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-aMs}ḡ(U_{t_n+s})d{W}_s$$ (11)

$$\underset{0}{\overset{\mathrm{\Delta }t}{\int }}d(e^{-aMs}{U}_{t_n+s})=\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-aMs}f(U_{t_n+s})ds+\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-aMs}ḡ(U_{t_n+s})d{W}_s$$ (12)

Taking the integral of the left side gives

$$e^{-aM\mathrm{\Delta }t}{U}_{t_n+1}-U_{t_n}=\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-aMs}f(U_{t_n+s})ds+\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-aMs}ḡ(U_{t_n+s})d{W}_s$$ (13)

Letting Δt = t_n+1 − t_n and with some more simplification, the equation above becomes

$$U_{t_{n+1}}=e^{aM\mathrm{\Delta }t}(U_{t_n}+\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-aMs}f(U_{t_n+s})ds+\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-aMs}ḡ(U_{t_n+s})d{W}_s)$$ (14)

All we have left is evaluating the right side of Eq.(14). Observe the following numerical approximation of the noise part of Eq.(14) [27]

$$\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-aMs}ḡ(U_{t_n+s})d{W}_s=ḡ(U_{t_n})(W_{t_{n+1}}-W_{t_n})$$ (15)

To approximate the deterministic part of Eq.(14), i.e $e^{aM\mathrm{\Delta }t}(U_{t_n}+\underset{0}{\overset{\mathrm{\Delta }t}{\int }}{e}^{-aMs}f(U_{t_n+s})ds)$, we apply the IIF strategy using Eqs.(5) and (6). Coupling this evaluation with Eq.(15), Eq.(14) becomes

$$U_{n+1}=e^{aM\mathrm{\Delta }t}{U}_n+\mathrm{\Delta }t({\alpha }_{n+1}f(U_{n+1})+\sum _{i=0}^{r-2}{\alpha }_{n-i}f(U_{n-1}))+e^{aM\mathrm{\Delta }t}ḡ(U_n)\mathrm{\Delta }{W}_n$$ (16)

where ΔW_n = W_tn+1 −W_tn, U_tn = U_n, and α_n+1,α_n, α_n−1,…,α_n−r+2 are as described in Eq.(6).

Let us denote ξ̄_n to be a standard normally-distributed random vector and n to be the indices of the temporal discretization points. We apply the standard Maruyama method to the noise term along with the first order IIF method, denoted as IIF1, or the second order IIF method, denoted as IIF2, to obtain

• IIF1-Maruyama method

  $$U_{n+1}=e^{aM\mathrm{\Delta }t}{U}_n+\mathrm{\Delta }tf(U_{n+1})+e^{aM\mathrm{\Delta }t}g(U_n)\frac{\sqrt{\mathrm{\Delta }t}{\overline{\xi }}_n}{\sqrt{\mathrm{\Delta }x}}$$ (17)
• IIF2-Maruyama method

  $$U_{n+1}=e^{aM\mathrm{\Delta }t}(U_n+\frac{1}{2}\mathrm{\Delta }tf(U_n))+\frac{1}{2}\mathrm{\Delta }tf(U_{n+1})+e^{aM\mathrm{\Delta }t}g(U_n)\frac{\sqrt{\mathrm{\Delta }t}{\overline{\xi }}_n}{\sqrt{\mathrm{\Delta }x}}$$ (18)

When the stochastic integral in Eq.(14) is approximated explicitly as in Eq.(15), the strong order of convergence of the overall scheme is dominated by the root mean-square order of the increments ΔW_n, which is one-half [15]. For this reason, the strong order of convergence for both of our methods will be consistent with those of most other methods with the same approximation of 5 the stochastic term, i.e the Euler Maruyama method. Let us illustrate this through approximating an SODE of the similar form

$$d{U}_t=-a{U}_{t}dt+b{U}_{t}dt+g(U_t)d{W}_t$$ (19)

using IIF1 and IIF2 methods respectively. By using the standard Maruyama approximation on the noise term, we obtain

• IIF1-Maruyama method:

  $$U_{n+1}=e^{-a\mathrm{\Delta }t}{U}_n+b\mathrm{\Delta }t{U}_{n+1}+g(U_n)e^{-a\mathrm{\Delta }t}\mathrm{\Delta }{W}_n$$ (20)
• IIF2-Maruyama method:

  $$U_{n+1}=e^{-a\mathrm{\Delta }t}{U}_n+\frac{1}{2}{e}^{-a\mathrm{\Delta }t}b\mathrm{\Delta }t{U}_n+\frac{1}{2}b\mathrm{\Delta }t{U}_{n+1}+g(U_n)e^{-a\mathrm{\Delta }t}\mathrm{\Delta }{W}_n$$ (21)

When the noise is additive, i.e.

$$g(U_t)=\sigma ,$$ (22)

the strong order of convergence for both methods is one, which is consistent with the strong order of convergence of the Euler Maruyama method [15].

When the noise is multiplicative, i.e.

$$g(U_t)=\sigma U_t$$ (23)

both methods share the same order of convergence of one-half with the Euler Maruyama method [15].

2.2. Stability Analysis

2.2.1. Additive noise

We first analyze the numerical stability of the IIF1-Maruyama and IIF2-Maruyama schemes in Eqs.(20) and (21) when the stochastic differential equation has additive noise Eq.(22).

Following a previous study [15], we define $U_{n_t}^{\delta }$ to be a time discrete approximation of the solution U(t) with maximum step size δ > 0 starting at time t₀ at $U_0^{\delta }$ and ${Ū}_{n_t}^{\delta }$ to be the corresponding approximation starting at ${Ū}_0^{\delta }$. Then $U_{n_t}^{\delta }$ is asymptotically numerically stable for a given stochastic differential equation if for any finite interval [t₀, T] there exists a positive constant Δ_a such that for each ε > 0 and δ ∈ (0, Δ_a) [15]:

$$\underset{|U_0^{\delta }={Ū}_0^{\delta }|\to 0}{\text{lim}}\underset{t_0\le t\le T}{\text{sup}}P(|U_{n_t}^{\delta }-{Ū}_{n_t}^{\delta }|\ge \epsilon )=0$$ (24)

and

$$\underset{|U_0^{\delta }-{Ū}_0^{\delta }|\to 0}{\text{lim}}\underset{T\to \infty }{\text{lim}}P(\underset{t_0\le t\le T}{\text{sup}}|U_{n_t}^{\delta }-{Ū}_{n_t}^{\delta }|\ge \epsilon )=0$$ (25)

with P(A) indicating the probability that event A occurs. We can analyze the asymptotic stability of a numerical stochastic scheme as we do for the A-stability of deterministic differential equations by studying the stability of the following class of complex-valued linear test equations [15]:

$$d{U}_t=\lambda U_{t}dt+d{W}_t$$ (26)

where λ is a complex number with ℝ(λ) < 0 and W is a real-valued standard Wiener process.

Suppose that a numerical scheme with equidistant step size Δt ≡ δ applied to test equation (26) with ℝ(λ) < 0 can be written recursively as:

$$U_{n+1}^{\mathrm{\Delta }t}=G(\lambda \mathrm{\Delta }t)U_n^{\mathrm{\Delta }t}+Z_n^{\mathrm{\Delta }t}$$ (27)

where G is a mapping of complex plane ℂ into itself and $Z_n^{\mathrm{\Delta }t}$ are random variables that do not depend on $U_n^{\mathrm{\Delta }t}$ for n = 0, 1, 2, …, then the set of complex values λΔt satisfying

$$ℝ(\lambda )<0\text{and}|G(\lambda \mathrm{\Delta }t)|<1$$ (28)

is the region of absolute stability of that scheme [15].

Our methods Eqs.(20) and (21) when applied to the linear test equation Eq.(26) are reduced to:

$$U_{n+1}^{\mathrm{\Delta }t}=e^{\lambda \mathrm{\Delta }t}{U}_n^{\mathrm{\Delta }t}+\sigma e^{\lambda \mathrm{\Delta }t}\mathrm{\Delta }{W}_n$$ (29)

Since |G(λΔt)| = |e^λΔt| < 1 for any arbitrarily large value Δt > 0 given ℝ(λ) < 0, we can claim that both the IIF1-Maruyama and IIF2-Maruyama methods are absolutely stable when noise is additive.

2.2.2. Multiplicative Noise

When the noise is multiplicative Eq.(23), we analyze the stability of each method using meansquare stability analysis [15]. A method is mean-square stable if lim_n→∞𝔼(|U_n|²) = 0 where 𝔼(․) denotes the expected value. To apply this technique to evaluating the stability region of both IIF-Maruyama methods aforementioned, we note that we can rewrite each method in the form:

$$U_{n+1}=h(a,b,\sigma ,\mathrm{\Delta }t,\mathrm{\Delta }{W}_n)U_n$$ (30)

Squaring and then taking expectation of both sides of Eq.(30) coupled with the fact that W_t is a standard Wiener process whose increment W(t) −W(s) is normally-distributed with mean 0 and variance t − s, we obtain

$$𝔼{|U_{n+1}|}^2=H(a,b,\sigma ,\mathrm{\Delta }t)𝔼{|U_n|}^2$$ (31)

where H(a, b, σ, Δt) = 𝔼(h(a, b, σ, Δt, W_n))².

Eq.(31) demonstrates that lim_n→∞𝔼(|U_n|²) = 0, i.e, the numerical method is mean-square stable if and only if H(a, b, σ, Δt) < 1 [17].

For the IIF1-Maruyama method, the mean-square stability condition becomes

$$e^{-2a\mathrm{\Delta }t}(1+{\sigma }^2\mathrm{\Delta }t)-{(1-b\mathrm{\Delta }t)}^2<0$$ (32)

For the IIF2-Maruyama method, the mean-square stability condition becomes

$${(2+b\mathrm{\Delta }t)}^2-{(2-b\mathrm{\Delta }t)}^2{e}^{2a\mathrm{\Delta }t}+4{\sigma }^2\mathrm{\Delta }t<0$$ (33)

We plot the stability regions of both IIF-Maruyama schemes on a plane whose axes are aΔt and bΔt in Figure 1 and vary the value of σ²Δt. Note that the stability region in Figure 1 for each method is the region under the respective colored curve. The desired absolute stability region is the region where the diffusion and reaction coefficients are negative. In terms of Eq.(19), this region is described as {(a, b) : a > 0 and b < 0}. In Figure 1A when there is no noise term, both methods are unconditionally stable with respect to this absolute stability region which is the inside of the square with dashed border. From Figure 1B and C, we observe that as the value of σ²Δt increases, the stability region of the IIF2-Maruyama method shrinks at a faster rate than the stability region of the IIF1-Maruyama method, resulting in the IIF1-Maruyama method having a larger stability region when the noise term is large enough. As a result, the IIF1-Maruyama method has a more desirable stability than the IIF2-Maruyama method in the case of more dominant noise.

Figure 1.

The stability regions of both IIF-Maruyama methods described in Eqs.(17) and (18) for multiplicative noise. The stability region lies below the corresponding colored curve. The desired absolute stability region is the region inside the square with dashed-border.

2.2.3. Comparison with other methods in the case of Multiplicative Noise

For the purpose of stability-region comparison, we present three other methods used to solve Eq.(19) and their constructions: The Euler Maruyama method [15] when it is applied to Eq.(19) takes the form

• Euler Maruyama method

  $$U_{n+1}=U_n-a\mathrm{\Delta }t{U}_n+b\mathrm{\Delta }t{U}_n+g(U_n)\mathrm{\Delta }{W}_n$$ (34)

The order of accuracy for this method is 1/2 [15] and the mean-square stability analysis when noise is multiplicative, i.e Eq.(23) gives the following stability condition:

$$-2a\mathrm{\Delta }t+{(a\mathrm{\Delta }t)}^2-2(a\mathrm{\Delta }t)(b\mathrm{\Delta }t)+2b\mathrm{\Delta }t+{(b\mathrm{\Delta }t)}^2+{\sigma }^2\mathrm{\Delta }t<0$$ (35)

The next method is designed in a similar fashion to the construction of the IIF-Maruyama methods with a modification on the approximation of the deterministic integral term of Eq.(14). Direct application of the exponential time differencing method of order 2 on this term leads to

• ETD2-Maruyama method

  $$U_{n+1}=(e^{-a\mathrm{\Delta }t}+\frac{b}{a}(1-e^{-a\mathrm{\Delta }t}))U_n+e^{-a\mathrm{\Delta }t}g(U_n)\mathrm{\Delta }{W}_n$$ (36)

Since the stochastic integral term is approximated explicitly as in the IIF-Maruyama methods, the order of accuracy for the overall scheme is 1/2. Mean-square stability analysis gives the following stability condition for the above method:

$$e^{-2a\mathrm{\Delta }t}+2(e^{-a\mathrm{\Delta }t}-e^{-2a\mathrm{\Delta }t})\frac{b\mathrm{\Delta }t}{a\mathrm{\Delta }t}+{((1-e^{-a\mathrm{\Delta }t})\frac{b\mathrm{\Delta }t}{a\mathrm{\Delta }t})}^2+{\sigma }^2\mathrm{\Delta }t{e}^{-2a\mathrm{\Delta }t}-1<0$$ (37)

The last scheme mentioned here is constructed similarly to the Euler Maruyama method with the exception of the deterministic term being approximated using second-order Runge Kutta method.

• RK2-Maruyama method

  $$U_{n+1}=(1+(b-a)\mathrm{\Delta }t+\frac{1}{2}{(b-a)}^2\mathrm{\Delta }{t}^2)U_n+g(U_n)\mathrm{\Delta }{W}_n$$ (38)

The construction of the RK2-Maruyama scheme also exploits the explicit approximation of the stochastic term as that of the Euler Maruyama scheme, so the order of accuracy for the RK2-Maruyama scheme is still 1/2. The scheme’s mean-square stability condition is:

$$2({(a\mathrm{\Delta }t)}^2+{(b\mathrm{\Delta }t)}^2)+2(b\mathrm{\Delta }t-a\mathrm{\Delta }t)-4(ab)(b\mathrm{\Delta }t)+{(b\mathrm{\Delta }t)}^3-3{(b\mathrm{\Delta }t)}^2(a\mathrm{\Delta }t)+3(b\mathrm{\Delta }t){(a\mathrm{\Delta }t)}^2-{(a\mathrm{\Delta }t)}^3+\frac{1}{4}{(b\mathrm{\Delta }t)}^4-{(b\mathrm{\Delta }t)}^3(a\mathrm{\Delta }t)+\frac{3}{2}{(a\mathrm{\Delta }t)}^2{(b\mathrm{\Delta }t)}^2-(b\mathrm{\Delta }t){(a\mathrm{\Delta }t)}^3+\frac{1}{4}{(a\mathrm{\Delta }t)}^4+{\sigma }^2\mathrm{\Delta }t<0$$ (39)

To illustrate the performance of the IIF-Maruyama schemes in terms of stability analysis in comparison with the above methods, we plot all the stability regions of each method for different values of σ²Δt on a plane whose axes are aΔt and bΔt (Figure 2). In the same figure, the region where unconditional stability is achieved for an ideal method is the region inside the box with dashed boundary. Figure 3 is the enlarged version of Figure 2 so we can observe better the changes in the absolute stability region for each method at different values of σ²Δt.

Figure 2.

Comparison of stability regions of the following methods: IIF1-Maruyama, IIF2-Maruyama, Euler Maruyama, RK2-Maruyama, and ETD2-Maruyama used to solve Eq.(19) with multiplicative noise. The stability region for each method is shaded blue. The ideal absolute stability region is the region inside the dashed box.

Figure 3.

Enlarged version of Figure 2. In this figure, the stability regions are enlarged for the following methods: IIF1-Maruyama, IIF2-Maruyama, and ETD2-Maruyama. The stability region for each method is shaded blue. The ideal absolute stability region is the region inside the dashed box.

In Figure 2A, i.e when there is no noise term, only the IIF-Maruyama methods are unconditionally stable, which is consistent with the stability of the deterministic IIF methods. For very positive values of the diffusion term and very negative values of the reaction terms (in terms of Eq.(19), this means that both a < 0 and b < 0), the Euler Maruyama, RK2-Maruyama, and ETD2-Maruyama methods achieve better stability than the IIF-Maruyama methods, as seen in the bottom left corner of each subplot of Figure 2A. However, the overall size of the absolute stability regions of the IIF-Maruyama methods is still larger than those of the other methods.

With the increasing size of the multiplicative noise term, the stability region for each method starts to shrink. More specifically, the stability regions for the Euler Maruyama and RK2-Maruyama methods start to shrink in width along the line b = −a (i.e along the direction in which the diffusion and reaction terms are equal) and become thin strips in Figure 2B and C. Both of these regions disappear completely in the next plot, i.e Figure 2D when the noise amplitude is high enough.

Meanwhile, the bottom-left corner of the stability region of the ETD2-Maruyama method recedes significantly as the noise amplitude increases, resulting in a greater loss of the absolute stability region than those of the IIF-Maruyama methods. Size comparison of the absolute stability regions from both Figure 2B–D and Figure 3B–D indicates that the stability region of the ETD2-Maruyama shrinks more than those of the IIF-Maruyama methods as σ²Δt = 1 increases from 0 to 1.

From Figure 2B–D, we observe that the IIF-Maruyama methods have the greatest region of absolute stability for any positive values of σ²Δt. Also, from Figure 3B–D, we observe that the stability regions for both IIF-Maruyama methods shrink at a slower rate than those of the other methods. As a result, both methods have the best absolute stability region for large noise amplitude, as demonstrated by Figure 2D, when σ²Δt = 1. In addition, the absolute stability region of the IIF1-Maruyama method shrinks more slowly than the IIF2-Maruyama, as evidenced by Figure 3B–D where the dashed box indicates the ideal absolute stability region. For this reason, the IIF1-Maruyama method achieves the largest absolute stability region for large noise amplitude out of all the methods.

We conclude that at different values of the noise term, the IIF-Maruyama methods outperform the other methods in terms of the region of absolute stability. In particular, at σ²Δt = 1, both the IIF-Maruyama methods have a much greater region of absolute stability than the rest of the methods.

3. Numerical Simulations

First, we compare the two IIF-Maruyama methods with the other methods when they are applied to Eq.(2) for both cases of additive noise and multiplicative noise. Through choosing different values of a, which corresponds to the size of diffusion, and different values of b, which corresponds to the strength and stiffness of reactions, we evaluate the convergence and stability of IIF-Maruyama methods.

3.1. Tests on Stochastic Ordinary Differential Equations

Here, we implement various methods to solve the linear stochastic ODE Eq.(19). Comparisons will be made between the two IIF methods, the Euler Maruyama, and the ETD2-Maruyama methods. The comparisons concern the accuracy of these methods in situations where the degree of stiffness is high or the noise amplitude is great. All the simulations are done over 1000 independent paths with a time frame from 0 to 1 unless specified otherwise. Numerical experiments were needed in order to decide on a sufficiently large number of Brownian paths that will yield the desirable orders of convergence. All the results obtained in this sub-section remain consistent for a greater number of Brownian paths. This notion is confirmed when we increase the number of paths from 1000 to 2000 and subsequently 10000.

3.1.1. Additive Noise

Denote U_Δt to be the solution obtained numerically from using time step Δt. The order of convergence for additive noise is the value γ such that there exists a constant C where

$$𝔼|U_{\mathrm{\Delta }t}-U_{\mathrm{\Delta }t/2}|\le C\mathrm{\Delta }{t}^{\gamma }$$ (40)

for Δt sufficiently small. The order of convergence for the case, in which the explicit solution (e.g to Eq.(19) with additive noise Eq.(22)) is unknown, is estimated by

$$\text{Order of Convergence}=\frac{\text{log}\left(\raisebox{1ex}{$𝔼|U_{\mathrm{\Delta }t}-U_{\mathrm{\Delta }t/2}|$}\!\left/ \!\raisebox{-1ex}{$𝔼|U_{\mathrm{\Delta }t/2}-U_{\mathrm{\Delta }t/4}|$}\right.\right)}{\text{log}2}.$$ (41)

In our simulations, we start with Δt = 2⁻⁵ and decrease Δt by half for a total of 6 times. 1000 independent Brownian paths are generated and the final solution U_Δt on each path for each time step Δt is calculated.

Next, we study the accuracy and stablity for both IIF-Maruyama methods and compare them with the Euler Maruyama and ETD2-Maruyama methods in different scenarios, especially, in the case in which the reaction term is dominant and the system becomes stiff.

In Figure 4, we plot all the mean errors of the numerical solutions obtained from the IIF-Maruyama, Euler Maruyama, and ETD2-Maruyama methods while using different time steps in the scenario where the reaction term is heavily stiff, i.e when the magnitude of the reaction term is relatively large compared to the magnitudes of the diffusion and noise terms. Here, the mean error is defined to be 𝔼|U_Δt − U_Δt/2| where U_Δt is the numerical solution resulted from using each of the above methods with the time step Δt. From this figure, we observe that both IIF-Maruyama methods maintain a low mean error as the time step Δt increases in size. When Δt becomes too large, i.e when Δt = 1/4, the mean errors of the solutions from using Euler Maruyama and ETD2-Maruyama methods explode out of reasonable bounds. Meanwhile, at the same time step, the mean errors of the numerical solutions resulted from the two IIF-Maruyama methods remain consistently small when larger step size Δt is used. This figure demonstrates that the IIF-Maruyama methods are highly effective whenever the reaction-diffusion system has a dominant reaction term.

Figure 4.

Comparison of the mean errors 𝔼|U_Δt −U_Δt/2| of the numerical solutions to Eq.(19) with additive noise obtained from the following methods: IIF1-Maruyama, IIF2-Maruyama, Euler Maruyama, and ETD2-Maruyama, when the reaction term is heavily stiff. The inserted figure shows the mean error comparison between different methods in more detail. The parameter values are as followed: a = 1, b = −10, and σ = 0.1.

Figure 5 shows the orders of convergence for IIF1-Maruyama, IIF2-Maruyama, Euler Maruyama, and ETD2-Maruyama methods in different scenarios. Figure 5A and B represent the scenario where the noise amplitude is great compared to the magnitudes of the diffusion and reaction terms. Figure 5C and D represent the scenario where the magnitude of diffusion term is relatively large compared to those of the reaction and noise terms. In both scenarios, all the methods share an order of convergence of one as expected and no single method outperform the others. Figure 5 shows that both the IIF-Maruyama methods are comparable to other methods in terms of accuracy in the additive-noise case.

Figure 5.

Comparison of the orders of convergence of the following methods: IIF1-Maruyama, IIF2-Maruyama, Euler Maruyama, and ETD2-Maruyama used to solve Eq.(19) with additive noise. Subplots A and B represent the orders of convergence of all the methods in the scenario that the noise amplitude is relatively large compared to the magnitudes of the diffusion and reaction terms, whose values are fixed to be a = 0.1 and b = −1. Subplots C and D represent the scenario where the magnitude of the diffusion term is great compared to those of the reaction and noise terms, whose values are fixed to be σ = 0.1 and b = −1.

3.1.2. Multiplicative Noise

For SODE with multiplicative noise, because the explicit solution for the linear stochastic ODE is known, the strong order of convergence can be estimated by the value γ if there exists a constant C such that:

$$𝔼|U_n-U(\tau )|\le C\mathrm{\Delta }{t}^{\gamma }$$ (42)

for any fixed τ = nΔt ∈ [0, T] where T is the final time and for Δt sufficiently small. Let T = LΔt for some time step Δt [27]. At τ = T, the order of convergence is calculated as the following

$$\text{Order of Convergence}=\frac{\text{log}\left(\raisebox{1ex}{$𝔼|U_{L,\mathrm{\Delta }t}-U(T)|$}\!\left/ \!\raisebox{-1ex}{$𝔼|U_{L,\mathrm{\Delta }t/2}-U(T)|$}\right.\right)}{\text{log}2}$$ (43)

where we denote U_L,Δt to be the numerical solution at t = T using time steps of size Δt each and U_L,Δt/2 the numerical solution at time T using time steps of size Δt/2 each. To acquire the value for 𝔼|U_L −U(T)|, we take the mean of |U_L −U(T)| over 1000 independent Brownian paths, hence we call 𝔼|U_L − U(T)| the mean error of the numerical solution.

First, we test the accuracy of the IIF-Maruyama, Euler Maruyama, and ETD2-Maruyama methods when the magnitude of the reaction term is large and plot the mean errors of the numerical solutions for each method in Figure 6A. The mean error for the multiplicative case is defined to be 𝔼|U_n − U(τ)| from Eq.(42). As in the additive case, when the reaction term is highly dominant, the IIF-Maruyama methods maintain low mean errors even when the time step Δt is relatively large. From this figure, we observe that when Δt is large enough, i.e Δt = 1/8, the mean errors of the solutions obtained from using Euler Maruyama and ETD2-Maruyama methods assume unreasonably large values and these two methods become unstable. At the same time step, both the IIF-Maruyama methods still maintain stability, as evidenced by the reasonable mean errors resulted from the numerical solutions. We conclude that the IIF-Maruyama methods give reliable results in the case where the reaction-diffusion system is highly stiff in the reaction term.

Figure 6.

Comparison of the mean errors and orders of convergence between the following methods: IIF1-Maruyama, IIF2-Maruyama, Euler Maruyama, and ETD2-Maruyama, used to solve Eq.(19) with multiplicative noise. Subplot A is the comparison of the mean errors 𝔼|U_n − U(τ)| of the numerical solutions when the reaction term is heavily stiff. The parameters used here are a = 1, b = −20, and σ = 0.1. In the scenario that the noise amplitude is relatively large compared to the magnitudes of the diffusion and reaction term, all methods display similar orders of convergence, as seen in subplot B. The parameter values for this subplot are a = 0.1, b = −0.02, and σ = 1. Finally, subplots C and D compare the mean errors 𝔼|U_n − U(τ)| of the numerical solutions when the diffusion term is dominant using fixed parameters σ = 0.1 and b = −2. The time span for the simulations in subplot A is one and the time span used in subplots B–D is one-half. For subplots A and D, the inserted images show the mean errors of each method in more detail.

Figure 6B shows that the orders of convergence for the IIF-Maruyama, Euler Maruyama, and ETD2-Maruyama methods are consistently one-half when the noise term is dominant. There is no real advantage of choosing one method over another in this scenario.

When the magnitude of the diffusion term is relatively large compared to those of the reaction and the noise terms, we notice that the mean errors 𝔼|U_n−U(τ)| obtained from the IIF-Maruyama methods take much smaller values than those of the Euler Maruyama and ETD2-Maruyama methods, as evidenced by Figure 6C and D. Although all the methods have relatively small mean errors, the IIF-Maruyama methods have the smallest mean error values and therefore are more accurate than the other methods.

3.2. Tests on Stochastic Partial Differential Equations

Now we apply IIF-Maruyama methods to the following stochastic PDEs and compare the IIF methods with two other methods.

$$\frac{\partial U}{\partial t}=a\frac{{\partial }^{2}U}{\partial x^2}+bU+g(U)\frac{{\partial }^{2}W}{\partial x\partial t}.$$ (44)

where 0 ≤ x ≤ 1 and t ∈ [0, 0.125] along with a periodic boundary condition U(0, t) = U(1, t).

We compare the orders of convergence from solving Eq.(44) in different scenarios among the following schemes: First-Order IIF-Maruyama method (17), Second-Order IIF-Maruyama method (18), Implicit-Euler Maruyama method, and Crank-Nicolson Maruyama method. The Implicit- Euler Maruyama method is constructed in the same way as the Euler Maruyama method with the deterministic term being approximated using the backward-Euler method of order one. The overall order of convergence for this method is 1/2 due to the explicit approximation of the stochastic term as in the case of the Euler Maruyama method. Letting M denote the diffusion matrix after applying finite difference to Eq.(44), the method takes the following form,

• Implicit-Euler Maruyama method

  $$U_{j+1}=U_j+aM\mathrm{\Delta }t{U}_{j+1}+\mathrm{\Delta }tf(U_{j+1})+g(U_j)\frac{\sqrt{\mathrm{\Delta }t}{\overline{\xi }}_j}{\sqrt{\mathrm{\Delta }x}}.$$ (45)

The construction of Crank-Nicolson method is also similar to that of the Euler Maruyama method, with the exception of the approximation of the deterministic term using the Crank-Nicolson method. For the same reason as the implicit-Euler Maruyama method, the overall order of convergence remains 1/2 for this method:

• Crank-Nicolson Maruyama method

  $$U_{j+1}=U_j+\frac{aM\mathrm{\Delta }t}{2}(U_j+U_{j+1})+\frac{1}{2}\mathrm{\Delta }t(f(U_j)+f(U_{j+1}))+g(U_j)\frac{\sqrt{\mathrm{\Delta }t}{\overline{\xi }}_j}{\sqrt{\mathrm{\Delta }x}}.$$ (46)

We do not show the numerical results of the explicit Euler Maruyama scheme Eq.(34) and ETD2-Maruyama scheme Eq.(36) due to their disadvantages in stability and the associated computational cost.

To compute the order of convergence for each scheme mentioned, we use five different values of the number of spatial steps: N₁, N₂,…N₅ where N₁ is a power of 2 and N_i+1 = 2N_i for i = 1, …, 4, and let the time step Δt = 1/(4N_i). The solutions are numerically calculated over the time frame [t₀, t_f] and generated over m different realizations of the Brownian sheet W(t, x). More information on how the Brownian sheet is generated can be found in [28]. We record the difference at the final time between two solutions obtained using N_i and N_i+1 spatial steps and store this difference under the variable S_i for i = 1, …4. The difference is the sum over m realizations of the sum of squared differences of the approximated solutions over N₁ spatial points, which are common to all solutions. Therefore, we obtain

$$S_i=\sum _{j=1}^m\sum _{k=1}^{N_1}{(U_{j,k}^i-U_{j,k}^{i+1})}^2,$$ (47)

which offers a mean to calculate the numerical error of the scheme. Note that $U_{j,k}^i$ indicates the approximated solution at space step x_k = k/N₁ and at final time, which is obtained from using N_i spatial steps and j^th independent realization of the Brownian sheet [7]. Then the order of convergence can be estimated by log₂ R where the ratio R = S_i/S_i+1. When using this method of computing the order of convergence, both Implicit-Euler Maruyama and Crank-Nicolson Maruyama schemes converge with an order of 1/2 for both additive and multiplicative noises [7]. As a result, we will use 1/2 as the standard value of the order of convergence in the subsequent numerical comparisons. In this sub-section, we fix m = 100. From our experimentation, increasing the value of m has no effects on the orders of the convergence of each scheme. However, the values of ${\{S_i\}}_{i=1}^4$ will increase since these quantities depend on the value of m. In our tests, when m = 500, the values of ${\{S_i\}}_{i=1}^4$ are roughly five times larger than those obtained with m = 100. Similarly, if we increase m to 1000, ${\{S_i\}}_{i=1}^4$ are about ten times larger than their corresponding values when m = 100.

To compare the orders of convergence, we observe the following scenarios with both noises: how the order of convergence for each scheme is affected when the degree of stiffness increases, and whether each method still performs satisfactorily with large noise amplitude.

3.2.1. Stiff reaction

Figure 7A and B plots the values ${\{S_i\}}_{i=1}^4$ obtained from the numerical solutions of Eq.(44) with both additive and multiplicative noises versus the size of the space step when the reaction term is stiff with respect to the diffusion and noise terms. The numerical methods that are applied here are the IIF1-Maruyama, IIF2-Maruyama, Implicit-Euler Maruyama, and Crank-Nicolson Maruyama methods. For both types of noise, all methods converge with a rate of approximately 1/2. The Implicit-Euler Maruyama scheme along with the two IIF-Maruyama schemes have an advantage over the Crank-Nicolson Maruyama scheme when the values ${\{S_i\}}_{i=1}^4$ are considered.

Figure 7.

Comparison of the values ${\{S_i\}}_{i=1}^4$ from Eq.(47) and orders of convergence of the following methods: IIF1-Maruyama, IIF2-Maruyama, Implicit-Euler Maruyama, and Crank-Nicolson Maruyama used to solve Eq.(44) with both additive and multiplicative noises in the scenario where the reaction term is heavily stiff. Subplots A and B show the plots of ${\{S_i\}}_{i=1}^4$ of all the methods when the reaction term b = −10. Subplots C and D display plots of ${\{S_i\}}_{i=1}^4$ of all the methods when b = −50. For all the subplots, the values of the reaction and noise terms are fixed to be a = 1 and σ = 0.1. In subplots A and B, the time frame is chosen to be t ∈ [0, 0.125] and for the remaining two subplots, t ∈ [0, 0.025]. Also, each plot contains the reference line of slope one-half for the purpose of order of convergence comparison.

In the interest of demonstrating the effectiveness of the IIF-Maruyama methods, we compare the orders of convergence between all of the aforementioned methods Eqs.(17), (18), (45), and (46) when the reaction term is extremely stiff for both additive and multiplicative noises. When noise is additive, all methods have an order of convergence of 1/2. However, the Implicit-Euler Maruyama and the two IIF-Maruyama methods perform much better than the Crank-Nicolson Maruyama method in terms of the mean errors ${\{S_i\}}_{i=1}^4$, as seen in Figure 7C. In Figure 7D, for multiplicative noise, the Implicit-Euler Maruyama and the IIF1-Maruyama methods converge at a much faster rate than 1/2 while the Crank-Nicolson Maruyama and the IIF2-Maruyama schemes maintain the 1/2 order of convergence. When using fewer number of space steps, the values S_i obtained from the Implicit-Euler Maruyama and IIF1-Maruyama methods are not as good as those of the other methods due to the large convergence rate. For example, when the spatial step sizes are 1/64, 1/128, and 1/256, the values S₁ and S₂ obtained from the IIF1-Maruyama and Implicit-Euler Maruyama methods are larger than those of the IIF2-Maruyama and Crank-Nicolson Maruyama methods. On the other hand, the IIF2-Maruyama method consistently has the smallest values for ${\{S_i\}}_{i=1}^4$, making it the most desirable method for solving a stochastic partial differential equation with an extremely stiff reaction term whose noise term can be either additive or multiplicative.

3.2.2. Strong diffusion

We obtain the numerical solutions for Eq.(44) using the four methods when the diffusion coefficient is large. Then we compare the values ${\{S_i\}}_{i=1}^4$ from Eq.(47) and the orders of convergence of all methods by plotting ${\{S_i\}}_{i=1}^4$ versus the size of the space step along with a reference line of slope one-half in Figure 8A and B. The Crank-Nicolson Maruyama method is slightly more stable than the other methods for some large values of spatial step size but does not maintain this stability if the space step assumes a larger value than those shown in this figure. Both the IIF-Maruyama methods and the Implicit-Euler Maruyama method achieve the best values for ${\{S_i\}}_{i=1}^4$, while the Crank-Nicolson Maruyama method has significantly larger ${\{S_i\}}_{i=1}^4$ compared to them. When the spatial step size assumes a small-enough value, the IIF-Maruyama schemes and the Implicit-Euler Maruyama schemes have comparable orders of convergence with that of the Crank-Nicolson Maruyama scheme. With both the mean errors and orders of convergence taken into consideration, it is more advantageous to choose the IIF-Maruyama methods and the Implicit-Euler Maruyama method over the Crank-Nicolson Maruyama scheme.

Figure 8.

Comparison of the values ${\{S_i\}}_{i=1}^4$ from Eq.(47) and orders of convergence of the following methods: IIF1-Maruyama, IIF2-Maruyama, Implicit-Euler Maruyama, and Crank-Nicolson Maruyama used to solve Eq.(44). Subplots A and B are the plots of ${\{S_i\}}_{i=1}^4$ of all the methods when the diffusion term is stiff. The values of the diffusion, reaction, and noise terms are fixed to be a = 20, b = −1, and σ = 0.1 for all subplots. Subplots C and D display plots of ${\{S_i\}}_{i=1}^4$ of all the methods when the noise term assumes a large value. For these subplots, the values of the diffusion, reaction, and noise terms are fixed to be a = 2, b = −1, and σ = 1. In this figure, all the simulations are run for 0.125 time units. Also, each plot contains the reference line of slope one-half for the purpose of order of convergence comparison.

3.2.3. Large noise amplitude

Figure 8C and D contains similar plots to Figure 8A and B in the case where Eq.(44) has a large noise term. For both additive and multiplicative noises, all methods have an order of convergence of one-half. Also, for all methods, the values of ${\{S_i\}}_{i=1}^4$ are slightly larger than the corresponding values obtained when the reaction term or diffusion term is stiff. Since the calculation of ${\{S_i\}}_{i=1}^4$ contains a double sum, the magnitude of ${\{S_i\}}_{i=1}^4$ could become quite large. Taking this into consideration, when the noise term has large magnitude, the IIF-Maruyama methods and the Implicit-Euler Maruyama method outperform the Crank-Nicolson Maruyama method significantly in terms of mean errors and thus are preferred.

4. A Turing patterning system with noise

4.1. 1-dimensional activator-substrate system

Finally we apply the IIF2-Maruyama scheme to a Turing patterning system that contains noise to study how noise may affect the formation of patterns. We use the activator-substrate system as an example, whose non-dimensional form is as followed [29] [30]:

$$\frac{\partial A}{\partial t}=D\frac{{\partial }^{2}A}{\partial x^2}+S{A}^2-A+\rho$$ (48)

$$\frac{\partial S}{\partial t}=\frac{{\partial }^{2}S}{\partial x^2}+\mu (1-S{A}^2).$$ (49)

The boundary conditions are no-flux. The constant D measures the diffusion coefficient ratio of activator to substrate. The parameters ρ and μ measures the production rates of activator and substrate, respectively. This system has known homogeneous solutions:

$$A^{*}=1+\rho ,S^{*}={(1+\rho )}^{-2}$$ (50)

One of the features of the activator-susbtrate system Eqs.(48) and (49) is that the substrate S can be consumed during the autocatalysis of activator A and the interactions between the activator and substrate of this system lead to the formation of spatially inhomogeneous patterns [29] [30]. Without noise, the inhomogeneous steady state patterns may be obtained in the parameter ranges

$$\rho \in (0,1),\mu \in (\frac{2}{1+\rho }-1,2),D\in (0.001,\frac{1}{\mu }(\sqrt{\frac{2}{1+\rho }}-1\left)\right).$$ (51)

The initial guesses for the pattern (A₀, S₀) are chosen as [29]

$$A_0=A^{*}(1+\gamma {\delta }_A(x)),S_0=S^{*}(1+\gamma {\delta }_S(x))$$ (52)

where δ_A(x) and δ_S(x) are standard normally-distributed random variables with zero mean and variance one. Different patterns can arise from a slight variations in initial conditions [31]. If we choose the following parameters,

$$t\in [0,101],x\in (0,10),\rho =0.01,\mu =1,D=0.1,\gamma =0.02,dx=10/2^7,dt=dx/4.$$ (53)

the different permutations of our initial conditions in Eq.(52) result in six distinct combinations of inhomogeneous steady state patterns for solutions A and S, similar to those in Figure 9.

Figure 9.

The six different combinations of steady-state patterns for long-term solutions A and S to the one-dimensional stochastic system in Eqs.(54) and (55). The values for the noise coefficients are ε_S = 0.01 and ε_A = 0.005.

Now, we add multiplicative noise to the Eqs.(48) (49) to obtain:

$$\frac{\partial A}{\partial t}=D\frac{{\partial }^{2}A}{\partial x^2}+S{A}^2-A+\rho +{\epsilon }_{A}A\frac{{\partial }^{2}W}{\partial x\partial t},$$ (54)

$$\frac{\partial S}{\partial t}=\frac{{\partial }^{2}S}{\partial x^2}+\mu (1-S{A}^2)+{\epsilon }_{S}S\frac{{\partial }^{2}W}{\partial x\partial t}.$$ (55)

4.1.1. Results

We implement the IIF2-Maruyama method described in Eq.(16) to Eqs.(54) and (55) and examine the changes to the deterministic steady-state patterns when a multiplicative noise is added to both Eqs.(48) and (49). For our implementation, the parameters from Eq.(53) are selected.

We want the time span for our simulations to be sufficiently long so that we can observe the long term behaviors of the solutions A and S. More specifically, in our cases, we want to see whether the long-term solutions assume any patterns observed in their deterministic steady-state counterparts. Unlike the previous study for the deterministic equations [29] in which fluctuations in the initial conditions are critically important in generating the patterns, we fix the initial conditions A₀ = A^* and S₀ = S^* defined in Eq.(50).

Interestingly, we obtain similar patterns in spite of the uniform initial conditions when the relatively small values to the noise coefficients, i.e ε_A = 0.005 and ε_S = 0.01, are given. The six different combinations of patterns for the long time solutions A and S that exist can be seen in Figure 9. We note that these six different combinations of patterns are the same inhomogeneous steady state patterns obtained from solving the deterministic equations Eqs.(48) and (49) [29]. Hence, adding multiplicative noise to the activator-substrate system enables us to obtain the inhomogeneous steady-state patterns that are otherwise obtained through the fluctuations of the initial conditions, as previously predicted [31].

Because each stochastic solution may reach a different steady state even with the same initial conditions in a deterministic form, we perform 100, 500, and 1000 simulations to see which combination of patterns shows up more frequently. In Table 1, we fix the initial conditions as in Eq.(50) and choose the spatial step size to be dx = 10/2⁶ and dx = 10/2⁷. This change in dx does not affect the frequency of presence of each combination of patterns. For Table 2, the values of the initial conditions are randomly permuted as in Eq.(52) for each independent path, while dx is fixed at 10/2⁷. From these two tables, we see that the first combination of patterns is consistently the most favored type of patterns, with a > 30% chance of occurrence, with the second combination of patterns being the second most typical combination of patterns. In addition, the frequency of appearance of each type of patterns is independent of the effects of extra fluctuations on the initial conditions, as evidenced by Table 2. This implies that the first combination of patterns is likely to make up the standard type of patterns that the long-term activator and substrate solutions are supposed to assume. The lack of robustness in pattern formation of the activator and substrate levels can be improved by adding growth factor to the system, in particular apical growth in the case of intrinsic noise [31]. As the domain grows, the space between the activated regions (characterized by the activator maxima or the substrate minima) is enlarged while the substrate concentration is quickly diffusing and increasing. This increase in substrate prompts a higher production of the activator at the side of the maxima in comparison to its center, resulting in the movement of the activator maxima to regions with higher substrate concentration [30]. Attributing the appropriate type of growth to the system can help stabilize the pattern formation over time where the robust patterns for both activator and substrate levels are the first combination of patterns.

Table 1.

Percentage of occurrence of each combination patterns for 100, 500, and 1000 different simulations. The initial conditions are uniformly fixed as in Eq.(50).

	Percentage of appearance
Spatial step size	dx = 10/26			dx = 10/27
No. simulations	100	500	1000	100	500	1000
Pattern 1	35%	33.4%	34.4%	39%	35.8%	36.7%
Pattern 2	28%	26.4%	29%	25%	22.8%	26.1%
Patterns 3 and 5	23%	19.8%	18.9%	15%	20.2%	18.8%
Patterns 4 and 6	14%	20.4%	17.7%	21%	21.2%	18.4%

Table 2.

Percentage of occurrence of each combination patterns for 100, 500, and 1000 different simulations. In this table, dx = 2⁻⁷ and the initial conditions are permuted as in Eq.(52).

	Percentage of appearance
No. simulations	100	500	1000
Pattern 1	41%	40.2%	37%
Pattern 2	30%	26.4%	26.7%
Patterns 3 and 5	15%	15.2%	18.1%
Patterns 4 and 6	14%	18.2%	18.2%

4.1.2. Computational efficiency

Next, we discuss the computational efficiency of the two IIF-Maruyama methods used to solve the Activator-Substrate system with multiplicative noise, Eqs.(54) and (55), by comparing their performances with that of an explicit method, which we choose to be the Euler Maruyama method. We keep all the parameter values as described previously Eq.(53) with the exception of time, which is changed to t ∈ [0, 1]. In Table 3, the mean errors, the order of convergence, and the computational time for each method are recorded. To calculate the mean errors and the order of convergence, we carry out 100 different simulations and apply Eq.(47). We choose dt = dx/2 for the IIF1-Maruyama method, dt = dx for the IIF2-Maruyama method, and dt = (dx)²/3 for the Euler Maruyama method to ensure convergence. In this table, we denote N to be the number of spatial grid points that partition the interval (0,10). Besides N = 2³, all methods display similar orders of convergence and mean errors of similar magnitude. Next, we discuss the computational time in seconds of each method. Each time listed in our table is the total time each method takes to compute the solution over 100 different Brownian paths. For the computational time of both IIF-Maruyama methods, we include the time it takes to calculate the exponential matrix using the Matlab function expm. We observe that with coarser spatial grids, i.e when N = 2³ and N = 2⁴, the Euler Maruyama method surpasses both IIF-Maruyama methods in terms of computational effort. With finer grids, i.e N = 2⁶ and 2⁷, it takes the Euler Maruyama method twice as long as it takes the IIF2-Maruyama method in computing the solutions over 100 Brownian paths. Similarly, the IIF1-Maruyama method is put at a disadvantage with respect to computational speed when the grids are coarse but quickly catches up to the Euler Maruyama method with refined grids. We note that the second-order IIF-Maruyama method is more efficient than the first-order IIFMaruyama method. In addition, the second-order IIF-Maruyama method catches up to the Euler Maruyama method much faster in improving its computational speed, which is demonstrated by the similar speeds between these two methods when N = 2⁴. Meanwhile, the computational speed of the IIF1-Maruyama method does not catch up to that of the Euler Maruyama method until N = 2⁵. In short, due to the restriction of the temporal step size that is required to maintain numerical stability, the Euler Maruyama scheme is less computationally efficient than the IIF1-Maruyama and the IIF2-Maruyama schemes when a finer spatial grid is required. Between the two IIF-Maruyama methods, the second-order IIF-Maruyama method is more desirable for its computational efficiency than the first-order IIF-Maruyama method.

Table 3.

The mean errors S defined in Eq.(47), orders of convergence, and computational cost when solving the activator-substrate system with noise described in Eqs.(54) and (55) obtained by each method: IIF1-Maruyama, IIF2-Maruyama, and Euler Maruyama. The results are computed over 100 Brownian paths with N denoting the number of spatial grid points that partition the interval (0,10).

	IIF1-Maruyama			IIF2-Maruyama			Euler Maruyama
	dt = dx/2			dt = dx			dt = (dx)2/3
N	S	Order	Time(s)	S	Order	Time(s)	S	Order	Time(s)
23	3.52e-2	2.36	0.158	1.27e-2	0.61	0.014	1.41e-2	0.28	0.009
24	6.88e-3	0.65	0.154	8.35e-3	0.62	0.101	1.02e-2	0.47	0.051
25	4.38e-3	0.37	0.233	5.42e-3	0.69	0.194	6.74e-3	0.60	0.230
26	3.38e-3	0.93	0.614	3.35e-3	0.86	0.481	3.70e-3	0.86	1.093
27	1.77e-3	0.58	2.019	1.84e-3	0.68	1.208	2.14e-3	0.79	4.431

4.2. 2-dimensional activator-substrate system

For two or three-dimensional systems, direct application of IIF is costly because the storage and computation of the exponential matrix e^aΔt in the IIF methods may become very large. Similarly to solving the deterministic systems in two or three dimensions, here we use the compact integration factor methods (cIIF) [24], in which the discretized diffusion operator is represented in a compact form that requires storage only proportional to the number of unknowns instead of the square of the number of unknowns in the case of non-compact IIF methods for the exponentials of matrices. cIIF methods can be combined with the Maruyama method in the same manner as the integration of the IIF methods with the Maruyama method, and both the cIIF and the IIF methods share the same desirable stability properties [24].

Here we construct the cIIF-Maruyama methods by estimating the deterministic diffusion and reaction terms using the cIIF methods and the stochastic term using the explicit Maruyama approximation. To demonstrate the efficiency of the cIIF-Maruyama methods, we apply the cIIF2-Maruyama method to the two-dimensional version of the activator-substrate system with no-flux boundary conditions Eqs.(54) and (55) presented in the previous section. Similarly to the one-dimensional case, we compute the solutions over the space (0, 10) × (0, 10) using the steady states in Eq.(50) as the initial conditions. The time window for the simulation is set to be t ∈ [0, 200] and dt = dx where dx = dy = 2⁻⁷ × 10. The rest of the parameter values in Eq.(53) remain the same. Figure 10 displays the contour plot of one of the final patterns of the solutions A and S.

Figure 10.

One of the final patterns obtained for long-term solutions A and S when solving the two-dimensional stochastic system in Eqs.(54) and (55). The values for the noise coefficients are ε_S = 0.01 and ε_A = 0.005.

The computational time it takes for the cIIF2-Maruyama method to compute one stochastic solution to the two-dimensional activator-substrate system is 126.789 seconds for this case. If we use the Euler-Maruyama method to solve this system using the same spatial step size, it takes 206.740 seconds due to the restriction on the temporal step size, which we set to be dt = (dx)²/5. In summary, due to the stability property, the cIIF-Maruyama methods are as efficient in solving two dimensional stochastic reaction-diffusion systems as the one-dimensional systems.

5. Conclusions and Discussion

By convention, stochastic stiffness is defined to be the result of the different time scales caused by the large discrepancies in the magnitudes of the Lyapunov exponents [15] [18]. For that reason, stochastic stiffness can occur in the deterministic term, stochastic term, or both. When solving a stochastic differential equation, the problem of stiffness that stems from the stochastic term has been studied previously [15] [16] [17] [18] [19] [20] [21] [22]. Here, we have focused on the treatment of stiffness of the reaction term for a stochastic reaction-diffusion system. By taking advantage of the existing semi-implicit integrating factor method that is both computationally efficient and absolutely stable at solving stiff deterministic reaction-diffusion systems, we have developed a new class of temporal schemes for reaction-diffusion systems with both additive and multiplicative noises. Similarly to the deterministic case, the new numerical schemes presented remove the restriction imposed on the temporal step size by the linear diffusion term by treating this term exactly while dealing with the stiff reaction term through an implicit approximation. Numerical comparisons show that the construction using the IIF technique to approximate the deterministic term allows the new methods to achieve better stability and good efficiency. While the explicit treatment of the diffusion in IIF naturally leads to good approximations on strong diffusion, the new IIF-Maruyama methods mainly offer an efficient approach to deal with stiff reactions in a reaction-diffusion systems. In general, this method is mostly effective when the reactions are very stiff while diffusion is still important in a stochastic reaction-diffusion system.

The approach used here in combining IIF for reaction and diffusion and Maruyama for the stochastic terms can be adapted in a straightforward fashion for compact IIF (cIIF) methods [24], which is effective for systems in two or three dimensions. With the compact representation for the differential operators, it would be more efficient in simulating 2D and 3D using cIIF than IIF-Maruyama. Another improvement on IIF-Maruyama is its order of convergence.

In the case of multiplicative noise, the order of convergence of the IIF-Maruyama methods is confined to a value of one-half. With the Euler Maruyama method, the order of convergence can be improved by adding more terms from the Ito-Taylor expansion to construct methods with higher order of accuracy such as the Milstein method [15]. Integrating such approach with IIF method might lead to higher order of accuracy with similar stability property of IIF-Maruyama. However, because the diffusion term is not well-defined for the Brownian sheet due to a lack of a well-defined spatial derivative for the Wiener process, a direct application of the Milstein method to stochastic PDEs may only lead to half-order of convergence. To deal with this difficulty, one might need to use the Q-Wiener process instead of the Brownian sheet to approximate the diffusion [32]. Similarly to the IIF methods, which can be used for the deterministic systems in various forms, spatial dimensions, and in combination with other approaches for treating additional terms such as convections, IIF-Maruyama methods may have broad applications in simulating stochastic partial differential equations in various forms and containing stiff reactions.