Entropy Production in Exactly Solvable Systems

Abstract

The rate of entropy production by a stochastic process quantifies how far it is from thermodynamic equilibrium. Equivalently, entropy production captures the degree to which global detailed balance and time-reversal symmetry are broken. Despite abundant references to entropy production in the literature and its many applications in the study of non-equilibrium stochastic particle systems, a comprehensive list of typical examples illustrating the fundamentals of entropy production is lacking. Here, we present a brief, self-contained review of entropy production and calculate it from first principles in a catalogue of exactly solvable setups, encompassing both discrete- and continuous-state Markov processes, as well as single- and multiple-particle systems. The examples covered in this work provide a stepping stone for further studies on entropy production of more complex systems, such as many-particle active matter, as well as a benchmark for the development of alternative mathematical formalisms.

1. Introduction

Stochastic thermodynamics has progressively evolved into an essential tool in the study of non-equilibrium systems as it connects the quantities of interest in traditional thermodynamics, such as work, heat and entropy, to the properties of microscopically resolved fluctuating trajectories [1,2,3]. The possibility of equipping stochastic processes with a consistent thermodynamic and information-theoretic interpretation has resulted in a number of fascinating works, with the interface between mathematical physics and the biological sciences proving to be a particularly fertile ground for new insights (e.g., [4,5,6,7,8]). The fact that most of the applications live on the small scale is not surprising, since it is precisely at the microscopic scale that fluctuations start to play a non-negligible role.

The concept of entropy and, more specifically, entropy production has attracted particular interest, as a consequence of the quantitative handle it provides on the distinction between equilibrium systems, passive systems relaxing to equilibrium and genuinely non-equilibrium, ‘active’ systems. While there exist multiple routes to the mathematical formulation of entropy production [9,10,11,12,13,14], the underlying physical picture is consistent: the entropy production associated with an ensemble of stochastic trajectories quantifies the degree of certainty with which we can assert that a particular event originates from a given stochastic process or from its suitably defined conjugate (usually, its time-reverse). When averaged over long times (or over an ensemble), a non-vanishing entropy production signals time-reversal symmetry breaking at the microscopic scale. This implies, at least for Markovian systems, the existence of steady-state probability currents in the state space, which change sign under time-reversal. When a thermodynamically consistent description is available, the average rate of entropy production can be related to the rate of energy or information exchange between the system, the heat bath(s) it is connected to, and any other thermodynamic entity involved in the dynamics, such as a measuring device [15,16,17]. Whilst the rate of energy dissipation is of immediate interest since it captures how ‘costly’ it is to sustain specific dynamics (e.g., the metabolism sustaining the development of an organism [18,19]), entropy production has also been found to relate non-trivially to the efficiency and precision of the corresponding process via uncertainty relations [3,20]. Entropy production along fluctuating trajectories also plays a fundamental role in the formulation of various fluctuation theorems [12].

Given the recent interest in stochastic thermodynamics and entropy production in particular, as well as the increasing number of mathematical techniques implemented for the quantification of the latter, it is essential to have available a few, well-understood reference systems, for which exact results are known. These can play the role of benchmarks for new techniques, while helping neophytes to develop intuition. This is not to say that exact results for more complicated systems are not available, see for example [21], however they are usually limited to small systems and/or require numerical evaluation. In this work, we will present results exclusively in the framework proposed by Gaspard [11], specifically in the form of Equations (4), (15) and (16), which we review and contextualise by deriving them via different routes in Section 2. In Section 3, we begin the analysis with processes in discrete state space (Section 3.1, Section 3.2, Section 3.3, Section 3.4, Section 3.5, Section 3.6, Section 3.7 and Section 3.8), and subsequently extend it to the continuous case (Section 3.9, Section 3.10 and Section 3.11). Finally, in Section 3.12 and Section 3.13 we consider processes that involve both discrete and continuous degrees of freedom. Time is taken as a continuous variable throughout.

2. Brief Review of Entropy Production

Entropy production of jump processes. The concept of time-dependent informational entropy associated with a given ensemble of stochastic processes was first introduced by Shannon [22]. For an arbitrary probability mass function $P_n\left(t\right)$ of time t over a discrete set of states $n\in \Omega$, the Shannon entropy is defined as

$$S\left(t\right)=-\sum _n{P}_n\left(t\right)ln{P}_n\left(t\right)$$ (1)

with the convention henceforth of $xlnx=0$ for $x=0$. It quantifies the inherent degree of uncertainty about the state of a process. In the microcanonical ensemble $P_n$ is constant in t and n and upon providing an entropy scale in the form of the Boltzmann constant $k_B$, Shannon’s entropy reduces to that of traditional thermodynamics given by Boltzmann’s $S=k_{B}ln|\Omega |$, where $|\Omega |=1/P_n$ is the cardinality of $\Omega$. In Markovian systems, the probability $P_n\left(t\right)$ depends on n and evolves in time t according to the master equation

$${\dot{P}}_n\left(t\right)=\sum _m\left(P_m\left(t\right)w_{mn}-P_n\left(t\right)w_{nm}\right)$$ (2)

with non-negative transition rates $w_{mn}$ from state m to state $n\ne m$. Equation (2) reduces to ${\dot{P}}_n\left(t\right)={\sum }_m{P}_m\left(t\right)w_{mn}$ by imposing the Markov condition ${\sum }_m{w}_{nm}=0$, equivalently $w_{nn}=-{\sum }_{m\ne n}{w}_{nm}$, which we will use in the following. For simplicity, we will restrict ourselves to time-independent rates $w_{nm}$ but as far as the following discussion is concerned, generalising to time-dependent rates is a matter of replacing $w_{nm}$ by $w_{nm}\left(t\right)$. The rate of change of entropy for a continuous time jump process can be derived by differentiating $S\left(t\right)$ in Equation (1) with respect to time and substituting (2) into the resulting expression [11,23], thus obtaining

$$\dot{S}\left(t\right)=-\sum _{m,n}{P}_m\left(t\right)w_{mn}ln\left(P_n\left(t\right)\right)=\sum _{m,n}{P}_n\left(t\right)w_{nm}ln\left(\frac{P_n\left(t\right)}{P_m\left(t\right)}\right)={\dot{S}}_e\left(t\right)+{\dot{S}}_i\left(t\right)$$ (3)

where we define

$$\begin{array}{cc}\hfill {\dot{S}}_e\left(t\right)& =-\frac{1}{2}\sum _{m,n}\left(P_n\left(t\right)w_{nm}-P_m\left(t\right)w_{mn}\right)ln\left(\frac{w_{nm}}{w_{mn}}\right)\hfill \\ \hfill & =-\sum _{m,n}{P}_n\left(t\right)w_{nm}ln\left(\frac{w_{nm}}{w_{mn}}\right)=-\sum _{m,n}\left(P_n\left(t\right)w_{nm}-P_m\left(t\right)w_{mn}\right)ln\left(\frac{w_{nm}}{w_0}\right)\hfill \end{array}$$ (4a)

$$\begin{array}{cc}\hfill {\dot{S}}_i\left(t\right)& =\frac{1}{2}\sum _{m,n}\left(P_n\left(t\right)w_{nm}-P_m\left(t\right)w_{mn}\right)ln\left(\frac{P_n\left(t\right)w_{nm}}{P_m\left(t\right)w_{mn}}\right)\hfill \\ \hfill & =\sum _{m,n}{P}_n\left(t\right)w_{nm}ln\left(\frac{P_n\left(t\right)w_{nm}}{P_m\left(t\right)w_{mn}}\right)=\sum _{m,n}(P_n\left(t\right)w_{nm}-P_m\left(t\right)w_{mn})ln\left(\frac{P_n\left(t\right)w_{nm}}{w_0}\right)\hfill \end{array}$$ (4b)

with arbitrary positive rate $w_0$ to restore dimensional consistency, that cancel trivially. Here we follow the convention [1] to split the rate of entropy change into two contributions: the first, Equation (4a), commonly referred to as “external” entropy production or entropy flow, is denoted by ${\dot{S}}_e$. It contains a factor $ln(w_{nm}/w_{mn})$ corresponding, for systems satisfying local detailed balance, to the net change in entropy of the reservoir(s) associated with the system’s transition from state n to state m. For such thermal systems, ${\dot{S}}_e$ can thus be identified as the rate of entropy production in the environment [9,24]. The second contribution, Equation (4b), termed “internal” entropy production and denoted by ${\dot{S}}_i$ is non-negative because $(x-y)ln(x/y)\ge 0$ for any two real, positive x, y and using the convention $zlnz=0$ for $z=0$. The internal entropy production vanishes when the global detailed balance condition $P_n\left(t\right)w_{nm}=P_m\left(t\right)w_{mn}$ is satisfied for all pairs of states. In this sense, a non-vanishing ${\dot{S}}_i$ is the fingerprint of non-equilibrium phenomena. Defining ${\pi }_n={lim}_{t\to \infty }{P}_n\left(t\right)$ as the probability mass function at steady-state, the internal entropy production rate can be further decomposed into two non-negative contributions, ${\dot{S}}_i\left(t\right)={\dot{S}}_i^{\mathrm{a}}\left(t\right)+{\dot{S}}_i^{\mathrm{na}}\left(t\right)$, of the form

$$\begin{array}{cc}\hfill {\dot{S}}_i^{\mathrm{a}}\left(t\right)& =\sum _{m,n}{P}_n\left(t\right)w_{nm}ln\left(\frac{{\pi }_n{w}_{nm}}{{\pi }_m{w}_{mn}}\right)\hfill \end{array}$$ (5a)

$$\begin{array}{cc}\hfill {\dot{S}}_i^{\mathrm{na}}\left(t\right)& =\sum _{m,n}{P}_n\left(t\right)w_{nm}ln\left(\frac{P_n\left(t\right){\pi }_m}{P_m\left(t\right){\pi }_n}\right)=-\sum _n{\dot{P}}_n\left(t\right)ln\left(\frac{P_n\left(t\right)}{{\pi }_n}\right).\hfill \end{array}$$ (5b)

These contributions are usually referred to as adiabatic (or housekeeping) and non-adiabatic (or excess) entropy production rates, respectively [1]. The non-adiabatic contribution vanishes at steady-state, ${lim}_{t\to \infty }{\dot{S}}_i^{\mathrm{na}}\left(t\right)=0$. While these quantities have received attention in the context of fluctuation theorems [23], they will not be discussed further here. At steady-state, namely when ${\dot{P}}_n\left(t\right)=0$ for all n, $\dot{S}\left(t\right)$ in Equation (3) vanishes by construction, so that the internal and external contributions to the entropy production cancel each other exactly, $\dot{S}\left(t\right)={\dot{S}}_e\left(t\right)+{\dot{S}}_i\left(t\right)=0$, while they vanish individually only for systems at equilibrium. Equation (4) will be used throughout the present work to compute the entropy productions of discrete-state processes.

Entropy production as a measure of time-reversal-symmetry breaking. As it turns out, a deeper connection between internal entropy production and time-reversal symmetry breaking can be established [11]. The result, which we re-derive below, identifies ${\dot{S}}_i$ as the relative dynamical entropy (i.e., the Kullback–Leibler divergence [25]) per unit time of the ensemble of forward paths and their time-reversed counterparts. To see this, we first need to define a path $\mathbf{n}=(n_0,n_1,\dots ,n_M)$ as a set of trajectories starting at time $t_0$ and visiting states $n_j$ at successive discrete times $t_j=t_0+j\tau$ with $j=0,1,\dots ,M$, equally spaced by a time interval $\tau$. For a time-homogeneous Markovian jump process in continuous time, the joint probability of observing a particular path is

$$\mathcal{P}(\mathbf{n};t_0,M\tau )=P_{n_0}\left(t_0\right)W(n_0\to n_1;\tau )W(n_1\to n_2;\tau )\dots W(n_{M-1}\to n_M;\tau )$$ (6)

where $P_{n_0}\left(t_0\right)$ is the probability of observing the system in state $n_0$ at time $t_0$, while $W(n_j\to n_{j+1};\tau )$ is the probability that the system is in state $n_{j+1}$ time $\tau$ after being in state $n_j$. This probability can be expressed in terms of the transition rate matrix w with elements $w_{mn}$. It is $W(n\to m;\tau )={\left[\exp \left(w\tau \right)\right]}_{nm}$, the matrix elements of the exponential of the matrix $w\tau$ with the Markov condition imposed. It can be expanded in small $\tau$ as

$$W(n\to m;\tau )={\delta }_{n,m}+w_{nm}\tau +O\left({\tau }^2\right),$$ (7)

where ${\delta }_{n,m}$ is the Kronecker-$\delta$ function. We can now define a dynamical entropy per unit time [22] as

$$h(t_0,\Delta t)=\underset{M\to \infty }{lim}-\frac{1}{\Delta t}\sum _{n_0,\dots ,n_M}\mathcal{P}(\mathbf{n};t_0,\Delta t)ln\mathcal{P}(\mathbf{n};t_0,\Delta t).$$ (8)

where the limit is to be considered a continuous time limit taken at fixed $\Delta t=t_M-t_0=M\tau$ [26], thus determining the sampling interval $\tau$, and the sum runs over all possible paths $\mathbf{n}$. Other than $\tau$, the paths are the only quantity on the right-hand side of Equation (8) that depend on M. The dynamical entropy $h(t_0,\Delta t)$ may be considered the expectation of $ln\left(\mathcal{P}(\mathbf{n};t_0,\Delta t)\right)$ across all paths. Similarly to the static Shannon entropy, the dynamical entropy $h(t_0,\Delta t)$ quantifies the inherent degree of uncertainty about the evolution over a time $\Delta t$ of a process starting at a given time $t_0$. To compare with the dynamics as observed under time-reversal, one introduces the time-reversed path ${\mathbf{n}}^R=(n_M,n_{M-1},\dots ,n_0)$ and thus the time-reversed dynamical entropy per unit time as

$$h^R(t_0,\Delta t)=\underset{M\to \infty }{lim}-\frac{1}{\Delta t}\sum _{n_0,\dots ,n_M}\mathcal{P}(\mathbf{n};t_0,\Delta t)ln\mathcal{P}({\mathbf{n}}^R;t_0,\Delta t).$$ (9)

While similar in spirit to $h(t_0,\Delta t)$, the physical interpretation of $h^R(t_0,\Delta t)$ as the expectation of $ln\left(\mathcal{P}({\mathbf{n}}^R;\Delta t)\right)$ under the forward probability $\mathcal{P}(\mathbf{n};t_0,\Delta t)$ is more convoluted since it involves the forward and the backward paths simultaneously, which have potentially different statistics. However, time-reversal symmetry implies precisely identical statistics of the two ensembles, whence $h(t_0,\Delta t)=h^R(t_0,\Delta t)$. The motivation for introducing $h^R(t_0,\Delta t)$ is that the difference of the two dynamical entropies defined above is a non-negative Kullback–Leibler divergence given by

$$h^R(t_0,\Delta t)-h(t_0,\Delta t)=\underset{M\to \infty }{lim}\frac{1}{\Delta t}\sum _{\mathbf{n}}\mathcal{P}(\mathbf{n};t_0,\Delta t)ln\left(\frac{\mathcal{P}(\mathbf{n};t_0,\Delta t)}{\mathcal{P}({\mathbf{n}}^R;t_0,\Delta t)}\right).$$ (10)

Using Equation (6) in (10) with Equation (7) provides the expansion

$$h^R(t_0,\Delta t)-h(t_0,\Delta t)=\sum _{nm}{P}_n\left(t_0\right)w_{nm}ln\left(\frac{P_n\left(t_0\right)w_{nm}}{P_m\left(t_0\right)w_{mn}}\right)+\mathcal{O}(\Delta t),$$ (11)

which is an instantaneous measure of the Kullback–Leibler divergence. The limit of $h^R(t_0,\Delta t)-h(t_0,\Delta t)$ in small $\Delta t$ is finite and identical to the internal entropy production (4b) derived above. This result establishes the profound connection between broken global detailed balance, Equation (4), and Kullback–Leibler divergence, Equation (11), both of which can thus be recognised as fingerprints of non-equilibrium systems. In light of this connection, it might not come as a surprise that the steady-state rate of entropy production is inversely proportional to the minimal time needed to decide on the direction of the arrow of time [27].

Entropy production for continuous degrees of freedom. The results above were obtained for Markov jump processes within a discrete state space. However, the decomposition of the rate of change of entropy in Equation (3) into internal and external contributions can be readily generalised to Markovian processes with continuous degrees of freedom, for example a spatial coordinate. For simplicity, we will restrict ourselves to processes in one dimension but as far as the following discussion is concerned, generalising to higher dimensions is a matter of replacing spatial derivatives and integrals over the spatial coordinate with their higher dimensional counterparts. The dynamics of such a process with probability density $P(x,t)$ to find it at x at time t are captured by a Fokker–Planck equation of the form $\dot{P}(x,t)=-{\partial }_{x}j(x,t)$, with j the probability current, augmented by an initial condition $P(x,0)$. Starting from the Gibbs–Shannon entropy for a continuous random variable $S\left(t\right)=-\int \mathrm{d}xP(x,t)ln(P(x,t)/P_0)$ with some arbitrary density scale $P_0$ for dimensional consistency, we differentiate with respect to time and substitute $-{\partial }_{x}j(x,t)$ for $\dot{P}(x,t)$ to obtain

$$\dot{S}\left(t\right)=-\int \mathrm{d}x\dot{P}(x,t)ln\left(\frac{P(x,t)}{P_0}\right)=-\int \mathrm{d}x\frac{\left({\partial }_{x}P(x,t)\right)j(x,t)}{P(x,t)},$$ (12)

where the second equality follows upon integration by parts using $\int \mathrm{d}x\dot{P}(x,t)=0$ by normalisation. For the paradigmatic case of an overdamped colloidal particle, which will be discussed in more detail below (Section 3.9, Section 3.10 and Section 3.11), the probability current is given by $j(x,t)=-D{\partial }_{x}P(x,t)+\mu F(x,t)P(x,t)$ with local, time-dependent force $F(x,t)$. We can then decompose the entropy production $\dot{S}\left(t\right)={\dot{S}}_i\left(t\right)+{\dot{S}}_e\left(t\right)$ into internal and external contributions as

$${\dot{S}}_i\left(t\right)=\int \mathrm{d}x\frac{j{(x,t)}^2}{DP(x,t)}\ge 0$$ (13)

and

$${\dot{S}}_e\left(t\right)=-\int \mathrm{d}x\frac{\mu }{D}F(x,t)j(x,t),$$ (14)

respectively. The Kullback–Leibler divergence between the densities of forward and time-reversed paths can be calculated as outlined above for discrete state systems, thus producing an alternative expression for the internal entropy production in the form

$$\begin{array}{cc}\hfill {\dot{S}}_i\left(t\right)& =\underset{\Delta t\to 0}{lim}{h}^R(t,\Delta t)-h(t,\Delta t)\hfill \\ & =\underset{\tau \to 0}{lim}\frac{1}{2\tau }\int \mathrm{d}x\mathrm{d}{x}^{\prime }(P(x,t)W(x\to x^{\prime },\tau )-P(x^{\prime },t)W(x^{\prime }\to x,\tau ))ln\frac{P(x,t)W(x\to x^{\prime },\tau )}{P(x^{\prime },t)W(x^{\prime }\to x,\tau )}.\hfill \end{array}$$ (15)

Here we have introduced the propagator $W(x^{\prime }\to x,\tau )$, the probability density that a system observed in state $x^{\prime }$ will be found at x time $\tau$ later. In general, here and above, the density $W(x\to x^{\prime },\tau )$ depends on the absolute time t, which we have omitted here for better readability. The corresponding expression for the entropy flow is obtained by substituting (15) into the balance equation ${\dot{S}}_e\left(t\right)=\dot{S}\left(t\right)-{\dot{S}}_i\left(t\right)$, whence

$${\dot{S}}_e\left(t\right)=-\underset{\tau \to 0}{lim}\frac{1}{2\tau }\int \mathrm{d}x\mathrm{d}{x}^{\prime }(P(x,t)W(x\to x^{\prime },\tau )-P(x^{\prime },t)W(x^{\prime }\to x,\tau ))ln\frac{W(x^{\prime }\to x,\tau )}{W(x\to x^{\prime },\tau )}.$$ (16)

Since ${lim}_{\tau \to 0}W(x\to x^{\prime },\tau )=\delta (x-x^{\prime })$ [28] and $P(x,t)\delta (x-x^{\prime })=P(x^{\prime },t)\delta (x^{\prime }-x)$, the factor in front of the logarithm in (15) and (16) vanishes in the limit of small $\tau$, ${lim}_{\tau \to 0}P(x,t)W(x\to x^{\prime };\tau )-P(x^{\prime },t)W(x^{\prime }\to x;\tau )=0$. Together with the prefactor $1/\tau$, this necessitates the use of L’Hôpital’s rule

$$\underset{\tau \to 0}{lim}\frac{1}{\tau }\left(P(x,t)W(x\to x^{\prime };\tau )-P(x^{\prime },t)W(x^{\prime }\to x;\tau )\right)=P(x,t)\dot{W}(x\to x^{\prime })-P(x^{\prime },t)\dot{W}(x^{\prime }\to x)$$ (17)

where we used the shorthand

$$\dot{W}(x\to x^{\prime }):=\underset{\tau \to 0}{lim}\frac{\mathrm{d}}{\mathrm{d}\tau }W(x\to x^{\prime };\tau ),$$ (18)

which is generally given by the Fokker–Planck equation of the process, so that

$$\dot{P}(x,t)=\int \mathrm{d}{x}^{\prime }P(x^{\prime },t)\dot{W}(x^{\prime }\to x).$$ (19)

In the continuum processes considered below, in particular Section 3.11, Section 3.12 and Section 3.13, $\dot{W}(x\to x^{\prime })$ is a kernel in the form of Dirac $\delta$-functions and derivatives thereof, acting under the integral as the adjoint Fokker–Planck operator on $P(x,t)$. With Equation (17) the internal entropy production of a continuous process (15) may conveniently be written as

$$\begin{array}{cc}\hfill {\dot{S}}_i\left(t\right)& =\frac{1}{2}\int \mathrm{d}x\mathrm{d}{x}^{\prime }\left(P(x,t)\dot{W}(x\to x^{\prime })-P(x^{\prime },t)\dot{W}(x^{\prime }\to x)\right)\times \underset{\tau \to 0}{lim}ln\left(\frac{P(x,t)W(x\to x^{\prime };\tau )}{P(x^{\prime },t)W(x^{\prime }\to x;\tau )}\right)\hfill \end{array}$$ (20a)

$$\begin{array}{cc}\hfill & =\int \mathrm{d}x\mathrm{d}{x}^{\prime }P(x,t)\dot{W}(x\to x^{\prime })\times \underset{\tau \to 0}{lim}ln\left(\frac{P(x,t)W(x\to x^{\prime };\tau )}{P(x^{\prime },t)W(x^{\prime }\to x;\tau )}\right)\hfill \end{array}$$ (20b)

$$\begin{array}{cc}\hfill & =\int \mathrm{d}x\mathrm{d}{x}^{\prime }\left(P(x,t)\dot{W}(x\to x^{\prime })-P(x^{\prime },t)\dot{W}(x^{\prime }\to x)\right)\times \underset{\tau \to 0}{lim}ln\left(\frac{P(x,t)W(x\to x^{\prime };\tau )}{W_0{P}_0}\right)\hfill \end{array}$$ (20c)

with suitable constants $W_0$ and $P_0$. Correspondingly, the (external) entropy flow (16) is

$$\begin{array}{cc}\hfill {\dot{S}}_e\left(t\right)& =-\frac{1}{2}\int \mathrm{d}x\mathrm{d}{x}^{\prime }\left(P(x,t)\dot{W}(x\to x^{\prime })-P(x^{\prime },t)\dot{W}(x^{\prime }\to x)\right)\times \underset{\tau \to 0}{lim}ln\left(\frac{W(x\to x^{\prime };\tau )}{W(x^{\prime }\to x;\tau )}\right)\hfill \end{array}$$ (21a)

$$\begin{array}{cc}\hfill & =-\int \mathrm{d}x\mathrm{d}{x}^{\prime }P(x,t)\dot{W}(x\to x^{\prime })\times \underset{\tau \to 0}{lim}ln\left(\frac{W(x\to x^{\prime };\tau )}{W(x^{\prime }\to x;\tau )}\right)\hfill \end{array}$$ (21b)

$$\begin{array}{cc}\hfill & =-\int \mathrm{d}x\mathrm{d}{x}^{\prime }\left(P(x,t)\dot{W}(x\to x^{\prime })-P(x^{\prime },t)\dot{W}(x^{\prime }\to x)\right)\times \underset{\tau \to 0}{lim}ln\left(\frac{W(x\to x^{\prime };\tau )}{W_0}\right).\hfill \end{array}$$ (21c)

All of these expressions assume that the limits of the logarithms exist. Naively replacing them by $ln(\delta (x-x^{\prime })/\delta (x^{\prime }-x))$ produces a meaningless expression with a Dirac $\delta$-function in the denominator. Equations (20) and (21) are identically obtained in the same manner as Equation (4) with the master Equation (2) replaced by the Fokker–Planck Equation (19). All of these expressions, Equations (4), (20) and (21), may thus be seen as Gaspard’s [11] framework.

Langevin description and stochastic entropy. We have seen in Equations (13) and (14) how the notion of entropy production can be extended to continuous degrees of freedom by means of a Fokker–Planck description of the stochastic dynamics. The Fokker–Plank equation is a deterministic equation for the probability density and thus provides a description at the level of ensembles, rather than single fluctuating trajectories. A complementary description can be provided by means of a Langevin equation of motion, which is instead a stochastic differential equation for the continuous degree of freedom [29]. The presence of an explicit noise term, which usually represents faster degrees of freedom or fluctuations induced by the contact with a heat reservoir, allows for a clearer thermodynamic interpretation. A paradigmatic example is that of the overdamped colloidal particle mentioned above, whose dynamics is described by

$$\dot{x}\left(t\right)=\mu F(x,t)+\zeta \left(t\right)$$ (22)

with $\mu$ a mobility, $F(x,t)$ a generic force and $\zeta \left(t\right)$ a white noise term with covariance $\langle \zeta \left(t\right)\zeta \left(t^{\prime }\right)\rangle =2D\delta (t-t^{\prime })$. For one-dimensional motion on the real line, the force $F(x,t)$ can always be written as the gradient of a potential $V(x,t)$, namely $F(x,t)=-{\partial }_{x}V(x,t)$, so that it is conservative. For time-independent, stable potentials, $V(x,t)=V\left(x\right)$, this leads over long timeframes to an equilibrium steady-state. This property does not hold in higher dimensions and for different boundary conditions (e.g., periodic), in which case the force $\mathbf{F}(\mathbf{x},t)$ need not have a corresponding potential $V(\mathbf{x},t)$ for which $\mathbf{F}(\mathbf{x},t)=-\nabla V(\mathbf{x},t)$ [30].

The concept of entropy is traditionally introduced at the level of ensembles. However, due to its role in fluctuation theorems [1,24], a consistent definition at the level of single trajectories is required. This can be constructed along the lines of [12] by positing the trajectory-dependent entropy $S(x^{\ast }\left(t\right),t)$ where $x^{\ast }\left(t\right)$ is a random trajectory as given by Equation (22) and

$$S(x,t)=-ln(P(x,t)/P_0).$$ (23)

Here, $P(x,t)$ denotes the probability density of finding a particle at position x at time t as introduced above and $P_0$ is a scale as used above to maintain dimensional consistency. Given that $x^{\ast }\left(t\right)$ is a random variable, so is $S(x^{\ast }\left(t\right),t)$, which may be regarded as an instantaneous entropy. Taking the total derivative with respect to t produces

$$\begin{array}{cc}\hfill \frac{\mathrm{d}}{\mathrm{d}t}S(x^{\ast }\left(t\right),t)& =-{\left.\frac{{\partial }_{t}P(x,t)}{P(x,t)}\right|}_{x=x^{\ast }\left(t\right)}-{\left.\frac{{\partial }_{x}P(x,t)}{P(x,t)}\right|}_{x=x^{\ast }\left(t\right)}\circ {\dot{x}}^{\ast }\left(t\right)\hfill \\ \hfill & =-{\left.\frac{{\partial }_{t}P(x,t)}{P(x,t)}\right|}_{x=x^{\ast }\left(t\right)}+\frac{j(x^{\ast }\left(t\right),t)}{DP(x^{\ast }\left(t\right),t)}\circ {\dot{x}}^{\ast }\left(t\right)-\frac{\mu }{D}F(x^{\ast }\left(t\right),t)\circ {\dot{x}}^{\ast }\left(t\right)\hfill \end{array}$$ (24)

where we have used the processes’ Fokker–Planck equation ${\partial }_{t}P(x,t)=-{\partial }_{x}j(x,t)$ with $j(x,t)=\mu F(x,t)P(x,t)-D{\partial }_{x}P(x,t)$. The total time derivative has been taken as a conventional derivative implying the Stratonovich convention indicated by ∘, which will become relevant below. The term in (24) containing ${\partial }_{t}P(x,t)$ accounts for changes in the probability density due to its temporal evolution, such as relaxation to a steady state, and any time-dependent driving protocol. The product $F(x^{\ast }\left(t\right),t)\circ {\dot{x}}^{\ast }\left(t\right)$ can be interpreted as a power expended by the force and in the absence of an internal energy of the particle, dissipated in the medium. With Einstein’s relation defining the temperature of $T=D/\mu$ of the medium, the last term may be written as

$${\dot{S}}_m\left(t\right)=\frac{F(x^{\ast }\left(t\right),t)\circ {\dot{x}}^{\ast }\left(t\right)}{T}$$ (25)

and thus interpreted as the entropy change in the medium. Together with the entropy change of the particle, this gives the total entropy change of particle and medium,

$${\dot{S}}_{\mathrm{tot}}\left(t\right)=\frac{\mathrm{d}}{\mathrm{d}t}S(x^{\ast }\left(t\right),t)+{\dot{S}}_m\left(t\right)=-{\left.\frac{{\partial }_{t}P(x,t)}{P(x,t)}\right|}_{x=x^{\ast }\left(t\right)}+\frac{j(x^{\ast }\left(t\right),t)}{DP(x^{\ast }\left(t\right),t)}\circ {\dot{x}}^{\ast }\left(t\right),$$ (26)

which is a random variable, as it depends on the position $x^{\ast }\left(t\right)$. It also draws on $P(x,t)$ and $j(x,t)$ which are properties of the ensemble. To make the connection to the entropies constructed above we need to take an ensemble average of the instantaneous ${\dot{S}}_{\mathrm{tot}}\left(t\right)$. To do so, we need an interpretation of the last term of (26), where the noise $\zeta \left(t\right)$ of ${\dot{x}}^{\ast }\left(t\right)$, Equation (22), multiplies $j(x^{\ast }\left(t\right),t)/P(x^{\ast }\left(t\right),t)$. Equivalently, we need the joint density $P(x,\dot{x};t)$ of position x and velocity $\dot{x}$ at time t. In the spirit of Ito, this density trivially factorises into a normally distributed $\dot{x}-\mu F(x,t)$ and $P(x,t)$ as the increment $\dot{x}dt$ on the basis of (22) depends only on the particle’s current position $x\left(t\right)$. However, this is not so in the Stratonovich interpretation of $P(x,\dot{x};t)$, as here the increment depends equally on $x\left(t\right)$ and $x(t+dt)$ [1,31,32]. Taking the ensemble average of ${\dot{S}}_{\mathrm{tot}}$ thus produces

$$\begin{array}{cc}\hfill \langle {\dot{S}}_{\mathrm{tot}}\left(t\right)\rangle & =\int \mathrm{d}{x}^{\ast }\mathrm{d}{\dot{x}}^{\ast }{\dot{S}}_{\mathrm{tot}}\left(t\right)P(x^{\ast },{\dot{x}}^{\ast };t)\hfill \\ \hfill & =-\int \mathrm{d}{x}^{\ast }\frac{{\partial }_{t}P(x^{\ast },t)}{P(x^{\ast },t)}\int \mathrm{d}{\dot{x}}^{\ast }P(x^{\ast },{\dot{x}}^{\ast };t)+\int \mathrm{d}{x}^{\ast }\mathrm{d}{\dot{x}}^{\ast }\frac{j(x^{\ast },t)}{DP(x^{\ast },t)}{\dot{x}}^{\ast }P(x^{\ast },{\dot{x}}^{\ast };t),\hfill \end{array}$$ (27)

where $x^{\ast }$ and ${\dot{x}}^{\ast }$ are now dummy variables. The first term on the right hand side vanishes, because $P(x^{\ast },t)=\int \mathrm{d}{\dot{x}}^{\ast }P(x^{\ast },{\dot{x}}^{\ast };t)$ is the marginal of $P(x^{\ast },{\dot{x}}^{\ast };t)$ and $\int \mathrm{d}{x}^{\ast }{\partial }_{t}P(x^{\ast },t)=0$ by normalisation. The integral over ${\dot{x}}^{\ast }$ in the second term produces the expected particle velocity conditional to its position,

$$\langle {\dot{x}}^{\ast }|x^{\ast },t\rangle =\int \mathrm{d}{\dot{x}}^{\ast }{\dot{x}}^{\ast }\frac{P(x^{\ast },{\dot{x}}^{\ast };t)}{P(x^{\ast },t)}$$ (28)

in the Stratonovich sense, where it gives rise to the current [12], $\langle {\dot{x}}^{\ast }|x^{\ast },t\rangle =j(x^{\ast },t)/P(x^{\ast },t)$, so that

$$\langle {\dot{S}}_{\mathrm{tot}}\left(t\right)\rangle =\int \mathrm{d}{x}^{\ast }\frac{j^2(x^{\ast },t)}{DP(x^{\ast },t)}\ge 0,$$ (29)

which vanishes only in the absence of any probability current, i.e., in thermodynamic equilibrium. In the Ito sense, the conditional expectation (28) would have instead given rise to the ensemble-independent drift, $\langle {\dot{x}}^{\ast }|x^{\ast },t\rangle =\mu F(x^{\ast },t)$. Comparing to Equation (13), the expectation $\langle {\dot{S}}_{\mathrm{tot}}\left(t\right)\rangle$ turns out to be the internal entropy production ${\dot{S}}_i\left(t\right)$, so that ${\dot{S}}_{\mathrm{tot}}\left(t\right)$ of Equation (26) may be regarded as its instantaneous counterpart.

Path integral methods. An interesting aspect of working with the Langevin description is the possibility of casting probability densities $p\left(\right[x];t)$ for paths $x\left(t^{\prime }\right)$ with $t^{\prime }\in [0,t]$ into path integrals, for example in the Onsager–Machlup formalism [33,34]. For the colloidal particle introduced in (22), it gives $p\left(\right[x];t)=\mathcal{N}exp\left(-\mathcal{A}\left(\right[x];t)\right)$ with the action functional

$$\mathcal{A}(\left[x\right];t)={\int }_0^t\mathrm{d}{t}^{\prime }\frac{{(\dot{x}\left(t^{\prime }\right)-\mu F(x\left(t^{\prime }\right),t^{\prime }))}^{\circ 2}}{4D}-\frac{\mu }{2}{\int }_0^t\mathrm{d}{t}^{\prime }{\partial }_{x}F(x\left(t^{\prime }\right),t^{\prime })$$ (30)

in the Stratonovich discretisation, which differs from the Ito form only by the second term ([34], Section 4.5), which is the Jacobian of the transform of the noise $\zeta \left(t\right)$ to $x\left(t\right)$, Equation (22). The Stratonovich form is needed so that the action does not give preference to a particular time direction [35]. This choice plays a role in every product of white noise, as is implicit to $\dot{x}$, and a random variable. We therefore indicate the choice by a ∘ also in powers, reminding us that $F(x\left(t^{\prime }\right),t^{\prime })$ should be read as $F((x\left(t^{\prime }\right)+x(t^{\prime }+\Delta t))/2,t^{\prime }+\Delta t)$ and $\dot{x}\left(t^{\prime }\right)$ as $(x(t^{\prime }+\Delta t)-x\left(t^{\prime }\right))/2$ with discretisation time step $\Delta t$. Evaluating the action for the reversed path $x^R\left(t^{\prime }\right)=x(t-t^{\prime })$ then gives

$$\begin{array}{cc}\hfill \mathcal{A}(\left[x^R\right];t)& ={\int }_0^t\mathrm{d}{t}^{\prime }\frac{{({\dot{x}}^R\left(t^{\prime }\right)-\mu F(x^R\left(t^{\prime }\right),t^{\prime }))}^{\circ 2}}{4D}-\frac{\mu }{2}{\int }_0^t\mathrm{d}{t}^{\prime }{\partial }_{x}F(x^R\left(t^{\prime }\right),t^{\prime })\hfill \end{array}$$ (31)

$$\begin{array}{cc}\hfill & ={\int }_0^t\mathrm{d}{t}^{\prime }\frac{{(\dot{x}\left(t^{\prime }\right)+\mu F(x\left(t^{\prime }\right),t-t^{\prime }))}^{\circ 2}}{4D}-\frac{\mu }{2}{\int }_0^t\mathrm{d}{t}^{\prime }{\partial }_{x}F(x\left(t^{\prime }\right),t-t^{\prime }).\hfill \end{array}$$ (32)

If the force is even under time reversal, $F(x,t^{\prime })=F(x,t-t^{\prime })$, in particular when it is independent of time, the path probability density obeys

$$ln\frac{p\left(\right[x];t)}{p(\left[x^R\right];t)}={\int }_0^t\mathrm{d}{t}^{\prime }\frac{F(x\left(t^{\prime }\right),t^{\prime })\circ \dot{x}\left(t^{\prime }\right)}{T}=S_m\left(t\right),$$ (33)

with random variables multiplied with Stratonovich convention. With Equation (25), the integral in Equation (33) can be identified as the entropy of the medium. When the driving is time-independent and the system’s probability distribution eventually becomes stationary, such that ${lim}_{t\to \infty }\langle \dot{S}(x^{\ast },t)\rangle =0$, Equation (23), the only contribution to the total entropy change is due to change of entropy in the medium, Equation (26). Assuming that the system is ergodic, we have the equivalence ${lim}_{t\to \infty }{S}_m\left(t\right)/t={lim}_{t\to \infty }\langle {\dot{S}}_{\mathrm{tot}}\left(t\right)\rangle$, where $\langle •\rangle$ denotes an ensemble average. Using Equations (13) and (29) gives ${lim}_{t\to \infty }{S}_m\left(t\right)/t={lim}_{t\to \infty }{\dot{S}}_i\left(t\right)$. Equation (33) can therefore be used directly to compute the steady-state internal entropy production rate. The equivalence between the long-time limit $t\to \infty$ and the ensemble average holds only for ergodic systems, whose unique steady-state does not depend on the specific initialisation $x\left(0\right)$. This connection between stochastic thermodynamics and field theory has stimulated a number of works aimed at characterising the non-equilibrium features of continuum models of active matter [13,36]. Extensions of this formalism to systems driven by correlated noise have also been proposed [37].

3. Systems

In this section, we calculate the entropy production rate on the basis of Gaspard’s framework [11], Equations (4), (15) and (16), for different particle systems. We cover the systems listed in Table 1, with both discrete and continuous states and with one or multiple particles.

Table 1.

List of particle systems for which we have calculated their entropy production ${\dot{S}}_i\left(t\right)$.

Section	System	${\dot{\mathit{S}}}_{\mathit{i}}\left(\mathit{t}\right)$
Section 3.1	Two-state Markov process	(37)
Section 3.2	Three-state Markov process	(41)
Section 3.3	Random walk on a complete graph	(44), (45)
Section 3.4	N independent, distinguishable Markov processes	(52)
Section 3.5	N independent, indistinguishable two-state Markov processes	(55b)
Section 3.6	N independent, indistinguishable d-state processes	(68)
Section 3.7	Random Walk on a lattice	(82)
Section 3.8	Random Walk on a ring lattice	(87), (89)
Section 3.9	Driven Brownian particle	(94)
Section 3.10	Driven Brownian particle in a harmonic potential	(100)
Section 3.11	Driven Brownian particle on a ring with potential	(113d)
Section 3.12	Run-and-tumble motion with diffusion on a ring	(121)
Section 3.13	Switching diffusion process on a ring	(128)

3.1. Two-State Markov Process

Consider a particle that hops between two states, 1 and 2, with transition rates $\dot{W}(1\to 2)=\alpha$ and $\dot{W}(2\to 1)=\beta$, see Figure 1 [23,38], and using the notation in Equation (18) for discrete states. The rate-matrix (see Equation (7)) may thus be

$$w=\left(\begin{array}{cc}-\alpha & \alpha \\ \beta & -\beta \end{array}\right),$$ (34)

with $\mathbf{P}\left(t\right)=(P_1\left(t\right),P_2\left(t\right))$ the probability of the particle to be in state 1 or 2 respectively as a function of time. By normalisation, $P_1\left(t\right)+P_2\left(t\right)=1$, with probabilistic initial condition $\mathbf{P}\left(0\right)=(p,1-p)$. Solving the master equation in Equation (2) yields

$$\mathbf{P}\left(t\right)=(P_1\left(t\right),P_2\left(t\right))=\frac{1}{\alpha +\beta }\left(\begin{array}{c}\beta +r{e}^{-(\alpha +\beta )t},\alpha -r{e}^{-(\alpha +\beta )t}\end{array}\right),$$ (35)

with $r=\alpha p-\beta (1-p)$, corresponding to an exponentially decaying probability current

$$P_1\left(t\right)\alpha -P_2\left(t\right)\beta =r{e}^{-(\alpha +\beta )t}.$$ (36)

Figure 1.

Two-state Markov chain in continuous time. The black blob indicates the current state of the system. Independently of the choice of $\alpha$ and $\beta$, this processes settles into an equilibrium steady-state over long timeframes (in the absence of an external time-dependent diving).

The internal entropy production (4b) is then

$${\dot{S}}_i\left(t\right)=[P_1\left(t\right)\alpha -P_2\left(t\right)\beta ]ln\left[\frac{P_1\left(t\right)\alpha }{P_2\left(t\right)\beta }\right]=r{\mathrm{e}}^{-(\alpha +\beta )t}ln\left[\frac{1+\frac{r}{\beta }{e}^{-(\alpha +\beta )t}}{1-\frac{r}{\alpha }{e}^{-(\alpha +\beta )t}}\right],$$ (37)

see Figure 2, and the entropy flow (4a),

$${\dot{S}}_e\left(t\right)=-r{\mathrm{e}}^{-(\alpha +\beta )t}ln\left(\frac{\alpha }{\beta }\right).$$ (38)

At stationarity, ${\dot{S}}_i={\dot{S}}_e=0$ and therefore the two-state Markov process reaches equilibrium. In this example, the topology of the transition network does not allow a sustained current between states, which inevitably leads to equilibrium in the steady state and, therefore, there is production of entropy only due to the relaxation of the system from the initial state.

Figure 2.

Entropy production of the two- and three-state Markov processes (black and grey lines respectively) discussed in Section 3.1 and Section 3.2 as a function of time. For the two-state system we plot Equation (37) with $p=1$ both for the symmetric, $\alpha =\beta$ (solid lines), and for the asymmetric case, $\alpha \ne \beta$ (dashed lines). In both, the entropy production decays exponentially over long timeframes. For the three-state system, Equation (41), the asymmetric case displays a finite entropy production rate over long timeframes, consistent with Equation (42) and the condition that at stationarity ${\dot{S}}_i\left(t\right)=-{\dot{S}}_e\left(t\right)$.

3.2. Three-State Markov Process

We extend the system in Section 3.1 to three states, 1, 2 and 3, with transition rates $\dot{W}(1\to 2)=\alpha$, $\dot{W}(2\to 3)=\alpha$, $\dot{W}(3\to 1)=\alpha$, $\dot{W}(2\to 1)=\beta$, $\dot{W}(3\to 2)=\beta$, and $\dot{W}(1\to 3)=\beta$, see Figure 3, and using the notation Equation (18) for discrete states. The rate matrix (see Equation (7)) is then

$$w=\left(\begin{array}{ccc}-(\alpha +\beta )& \alpha & \beta \\ \beta & -(\alpha +\beta )& \alpha \\ \alpha & \beta & -(\alpha +\beta )\end{array}\right).$$ (39)

Assuming the initial condition $\mathbf{P}\left(0\right)=(1,0,0)$, the probabilities of states 1, 2 and 3 respectively, evolve according to Equation (2), which has solution

$$\begin{array}{cc}\hfill P_1\left(t\right)=& \frac{1}{3}\left(1+2{\mathrm{e}}^{-3\varphi t}cos\left(\sqrt{3}\psi t\right)\right),\hfill \end{array}$$ (40a)

$$\begin{array}{cc}\hfill P_2\left(t\right)=& \frac{1}{3}\left(1-2{\mathrm{e}}^{-3\varphi t}cos(\sqrt{3}\psi t-\pi /3)\right),\hfill \end{array}$$ (40b)

$$\begin{array}{cc}\hfill P_3\left(t\right)=& \frac{1}{3}\left(1-2{\mathrm{e}}^{-3\varphi t}cos(\sqrt{3}\psi t+\pi /3)\right),\hfill \end{array}$$ (40c)

with $\varphi =(\alpha +\beta )/2$ and $\psi =(\alpha -\beta )/2$.

Figure 3.

Three-state Markov chain in continuous time. The black blob indicates the current state of the system. Symmetry under cyclic permutation is introduced by imposing identical transition rates $\alpha$ and $\beta$ for counter-clockwise and clockwise transition, respectively.

The entropy production (4b) is then, using (40),

$$\begin{array}{cc}\hfill {\dot{S}}_i\left(t\right)& =(\alpha -\beta )ln\frac{\alpha }{\beta }+\left(P_1\left(t\right)\alpha -P_2\left(t\right)\beta \right)ln\left(\frac{P_1\left(t\right)}{P_2\left(t\right)}\right)\hfill \\ \hfill & +\left(P_2\left(t\right)\alpha -P_3\left(t\right)\beta \right)ln\left(\frac{P_2\left(t\right)}{P_3\left(t\right)}\right)+\left(P_3\left(t\right)\alpha -P_1\left(t\right)\beta \right)ln\left(\frac{P_3\left(t\right)}{P_1\left(t\right)}\right),\hfill \end{array}$$ (41)

see Figure 2, and the entropy flow (4a),

$${\dot{S}}_e\left(t\right)=-(\alpha -\beta )ln\left(\frac{\alpha }{\beta }\right),$$ (42)

which is constant throughout. At stationarity, the system is uniformly distributed and, if $\alpha \ne \beta$, the entropy production and flow satisfy ${\dot{S}}_i=-{\dot{S}}_e\ne 0$. If $\alpha \ne \beta$, the particle has a net drift that sustains a probability current $(\alpha -\beta )/3$ in the system, which prevents the system from reaching equilibrium. This setup can be generalised straightforwardly to M-state Markov processes characterised by the same cyclic structure $\dot{W}(i\to i+1(modM))=\alpha$ and $\dot{W}(i\to i-1(modM))=\beta$, to find that the steady-state entropy production is independent of M. These can be seen as simple models of protein conformational cycles driven by instantaneous energy inputs of order $k_{B}Tln(\alpha /\beta )$, for example ATP hydrolysis [39].

3.3. Random Walk on a Complete Graph

Consider a random walker on a complete graph with d nodes, where each node is connected to all other nodes, and the walker jumps from node $j\in \{1,2,\dots ,d\}$ to node $k\in \{1,2,\dots ,d\}$, $k\ne j$, with rate $w_{jk}$, see Figure 4. These are the off-diagonal elements of the corresponding Markov matrix whose diagonal elements are $w_{jj}=-{\sum }_{i=1,i\ne j}^d{w}_{ji}$. The probability vector $\mathbf{P}\left(t\right)=(P_1\left(t\right),P_2\left(t\right),\dots ,P_d\left(t\right))$ has components $P_j\left(t\right)$ that are the probability that the system is in state j at time t. The general case of arbitrary transition rates is impossible to discuss exhaustively. In the uniform case, $w_{jk}=\alpha$, the Markov matrix has only two distinct eigenvalues, namely eigenvalue $\alpha d$ with degeneracy $d-1$ and eigenvalue 0 with degeneracy 1. Assuming an arbitrary initial condition $\mathbf{P}\left(0\right)$, the probability distribution at a later time t is

$$P_j\left(t\right)=\frac{1}{d}+e^{-d\alpha t}\left(P_j\left(0\right)-\frac{1}{d}\right).$$ (43)

The steady state, which is associated with the vanishing eigenvalue, is the uniform distribution ${lim}_{t\to \infty }{P}_j\left(t\right)=1/d$ for all $j\in \{1,2,\dots ,d\}$. The entropy production (4b) of the initial state relaxing to the uniform state is

$${\dot{S}}_i\left(t\right)=\frac{1}{2}\alpha e^{-d\alpha t}\sum _{j,k}\left(P_j\left(0\right)-P_k\left(0\right)\right)ln\left(\frac{1+e^{-d\alpha t}\left(P_j\left(0\right)d-1\right)}{1+e^{-d\alpha t}\left(P_k\left(0\right)d-1\right)}\right),$$ (44)

and the entropy flow (4a) is ${\dot{S}}_e=0$ throughout. If the walker is initially located on node k, so that $P_j\left(0\right)={\delta }_{j,k}$, the entropy production simplifies to

$${\dot{S}}_i\left(t\right)=(d-1)\alpha e^{-d\alpha t}ln(1+\frac{d{e}^{-d\alpha t}}{1-e^{-d\alpha t}}).$$ (45)

We can see that the system reaches equilibrium at stationarity, since ${lim}_{t\to \infty }{\dot{S}}_i\left(t\right)={\dot{S}}_e\left(t\right)=0$. Over long timeframes ($d{e}^{-d\alpha t}\ll 1$), the asymptotic behaviour of ${\dot{S}}_i$ is

$${\dot{S}}_i\left(t\right)=d(d-1)\alpha e^{-2d\alpha t}+\mathcal{O}\left(e^{-3d\alpha t}\right),$$ (46)

by expanding the logarithm in the small exponential.

Figure 4.

Random walk on a complete graph of d nodes (here shown for $d=6$). The black blob indicates the current state of the system. For uniform transition rates, the symmetry under node relabelling leads to an equilibrium, homogeneous steady-state with $P_j=1/d$ for all j.

3.4. N Independent, Distinguishable Markov Processes

In the following, we consider N non-interacting, distinguishable particles undergoing Markovian dynamics on a discrete state space, see Figure 5. Each of the N particles carries an index $\ell \in \{1,2,\dots ,N\}$ and is in state $n_{\ell }\in \{1,2,\dots ,d_{\ell }\}$, so that the state of the entire system is given by an N-particle state $\mathbf{n}=(n_1,n_2,\dots ,n_N)$. Particle distinguishability implies the factorisation of state and transition probabilities into their single-particle contributions, whence the joint probability $P_{\mathbf{n}}\left(t\right)$ of an N-particle state $\mathbf{n}$ factorises into a product of single particle probabilities $P_{n_{\ell }}^{(\ell )}\left(t\right)$ of particle ℓ to be in state $n_{\ell }$,

$$P_{\mathbf{n}}\left(t\right)=\prod _{\ell =1}^N{P}_{n_{\ell }}^{(\ell )}\left(t\right).$$ (47)

Further, the Poissonian rate $w_{\mathbf{n}\mathbf{m}}$ from N-particle state $\mathbf{n}$ to N-particle state $\mathbf{m}\ne \mathbf{n}$ vanishes for all transitions $\mathbf{n}\to \mathbf{m}$ that differ in more than one component ℓ, i.e., $w_{\mathbf{n}\mathbf{m}}=0$ unless there exists a single $\ell \in \{1,2,\dots ,N\}$ such that $m_k=n_k$ for all $k\ne \ell$, in which case $w_{\mathbf{n}\mathbf{m}}=w_{n_{\ell }{m}_{\ell }}^{(\ell )}$, the transition rates of the single particle transition of particle ℓ.

Figure 5.

Example of $N=5$ non-interacting, distinguishable processes with $d_1=4$, $d_2=2$, $d_3=3$, $d_4=5$ and $d_5=5$. The black blobs indicate the current state of each sub-system.

The entropy production of this N-particle system according to Equation (4b),

$${\dot{S}}_i\left(t\right)=\frac{1}{2}\sum _{\mathbf{n}\mathbf{m}}(P_{\mathbf{n}}\left(t\right)w_{\mathbf{n}\mathbf{m}}-P_{\mathbf{m}}\left(t\right)w_{\mathbf{m}\mathbf{n}})ln\left(\frac{P_{\mathbf{n}}\left(t\right)w_{\mathbf{n}\mathbf{m}}}{P_{\mathbf{m}}\left(t\right)w_{\mathbf{m}\mathbf{n}}}\right)$$ (48)

simplifies considerably due to $w_{\mathbf{n}\mathbf{m}}$, as the sum may be re-written as

$$\sum _{\mathbf{n}\mathbf{m}}\dots w_{\mathbf{n}\mathbf{m}}\dots =\sum _{\mathbf{n}}\sum _{\ell }\sum _{m_{\ell }}\dots w_{\mathbf{n}{\mathbf{m}}_{\ell }}\dots$$ (49)

with ${\mathbf{m}}_{\ell }=(n_1,n_2,\dots ,n_{\ell -1},m_{\ell },n_{\ell +1},\dots ,n_N)$ so that $w_{\mathbf{n}{\mathbf{m}}_{\ell }}=w_{n_{\ell }{m}_{\ell }}^{(\ell )}$ and

$${\dot{S}}_i\left(t\right)=\frac{1}{2}\sum _{\mathbf{n}}\sum _{\ell =1}^N\sum _{m_{\ell }}\left\{\left(\prod _{k=1}^N{P}_{n_k}^{\left(k\right)}\left(t\right)\right)w_{n_{\ell }{m}_{\ell }}^{(\ell )}-\left(\prod _{k=1}^N{P}_{m_k}^{\left(k\right)}\left(t\right)\right)w_{m_{\ell }{n}_{\ell }}^{(\ell )}\right\}ln\left(\frac{{\prod }_{k=1}^N{P}_{n_k}^{\left(k\right)}\left(t\right)w_{n_{\ell }{m}_{\ell }}^{(\ell )}}{{\prod }_{k=1}^N{P}_{m_k}^{\left(k\right)}\left(t\right)w_{m_{\ell }{n}_{\ell }}^{(\ell )}}\right).$$ (50)

Since $m_k=n_k$ for any $k\ne \ell$ inside the curly bracket, we may write

$$\begin{array}{ccccc}\hfill \prod _{k=1}^N{P}_{n_k}^{\left(k\right)}\left(t\right)=P_{n_{\ell }}^{(\ell )}\left(t\right)\prod _{\begin{array}{c}k=1\\ k\ne \ell \end{array}}^N{P}_{n_k}^{\left(k\right)}\left(t\right)& \hfill & \hfill \mathrm{and}& \hfill & \hfill \prod _{k=1}^N{P}_{m_k}^{\left(k\right)}\left(t\right)=P_{m_{\ell }}^{(\ell )}\left(t\right)\prod _{\begin{array}{c}k=1\\ k\ne \ell \end{array}}^N{P}_{n_k}^{\left(k\right)}\left(t\right).\end{array}$$ (51)

The product ${\prod }_{k\ne \ell }^N{P}_{n_k}^{\left(k\right)}\left(t\right)$ can thus be taken outside the curly bracket in Equation (50) and be summed over, as well as cancelled in the logarithm. After changing the dummy variables in the remaining summation from $n_{\ell }$ and $m_{\ell }$ to n and m respectively, the entropy production is

$${\dot{S}}_i\left(t\right)=\frac{1}{2}\sum _{\ell =1}^N\sum _{nm}\left(P_n^{(\ell )}\left(t\right)w_{nm}^{(\ell )}-P_m^{(\ell )}\left(t\right)w_{mn}^{(\ell )}\right)ln\left(\frac{P_n^{(\ell )}\left(t\right)w_{nm}^{(\ell )}}{P_m^{(\ell )}\left(t\right)w_{mn}^{(\ell )}}\right),$$ (52)

which is the sum of the entropy productions of the single particle $\ell \in \{1,2,\dots ,N\}$, Equation (4b), irrespective of how each particle is initialised. The same argument applies to ${\dot{S}}_e$, the entropy flow Equation (4a). The entropy production and flow obviously simplify to an N-fold product of the single particle expressions if $w_{nm}^{(\ell )}$ do not depend on ℓ and all particles are initialised by the same $P_n^{\ell }\left(0\right)$ independent of ℓ. This result may equally be found from the dynamical entropy per unit time, Equation (8).

3.5. N Independent, Indistinguishable Two-State Markov Processes

Suppose that N identical, indistinguishable, non-interacting particles follow the two-state Markov process described in Section 3.1, Figure 6 [23]. There are $\Omega =N+1$ distinct states given by the occupation number $n\in \{0,1,\dots ,N\}$ of one of the two states, say state 1, as the occupation number of the other state follows as $N-n$ given the particle number N is fixed under the dynamics. In the following, $P(n,t)$ denotes the probability of finding n particles in state 1 at time t. The master equation is then

$$\dot{P}(n,t)=-\alpha nP(n,t)+\alpha (n+1)P(n+1,t)-\beta (N-n)P(n,t)+\beta (N-n+1)P(n-1,t).$$ (53)

Figure 6.

N independent, indistinguishable two-state Markov processes in continuous time. The black blobs indicate the current state of the single-particle sub-system. Since processes are indistinguishable, states are fully characterised by the occupation number of either state, if the total number of particles is known.

The state space and the evolution in it can be thought of as a hopping process on a one-dimensional chain of states with non-uniform rates. Provided $P(n,0)$ initially follows a binomial distribution, $P(n,0)=\left(\genfrac{}{}{0pt}{}{N}{n}\right)p^n{(1-p)}^{N-n}$ with probability p for a particle to be placed in state 1 initially, the solution of Equation (53) is easily constructed from the solution $P_1\left(t\right)$ in Equation (35) of Section 3.1 via

$$P(n,t)=\left(\genfrac{}{}{0pt}{}{N}{n}\right)P_1^n\left(t\right){(1-P_1\left(t\right))}^{N-n}\mathrm{for}0\le n\le N$$ (54)

with $P_1\left(0\right)=p$, as ${\dot{P}}_1\left(t\right)=-\alpha P_1\left(t\right)+\beta (1-P_1\left(t\right))$, which can be verified by substituting Equation (54) into Equation (53). Using Equations (34) and (54) in (4b), the entropy production reads

$$\begin{array}{cc}\hfill {\dot{S}}_i\left(t\right)& =\sum _{n=1}^N[P(n,t)\alpha n-P(n-1,t)\beta (N-n+1)]ln\left[\frac{P(n,t)\alpha n}{P(n-1,t)\beta (N-n+1)}\right]\hfill \end{array}$$ (55a)

$$\begin{array}{cc}\hfill & =N[P_1\left(t\right)\alpha -(1-P_1\left(t\right))\beta ]ln\left[\frac{P_1\left(t\right)\alpha }{(1-P_1\left(t\right))\beta }\right],\hfill \end{array}$$ (55b)

which is the N-fold multiple of the result of the corresponding single particle system, Equation (37). This result, Equation (55b), depends on the initialisation being commensurable with Equation (54) which otherwise is recovered only asymptotically and only if the stationary distribution is unique.

Further, the entropy production of N indistinguishable particles being the N-fold entropy production of a single particle does not extend to the external entropy flow, which lacks the simplification of the logarithm and gives

$$\begin{array}{c}\hfill {\dot{S}}_e\left(t\right)=-N[\alpha P_1\left(t\right)-\beta (1-P_1\left(t\right))]\left\{ln\left(\frac{\alpha }{\beta }\right)+\sum _{n=0}^{N-1}{P}_1^n\left(t\right){(1-P_1\left(t\right))}^{N-1-n}\left(\genfrac{}{}{0pt}{}{N-1}{n}\right)ln\left(\frac{n+1}{N-n}\right)\right\}\end{array}$$ (56)

thus picking up a correction in the form of the additional sum in the curly bracket that vanishes only at $N=1$ or $P_1\left(t\right)=1/2$, but does not contribute at stationarity because of the overall prefactor $\alpha P_1-\beta (1-P_1)$ that converges to 0. To make sense of this correction in relation to particle indistinguishability, with the help of Equation (54) we can rewrite the difference between the right hand side of Equation (56) and the N-fold entropy flow of a single two-state system (38) as

$$\begin{array}{cc}\hfill -& N[\alpha P_1\left(t\right)-\beta (1-P_1\left(t\right))]\sum _{n=0}^{N-1}{P}_1^n\left(t\right){(1-P_1\left(t\right))}^{N-1-n}\left(\genfrac{}{}{0pt}{}{N-1}{n}\right)ln\left(\frac{n+1}{N-n}\right)\hfill \\ \hfill & =-\sum _{n=0}^{N-1}[\alpha (n+1)P(n+1,t)-\beta (N-n)P(n,t)]ln\left(\frac{n+1}{N-n}\right)\hfill \end{array}$$ (57)

which now explicitly involves the net probability current from the occupation number state with $n+1$ particles in state A to that with n particles in state A, as well as a the logarithm

$$ln\left(\frac{n+1}{N-n}\right)=ln\left[\left(\genfrac{}{}{0pt}{}{N}{n}\right)\right]-ln\left[\left(\genfrac{}{}{0pt}{}{N}{n+1}\right)\right].$$ (58)

Written in terms of the same combinatorial factors appearing in Equation (54), the logarithm (58) can be interpreted as a difference of microcanonical (Boltzmann) entropies, defined as the logarithm of the degeneracy of the occupation number state if we were to assume that the N particles are distinguishable. With the help of the master Equation (53) as well as Equation (54) and (58), the term Equation (57) may be rewritten to give

$${\dot{S}}_e\left(t\right)=-N[\alpha P_1\left(t\right)-\beta (1-P_1\left(t\right))]ln\left(\frac{\alpha }{\beta }\right)-\sum _{n=0}^{N-1}\dot{P}(n,t)ln\left[\left(\genfrac{}{}{0pt}{}{N}{n}\right)\right]$$ (59)

This result is further generalised in Equation (70).

3.6. N Independent, Indistinguishable d-State Processes

We generalise now the results in Section 3.3 and Section 3.5 to N independent d-state Markov processes, see Figure 7. These results represent a special case of those obtained in [40] when the N processes are non-interacting. In this section, we consider non-interacting, indistinguishable particles hopping on a graph of d nodes with edge-dependent hopping rates $w_{jk}$. As in the two-state system in Section 3.5, we find that the internal (but not the external) entropy production of the d-state system ${\dot{S}}_i$ is N times the entropy production of the individual processes assuming the initial condition is probabilistically identical for all single-particle sub-systems. The entropy productions of a single such process according to Equation (4) read

$$\begin{array}{cc}\hfill {\dot{S}}_i^{\left(1\right)}\left(t\right)& =\frac{1}{2}\sum _{jk}[P_j\left(t\right)w_{jk}-P_k\left(t\right)w_{kj}]ln\left(\frac{P_j\left(t\right)w_{jk}}{P_k\left(t\right)w_{kj}}\right),\hfill \end{array}$$ (60a)

$$\begin{array}{cc}\hfill {\dot{S}}_e^{\left(1\right)}\left(t\right)& =-\frac{1}{2}\sum _{jk}[P_j\left(t\right)w_{jk}-P_k\left(t\right)w_{kj}]ln\left(\frac{w_{jk}}{w_{kj}}\right),\hfill \end{array}$$ (60b)

where $P_j\left(t\right)$ is the time-dependent probability of a single-particle process to be in state j, Section 3.3.

Figure 7.

N independent, indistinguishable d-state Markov processes (here shown for $d=6$ and $N=8$) in continuous time. Black blobs indicate the current state of the single-particle sub-systems. Due to indistinguishability, multi-particle states are fully characterised by the occupation number of an arbitrary subset of $d-1$ states, if the total number of particles is known.

To calculate the entropy production of the N concurrent indistinguishable processes using the occupation number representation, we first derive the probability of an occupation number configuration $\mathbf{n}=(n_1,n_2,\dots ,n_d)$, with ${\sum }_{j=1}^d{n}_j=N$, which similarly to Equation (54) is given by the multinomial distribution

$$P_{\mathbf{n}}\left(t\right)=N!\prod _{j=1}^d\frac{P_j^{n_j}\left(t\right)}{n_j!}$$ (61)

for the probability $P_{\mathbf{n}}\left(t\right)$ of the system to be in state $\mathbf{n}$ at time t assuming that each particle is subject to the same single-particle distribution $P_j\left(t\right)$, $j\in \{1,2,\dots ,d\}$ for all t, i.e., in particular assuming that all particles are initialised identically, by placing them all at the same site or, more generally, by placing them initially according to the same distribution $P_j\left(0\right)$. Given this initialisation, Equation (61) solves Equation (2)

$${\dot{P}}_{\mathbf{n}}\left(t\right)=\sum _{\mathbf{m}}{P}_{\mathbf{m}}\left(t\right)w_{\mathbf{m}\mathbf{n}}-P_{\mathbf{n}}\left(t\right)w_{\mathbf{n}\mathbf{m}}$$ (62)

with the transition rates $w_{\mathbf{m}\mathbf{n}}$ discussed below.

For non-interacting processes with a unique stationary distribution, Equation (61) is always obeyed in the limit of long times after initialisation, since the single-particle distributions $P_j\left(t\right)$ are identical at steady state. The entropy production Equation (4b) of the entire system has the same form as Equation (48) of Section 3.4 (N independent, distinguishable particles) with $w_{\mathbf{n}\mathbf{m}}$, however now the transition rate between the occupation number state $\mathbf{n}=(n_1,n_2,\dots ,n_d)$ with $0\le n_k\le N$ to occupation number state $\mathbf{m}=(m_1,m_2,\dots ,m_d)$. The rate $w_{\mathbf{n}\mathbf{m}}$ vanishes except when $\mathbf{m}$ differs from $\mathbf{n}$ in exactly two distinct components, say $m_j=n_j-1\ge 0$ and $m_k=n_k+1\ge 1$ in which case $w_{\mathbf{n}\mathbf{m}}=n_j{w}_{jk}$ with $w_{jk}$ the transition rates of a single particle from j to k as introduced above. For such $\mathbf{m}$, the rate obeys $w_{\mathbf{m}\mathbf{n}}=m_k{w}_{kj}$ and the probability $P_{\mathbf{m}}\left(t\right)$ fulfills

$$P_{\mathbf{m}}\left(t\right)=P_{\mathbf{n}}\left(t\right)\frac{P_k\left(t\right)n_j}{P_j\left(t\right)m_k}=P_{\mathbf{n}}\left(t\right)\frac{P_k\left(t\right)w_{\mathbf{n}\mathbf{m}}{w}_{kj}}{P_j\left(t\right)w_{\mathbf{m}\mathbf{n}}{w}_{jk}},$$ (63)

which simplifies the entropy production Equation (48) to

$$\begin{array}{cc}\hfill {\dot{S}}_i\left(t\right)& =\frac{1}{2}\sum _{\mathbf{n}\mathbf{m}}(P_{\mathbf{n}}\left(t\right)w_{\mathbf{n}\mathbf{m}}-P_{\mathbf{m}}\left(t\right)w_{\mathbf{m}\mathbf{n}})ln\left(\frac{P_{\mathbf{n}}\left(t\right)w_{\mathbf{n}\mathbf{m}}}{P_{\mathbf{m}}\left(t\right)w_{\mathbf{m}\mathbf{n}}}\right)\hfill \\ \hfill & =\frac{1}{2}\sum _{\mathbf{n}}\sum _{jk}(P_{\mathbf{n}}\left(t\right)n_j{w}_{jk}-P_{\mathbf{m}}\left(t\right)m_k{w}_{kj})ln\left(\frac{P_j\left(t\right)w_{jk}}{P_k\left(t\right)w_{kj}}\right)\hfill \end{array}$$ (64)

where the sum ${\sum }_{\mathbf{n}}$ runs over all allowed configurations, namely $0\le n_j\le N$ for $j=1,2,\dots ,d$ with ${\sum }_j{n}_j=N$ and $\mathbf{m}=(n_1,n_2,\dots ,n_j-1,\dots ,n_k+1,\dots ,n_d)$ is derived from $\mathbf{n}$ as outlined above. Strictly, $P_{\mathbf{n}}\left(t\right)$ has to be defined to vanish for invalid states $\mathbf{n}$, so that the first bracket in the summand of Equation (64) vanishes, in particular when $n_j=0$, in which case $m_j=-1$. To proceed, we introduce the probability

$${\overline{P}}_{{\overline{\mathbf{n}}}_j}\left(t\right)=(N-1)!\frac{P_j^{n_j-1}}{(n_j-1)!}\prod _{i=1,i\ne j}^d\frac{P_i^{n_i}}{n_i!},$$ (65)

defined to vanish for $n_j=0$, so that $P_{\mathbf{n}}\left(t\right)n_j=N{P}_j\left(t\right){\overline{P}}_{{\overline{\mathbf{n}}}_j}\left(t\right)$. The probability ${\overline{P}}_{{\overline{\mathbf{n}}}_j}\left(t\right)$ is that of finding $n_i$ particles at states $i\ne j$ and $n_j-1$ particles at state j. It is Equation (61) evaluated in a system with only $N-1$ particles and configuration ${\overline{\mathbf{n}}}_j=(n_1,n_2,\dots ,n_{j-1},n_j-1,n_{j+1},\dots ,n_d)={\overline{\mathbf{m}}}_k$ a function of $\mathbf{n}$. Equation (64) may now be rewritten as

$${\dot{S}}_i\left(t\right)=\frac{N}{2}\sum _{jk}\left\{\left(\sum _{\mathbf{n}}{\overline{P}}_{{\overline{\mathbf{n}}}_j}\left(t\right)\right)P_j\left(t\right)w_{jk}-\left(\sum _{\mathbf{n}}{\overline{P}}_{{\overline{\mathbf{m}}}_k}\left(t\right)\right)P_k\left(t\right)w_{kj}\right\}ln\left(\frac{P_j\left(t\right)w_{jk}}{P_k\left(t\right)w_{kj}}\right)$$ (66)

where we have used that the arguments of the logarithm are independent of $\mathbf{n}$ and $\mathbf{m}$. The summation over $\mathbf{n}$ gives

$$\sum _{\mathbf{n}}{\overline{P}}_{{\overline{\mathbf{n}}}_j}\left(t\right)=\sum _{\mathbf{n}}{\overline{P}}_{{\overline{\mathbf{m}}}_k}\left(t\right)=1$$ (67)

so that

$${\dot{S}}_i\left(t\right)=\frac{N}{2}\sum _{jk}(P_j\left(t\right)w_{jk}-P_k\left(t\right)w_{kj})ln\left(\frac{P_j\left(t\right)w_{jk}}{P_k\left(t\right)w_{kj}}\right)=N{\dot{S}}_i^{\left(1\right)}\left(t\right)$$ (68)

which is the N-fold entropy production of the single particle system ${\dot{S}}_i\left(t\right)$, Equation (60a), or equivalently that of N distinguishable particles, Equation (52), Section 3.4. As in Section 3.5, this dramatic simplification does not carry over to the external entropy flow Equation (4a)

$$\begin{array}{cc}\hfill {\dot{S}}_e\left(t\right)& =-\frac{N}{2}\sum _{jk}\sum _{\mathbf{n}}{\overline{P}}_{{\overline{\mathbf{n}}}_j}\left(t\right)(P_j\left(t\right)w_{jk}-P_k\left(t\right)w_{kj})ln\left(\frac{n_j{w}_{jk}}{(n_k+1)w_{kj}}\right)\hfill \\ \hfill & =-\frac{N}{2}\sum _{jk}(P_j\left(t\right)w_{jk}-P_k\left(t\right)w_{kj})ln\left(\frac{w_{jk}}{w_{kj}}\right)\hfill \\ \hfill & \phantom{m}-\frac{N}{2}\sum _{jk}\sum _{\mathbf{n}}{\overline{P}}_{{\overline{\mathbf{n}}}_j}\left(t\right)(P_j\left(t\right)w_{jk}-P_k\left(t\right)w_{kj})ln\left(\frac{n_j}{n_k+1}\right),\hfill \end{array}$$ (69)

where of the last two terms only the first is the N-fold entropy flow of the single particle system ${\dot{S}}_e\left(t\right)$, Equation (60b). The reason for the second term is the lack of a cancellation mechanism to absorb the $n_j$ and $n_k+1$ from the logarithm. Rewriting the second term as

$$\begin{array}{cc}\hfill & -\frac{N}{2}\sum _{jk}\sum _{\mathbf{n}}{\overline{P}}_{{\overline{\mathbf{n}}}_j}\left(t\right)(P_j\left(t\right)w_{jk}-P_k\left(t\right)w_{kj})ln\left(\frac{n_j}{n_k+1}\right)\hfill \end{array}$$

$$\begin{array}{cc}\hfill =& -\frac{1}{2}\sum _{\mathbf{n}}\sum _{jk}\left(P_{\mathbf{n}}\left(t\right)n_j{w}_{jk}-P_{\mathbf{n}}\left(\frac{P_k\left(t\right)n_j}{P_j\left(t\right)(n_k+1)}\right)(n_k+1)w_{kj}\right)ln\left(\frac{n_j}{n_k+1}\right)\hfill \end{array}$$ (70)

$$\begin{array}{cc}\hfill =& -\sum _{\mathbf{n}}{\dot{P}}_{\mathbf{n}}\left(t\right)ln\left[\left(\genfrac{}{}{0pt}{}{N}{n_1,...,n_d}\right)\right],\hfill \end{array}$$ (71)

using Equation (62) where we re-expressed the logarithm as

$$ln\left(\frac{n_j}{n_k+1}\right)=ln\left[\left(\genfrac{}{}{0pt}{}{N}{n_1,\dots ,n_j-1,\dots ,n_k+1,\dots ,n_d}\right)\right]-ln\left[\left(\genfrac{}{}{0pt}{}{N}{n_1,\dots ,n_d}\right)\right],$$ (72)

shows that the correction term has the same form as the corresponding term in the two-state system, Equation (57), namely that of a difference of microcanonical (Boltzmann) entropies of the multi-particle states. It vanishes when all $n_j$ are either 0 or 1, as expected for $d\gg N$ and also at stationarity when ${\dot{P}}_{\mathbf{n}}\left(t\right)=0$. In that limit ${\dot{S}}_e=-{\dot{S}}_i$ when indeed Equation (60a) gives

$$\underset{t\to \infty }{lim}{\dot{S}}_i^{\left(1\right)}\left(t\right)=\frac{1}{2}\sum _k(P_j{w}_{jk}-P_k{w}_{kj})ln\left(\frac{w_{jk}}{w_{kj}}\right),$$ (73)

with $P_j={lim}_{t\to \infty }{P}_j\left(t\right)$. As far as the entropy production ${\dot{S}}_i\left(t\right)$ is concerned, we thus recover and generalise the result in Section 3.5 on indistinguishable particles in a two-state system, which produce N times the entropy of a single particle. In Section 3.4, it was shown that N distinguishable particles have the same entropy production and flow as the sum of the entropy productions of individual particles. In Section 3.5 and Section 3.6, it was shown that the entropy production of indistinguishable particles, which require the states to be represented by occupation numbers, show the N-fold entropy production of the single particle system, provided suitable initialisation, but asymptotically independent of initialisation, provided the stationary state has a unique distribution. The same does not apply to the entropy flow, which generally acquires additional logarithmic terms accounting for the degeneracy of the occupation number states. The extra terms, however, are bound to vanish at stationarity, when ${\dot{S}}_e\left(t\right)=-{\dot{S}}_i\left(t\right)$.

3.7. Random Walk on a Lattice

In this section, we study a particle on a one-dimensional lattice that hops to the right nearest neighbouring site with rate r and to the left with rate ℓ, see Figure 8. The case of unequal hopping rates, $\ell \ne r$, is generally referred to as an asymmetric random walk and can be seen as a minimal model for the directed motion of a molecular motor on a cytoskeletal filament [41]. The position x of the particle at time t, after $N\left(t\right)$ jumps, is

$$x=x_0+\sum _{i=1}^{N\left(t\right)}\Delta x_i,$$ (74)

where the random hops $\Delta x_i$ are independent and identically distributed, and $x_0$ is the initial position at time $t=0$. If a is the lattice spacing, the distance increments are $\Delta x_i=+a$ with probability $r/(\ell +r)$ and $\Delta x_i=-a$ with probability $\ell /(\ell +r)$. The probability distribution of the particle position is

$$P(x,t;x_0)=\sum _{n=0}^{\infty }H(n,t)P_n(x;x_0),$$ (75)

where $H(n,t)$ is the probability that by time t, the particle has hopped $N\left(t\right)=n$ times, and $P_n(x;x_0)$ is the probability that the particle is at position x after n hops starting from $x_0$. Since jumping is a Poisson process with rate $r+\ell$, the random variable $N\left(t\right)$ has a Poisson distribution,

$$H(n,t)=\frac{{\left((\ell +r)t\right)}^n}{n!}{\mathrm{e}}^{-(\ell +r)t}.$$ (76)

On the other hand, the distribution of the position x after n jumps is the binomial distribution

$$P_n(x;x_0)=\left(\genfrac{}{}{0pt}{}{n}{k_x}\right)\frac{r^{k_x}{\ell }^{n-k_x}}{{(\ell +r)}^n},$$ (77)

where $k_x=(n+(x-x_0)/a)/2$ is the number of jumps to the right, $0\le k_x\le n$ with (77) implied to vanish if $k_x$ is not integer. From Equation (74), the parity of $(x-x_0)/a$ and $N\left(t\right)$ are identical. Using (76) and (77), the probability distribution in (75) reads

$$\begin{array}{cc}\hfill P(x,t;x_0)=& {\mathrm{e}}^{-(\ell +r)t}{\left(\frac{r}{\ell }\right)}^{\frac{x-x_0}{2a}}\mathcal{I}\left(\frac{|x-x_0|}{a},2t\sqrt{r\ell }\right),\hfill \end{array}$$ (78)

where $\mathcal{I}(n,z)$ is the modified Bessel function of the first kind of $n,z\in \mathbb{C}$, which is defined as [42]

$$\mathcal{I}(m,z)=\sum _{j=0}^{\infty }\frac{1}{j!\mathsf{\Gamma }(j+m+1)}{\left(\frac{z}{2}\right)}^{2j+m}.$$ (79)

The transition probability is then

$$W(x\to y;\tau )={\mathrm{e}}^{-(\ell +r)\tau }{\left(\frac{r}{\ell }\right)}^{\frac{y-x}{2a}}\mathcal{I}\left(\frac{|y-x|}{a},2\tau \sqrt{r\ell }\right).$$ (80)

Figure 8.

Simple random walk on an infinite, one-dimensional lattice in continuous time. The black blob indicates the current position of the random walker. The left and right hopping rates, labelled ℓ and r respectively, are assumed to be homogeneous but not equal in general, thus leading to a net drift of the average position.

Using (78) and (80) to calculate the entropy production (4b), we need the following identity for $|y-x|/a=\left|m\right|\ge 1$,

$$\underset{\tau \to 0}{lim}\frac{1}{\tau }\mathcal{I}\left(\left|m\right|,2\tau \sqrt{r\ell }\right)=\sqrt{r\ell }{\delta }_{\left|m\right|,1},$$ (81)

which follows immediately from the definition of the modified Bessel function, Equation (79). It indicates that the only transitions that contribute to the entropy production are those where the particle travels a distance equal to the lattice spacing a. Then, the entropy production reads

$$\begin{array}{cc}\hfill {\dot{S}}_i\left(t\right)=& \frac{1}{2}(r-\ell )ln\left(\frac{r}{\ell }\right)+{\mathrm{e}}^{-(\ell +r)t}\sum _{m=-\infty }^{\infty }{\left(\frac{r}{\ell }\right)}^{\frac{m}{2}}\mathcal{I}\left(\left|m\right|,2t\sqrt{r\ell }\right)\hfill \\ & \times \left[rln\left(\frac{\mathcal{I}\left(\left|m\right|,2t\sqrt{r\ell }\right)}{\mathcal{I}\left(|m+1|,2t\sqrt{r\ell }\right)}\right)+\ell ln\left(\frac{\mathcal{I}\left(\left|m\right|,2t\sqrt{r\ell }\right)}{\mathcal{I}\left(|m-1|,2t\sqrt{r\ell }\right)}\right)\right],\hfill \end{array}$$ (82)

see Figure 9. The entropy flow is ${\dot{S}}_e\left(t\right)=-(r-\ell )ln(r/\ell )$ independent of t, which owes its simplicity to the transition rates being independent of the particle’s position. We are not aware of a method to perform the sum in (82) in closed form and, given that this expression involves terms competing at large times t, we cannot calculate the stationary entropy production ${lim}_{t\to \infty }{\dot{S}}_i\left(t\right)$. If we naively assume that the sum in Equation (82) converges such that it is suppressed by the exponential $exp\left(-(r+\ell )t\right)$, then the entropy production ${\dot{S}}_i$ appears to converge to $\frac{1}{2}(r-\ell )ln(r/\ell )$. If that were the case, $\dot{S}={\dot{S}}_i+{\dot{S}}_e$ would converge to a negative constant, while $S\left(t\right)$, Equation (1), which vanishes at $t=0$ given the initialisation of $P(x,t;x_0)={\delta }_{\left(x-x_0\right)/a,0}$, is bound to be strictly positive at all finite t. Given that $P(x,t;x_0)$ does not converge, not much else can be said about $S\left(t\right)$ or $\dot{S}$. Numerically, using the GNU Scientific Library [43] implementation of Bessel functions, we find that, asymptotically for large times, Equation (82) is

$${\dot{S}}_i\left(t\right)\simeq (r-\ell )ln\left(\frac{r}{\ell }\right)+\frac{1}{2t}+\mathcal{O}\left(t^{-2}\right)$$ (83)

if $r\ne \ell$ and ${\dot{S}}_i\left(t\right)\simeq 1/\left(2t\right)+\mathcal{O}\left(t^{-3}\right)$ if $r=\ell$, see Figure 9.

Figure 9.

Entropy production of a random walk on a one-dimensional lattice (RW on $\mathbb{Z}$), for symmetric and asymmetric hopping rates, as a function of time, Equation (82) (solid lines). The asymptotic behaviour at large t, Equation (83) (dotted lines), decays algebraically in the symmetric case ($r=\ell$) and converges to a positive constant in the asymmetric case ($r\ne \ell$).

To take the continuum limit $a\to 0$ of the probability distribution (78), we define v and D such that $r+\ell =2D/a^2$ and $r-\ell =v/a$. Using the asymptotic expansion of $\mathcal{I}(m,z)$ in m [42]

$$\mathcal{I}(m,z)\sim {\displaystyle \frac{exp\left(z\right)}{\sqrt{2\pi z}}}\left(1-{\displaystyle \frac{4m^2-1}{8z}}+{\displaystyle \frac{\left(4m^2-1\right)\left(4m^2-9\right)}{2!{\left(8z\right)}^2}}-{\displaystyle \frac{\left(4m^2-1\right)\left(4m^2-9\right)\left(4m^2-25\right)}{3!{\left(8z\right)}^3}}+\dots \right),$$ (84)

which is valid for $|argz|<\pi /2$, we obtain in fact the Gaussian distribution,

$$\underset{a\to 0}{lim}\frac{1}{a}P\left(\frac{x}{a},t;\frac{x_0}{a};r(v,D,a),\ell (v,D,a)\right)=\frac{1}{\sqrt{4\pi Dt}}{\mathrm{e}}^{-\frac{{(x-x_0-vt)}^2}{4Dt}},$$ (85)

which corresponds to the distribution of a drift–diffusive particle, which is studied in Section 3.9. Therefore, all results derived in Section 3.9, apply to the present system in the continuum limit.

3.8. Random Walk on a Ring Lattice

In this section, we extend the system in Section 3.7 to a random walk on a ring lattice of length $L>2$, so that $1\le x\le L$, see Figure 10. The probability distribution $P_L(x,t)$ of the particle on the ring follows from the distribution on the one-dimensional lattice $P(x,t)$ in (78), by mapping all positions $x+jL$ on the one-dimensional lattice to position $x\in \{1,2,\dots ,L\}$ on the ring with j being the winding number irrelevant to the evolution of the walker. Then, the distribution on the ring lattice reads,

$$P_L(x,t;x_0)=\sum _{j=-\infty }^{\infty }P(x+jL,t;x_0)$$ (86)

and similarly for the transition probability $W(x\to y,\tau )=P_L(y,\tau ;x)$. To calculate the entropy production (4b), each pair of points $x,y$ on the lattice is mapped to a pair of points on the ring.

Figure 10.

Simple random walk on an periodic, one-dimensional ‘ring’ lattice in continuous time. This model generalises the three-state Markov chain discussed in Section 3.2 to L states. The black blob indicates the current position of the random walker. Due to the finiteness of the state space, this process is characterised by a well defined steady-state, which is an equilibrium one for symmetric rates $\ell =r$.

For $L>2$, as $\tau \to 0$ only transitions to distinct, nearest neighbours contribute and the expression for the entropy production simplifies dramatically,

$${\dot{S}}_i\left(t\right)=(r-\ell )ln\left(\frac{r}{\ell }\right)+\sum _{m=1}^{L/a}{P}_L(ma,t;x_0)\left\{rln\left(\frac{P_L(ma,t;x_0)}{P_L((m+1)a,t;x_0)}\right)+\ell ln\left(\frac{P_L(ma,t;x_0)}{P_L((m-1)a,t;x_0)}\right)\right\}$$ (87)

and similarly for

$${\dot{S}}_e\left(t\right)=-(r-\ell )ln\left(\frac{r}{\ell }\right).$$ (88)

While the entropy flow ${\dot{S}}_e$ on a ring is thus identical to that of a particle on a one-dimensional lattice, the entropy production ${\dot{S}}_i$ on a ring is in principle more complicated, but with a lack of cancellations of $\sqrt{r/\ell }$ in the logarithm as found in Section 3.7 and $P_L$ reaching stationarity comes the asymptote

$$\underset{t\to \infty }{lim}{\dot{S}}_i\left(t\right)=(r-\ell )ln\left(\frac{r}{\ell }\right).$$ (89)

This is easily derived from ${lim}_{t\to \infty }{P}_L(x,t;x_0)=1/L$ taken into the finite sum of Equation (87). It follows that $\dot{S}\left(t\right)={\dot{S}}_i\left(t\right)+{\dot{S}}_e\left(t\right)$ converges to 0 at large t, as expected for a convergent stationary distribution.

The case $L=2$ and the less interesting case $L=1$ are not covered above, because of the different topology of the phase space of $L>2$ compared to $L=2$. The difference can be observed in the different structure of the transition matrices (34) and (39). The framework above is based on each site having two outgoing and two incoming rates, $2L$ in total. However, for $L=2$ there are only two transitions, which cannot be separated into four to fit the framework above, because even when rates of concurrent transitions between two given states are additive, their entropy production generally is not. The case of $L=2$ is recovered in the two-state system of Section 3.1 with $\alpha =\beta =r+\ell$, which is at equilibrium in the stationary state.

3.9. Driven Brownian Particle

In continuum space, the motion of a freely diffusive particle with diffusion constant D and drift v is governed by the Langevin equation $\dot{x}=v+\sqrt{2D}\xi \left(t\right)$, where $\xi \left(t\right)$ is a Gaussian white noise with zero mean, $〈\xi \left(t\right)〉=0$, and covariance $〈\xi \left(t\right)\xi \left(t^{\prime }\right)〉=\delta (t-t^{\prime })$, see Figure 11 [44]. The corresponding Fokker–Planck equation for the probability distribution $P(x,t)$ is [45]

$${\partial }_{t}P(x,t)=-v{\partial }_{x}P(x,t)+D{\partial }_x^{2}P(x,t).$$ (90)

Assuming the initial condition $P(x,0)=\delta (x-x_0)$, the solution to the Fokker–Planck equation is the Gaussian distribution

$$P(x,t)=\frac{1}{\sqrt{4D\pi t}}{\mathrm{e}}^{-\frac{{(x-x_0-vt)}^2}{4Dt}},$$ (91)

which is also the Green function of the Fokker–Planck Equation (90). We therefore also have the transition probability density from state x to state y over an interval $\tau$,

$$W(x\to y,\tau )=\frac{1}{\sqrt{4D\pi \tau }}{\mathrm{e}}^{-\frac{{(y-x-v\tau )}^2}{4D\tau }}.$$ (92)

Figure 11.

Driven Brownian particle on the real line. The black blob indicates the particle’s current position.

Substituting (91) and (92) into Equation (15) for the internal entropy production of a continuous system gives

$${\dot{S}}_i=\underset{\tau \to 0}{lim}\frac{1}{\tau }\int \mathrm{d}x\mathrm{d}y\frac{1}{\sqrt{4D\pi t}}{\mathrm{e}}^{-\frac{{(x-x_0-vt)}^2}{4Dt}}\frac{1}{\sqrt{4D\pi \tau }}{\mathrm{e}}^{-\frac{{(y-x-v\tau )}^2}{4D\tau }}\left(\frac{{(y-x_0)}^2-{(x-x_0)}^2}{4Dt}+\frac{(y-x)v}{2D}\right),$$ (93)

where the Gaussian integrals can be evaluated in closed form, ${\dot{S}}_i\left(t\right)={lim}_{\tau \to 0}\left[1/\left(2t\right)+v^2/D+v^2\tau /\left(2Dt\right)\right]$. Taking the limit $\tau \to 0$ then gives the entropy production rate [44,46,47],

$${\dot{S}}_i\left(t\right)=\frac{1}{2t}+\frac{v^2}{D},$$ (94)

see Figure 12. Similarly, following (16), the entropy flow reads ${\dot{S}}_e\left(t\right)=-v^2/D$ independent of time t. As ${\dot{S}}_i\left(t\right)\ne 0$, we see that for finite t or $v\ne 0$, the system is out of equilibrium with a sustained probability current, so that there is in fact no steady-state distribution. We can verify Equation (94) for the time-dependent internal entropy production by computing the probability current

$$j(x,t)=(v-D{\partial }_x)P(x,t)=\left(\frac{v}{2}+\frac{(x-x_0-vt)}{4t}\right)\frac{e^{-\frac{{(x-x_0-vt)}^2}{4Dt}}}{\sqrt{\pi Dt}}$$ (95)

and substituting it together with (91), into (29). As expected, the two procedures return identical results. The independence of the transient contribution $1/\left(2t\right)$ to the internal entropy production on the diffusion constant is remarkable although necessary on dimensional grounds, as a consequence of ${\dot{S}}_i$ having dimensions of inverse time. The diffusion constant characterising the spatial behaviour of diffusion suggests that it is the temporal, rather than the spatial, features of the process that determine its initial entropy production.

Figure 12.

Entropy production of the drift–diffusion process in an external potential, Equation (100), as a function of time for different parameter combinations. For vanishing potential stiffness, $k\to 0$, we recover Equation (94) for a free drift–diffusion particle. In particular, for $v=0$ the entropy production decays algebraically, while for $v\ne 0$ it converges to the constant value $v^2/D$. For $k>0$, the algebraic decay is suppressed exponentially over long timeframes as the process settles into its equilibrium steady-state.

3.10. Driven Brownian Particle in a Harmonic Potential

Consider a drift–diffusive particle such as in Section 3.9 that is confined in a harmonic potential $V\left(x\right)=\frac{1}{2}k{x}^2$, where k is the potential stiffness, see Figure 13 [48]. The Langevin equation is $\dot{x}=v-kx+\sqrt{2D}\xi \left(t\right)$, where $〈\xi \left(t\right)〉=0$ and $〈\xi \left(t\right)\xi \left(t^{\prime }\right)〉=\delta (t-t^{\prime })$ and the Fokker–Planck equation for $P(x,t)$ is [45]

$${\partial }_{t}P(x,t)=-{\partial }_x\left((v-kx)P(x,t)\right)+D{\partial }_x^{2}P(x,t).$$ (96)

Assuming the initial condition $P(x,0)=\delta (x-x_0)$, the solution to the Fokker–Plank equation is the Gaussian distribution

$$P(x,t)=\sqrt{\frac{k}{2\pi D(1-exp\left(-2kt\right))}}{\mathrm{e}}^{-\frac{{\left(kx-v-\left(k{x}_0-v\right)exp\left(-kt\right)\right)}^2}{2Dk(1-exp\left(-2kt\right))}},$$ (97)

corresponding to a probability current $j(x,t)=(v-kx-D{\partial }_x)P(x,t)$ of the form

$$j\left(x\right)=\frac{\sqrt{\frac{k}{2\pi D(1-e^{-2kt})}}{e}^{-kt}\left(v\left(1-e^{-kt}\right)-k\left(x_0-x{e}^{-kt}\right)\right)}{1-e^{-2kt}}{\mathrm{e}}^{-\frac{(v(1-e^{-kt})-k{(x-x_0{e}^{-kt})}^2}{2Dk(1-e^{-2kt})}}.$$ (98)

The transition probability density within $\tau$ is then also of Gaussian form, namely

$$W(x\to y,\tau )=\sqrt{\frac{k}{2\pi D(1-exp\left(-2k\tau \right))}}{\mathrm{e}}^{-\frac{{\left(ky-v-\left(kx-v\right)exp\left(-k\tau \right)\right)}^2}{2Dk(1-exp\left(-2k\tau \right))}}.$$ (99)

Figure 13.

Driven Brownian particle in a harmonic potential. This process reduces to the standard Ornstein–Uhlenbeck process upon rescaling $x\to x^{\prime }+v/k$. The black blob indicates the particle’s current position. The presence of a binding potential implies that the system relaxes to an equilibrium steady-state over long timeframes.

Using (97) and (99) in (15) gives the entropy production rate

$${\dot{S}}_i=\left(\frac{{(v-k{x}_0)}^2}{D}-k\right){\mathrm{e}}^{-2kt}+\frac{kexp\left(-2kt\right)}{1-exp\left(-2kt\right)},$$ (100)

see Figure 12, and in (16) the external entropy flow

$${\dot{S}}_e=-\left(\frac{{(v-k{x}_0)}^2}{D}-k\right){\mathrm{e}}^{-2kt}.$$ (101)

In the limit $t\to \infty$, the system will reach equilibrium as $P(x,t)$ in Equation (97) converges to the Boltzmann distribution $\sqrt{\frac{k}{2\pi D}}exp\left(-\frac{{(kx-v)}^2}{2Dk}\right)$ of the effective potential $\frac{1}{2}k{x}^2-vx$ at temperature D. This is consistent with (100) and (101), since ${lim}_{t\to \infty }{\dot{S}}_i\left(t\right)={lim}_{t\to \infty }{\dot{S}}_e\left(t\right)=0$. Similarly to drift diffusion on the real line, Equation (94), there is a transient contribution to the entropy production that is independent of the diffusion constant D but does now depend on the stiffness k, which has dimensions of inverse time, through the rescaled time $kt$.

3.11. Driven Brownian Particle on a Ring with Potential

Consider a drift–diffusive particle on a ring $x\in [0,L)$ in a smooth potential $V\left(x\right)$, Figure 14, initialised at position $x_0$. The Langevin equation of the particle is [49,50,51] $\dot{x}=v-{\partial }_{x}V\left(x\right)+\sqrt{2D}\xi \left(t\right)$, where $\xi \left(t\right)$ is Gaussian white noise. The Fokker–Planck equation is then

$${\partial }_{t}P(x,t;x_0)=-{\partial }_x\left((v-V^{\prime }\left(x\right))P(x,t;x_0)\right)+D{\partial }_x^{2}P(x,t;x_0)$$ (102)

with $V^{\prime }\left(x\right)=\frac{d}{dx}V\left(x\right)$ and boundary condition $P^{\left(n\right)}(0,t;x_0)=P^{\left(n\right)}(L,t;x_0)$ for for all $n\ge 0$ derivatives and $t\ge 0$. At stationarity, in the limit $t\to \infty$, where ${\partial }_{t}P(x,t;x_0)=0$, the solution to the Fokker–Planck Equation (102) is [29,51,52]

$$P_s\left(x\right)=\underset{t\to \infty }{lim}P(x,t)=\mathcal{Z}{\mathrm{e}}^{-\frac{V\left(x\right)-vx}{D}}{\int }_x^{x+L}\mathrm{d}y{\mathrm{e}}^{\frac{V\left(y\right)-vy}{D}},$$ (103)

where $\mathcal{Z}$ is the normalisation constant. The corresponding steady-state probability current $j=(v-{\partial }_{x}V)P_s-D{\partial }_x{P}_s$ is independent of x by continuity, $0={\partial }_{t}P=-{\partial }_{x}j$, and reads [45]

$$\begin{array}{cc}\hfill j& =\mathcal{Z}\left({\mathrm{e}}^{-\frac{vL}{D}}-1\right).\hfill \end{array}$$ (104)

Figure 14.

Driven Brownian particle on a ring $x\in [0,L)$ with a periodic potential satisfying $V\left(x\right)=V(x+L)$. Any finite diffusion constant $D>0$ results in a stationary state over long timeframes that is non-equilibrium for $v\ne 0$. The black blob indicates the particle’s current position.

In order to calculate the entropy production according to (15) and (16) using (17), we need $W(x\to y;\tau )$ for small $\tau$. As discussed after Equation (17), $W(x\to y;\tau )$ obeys the Fokker–Planck Equation (102) in the form

$${\partial }_{\tau }W(x\to y;\tau )=-{\partial }_y\left[\left(v-V^{\prime }\left(y\right)\right)W(x\to y;\tau )\right]+D{\partial }_y^{2}W(x\to y;\tau )$$ (105)

with ${lim}_{\tau \to 0}W(x\to y;\tau )=\delta (y-x)$, so that

$$\dot{W}(x\to y)=\underset{\tau \to 0}{lim}{\partial }_{\tau }W(x\to y;\tau )=V^{″}\left(y\right)\delta (y-x)-(v-V^{\prime }\left(y\right)){\delta }^{\prime }(y-x)+D{\delta }^{″}(y-x)$$ (106)

to be evaluated under an integral, where ${\delta }^{\prime }(y-x)=\frac{d}{dy}\delta (y-x)$ will require an integration by parts. As for the logarithmic term, we use [28,45]

$$W(x\to y;\tau )=\frac{1}{\sqrt{4\pi D\tau }}{e}^{-\frac{{\left(y-x-\tau \left(v-V^{\prime }\left(x\right)\right)\right)}^2}{4D\tau }}\left(1+\mathcal{O}\left(\tau \right)\right)$$ (107)

so that

$$ln\left(\frac{W(x\to y;\tau )}{W(y\to x;\tau )}\right)=\frac{y-x}{2D}\left(2v-V^{\prime }\left(x\right)-V^{\prime }\left(y\right)\right)+\mathcal{O}\left(\tau \right).$$ (108)

The entropy flow Equation (16) in the more convenient version Equation (21a) can be obtained easily using Equations (106) and (108)

$$\begin{array}{cc}\hfill {\dot{S}}_e\left(t\right)& =-{\int }_0^L\mathrm{d}x\mathrm{d}yP(x,t)\left(V^{″}\left(y\right)\delta (y-x)-(v-V^{\prime }\left(y\right)){\delta }^{\prime }(y-x)+D{\delta }^{″}(y-x)\right)\hfill \end{array}$$ (109)

$$\begin{array}{cc}\hfill & \times \frac{y-x}{2D}\left(2v-V^{\prime }\left(x\right)-V^{\prime }\left(y\right)\right)\hfill \end{array}$$ (110)

$$\begin{array}{cc}\hfill & =-{\int }_0^L\mathrm{d}xP(x,t)\left(\frac{1}{D}{\left(v-V^{\prime }\left(x\right)\right)}^2-V^{″}\left(x\right)\right)\hfill \end{array}$$ (111)

after suitable integration by parts, whereby derivatives of the $\delta$-function are conveniently interpreted as derivatives with respect to y to avoid subsequent differentiation of $P(x,t)$. Since $\delta (y-x)(y-x)=0$, the factor $(y-x)/\left(2D\right)$ needs to be differentiated for a term to contribute. In the absence of a potential, $P(x,t)=1/L$ at stationarity, so that Equation (111) simplifies to ${\dot{S}}_e\left(t\right)=-v^2/D$ and ${lim}_{t\to \infty }{\dot{S}}_i\left(t\right)=v^2/D$, Equation (94). Using the probability current $j(x,t)=-D{\partial }_{x}P(x,t)+\left(v-V^{\prime }\left(x\right)\right)P(x,t)$, the entropy flow simplifies further to

$${\dot{S}}_e\left(t\right)=-{\int }_0^L\mathrm{d}xj(x,t)\frac{v-V^{\prime }\left(x\right)}{D}$$ (112)

so that at stationarity, when the current is spatially uniform, ${lim}_{t\to \infty }{\dot{S}}_e\left(t\right)=-{lim}_{t\to \infty }j(x,t)vL/D$ as the potential is periodic, entering only via the current.

An equivalent calculation of ${\dot{S}}_i$ on the basis of (20a) gives

$$\begin{array}{cc}\hfill {\dot{S}}_i\left(t\right)& =-{\dot{S}}_e+{\int }_0^L\mathrm{d}x\mathrm{d}yP(x,t)\left(V^{″}\left(y\right)\delta (y-x)-(v-V^{\prime }\left(y\right)){\delta }^{\prime }(y-x)+D{\delta }^{″}(y-x)\right)ln\left(\frac{P(x,t)}{P(y,t)}\right)\hfill \end{array}$$ (113a)

$$\begin{array}{cc}\hfill & =-{\dot{S}}_e+{\int }_0^L\mathrm{d}x\left\{D\frac{{\left(P^{\prime }(x,t)\right)}^2}{P(x,t)}-P(x,t)V^{″}\left(x\right)\right\}\hfill \end{array}$$ (113b)

$$\begin{array}{cc}\hfill & =-{\dot{S}}_e-{\int }_0^L\mathrm{d}xj(x,t){\partial }_{x}lnP(x,t)\hfill \end{array}$$ (113c)

$$\begin{array}{cc}\hfill & ={\int }_0^L\mathrm{d}x\frac{j^2(x,t)}{DP(x,t)},\hfill \end{array}$$ (113d)

with the last line identical to Equation (13).

By considering the functional derivative $\delta Z/\delta V\left(z\right)$ in ${\int }_0^L\mathrm{d}xP\left(x\right)=1$ of Equation (103), one can show that the stationary current $j(x,t)$ Equation (104) is extremal for constant $V\left(x\right)$, indicating that the magnitude of the stationary entropy flow Equation (113d) is maximised in a constant potential.

3.12. Run-and-Tumble Motion with Diffusion on A ring

Consider the dynamics of a run-and-tumble particle on a ring $x\in [0,L)$ with Langevin equation $\dot{x}=v\left(t\right)+\sqrt{2D}\xi \left(t\right)$, where the drift $v\left(t\right)$ is a Poisson process with rate $\alpha$ that alternates the speed of the particle between the constants $v_1$ and $v_2$, and $\xi \left(t\right)$ is Gaussian white noise, Figure 15. Run-and-tumble particles are widely studied as a model of bacterial motion [53]. The drift being $v\left(t\right)=v_1$ or $v\left(t\right)=v_2$ will be referred to as the mode of the particle being 1 or 2 respectively. Defining $P_1(x,t)$ and $P_2(x,t)$ as the joint probabilities that the particle is at position x at time t and in mode 1 or 2 respectively, the coupled Fokker–Planck equations for $P_1$ and $P_2$ are

$$\begin{array}{cc}\hfill {\partial }_t{P}_1(x,t)=& -v_1{\partial }_x{P}_1(x,t)+D{\partial }_x^2{P}_1(x,t)-\alpha (P_1(x,t)-P_2(x,t))\hfill \end{array}$$ (114a)

$$\begin{array}{cc}\hfill {\partial }_t{P}_2(x,t)=& -v_2{\partial }_x{P}_2(x,t)+D{\partial }_x^2{P}_2(x,t)-\alpha (P_2(x,t)-P_1(x,t))\hfill \end{array}$$ (114b)

whose stationary solution is the uniform distribution ${lim}_{t\to \infty }{P}_1(x,t)={lim}_{t\to \infty }{P}_2\left(x\right)=1/\left(2L\right)$, as is easily verified by direct substitution. The corresponding steady-state probability currents thus read $j_1=v_1/\left(2L\right)$ and $j_2=v_2/\left(2L\right)$.

Figure 15.

Run-and-tumble motion with diffusion on a ring $x\in [0,L)$. A run-and-tumble particle switches stochastically, in a Poisson process with rate $\alpha$, between two modes 1 and 2 characterised by an identical diffusion constant D but distinct drift velocities $v_1$ and $v_2$. The two modes are here represented in black and grey, respectively. For arbitrary positive diffusion constant D or tumbling rate $\alpha$ with $v_1\ne v_2$, the steady state is uniform but generally non-equilibrium.

In the following, we denote by the propagator $W(x\to y,Q\to R;\tau )$ the probability density that a particle at position x in mode Q is found time $\tau$ later at position y in mode R. For $Q=R$, this propagation is a sum over all even numbers m of Poissonian switches, that occur with probability ${\left(\alpha \tau \right)}^{m}exp\left(-\alpha \tau \right)/m!$, which includes the probability $exp\left(-\alpha \tau \right)$ of not switching at all over a total of time $\tau$. For $Q\ne R$, the propagation is due to an odd number of switches.

For $m=0$, the contribution to $W(x\to y,Q\to R;\tau )$ is thus $exp\left(-\alpha \tau \right)W(x\to y;\tau )$, with $W(x\to y;\tau )$ of a drift diffusion particle on a ring, Section 3.11, but without potential, approximated at short times $\tau$ by the process on the real line, Equation (92) with drift $v=v_1$ or $v=v_2$ according to the particle’s mode. For $m=1$, the contribution is a single convolution over the time $t^{\prime }\in [0,\tau )$ at which the particle changes mode, most easily done after Fourier transforming. Before presenting this calculation in real space, we argue that any such convolution will result in some approximate Gaussian with an amplitude proportional to $1/\sqrt{\tau }$ multiplied by a term of order ${\left(\alpha \tau \right)}^m$. In small $\tau$, therefore only the lowest orders need to be kept, $m=0$ for $Q=R$ and $m=1$ for $Q\ne R$.

More concretely,

$$\begin{array}{cc}\hfill & W(x\to y,1\to 2;\tau )\hfill \end{array}$$

$$\begin{array}{cc}\hfill =& {\int }_{-\infty }^{\infty }\mathrm{d}z{\int }_0^{\tau }\mathrm{d}{\tau }^{\prime }\frac{1}{\sqrt{4\pi D{\tau }^{\prime }}}{\mathrm{e}}^{-\frac{{(z-x-v_1{\tau }^{\prime })}^2}{4D{\tau }^{\prime }}}{\mathrm{e}}^{-\alpha {\tau }^{\prime }}\frac{1}{\sqrt{4\pi D(\tau -{\tau }^{\prime })}}{\mathrm{e}}^{-\frac{{\left(y-z-v_2(\tau -{\tau }^{\prime })\right)}^2}{4D(\tau -{\tau }^{\prime })}}{\mathrm{e}}^{-\alpha (\tau -{\tau }^{\prime })}+\dots \hfill \end{array}$$ (115)

$$\begin{array}{cc}\hfill =& \frac{\alpha exp\left(-\alpha \tau \right)}{2(v_1-v_2)}\left[\mathrm{erf}\left(\frac{x-y+v_1\tau }{\sqrt{4D\tau }}\right)-\mathrm{erf}\left(\frac{x-y+v_2\tau }{\sqrt{4D\tau }}\right)\right]+\dots \hfill \end{array}$$ (116)

which in small $\tau$, when $v_{1,2}\tau /\sqrt{4D\tau }\ll 1$, so that $\mathrm{erf}(r+\epsilon )=\mathrm{erf}\left(r\right)+2\epsilon e^{-r^2}/\sqrt{\pi }+\dots$, expands to

$$\begin{array}{cc}\hfill W(x\to y,1\to 2;\tau )& =\frac{\alpha \tau }{\sqrt{4\pi D\tau }}{e}^{-\frac{{(y-x)}^2}{4D\tau }}\left(1+\mathcal{O}\left(\tau \right)\right)\hfill \\ \hfill & =W(x\to y,2\to 1;\tau ),\hfill \end{array}$$ (117)

whereas $W(x\to y,Q\to Q;\tau )$, the propagator with an even number of mode switches, is given by Equation (92) to leading order in $\tau$,

$$W(x\to y,Q\to Q;\tau )=\frac{1}{\sqrt{4\pi D\tau }}{e}^{-\frac{{(y-x-v_Q\tau )}^2}{4D\tau }-\alpha \tau }\left(1+\mathcal{O}\left(\tau \right)\right).$$ (118)

Much of the calculation of the entropy production follows the procedure in Section 3.9 and Section 3.11 to be detailed further below. To this end, we also need

$$\begin{array}{cc}\hfill \underset{\tau \to 0}{lim}\frac{\mathrm{d}}{\mathrm{d}\tau }W(x\to y,1\to 2;\tau )& =\dot{W}(x\to y,1\to 2)=\alpha \delta (x-y)\hfill \\ \hfill & =\dot{W}(x\to y,2\to 1).\hfill \end{array}$$ (119)

As far as processes are concerned that involve a change of particle mode, therefore only the transition rates enter, not diffusion or drift. Given a uniform stationary spatial distribution of particles of any mode, mode changes between two modes cannot result in a sustained probability current, even when the switching rates differ,

$$\left(P_1\dot{W}(1\to 2)-P_2\dot{W}(2\to 1)\right)ln\left(\frac{P_1\dot{W}(1\to 2)}{P_2\dot{W}(2\to 1)}\right)=0$$ (120)

for $P_1\dot{W}(1\to 2)=P_2\dot{W}(2\to 1)$ at stationarity as in the process discussed in Section 3.1. A probability current and thus entropy production can occur when different particle modes result in a different distribution, Section 3.10, or when mode switching between more than two modes results in a current in its own rights, Section 3.2 and Section 3.13.

Since the full time-dependent density is beyond the scope of the present work, we calculate entropy flow and production at stationary on the basis of a natural extension of Equations (4), (16) and (21a) to a mixture of discrete and continuous states

$$\begin{array}{cc}\hfill \phantom{=}& -\underset{t\to \infty }{lim}{\dot{S}}_e\left(t\right)=\underset{t\to \infty }{lim}{\dot{S}}_i\left(t\right)\hfill \\ \hfill & =\sum _{Q,R\in \{1,2\}}{\int }_0^L\mathrm{d}x\mathrm{d}y{P}_Q(x,t)\dot{W}(x\to y,Q\to R)\underset{\tau \to 0}{lim}ln\left(\frac{W(x\to y,Q\to R;\tau )}{W(y\to x,R\to Q;\tau )}\right)\hfill \end{array}$$ (121)

$$\begin{array}{cc}\hfill & =\frac{v_1^2+v_2^2}{2D}\hfill \end{array}$$ (122)

which immediately follows from Section 3.9 and Section 3.11, as the stationary density is constant, $P_Q=P_R=1/\left(2L\right)$, and only $Q=R$ contribute, with

$$\underset{\tau \to 0}{lim}ln\left(\frac{W(x\to y,1\to 2;\tau )}{W(y\to x,2\to 1;\tau )}\right)=0.$$ (123)

If the drifts are equal in absolute value $|v_1|=|v_2|=v$, then we recover the entropy production of a simple drift–diffusive particle, ${\dot{S}}_i=v^2/D$. This is because we can think of run-and-tumble as a drift–diffusion particle that changes direction instantly. Since changing the direction produces no entropy, the total entropy production rate should be the same as a drift–diffusion particle. The entropy production can alternatively be derived via (29) by computing ${\dot{S}}_i=\int \mathrm{d}x\left(j_1^2/\left(D{P}_1\right)+j_2^2/\left(D{P}_2\right)\right)$ with the steady-state currents stated above.

3.13. Switching Diffusion Process on a Ring

The dynamics of a one-dimensional run-and-tumble particle discussed above can be readily generalised to the so called switching diffusion process [54] by allowing for an extended set $\left\{v_i\right\}$ of drift modes $i=1,\cdots ,M$, Figure 16. The corresponding Langevin equation for the particle position on a ring $x\in [0,L]$ is almost identical to that of run-and-tumble, namely $\dot{x}=v\left(t\right)+\sqrt{2D}\xi \left(t\right)$, with the exception that the process $v\left(t\right)$ is now an M-state Markov process. In the general case, a single switching rate $\alpha$ is thus not sufficient and the full transition rate matrix ${\alpha }_{ij}$ needs to be provided. In this formulation, the run-and-tumble dynamics Section 3.12 correspond to the choice $M=2$ with symmetric rates ${\alpha }_{12}={\alpha }_{21}=\alpha$. Defining $P_i(x,t)$ as the joint probability that at time t the particle is at position x and in mode i, thereby moving with velocity $v_i$, the system (114) of Fokker–Planck equations generalises to

$${\partial }_t{P}_i(x,t)=-{\partial }_x\left[(v_i-D{\partial }_x)P_i(x,t)\right]+\sum _j{P}_j(x,t){\alpha }_{ji}$$ (124)

where the transmutation rates ${\alpha }_{ij}$ from mode i to mode j are assumed to be independent of position. To ease notation we use the convention ${\alpha }_{jj}=-{\sum }_{i\ne j}{\alpha }_{ji}$. For a non-vanishing diffusion constant, the stationary solution is uniform for all modes and given by ${lim}_{t\to \infty }{P}_i(x,t)=z_i/L$, where $z_i$ is the ith element of the eigenvector $\mathbf{z}$ satisfying ${\sum }_j{z}_j=1$ and the eigenvalue relation ${\sum }_j{z}_j{\alpha }_{ji}=0$, which we assume to be unique for simplicity.

Figure 16.

Switching diffusion process on a ring $x\in [0,L]$ in continuous time. A switching diffusion process involves a stochastic switching between M modes characterised by an identical diffusion constant D but distinct drifts $v_i$ ($i=1,2,\cdots ,M$). The marginal switching dynamics are characterised as an M-state Markov process with transition rates ${\alpha }_{ij}$ from mode i to mode j.

The calculation of the steady-state entropy production follows very closely that of run-and-tumble presented above. The conditional transition probabilities including up to one transmutation event read to leading order

$$W(x\to y,i\to j;\tau )=\left\{\begin{array}{cc}\frac{e^{{\alpha }_{ii}\tau }}{\sqrt{4\pi D\tau }}{\mathrm{e}}^{-\frac{{\left(y-x-v_i\tau \right)}^2}{4D\tau }}\left(1+\mathcal{O}\left(\tau \right)\right)\hfill & \mathrm{for}i=j\hfill \\ \frac{{\alpha }_{ij}}{2(v_i-v_j)}\left[\mathrm{erf}\left(\frac{x-y+v_i\tau }{\sqrt{4D\tau }}\right)-\mathrm{erf}\left(\frac{x-y+v_j\tau }{\sqrt{4D\tau }}\right)\right]\left(1+\mathcal{O}\left(\tau \right)\right)\hfill & \mathrm{for}i\ne j,\hfill \end{array}\right.$$ (125)

so that

$$\underset{\tau \to 0}{lim}\frac{\mathrm{d}}{\mathrm{d}\tau }W(x\to y,i\to j;\tau )=\left\{\begin{array}{cc}D{\partial }_y^2\delta (y-x)-v_i{\partial }_y\delta (y-x)+{\alpha }_{ii}\delta (y-x)\hfill & \mathrm{for}i=j\hfill \\ {\alpha }_{ij}\delta (y-x)\hfill & \mathrm{for}i\ne j.\hfill \end{array}\right.$$ (126)

We could perform the calculation of the entropy production using the procedure of Section 3.9 rather than drawing on the operator for $i=j$, which, however, is used in the following for convenience, see Section 3.11. Substituting (125) and (126) into (20a) and assuming steady-state densities, we arrive at

$$\begin{array}{cc}\hfill \underset{t\to \infty }{lim}{\dot{S}}_i\left(t\right)=& -\underset{t\to \infty }{lim}{\dot{S}}_e\left(t\right)\hfill \\ \hfill =& {\int }_0^L\mathrm{d}x\mathrm{d}y\sum _i\frac{z_i}{L}\left(D{\partial }_y^2\delta (y-x)-v_i{\partial }_y\delta (y-x)+{\alpha }_{ii}\delta (y-x)\right)(y-x)\frac{v_i}{D}\hfill \\ \hfill & +{\int }_0^L\mathrm{d}x\mathrm{d}y\sum _{i,j\ne i}\frac{z_i}{L}{\alpha }_{ij}\delta (y-x)ln\left(\frac{{\alpha }_{ij}}{{\alpha }_{ji}}\right),\hfill \end{array}$$ (127)

where we have used Equation (126) in the operators containing the $\delta$-functions and Equation (125) in the logarithms. The term $ln({\alpha }_{ij}/{\alpha }_{ji})$ is obtained by the same expansion as used in Equation (117), Section 3.12. Both terms contributing to the entropy production above are familiar from previous sections: the first is a sum over the entropy production of M drift–diffusion processes with characteristic drift $v_i$, Section 3.11 without potential, weighted by the steady-state marginal probability $z_i$ for the particle to be in state i; the second is the steady-state entropy production of an M-state Markov process with transition rate matrix ${\alpha }_{ij}$, which reduces to Equation (4) after integration. Carrying out all integrals, we finally have

$$\underset{t\to \infty }{lim}{\dot{S}}_i\left(t\right)=\underset{t\to \infty }{lim}-{\dot{S}}_e\left(t\right)=\sum _i{z}_i\frac{v_i^2}{D}+\frac{1}{2}\sum _{i,j}(z_i{\alpha }_{ij}-z_j{\alpha }_{ji})ln\left(\frac{{\alpha }_{ij}}{{\alpha }_{ji}}\right).$$ (128)

Unlike run-and-tumble, Section 3.12, the transmutation process in switching diffusion does in general contribute to the entropy production for $M>2$, since the stationary state generally does not satisfy global detailed balance. However, contributions to the total entropy production originating from the switching and those from the diffusion parts of the process are effectively independent at steady state, as only the stationary marginal probabilities $z_i$ of the switching process feature as weights in the entropy production of the drift–diffusion. Otherwise, the parameters characterising the two processes stay separate in Equation (128). Further, the drift–diffusion contributions of the form $v_i^2/D$ are invariant under the time-rescaling ${\alpha }_{ij}\to T{\alpha }_{ij}$. This property originates from the steady-state distributions $P_i\left(x\right)$ being uniform and would generally disappear in a potential, Section 3.10.

4. Discussion and Concluding Remarks

In this work, we calculate the rate of entropy production within Gaspard’s framework [11] from first principles in a collection of paradigmatic processes, encompassing both discrete and continuous degrees of freedom. Based on the Markovian dynamics of each system, where we can, we derive the probability distribution of the particle (or particles) as a function of time $P(x,t)$ from Dirac or Kronecker-$\delta$ initial conditions $P(x,0)=\delta (x-x_0)$, from which the transition probability $W(x\to y;\tau )$ follows straightforwardly. In some cases, we determine only the stationary density and the (short-time) propagator $W(x\to y;\tau )$ to leading order in $\tau$. We then use Equation (4) for discrete systems or Equations (20) and (21) for continuous systems to calculate the time-dependent entropy production. We set out to give concrete, exact results in closed form, rather than general expressions that are difficult to evaluate, even when we allowed for general potentials in Section 3.11. In summary, the ingredients that are needed to calculate the entropy production in closed form in the present framework are: (a) the probability (density) $P(x,t)$ to find the system in state x ideally as a function of time t and (b) the propagator $W(x\to y;\tau )$, the probability (density) that the system is found at a certain state y after some short time $\tau$ given an initial state x. If the propagator is known for any time $\tau$, it can be used to calculate the probability $P(x,t;x_0)=W(x_0\to x;t)$ for some initial state $x_0$. However, this full time dependence is often difficult to obtain. The propagator is further needed in two forms, firstly ${lim}_{\tau \to 0}{\partial }_{\tau }W(x\to y;\tau )$ when it is most elegantly written as an operator in continuous space, and secondly ${lim}_{\tau \to 0}ln(W(x\to y;\tau )/W(y\to x;\tau ))$.

For completeness, where feasible, we have calculated the probability current $j(x,t)$ in continuous systems at position x. The mere presence of such a flow indicates broken time-reversal symmetry and thus non-equilibrium. Our results on the discrete systems (Section 3.1, Section 3.2, Section 3.3, Section 3.4, Section 3.5, Section 3.6, Section 3.7 and Section 3.8) illustrate two important aspects of entropy production. First, the need of a probability flow $P_A\dot{W}(A\to B)-P_B\dot{W}(B\to A)$ between states: in the two-state system Section 3.1 there are no transition rates $\alpha$ and $\beta$ such that there is a sustained probability flow and therefore, the system inevitably relaxes to equilibrium. However, in the three-state system Section 3.2 the transition rates can be chosen so that there is a perpetual flow $(\alpha -\beta )/3$ between any two states and therefore there is entropy production not only during relaxation but also at stationarity. Hence, we can ascertain these as non-equilibrium steady states in the long term limit due to the non-vanishing rate of internal entropy production. Uniformly distributed steady states can be far from equilibrium, as a rigorous analysis on the basis of the microscopic dynamics reveals, although an effective dynamics may suggest otherwise.

Second, we see how the extensivity of entropy production arises in the N-particle systems (Section 3.4, Section 3.5 and Section 3.6), independently of whether the particles are distinguishable or not. We therefore conclude that the number of particles in the system must be accounted for when calculating the entropy production, and doing otherwise will not lead to a correct result. This is sometimes overlooked, especially when using effective theories. In the continuous systems (Section 3.9, Section 3.10 and Section 3.11), which involve a drift v and a diffusion constant D, we always find the contribution $v^2/D$ to the entropy production emerging one way or another. Moreover, in the case of drift–diffusion on the real line (Section 3.9) we find that the contribution due to the relaxation of the system $1/\left(2t\right)$ is independent of any of the system parameters.

Finally, we have studied two systems (Section 3.12 and Section 3.13) where the state space has a discrete and a continuous component. The discrete component corresponds to the transmutation between particle species, i.e., their mode of drifting, whereas the continuous component corresponds to the particle motion. We find that both processes, motion and transmutation, contribute to the entropy production rate essentially independently since any term that combines both processes is a higher-order term contribution in $\tau$, and therefore vanishes in the limit $\tau \to 0$.

This work has applications to the field of active particle systems, where particles are subject to local non-thermal forces. In fact, the systems studied in Section 3.2, Section 3.8, Section 3.9, Section 3.10, Section 3.11, Section 3.12 and Section 3.13 are prominent examples of active systems. We have shown that their entropy production crucially relies on the microscopic dynamics of the system, which are captured by the Fokker–Planck equation (or the master equation for discrete systems) and its solution. However, in interacting many-particle systems, such a description is not available in general. Instead, we may choose to use the Doi–Peliti formalism [55,56,57,58,59,60,61,62,63] to describe the system, since it provides a systematic approach based on the microscopic dynamics and which retains the particle entity.

Author Contributions

Formal analysis, L.C., R.G.-M., Z.Z., B.B. and G.P.; Supervision, G.P.; Writing—original draft, L.C., R.G.-M., Z.Z. and B.B.; Writing—review & editing, G.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.