Accelerated First-Passage Dynamics in a Non-Markovian Feedback Ornstein–Uhlenbeck Process

Abstract

We study the first-passage dynamics of a non-Markovian stochastic process with time-averaged feedback, which we model as a one-dimensional Ornstein–Uhlenbeck process wherein the particle drift is modified by the empirical mean of its trajectory. This process maps onto a class of self-interacting diffusions. Using weak-noise large deviation theory, we calculate the leading order asymptotics of the time-dependent distribution of the particle position, derive the most probable paths that reach the specified position at a given time and quantify their likelihood via the action functional. We compute the feedback-modified Kramers rate and its inverse, which approximates the mean first-passage time, and show that the feedback accelerates dynamics by storing finite-time fluctuations, thereby lowering the effective energy barrier and shifting the optimal first-passage time from infinite to finite. Although we identify alternative mechanisms, such as slingshot and ballistic trajectories, we find that they remain sub-optimal and hence do not accelerate the dynamics. These results show how memory feedback reshapes rare event statistics, thereby offering a mechanism to potentially control first-passage dynamics.

Introduction

Many real-world systems exhibit memory-feedback dynamics [1–4], where the evolution of a process depends on its past trajectory. Unlike standard Markovian systems, these processes are influenced by accumulated memory. Such feedback mechanisms introduce rich dynamical features, altering long-time behavior and fluctuation properties in ways not captured by memoryless models [5–13]. In particular, memory-feedback mechanisms are relevant for designing control strategies to achieve specific tasks [14–18] with specific thermodynamic properties [19–25].

Self-interacting systems [26–32] constitute a notable class of memory-feedback processes in which past states, encoded in the empirical occupation measure, or in a time-integrated functional of the process, influence future evolution. Examples include models in statistical physics relevant to biological [33–37] and colloidal systems [38–40], where interactions with the occupation measure affect the dynamics.

Although the stationary properties of self-interacting processes have been studied extensively, the role of feedback in shaping rare event statistics, and in particular first-passage dynamics, remains poorly understood. However, understanding the finite-time stochastic dynamics of feedback-driven systems has many important implications in, among other problems, reinforcement learning algorithms [14].

Only recently have large-deviation results for self-interacting systems begun to emerge, both in the long-time regime for discrete Markov chains [41, 42] and for diffusions [43]. For example, first-passage dynamics has been addressed in self-interacting discrete-time random walks [44], where universal scaling laws for first-passage time distributions were identified, and in diffusive systems [45], where by analyzing the asymptotic behavior of the first-passage time in the weak-noise limit a Kramers-type law was derived.

Here we analyze a representative case: a one-dimensional Ornstein–Uhlenbeck process where the drift term is influenced by its own time-averaged position. This feedback mechanism breaks Markovianity and defines a self-interacting diffusion process, fundamentally altering the first-passage dynamics of the system.

By using weak-noise large deviation theory, we show that self-interaction can lead to accelerated first-passage dynamics, and thus fluctuation-driven mechanisms induced by memory. In particular, we identify and characterize different first-passage scenarios and quantify their likelihood. Our large deviation approach allows us to derive a Kramers-type law for the mean first-passage time in the feedback Ornstein–Uhlenbeck process, which complements and extends the results in [44, 45]. We show that for a positive feedback parameter $\beta >0$ (see Eq. 1), the leading-order noise asymptotics of the first-passage time distribution develops a new global minimum in time. As a result, the effective energy barrier is reduced, causing exponential speed-up and an exponentially smaller mean first-passage time. Furthermore, our findings generalize recent results on first-passage dynamics in other classes of memory-driven systems, such as generalized Langevin equations [46, 47].

The paper is organized as follows. In Section 2, we introduce the Ornstein–Uhlenbeck process with empirical average position feedback, establish its mapping to self-interacting diffusion, and discuss its deterministic and long-time dynamics. In Section 3, we present the weak-noise large deviation framework used to derive our results. We apply it to both the standard and feedback Ornstein–Uhlenbeck processes, and derive the fundamental equation (61) governing finite-time dynamics in the weak-noise regime, which we have not found in the literature. In Section 4, we detail the consequences of solving (61). We describe the finite-time mechanisms driving first-passage dynamics in the feedback process, compare them to the standard Ornstein–Uhlenbeck case, and analyze their relative and absolute likelihoods. We characterize the quasi-potential (i.e., the large deviation rate function associated with the Arrhenius law) and the optimal first-passage times. Finally, in Section 5 we summarize our findings, highlight open questions, and discuss potential applications, particularly in the context of reinforcement learning.

Stochastic Feedback Ornstein–Uhlenbeck Processes and Deterministic Dynamics

Ornstein–Uhlenbeck Process with Empirical Average Position Feedback

We consider a one-dimensional process ${(X_t)}_{t\ge 0}$ governed by the following stochastic differential equation (SDE):

$$\begin{array}{c}\hfill d{X}_t=-X_{t}dt+\frac{\beta }{t}\left({\int }_0^t,X_s,,d,s\right)dt+\sqrt{ϵ}d{W}_t,\end{array}$$ 1

with initial condition $X_0=0$, noise intensity $ϵ>0$, and feedback parameter $\beta \in (-\infty ,1)$, the range of which we will justify in detail below. Here, $W_t$ denotes a standard Wiener process, or Brownian motion, in $\mathbb{R}$, characterized by independent Gaussian increments: $\mathbb{E}[d{W}_t]=0$ and $\mathbb{E}[d{W}_{s}d{W}_t]=\delta (t-s)$.

In the absence of the additional feedback term involving the time-averaged position $\frac{1}{t}{\int }_0^t{X}_{s}ds$, Eq. (1) is the classical Ornstein–Uhlenbeck (OU) process, describing a Brownian agent moving in a parabolic potential well centered on the origin. As we will show, the inclusion of the empirical time-averaged position introduces a non-Markovian feedback that can give rise to unique dynamical behavior, effectively modifying the potential in which the particle evolves. This results in an effective potential whose minimum shifts along $\mathbb{R}$ depending on the cumulative history of the process. We discuss this in more detail in Section 2.2.

Our objective is to analyze how this non-Markovian feedback influences the first-passage dynamics of $X_t$ through a threshold $x_f>0$, with a particular focus on the role of noise and memory. The feedback term can be expressed as

$$\begin{array}{cc}\hfill {\overline{X}}_t& :=\frac{1}{t}{\int }_0^t{X}_{s}ds\hfill \end{array}$$ 2

$$\begin{array}{cc}& ={\int }_{\mathbb{R}}x{\rho }_t(x)dx,\mathrm{where}\hfill \end{array}$$ 3

$$\begin{array}{c}\hfill {\rho }_t(x)=\frac{1}{t}{\int }_0^t\delta (X_s-x)ds\end{array}$$ 4

is the empirical occupation measure of the process. This measure captures the fraction of time the system has spent in the vicinity of each position x up to time t.

Although because its evolution depends on its entire history, the process ${(X_t)}_{t\ge 0}$ is non-Markovian, the joint two-dimensional process ${(X_t,{\overline{X}}_t)}_{t\ge 0}=:{(Z_t)}_{t\ge 0}$ is Markovian and satisfies:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}d{X}_t\hfill & =-X_{t}dt+\beta {\overline{X}}_{t}dt+\sqrt{ϵ}d{W}_t\hfill \\ d{\overline{X}}_t\hfill & =\frac{X_t-{\overline{X}}_t}{t}dt.\hfill \end{array}\right)\end{array}$$ 5

Despite the fact that the process ${(Z_t)}_{t\ge 0}$ is time-inhomogeneous, the Markovian formulation (5) allows us to study the first-passage dynamics of the feedback OU process semi-analytically in the weak-noise regime, as shown in Section 3.3.

Mapping to Self-Interacting Diffusion Processes

The structure of the feedback embodied in Eqs. (2)–(4), allows the process ${(X_t)}_{t\ge 0}$ in Eq. (1) to be interpreted as a specific case of a one-dimensional self-interacting diffusion (SID). A general SID in ${\mathbb{R}}^d$ is typically expressed as

$$\begin{array}{c}\hfill d{X}_t=-\mathrm{\nabla }V(X_t)dt-\frac{1}{t}\left({\int }_0^t,\mathrm{\nabla },K,(X_s,X_t),,d,s\right)dt+\sqrt{ϵ}d{W}_t,\end{array}$$ 6

where $V(X_t)$ is a background potential and $K(X_s,X_t)$ is an interaction potential whose gradient is taken with respect to $X_t$. The interaction term describes how the current position is influenced by the history of past positions, functionally through the empirical occupation measure defined in Eq. (4).

The long-time behavior of self-interacting processes, including random walks and diffusions, has been an active area of research in probability theory for several decades. Self-interacting random walks were first introduced in the discrete-time setting by Norris et al. [48], and continuous-time counterparts appeared soon after as models for polymer growth [49]. The normalized SID considered here (see Eq. (6)) was introduced by Benaïm et al. [27] as an example of a continuous-time path-interaction (or reinforcement) process. It extends earlier classes of self-interacting processes based on non-normalized occupation measures [50, 51] and discrete-time reinforced random walks. A thorough historical overview is given in [28].

The asymptotic properties of SIDs were first studied in [27, 52–54], with results established for compact spaces and symmetric interaction potentials. These were later extended to open domains, in particular to ${\mathbb{R}}^d$ under confining potentials [30, 55, 56], which is directly relevant for our feedback OU process. Stochastic approximation methods were used in [27, 30] to describe the asymptotic behavior of the occupation measure ${\rho }_t$ via a limiting non-autonomous differential equation. A key result of [53] shows that for symmetric potentials ${\rho }_t$ converges almost surely to a local minimum of a nonlinear free-energy functional, which typically admits multiple critical points, each selected at random with some probability. In the special case of symmetric self-repelling potentials this functional is strictly convex, so ${\rho }_t$ converges almost surely to a unique global minimum in the long-time limit.

More recently, refinements of Kramers’ law for SIDs have been obtained [45, 57] (see also [46, 47] for generalized overdamped Langevin systems), and these results extend to systems with general additive noise [45, 57, 58]. A recent contribution by one of us establishes large-deviation results for SIDs on the ring [43]. We will detail below (Sec. 2.4) those results that are most relevant for the present work.

For the case of Eq. (1), the potentials in Eq. (6) take the form

$$\begin{array}{cc}\hfill V(X_t)& =\left(1,-,\beta \right)\frac{X_t^2}{2}\mathrm{and}\hfill \end{array}$$ 7

$$\begin{array}{cc}\hfill K(X_s,X_t)& =K(X_s,X_t)=\beta \frac{{(X_t-X_s)}^2}{2}.\hfill \end{array}$$ 8

Therefore, a quadratic interaction potential K describes the attraction of the current position $X_t$ to the past positions $X_s$, so that the general behavior of the process is determined by the combination of this interaction and the background potential V.

By rewriting Eq. (1), we can define an effective potential

$$\begin{array}{c}\hfill \tilde{V}(X_t)=V(X_t)+\frac{1}{t}{\int }_0^{t}K(X_s,X_t)ds,\end{array}$$ 9

whose shape evolves according to the past trajectory of the process. For example, the minimum of this effective potential may shift to the left or right of the origin depending on whether the integral contribution in Eq. (9) is negative or positive, respectively.

Eqs. (7) and (8) make clear that for $\beta <1$ ($\beta >1$), the background potential remains quadratic and attractive (repulsive) around the origin. Although the process is still attracted to its past positions through the interaction potential K, the repulsive background potential drives first-passage dynamics towards infinity, which justifies the choice $\beta <1$ described when we introduced Eq. (1).

More generally, Eq. (1) can be extended to incorporate alternative forms of feedback, such as

$$\begin{array}{c}\hfill {\overline{K}}_t:=t^{-\alpha }{\int }_0^{t}f(X_s)ds,\end{array}$$ 10

with $\alpha >0$, which ensures a decaying contribution, with slower decay as $\alpha$ decreases, and f is an arbitrary function on $\mathbb{R}$. Although the methods developed here could be generalized to such circumstances, we focus on the simplest and most natural case: feedback via the empirical average position, corresponding to $\alpha =1$ and $f(X_s)=X_s$. Importantly, this choice enables an exact mapping to SID processes. Alternative values of $\alpha$, or more general time-dependent functions, would typically not allow this mapping and thus preclude comparison with well-established convergence properties of SIDs, which we discuss in Section 2.4.

Exact Results for the Deterministic Dynamics

Before analyzing the stochastic dynamics of the feedback OU process, we examine some essential features of the deterministic dynamics, obtained by setting $ϵ=0$ in Eqs. (1) or (5). This analysis provides insight into the behavior of the system and helps clarify the choice of the parameter range for $\beta$.

We set $ϵ=0$ in Eq. (5), which becomes the following first-order, time-inhomogeneous linear system of differential equations;

$$\begin{array}{c}\hfill {\dot{Z}}_t=A_t(\beta )Z_t,\end{array}$$ 11

where $\dot{()}$ denotes the time derivative, and the time-dependent drift matrix is

$$\begin{array}{c}\hfill A_t(\beta )=\left(\begin{array}{cc}-1& \beta \\ \frac{1}{t}& -\frac{1}{t}\end{array}\right).\end{array}$$ 12

For $\beta \in \mathbb{R}\backslash \{1\}$, the only fixed point of the system is the origin (0, 0). However, when $\beta =1$, all points satisfying $X=\overline{X}$ are fixed points.

The stability of the origin depends on both $\beta$ and time as follows (see Fig. 1 for comparison):

$\beta <0$:

The origin is stable, but exhibits a spiral-like behavior in the finite time window $[t_{-},t_{+}]$ defined below.

$\beta \in [0,1)$:

The origin is a stable node.

$\beta >1$:

The origin becomes a saddle point, and an instability develops along a specific direction. At the bifurcation point $\beta =1$, all points $X=\overline{X}$ are neutrally stable.

Fig. 1.

Phase portraits of the deterministic dynamical systems in Eq. (11) with (12) for three illustrative values of $\beta =-0.5$ (left), $\beta =0.25$ (middle), and $\beta =1.5$ (right)

Due to the linearity of Eq. (11), stability is fully characterized by the eigenvalues of the $2\times 2$ matrix $A_t(\beta )$, which are always real and given by:

$$\begin{array}{c}\hfill {\lambda }_t^{\pm }=\frac{-(1+t)\pm \sqrt{{(1+t)}^2-4t(1-\beta )}}{2t}.\end{array}$$ 13

For $\beta \in [0,1)$, the square root term in Eq. (13) satisfies:

$$\begin{array}{c}\hfill \sqrt{{(1+t)}^2-4t(1-\beta )}\le 1+t,\end{array}$$ 14

implying that both eigenvalues ${\lambda }_t^{\pm }$ are negative for all $t>0$, and the origin is a stable node.

For $\beta <0$, the discriminant in Eq. (13) can become negative, leading to complex eigenvalues. This occurs when $t\in [t_{-},t_{+}]$, where

$$\begin{array}{c}\hfill t_{\pm }=-(2\beta -1)\pm 2\sqrt{\beta (\beta -1)}.\end{array}$$ 15

Since the real parts of both eigenvalues remain negative, the origin remains stable, but the dynamics exhibit a spiral behavior during this time interval.

For $\beta >1$, the situation is reversed: one eigenvalue ${\lambda }_t^{-}<0$, while the other ${\lambda }_t^{+}>0$. An unstable direction thus emerges, associated with the eigenvector

$$\begin{array}{c}\hfill u_t^{+}=\left(\begin{array}{c}\frac{1}{2}\left[1,-,t,+,\sqrt{{(1+t)}^2-4t(1-\beta )}\right]\\ 1\end{array}\right).\end{array}$$ 16

Here, the fixed point at the origin becomes a saddle. When $\beta =1$, the direction $X=\overline{X}$ is neutrally stable with ${\lambda }_t^{+}=0$. Thus, any small perturbation to (0, 0) not orthogonal to $u_0^{+}$ will drive the system away from the fixed point, growing asymptotically along the direction of $u_t^{+}$, with X and $\overline{X}$ increasing proportionally, as in Eq. (16).

The key features of the deterministic dynamics that have important implications for the stochastic system are as follows:

1. If one conditions on $X_T=x_f>0$, then when $\beta >1$ even a small perturbation due to noise will cause the process to escape the origin deterministically towards $x_f$. In such a regime, noise merely initiates the escape, but deterministic dynamics dominate the evolution. We will not focus on this regime in our work. However, it would be interesting to study the fluctuations of the escape time from the origin, similarly to that done in [59].

2. The interesting regime for our work is when $\beta <1$. For $\beta \in [0,1)$, the feedback in Eq. (1) is positive, so the most likely first-passage to $x_f>0$ occurs through the accumulation of positive fluctuations. However, for $\beta <0$, this intuitive picture breaks down: as positive fluctuations accumulate, the feedback becomes increasingly negative, pulling the process back toward the origin and making first-passage through $x_f$ highly unlikely. Therefore, two alternative first-passage mechanisms for the stochastic dynamics emerge:

3. (a) A one-stage mechanism, whereby large fluctuations are sufficient to overcome the negative feedback. This is an extremely rare event in the weak-noise limit. (b) A two-stage mechanism, whereby the stochastic process first accumulates negative fluctuations, generating a strong positive feedback, which then drives a “slingshot” first-passage to the right; $x_f>0$.

These $\beta$-dependent qualitatively distinct first-passage pathways raise a natural question: In a symmetric domain $[-x_f,x_f]$, with $\beta \in [0,1)$, is the agent more likely to escape to the right via the accumulation of positive fluctuations, or to the left by exploiting a slingshot mechanism? We return to these questions in Section 4.

In the next section (2.4), we characterize the long-time behavior of general SIDs and feedback processes, which will serve as a useful reference for our analysis of finite-time first-passage dynamics.

Long-Time Behavior of the Feedback Ornstein–Uhlenbeck Processes

Having established in Section 2.2 that the feedback OU process in Eq. (1) belongs to the class of SIDs, we now draw on results from SID theory to understand the important asymptotic behavior. This will prove useful later, particularly in Sections 4.2 and 4.3, when interpreting our results.

Assuming stationary behavior at long times (as in [27, 43, 57]), one can formally derive the stationary occupation measure, or invariant density, ${\rho }_{\mathrm{inv}}$ of the general SID in Eq. (6) from the stationary Fokker–Planck equation (see [43]):

$$\begin{array}{c}\hfill {\rho }_{\mathrm{inv}}(x)=\frac{e^{-\frac{2}{ϵ}\left(V,(x),+,{\int }_{{\mathbb{R}}^d},K,(y,x),{\rho }_{\mathrm{inv}},(y),,d,y\right)}}{Z},\end{array}$$ 17

where Z is a normalization constant. The symmetry of the potential V in Eq. (7) is inherited by the invariant density ${\rho }_{\mathrm{inv}}$ [45, 57, 60], and in the case of the feedback OU process Eq. (1), Eq. (17) becomes

$$\begin{array}{c}\hfill {\rho }_{\mathrm{inv}}(x)=\sqrt{\frac{1}{\pi ϵ}}{e}^{-\frac{x^2}{ϵ}},\end{array}$$ 18

which coincides with the stationary distribution of the standard OU process. In consequence, the memory feedback term in Eq. (2) converges in probability to

$$\begin{array}{c}\hfill {\overline{X}}_t\stackrel{t\to \infty }{\to }0.\end{array}$$ 19

Therefore, the classical OU process is recovered as the long-time limit of the feedback OU process. However, it is important to note that the convergence to this fixed point occurs very-slowly; slower than any algebraic rate. This has been shown in [45, 56, 57]: see Theorems 1.6 and 1.12 in [56], Theorem 2.7 in [57], and Theorem 2.10 and Propositions 2.11–2.12 in [45]. These long-time results serve as a valuable benchmark for understanding the finite-time dynamics studied in the remainder of this paper, where the memory feedback has a controlling influence.

We conclude this section by highlighting Theorem 2.13 from Aleksian et al. [45], which is relevant to our study. In the weak-noise limit $ϵ\to 0$, they established a Kramers-type law for the first-passage time OF the process in Eq. (6) THAT exits a domain $\mathcal{D}\subset {\mathbb{R}}^d$, through the boundary $\partial \mathcal{D}\subset {\mathbb{R}}^d$, defined as $T_0(x):=inf\{t\ge 0:X_t\notin \mathcal{D}|X_0=x\in \mathcal{D}\}$. They assumed that the potentials V and W are uniformly convex with at most polynomial growth and V such that ${lim}_{|x|\to \infty }{|\mathrm{\nabla }V(x)|}^2/V(x)=+\infty$ (this last condition does not hold for the V in Eq. (7)). They established that

$$\begin{array}{c}\hfill \underset{ϵ\to 0}{lim}\frac{ϵ}{2}log{T}_0(x)=\underset{x^{\prime }\in \partial \mathcal{D}}{inf}H(x^{\prime }),\end{array}$$ 20

where

$$\begin{array}{c}\hfill H(x^{\prime }):=V(x^{\prime })+K(x^{\prime }-m)-V(m),\end{array}$$ 21

and m is the point at which ${\rho }_{\mathrm{inv}}$ attains its maximum. For the feedback OU process in Eq. (1), we have $m=0$ and we typically consider a single first-passage boundary $\partial \mathcal{D}=\{x_f\}$. Therefore, we find:

$$\begin{array}{c}\hfill H(x_f)=x_f^2.\end{array}$$ 22

We note that this result was derived under the further assumption that the empirical occupation measure stabilizes prior to first-passage dynamics, which we believe is the reason the right hand side of Eq. (20) depends neither on the starting point x nor on $\beta$. As we show in Section 3.1 using a weak-noise large deviation approach, our analysis does not require this assumption, which leads to a strong dependence of the first-passage dynamics on the value of $\beta \in [0,1)$, as we will explore in detail.

Finally, we emphasize that Eq. (20) holds in probability. As such, it does not constitute an Arrhenius-type law, which would describe the expected time to reach a specified position (this is also noted in the Appendix of [45]). The quantification of an Arrhenius law for the feedback OU process, and its relation to the underlying large deviation principles, is one of the main results of our work.

First-passage Dynamics and Weak-Noise Large Deviations

Given the Markovian structure of the dynamics of the process ${(Z_t)}_{t\ge 0}$ in Eq. (5), the associated time-dependent infinitesimal generator takes the form:

$$\begin{array}{c}\hfill L_t=A_t(\beta )z·\mathrm{\nabla }+\frac{ϵ}{2}\frac{{\partial }^2}{\partial x^2},\end{array}$$ 23

where $z:=(x,\overline{x})$ and $\mathrm{\nabla }$ denotes the gradient with respect to the variables $(x,\overline{x})$. In principle, this generator can be used to extract information on the first-passage dynamics, in particular: (i) the first-passage time probability density, and (ii) the mean first-passage time.

For the non-homogeneous Markov process with time-dependent infinitesimal generator given in (23), the first-passage time is defined as

$$\begin{array}{c}\hfill T_s(z_s)=inf\left\{t,-,s,\ge ,0,:,X_t,>,x_f\right\},\end{array}$$ 24

with initial condition $Z_s=z_s:=(x_s,{\overline{x}}_s)$ at time s. Its average, the mean first-passage time, is given by

$$\begin{array}{c}\hfill {\overline{T}}_s(z_s)={\mathbb{E}}_{s,z_s}\left[T,-,s\right]=\mathbb{E}\left[inf,\left\{t,-,s,\ge ,0,:,X_t,>,x_f\right\},|,Z_s,=,z_s\right],\end{array}$$ 25

which is the expected time (from time s) it takes for the process to hit $x_f$. Notice that both the first-passage time and its average depend on the starting time s and starting position $z_s$ (due to the non-homogeneity of the process).

In order to describe a general procedure to calculate the first-passage time probability density and its first moment ${\overline{T}}_s(z_s)$, we introduce the transition probability density of the process ${\left(Z_t\right)}_{t\ge 0}$, with absorbing boundary at $x_f$, which is written as

$$\begin{array}{c}\hfill \rho (z,t|z_s,s):=P(Z_t=z|Z_s=z_s),\end{array}$$ 26

for $t\ge s$. Given the pointwise initial condition, the transition probability density above is already normalized. We remark that $P(Z_t=z|Z_s=z_s)$ is shorthand for the probability that $Z_t$ lies in a small interval $[x,x+dx)\times [\overline{x},\overline{x}+d\overline{x})$ conditioned on having started the process at $z_s$ at time s, so that the probability density satisfies $\mathbb{P}(Z_t=z|Z_s=z_s)=P(Z_t=z|Z_s=z_s)dxd\overline{x}$, where $\mathbb{P}$ is the probability distribution.

Returning to the transition probability density function $\rho (z,t|z_s,s)$ in (26), we remark that it satisfies the following Fokker–Planck boundary value problem:

$$\begin{array}{cc}& \frac{\partial \rho (z,t|z_s,s)}{\partial t}=L_t^{†}\rho (z,t|z_s,s),\mathrm{with}\rho (\mathrm{z},\mathrm{s}|{\mathrm{z}}_{\mathrm{s}},\mathrm{s})=\delta (\mathrm{z}-{\mathrm{z}}_{\mathrm{s}})\hfill \\ \hfill & \mathrm{and}\rho (({\mathrm{x}}_{\mathrm{f}},\overline{\mathrm{x}}),\mathrm{t}|{\mathrm{z}}_{\mathrm{s}},\mathrm{s})=0\forall \overline{\mathrm{x}}\in \mathbb{R},\hfill \end{array}$$ 27

with $t\ge s$ and where $L_t^{†}$ denotes the adjoint of the generator $L_t$ in Eq. (23). We also have that the first-passage time probability density is given by

$$\begin{array}{c}\hfill f_{s,t}(z_s):=-\frac{\partial Q_{s,t}(z_s)}{\partial t},\end{array}$$ 28

where

$$\begin{array}{c}\hfill Q_{s,t}(z_s):={\int }_{-\infty }^f{\int }_{-\infty }^{\infty }\rho (z,t|z_s,s)d\overline{x}dx,\end{array}$$ 29

is the survival probability density function, which is the probability that the particle starting at $z_s$ at time s has not hit $x_f$ by time $t\ge s$ (notice that the variable $\overline{x}$ is also integrated out).

Therefore, the mean first-passage time ${\overline{T}}_s(z_s)$ satisfies the following boundary value problem:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}{L}_s{\overline{T}}_s(z_s)=-1,\hfill & \mathrm{for}{\mathrm{x}}_{\mathrm{s}}<{\mathrm{x}}_{\mathrm{f}},{\overline{\mathrm{x}}}_{\mathrm{s}}<{\mathrm{x}}_{\mathrm{f}}\hfill \\ {\overline{T}}_s(z_s)=0,\hfill & \mathrm{for}{\mathrm{x}}_{\mathrm{s}}={\mathrm{x}}_{\mathrm{f}},{\overline{\mathrm{x}}}_{\mathrm{s}}<{\mathrm{x}}_{\mathrm{f}},\hfill \end{array}\right)\end{array}$$ 30

whose derivation is given in Appendix 6.1. Because the infinitesimal generator $L_t$ is explicitly time-dependent due to the drift term, and the diffusion is degenerate since no noise acts directly on the dynamics of ${\overline{X}}_t$, exact analytical solutions to Eqs. (27) and (30) are generally not available.

However, asymptotic approaches are possible. For instance, approximate analytical solutions for both the autonomous and non-autonomous one-dimensional Ornstein–Uhlenbeck processes and for stochastic resonance have recently been obtained using matched asymptotic expansions [61–64]. While this approach could in principle be extended to the two-dimensional setting considered here, identifying the appropriate boundary layers in two dimensions is highly non-trivial and would require a separate in-depth study.

Instead, here we use the framework of weak-noise large deviation theory to derive asymptotic results for the first-passage dynamics of the feedback OU process. In the weak-noise regime, we can analyze the asymptotic behavior of the probability density $\rho \left(z,,,t,\mid ,z_0,,,0\right)$ by solving an effective approximation of Eq. (27) that is the associated Hamilton–Jacobi equation.

In Section 3.1, we briefly review the general weak-noise large deviation framework for Markov processes—see [65–67] for more comprehensive treatments. We then illustrate how this framework can be used to extract relevant information about first-passage dynamics and apply it to both the standard OU process (Section 3.2) and the feedback OU process (Section 3.3).

Weak-Noise Large Deviations

For simplicity, we introduce the weak-noise large deviation framework by considering a general two-dimensional Markov process, denoted, with slight abuse of notation, ${(Z_t)}_{t\ge 0}$. We initially keep the notation as general as possible, since the framework extends to any d-dimensional Markov process, and then specialize to the case of interest in Sec. 3.3. The general process is governed by the stochastic differential equation,

$$\begin{array}{c}\hfill d{Z}_t=F_t(Z_t)dt+\sqrt{ϵ}\sigma d{W}_t,\end{array}$$ 31

where $F_t$ is a (possibly time-dependent) 2-dimensional drift vector field, $\sigma \in {\mathbb{R}}^{2\times 2}$ is a constant noise matrix, $D={\sigma }^T\sigma$ is the noise covariance (or diffusion) matrix, and $W_t$ denotes a 2-dimensional Brownian motion. For simplicity, we assume additive noise, i.e., $\sigma$ does not depend on position, and we consider a pointwise initial condition $Z_0=z_0$. We are interested in events wherein the process ${(Z_t)}_{t\ge 0}$ exhibits a specific outcome—for instance, when the trajectory reaches a prescribed set $A\subset {\mathbb{R}}^2$ at time T, that is $Z_T\in A$. While such events are impossible in the deterministic case ($ϵ=0$), they typically occur in the presence of noise ($ϵ>0$), though of course with decreasing frequency as the noise intensity vanishes, $ϵ\to 0$.

Similarly to the previous treatment, we introduce the normalized transition probability density function $\rho (z,t|z_0,0)$ as in Eq. (26) with absorbing boundary $A\subset {\mathbb{R}}^2$. This function satisfies a Fokker–Planck boundary value problem equivalent to (27) and written below for thoroughness

$$\begin{array}{c}\hfill {\partial }_t\rho (z,t|z_0,0)=-\mathrm{\nabla }·\left(F_t,(z),,\rho ,(z,t|z_0,0)\right)+\frac{ϵ}{2}\mathrm{\nabla }·\left(D,,\mathrm{\nabla },\rho ,(,z,,,t,|,z_0,,,0,)\right),\end{array}$$ 32

with initial condition $\rho (z,0|z_0,0)=\delta (z-z_0)$ and absorbing boundary $\rho (z,t|z_0,0)=0$ for $z\in A$ and $t\ge 0$.

Now, by assuming an exponential scaling in $ϵ$ we make the following change of variable,

$$\begin{array}{c}\hfill \rho (z,t|z_0,0)=exp\left[-,{ϵ}^{-1},S_t,(z)\right],\end{array}$$ 33

where, for parsimony of notation, on the right hand side the dependence on the initial time and point is implicit. Taking the weak-noise limit, $ϵ\to 0$, and retaining terms up to order $O({ϵ}^{-1})$, gives the following Hamilton–Jacobi equation [67]:

$$\begin{array}{c}\hfill -{\partial }_t{S}_t(z)=H_t(z,\mathrm{\nabla }{S}_t(z)).\end{array}$$ 34

Here, the time-dependent Hamiltonian is given by

$$\begin{array}{c}\hfill H_t(z,p)=\frac{1}{2}p·Dp+F_t(z)·p,\end{array}$$ 35

where p denotes a vector of adjoint variables, or momentum coordinates. The approximation introduced through the change of variable (33) with $ϵ\ll 1$ is commonly referred to as the WKBJ (after Wentzel, Kramers, Brillouin and Jeffreys) ansatz in many areas of physics, including stochastic processes [61, 68, 69].

The Hamilton–Jacobi equation (34) is a first-order partial differential equation whose solution can be obtained from the characteristic curves of the associated Hamiltonian dynamics, which are

$$\begin{array}{c}\hfill {\dot{z}}_t={\mathrm{\nabla }}_p{H}_t(z,p)\mathrm{and}{\dot{\mathrm{p}}}_{\mathrm{t}}=-{\mathrm{\nabla }}_{\mathrm{z}}{\mathrm{H}}_{\mathrm{t}}(\mathrm{z},\mathrm{p}),\end{array}$$ 36

subject to appropriate boundary conditions. By integrating along these characteristic trajectories—often referred to as instantons in the large deviation literature [67, 68]—the action functional in Eq. (34) takes the following form;

$$\begin{array}{c}\hfill S_t(z)={\int }_0^t\left[p,·,\dot{z},-,H_t,(z,p)\right]dt+S_0(z_0).\end{array}$$ 37

This expression reflects the fact that the action is the generating function of a canonical transformation, linking the first-order PDE (34) with the second-order Hamiltonian dynamics of Eq. (36) [70, 71].

In the weak-noise regime, the action so obtained represents the dominant contribution to the probability density function $\rho (z,t|z_0,0)$ in Eq. (33), wherein, with a slight abuse of notation, we denote both the full action and its leading-order contribution at $O({ϵ}^{-1})$ with the same symbol. This action determines the asymptotic behavior in ${ϵ}^{-1}$ of the probability of finding the process at $z_f\in A$ for the first time T, viz.

$$\begin{array}{c}\hfill \rho (z_f,T|z_0,0)\approx exp\left[-,{ϵ}^{-1},S_T,(z_f)\right].\end{array}$$ 38

At leading order in ${ϵ}^{-1}$, this characterizes the survival probability density function. Therefore, upon taking the time derivative of Eq. (38), we find that in the weak-noise limit the first-passage time distribution in Eq. (28) is also approximated by $\rho (z_f,T|z_0,0)$ of Eq. (38).

The optimal trajectory $z_t$ is that which minimizes the action and thereby defines the instanton. Thus, the instanton corresponds to the most likely path connecting the initial point $z_0$ to the final point $z_T=z_f$ as $ϵ\to 0$, thereby providing the dominant path through which the system reaches $z_f$ at time T. By integrating over time, Eq. (38) leads to the Kramers-type, or Arrhenius-type, formula as

$$\begin{array}{c}\hfill \mu (z_f|z_0,0)\approx exp\left[-,{ϵ}^{-1},U,(z_f)\right],\end{array}$$ 39

where $\mu$ is the probability density on the state space induced by time integration and

$$\begin{array}{c}\hfill U(z_f):=\underset{T\ge 0}{inf}{S}_T(z_f)\end{array}$$ 40

is the quasi-potential, or the large deviation rate function. We interpret $U(z_f)$ as an effective energy barrier that the system must overcome to reach $z_f$, and it governs the leading-order behavior of the first-passage probability. Additionally, the optimal first-passage time can be determined as

$$\begin{array}{c}\hfill \tau (z_f)=\underset{T\ge 0}{\mathrm{arginf}}{S}_T(z_f).\end{array}$$ 41

We stress that the equalities in Eq. (38) and Eq. (39) are exact only at the logarithmic scale. Consequently, the right hand side of both equations are not normalized, since terms of order $O({ϵ}^{-1})$ are neglected. This explains the apparent divergence obtained when integrating Eq. (38) over time.

We note that in general this optimal first-passage time $\tau (z_f)$ is not equivalent to the mean first-passage time ${\overline{T}}_0(z_0)$ defined in Eq. (25) and approximated by the inverse Kramers rate; ${\overline{T}}_0(z_0)\approx exp\left[{ϵ}^{-1},,U,(z_f)\right]$. Although the two quantities are related, they have distinct interpretations. The optimal first-passage time is defined in the limit $ϵ\to 0$, where it is given by the value of T at which the action $S_T(z_f)$ reaches its minimum. It is therefore independent of $ϵ$ and should not be confused with either the mean or the typical first-passage time with finite noise. Instead, it captures the intrinsic tendency of the process in the absence of noise. For example, in the Ornstein–Uhlenbeck process of Sec. 3.2 below, the minimum of the action is attained only as $T\to \infty$, reflecting the fact that the noiseless dynamics keeps the process at the fixed point indefinitely (before escaping). In contrast, real first-passage events are noise-induced and occur asymptotically on the finite timescale set by the inverse Kramers rate in Eq. (39).

The Weak-Noise Regime of the Ornstein–Uhlenbeck Process

As noted above, a canonical analytically tractable stochastic process in statistical physics is the Ornstein–Uhlenbeck process, which is obtained by removing the feedback term, $\frac{1}{t}{\int }_0^t{X}_{s}ds$, from Eq. (1). The result derived in the following can be found in the standard literature of stochastic processes, see for instance [72, 73]. The transition probability density function is a Gaussian that can be computed explicitly by solving the Fokker–Planck equation

$$\begin{array}{c}\hfill {\partial }_t\rho (x,t|0,0)=L^{†}\rho (x,t|0,0),\end{array}$$ 42

with initial condition $\rho (x,0|0,0)=\delta (x)$, where L is the infinitesimal Markov generator given by

$$\begin{array}{c}\hfill L=-x\frac{d}{\mathit{dx}}+\frac{ϵ}{2}\frac{d^2}{d{x}^2}.\end{array}$$ 43

Equation (42) can be solved using Fourier transform or heat kernel methods giving the transition probability density as

$$\begin{array}{c}\hfill \rho (x,t|0,0)=\sqrt{\frac{1}{\pi ϵ(1-e^{-2t})}}exp\left[-,\frac{1}{ϵ},\frac{x^2}{(1-e^{-2t})}\right],\end{array}$$ 44

which is the probability of finding the process at position x at time t, starting at $x=0$ at $t=0$.

Because Eq. (44) takes a large deviation form, we identify the action $S_T^{\mathrm{OU}}$ in Eq. (38) as

$$\begin{array}{c}\hfill S_T^{\mathrm{OU}}(x_f):=\frac{x_f^2}{1-e^{-2T}}.\end{array}$$ 45

As discussed in Section 3.1, this gives an asymptotic characterization of the probability of finding the OU process at $x_f$ for the first time T; it provides the leading-order asymptotic form of the first-passage time distribution in Eq. (28). By solving Eqs. (36) for the Hamiltonian in Eq. (35), which in this case is $H(x,p)=\frac{1}{2}{p}^2-px$, we obtain the instanton equation

$$\begin{array}{c}\hfill x_t^{\mathrm{OU}}=x_f\frac{sinht}{sinhT},\end{array}$$ 46

which gives the most probable path from $x_0^{\mathrm{OU}}=0$ to $x_T^{\mathrm{OU}}=x_f$. Integrating Eq. (44) in time yields the probability of being at x as an implicit function of time;

$$\begin{array}{c}\hfill \mu (x|0,0)=\frac{|x|}{\sqrt{\pi }ϵ}{\int }_{\frac{|x|}{\sqrt{ϵ}}}^{\infty }\frac{e^{-t^2}}{t^2-\frac{|x|}{\sqrt{ϵ}}}dt,\end{array}$$ 47

and its leading-order behavior in ${ϵ}^{-1}$, which can also be obtained from Eq. (40), is given by the following quasi-potential

$$\begin{array}{c}\hfill U^{\mathrm{OU}}(x_f)=x_f^2,\end{array}$$ 48

which, as expected, is quadratic in the final position. Moreover, we note that the quasi-potential for the OU process coincides with $H(x_f)$ in Eq. (22).

Finally, the first-passage time of the process generally depends on the noise intensity $ϵ$. Although the mean first-passage time is finite and given by the inverse Kramers rate, in the weak-noise limit inspection of Eq. (41) shows that the optimal first-passage time diverges; $\tau \to \infty$. Therefore, the weak-noise OU process tends to remain near the bottom of the potential well for as long as possible in order to minimize the action, eventually escaping the quadratic trap through a single large fluctuation.

The Weak-Noise Regime of Feedback Ornstein–Uhlenbeck Processes

We now turn our attention to the feedback OU process. Rather than analyzing the non-Markovian process $X_t$ defined by Eq. (1), we consider the two-dimensional Markovian process $Z_t=(X_t,{\overline{X}}_t)$ defined by Eq. (5). Using the notation of Eq. (31), the deterministic time-dependent drift is given by

$$\begin{array}{c}\hfill F_t(Z_t)=A_t(\beta )Z_t,\end{array}$$ 49

and a noise matrix

$$\begin{array}{c}\hfill \sigma =D=\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right),\end{array}$$ 50

showing that noise acts only on the $X_t$ component. (We discuss the consequences of the structure of the noise matrix below.) We consider the initial condition $Z_0=(x_0,{\overline{x}}_0)=(0,0)$, which also regularizes the singularity at $t=0$, and we aim to study the first-passage dynamics through $x_f>0$.

The time-dependent Hamiltonian is given by

$$\begin{array}{c}\hfill H_t(z,p)=\frac{p_x^2}{2}-\left(x,-,\beta ,\overline{x}\right)p_x+\frac{(x-\overline{x})}{t}{p}_{\overline{x}},\end{array}$$ 51

where $z=(x,\overline{x})$ and $p=(p_x,p_{\overline{x}})$. The solutions to the Hamilton–Jacobi equation (34) are obtained by solving the associated Hamilton equations (36) and are:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}\dot{x}=p_x-x+\beta \overline{x}\hfill \\ \dot{\overline{x}}={\displaystyle \frac{x-\overline{x}}{t}}\hfill \\ {\dot{p}}_x=p_x-{\displaystyle \frac{p_{\overline{x}}}{t}}\hfill \\ {\dot{p}}_{\overline{x}}=-\beta p_x+{\displaystyle \frac{p_{\overline{x}}}{t}}\hfill \end{array}\right)\left\{\begin{array}{c}{x}_0=0\hfill \\ {\overline{x}}_0=0\hfill \\ p_{x,T}=\lambda \hfill \\ p_{\overline{x},T}=0\hfill \end{array}\right).\end{array}$$ 52

In Eqs. (52), $\lambda$ is a parameter implicitly determined. by the final condition $x_T=x_f$ through the Hamiltonian dynamics. As we highlight in the derivation below, it is more convenient to impose final conditions on the adjoint variables rather than initial conditions on the adjoint variables or final conditions on the coordinate variables. We also refer the reader to Section 2.4.1 of [74] for an explanation of how to transform final conditions on position variables into final conditions on adjoint variables within a Lagrangian framework.

We note that, in principle, for non-autonomous driving, the choice of boundary conditions in the propagator (see Eq. (32)) can influence even the leading-order behavior in the large-deviation regime ($O({ϵ}^{-1})$), because first-passage events may depend on whether earlier crossings are excluded. This would in turn modify the boundary conditions in (52). However, in the feedback OU process considered here the boundary conditions do not affect the rate function. The process starts at the stable fixed point $(x,\overline{x})=(0,0)$, and we ask for a crossing at a distant value $x_f>0$. In the weak-noise limit the deterministic drift drives trajectories into the fixed point where they vanish, so waiting before departure incurs no cost in the action. As a result, the minimization problem leading to the instanton is the same with or without a “no earlier crossing” constraint: the optimal path leaves the fixed point once, at the time that minimizes the action, and reaches $x_f$ at $T$.

Starting from Eqs. (52), we derive a simplified differential equation the solution to which gives the instanton. Importantly, the adjoint equations are decoupled from the coordinate variables, which allows us to solve them independently and substitute their solution into the coordinate equations. We begin with the equation for ${\dot{p}}_x$. The formal solution is obtained by treating $p_{\overline{x}}$ as a source term, and using separation of variables and variation of constants, and is given by

$$\begin{array}{c}\hfill p_{x,t}=\lambda e^{-(T-t)}-{\int }_t^T\frac{p_{\overline{x},s}{e}^{(t-s)}}{s}ds,\end{array}$$ 53

where the constant of integration is fixed by the boundary condition $p_{x,T}=\lambda$. Note that, because the momentum grows exponentially in time, this is why it is preferable to impose a final rather than an initial condition on the momentum, which naturally constrains the unbounded growth of the solution.

Substituting Eq. (53) into the equation for $p_{\overline{x}}$ and differentiating both sides with respect to t, we obtain the following second-order differential equation:

$$\begin{array}{c}\hfill {\ddot{p}}_{\overline{x}}={\dot{p}}_{\overline{x}}\left(1,+,\frac{1}{t}\right)+\frac{p_{\overline{x}}}{t}\left(\beta ,-,1,-,\frac{1}{t}\right).\end{array}$$ 54

This equation must be solved subject to the boundary conditions that $p_{\overline{x},T}=0$, as in Eqs. (52), and ${\dot{p}}_{\overline{x},T}=-\beta \lambda$, which follows directly from the evolution equation for $p_{\overline{x}}$. The general solution of Eq. (54) is given in terms of special functions as

$$\begin{array}{c}\hfill p_{\overline{x},t}=t\left(c_1,,\mathrm{HypU},\left[\beta ,,,1,,,\mathrm{t}\right],+,{\mathrm{c}}_2,,\mathrm{LagL},\left[-,\beta ,,,\mathrm{t}\right]\right),\end{array}$$ 55

where $c_1$ and $c_2$ are constants, the first term on the right-hand side is the Tricomi confluent hypergeometric function,

$$\begin{array}{c}\hfill \mathrm{HypU}[\mathrm{a},\mathrm{b},\mathrm{t}]=\frac{1}{\mathrm{\Gamma }(\mathrm{a})}{\int }_0^{\infty }{\mathrm{e}}^{-\mathrm{st}}{\mathrm{s}}^{\mathrm{a}-1}{(1+\mathrm{s})}^{\mathrm{b}-\mathrm{a}-1}\mathrm{ds},\end{array}$$ 56

and $\mathrm{LagL}[\mathrm{n},\mathrm{t}]$ is the Laguerre function, defined as the solution of Laguerre’s differential equation,

$$\begin{array}{c}\hfill t{y}^{\prime \prime }+(1-t)y^{\prime }+ny=0,\end{array}$$ 57

WHICH is non-singular at $t=0$. It can be shown that Eq. (55) satisfies $p_{\overline{x},0}=0$, ensuring regularity in the limit $t\to 0$ in Eqs. (52). The same regularity is obtained by formally considering the asymptotic limit of $\beta \to 0$ or for $t\ll 1$ or both. In both cases, ${\dot{p}}_{\overline{x}}=-\beta p_x+{\displaystyle \frac{p_{\overline{x}}}{t}}$ becomes ${\dot{p}}_{\overline{x}}={\displaystyle \frac{p_{\overline{x}}}{t}}$, so that $p_{\overline{x}}=Ct$ and hence ${\dot{p}}_x=p_x-C$.

Imposing the two boundary conditions above determines the constants in Eq. (55) as

$$\begin{array}{cc}\hfill c_1=& \frac{\beta \lambda \mathrm{LagL}\left[-,\beta ,,,\mathrm{T}\right]}{\beta T\mathrm{HypU}\left[1,+,\beta ,,,2,,,\mathrm{T}\right]\mathrm{LagL}\left[-,\beta ,,,\mathrm{T}\right]-\mathrm{T}\mathrm{HypU}\left[\beta ,,,1,,,\mathrm{T}\right]\mathrm{LagL}\left[-,1,-,\beta ,,,1,,,\mathrm{T}\right]}\mathrm{and}\hfill \end{array}$$ 58

$$\begin{array}{cc}\hfill c_2=& -\frac{\beta \lambda \mathrm{HypU}\left[\beta ,,,1,,,\mathrm{T}\right]}{\beta T\mathrm{HypU}\left[1,+,\beta ,,,2,,,\mathrm{T}\right]\mathrm{LagL}\left[-,\beta ,,,\mathrm{T}\right]-\mathrm{T}\mathrm{HypU}\left[\beta ,,,1,,,\mathrm{T}\right]\mathrm{LagL}\left[-,1,-,\beta ,,,1,,,\mathrm{T}\right]}.\hfill \end{array}$$ 59

This completes the solution Eq. (55) of Eq. (54), thereby providing an explicit expression for $p_{x,t}$ in Eq. (53).

Differentiating the first equation in (52), substituting the expression for $\dot{\overline{x}}$, and using the first equation again to eliminate $\overline{x}$, we obtain the following second-order differential equation for $x_t$,

$$\begin{array}{c}\hfill \ddot{x}={\dot{p}}_x-\left(1,+,\frac{1}{t}\right)\dot{x}+\frac{p_x}{t}+(\beta -1)\frac{x}{t},\end{array}$$ 60

which can be rewritten, using Eq. (53), as

$$\begin{array}{c}\hfill \ddot{x}=\left(1,+,\frac{1}{t}\right)\left(p_x,-,\dot{x}\right)-\frac{1}{t}\left(p_{\overline{x}},-,(\beta -1),x\right).\end{array}$$ 61

This equation must be solved using $p_{x,t}$ from Eq. (53), $p_{\overline{x},t}$ from Eq. (55), and boundary conditions $x_0=0$ and ${\dot{x}}_0=p_{x,0}$. The latter condition follows directly from the initial condition ${\overline{x}}_0=0$ and the Hamiltonian equation for $\dot{x}$. The dynamics of ${\overline{x}}_t$ can then be recovered by inverting the equation for $\dot{x}$ once the solution to Eq. (61) is known.

In summary, we have reduced the full system Eqs. (52) to a single second-order differential equation (61), whose solution provides the instanton $x_t$ connecting $x_0=0$ to $x_T=x_f$, which indirectly determines ${\overline{x}}_t$. The solution to Eq. (61) must be computed numerically. We compare the instantons of the OU process and the feedback OU process in Fig. 2 of Section 4.1.

Fig. 2.

Comparison between the instanton $x_t$ solving Eq. (61) for $T=10$ and numerical simulations of the feedback OU process governed by Eqs. (5), for three values of the feedback parameter: $\beta <0$ (a), $\beta =0$ (b), and $\beta >0$ (c). In the small-noise regime $ϵ=0.01$, the average simulated trajectory (solid orange line), computed over ${10}^7$ realizations and post-selected on reaching $x_f=0.0375$ at time $T=10$, closely follows the instanton. The shaded area shows one standard deviation around the average. The corresponding memory trajectory ${\overline{x}}_t$, obtained by inverting the equation for $\dot{\overline{x}}$ in Eq. (52), is also shown. These results support the accuracy of the analytical approximation

Once the instanton solution is obtained, we can compute the action using Eq. (37). Interestingly, by substituting the Hamiltonian (51) along with Hamilton’s equations (52) into Eq. (37), we obtain the following simplified form for the action:

$$\begin{array}{c}\hfill S_t(z)={\int }_0^t\frac{p_{x,s}^2}{2}ds+S_0(z_0),\end{array}$$ 62

with $S_0(z_0)=0$, since the process starts at the minimum of the potentials V and W given by Eqs. (7) and (8). The weak-noise approximation of the first-passage probability through $x_f$ is then given by Eq. (39), with:

$$\begin{array}{c}\hfill z_f=(x_f,{\overline{x}}_f),\end{array}$$ 63

where $x_f$ is prescribed, and ${\overline{x}}_f$ is determined from the solution of Hamilton’s equations. The quasi-potential and optimal first-passage time are finally obtained using Eqs. (40) and (41), respectively.

We conclude this section by highlighting the following. A natural approach to deriving the instanton equation would have been to consider the Lagrangian $L(z,\dot{z})=⟨\dot{z}-F_t(z),D^{-1}(\dot{z}-F_t(z))⟩/2$, where angle brackets denote the Euclidean scalar product. This dual Lagrangian derivation would have ensured that we would have to grapple with consequences of the singular nature of the matrix D. Indeed, the kernel of D is one-dimensional and may encode important information about the dynamics. For instance, using the Moore–Penrose inverse1 (pseudo-inverse), one would obtain an equation of the form $\ddot{x}=-(1+t^{-1})\dot{x}+t^{-1}(\beta -1)x$, the solution of which is the instanton. This equation for $\ddot{x}$, in contrast to Eq. (61), entirely misses the contribution of the adjoint dynamics. Thus, in such a formulation, it would be necessary to supplement the Euler–Lagrange equations with an additional kernel-dependent term, the analysis of which would require alternative methods. On the contrary, within the Hamiltonian framework no such issue arises: the degeneracy of D is not a problem and the adjoint dynamics are fully incorporated into the solution.

Numerical Results and Discussion

Instanton Dynamics

We now compare and contrast the classical and feedback OU processes allowing us to characterize the physical mechanisms responsible for first-passage dynamics. In particular, we present our results on the instanton dynamics leading the feedback OU process to first-passage through $x_f>0$, and compare them with the classical OU case.

To solve Eq. (61) numerically we use the solve_ivp solver from the scipy.integrate Python package, employing RK45, which corresponds to the explicit fourth order Runge–Kutta method of integration. In Fig. 2 we compare the numerical solution for the instanton $x_t$, obtained from the analytical approximation of Eq. (61), and the memory component ${\overline{x}}_t$, with simulations for three representative cases: $\beta <0$ in Fig. 2(a), $\beta =0$ (OU process) in Fig. 2(b), and $\beta >0$ in Fig. 2(c). Clearly, the average behavior of sample-path simulations conducted at finite but small noise $ϵ$ compares well with the instanton $x_t$, thereby supporting our analytical derivation of Eq. (61).

Firstly, the case $\beta =0$ shown Fig. 2(b) exhibits the classical OU instanton $x_t^{\mathrm{OU}}$ given by Eq. (46). For completeness, we also show the memory ${\overline{x}}_t$, defined in Eq. (2), which does not influence the dynamics in this case. As expected, the classical OU instanton corresponds to a monotonic exponential approach to $x_f$, as discussed in Section 3.2.

Secondly, Fig. 2 reveals qualitatively distinct first-passage mechanisms for negative and positive values of $\beta$, although both resemble the classical OU first-passage at late times. For $\beta <0$, Fig. 2(a) shows that since it is detrimental to accumulate positive fluctuations toward $x_f$, the process instead accumulates fluctuations in the opposite direction. Because $\beta <0$, this effectively moves the potential minimum toward $x_f$, storing a form of ‘potential energy’ that is then released through a monotonic exponential approach to $x_f$. We refer to this behavior as the ‘slingshot’ effect.

For $\beta >0$, Fig. 2(c) shows that the process escapes optimally by initially accumulating small fluctuations toward $x_f$, which are stored in the memory ${\overline{x}}_t$, effectively shifting the minimum of the potential closer to $x_f$. As time progresses, the memory feedback becomes increasingly rigid due to averaging, which leads to a plateau in the intermediate time regime. However, since the effective potential minimum moves closer to $x_f$, first-passage becomes less costly in terms of the action (see Fig. 3) and we see the final exponential rise, again akin to the late time behavior of the classical OU case in Fig. 2(b).

Fig. 3.

(a) The action $S_T(z_f)$ (Eq. 37) as a function of time T, at a fixed $x_f=0.3$, for several values of $\beta$. The classical OU action is the black dashed line ($\beta =0$), with the feedback OU process curves above ($\beta <0$) and below ($\beta >0$). (b) The action plotted as a function of $\beta$, at fixed $x_f=0.3$, for several values of T. (c) The instanton $x_t$ and memory ${\overline{x}}_t$ for $\beta =-0.8$ and $T=1.0$, showing nearly ballistic dynamics associated with the local minimum in (a) and the non-monotonic time behavior in (b). (d) A heatmap of the action plotted as a function of first-passage time T and feedback parameter $\beta$, at fixed $x_f=0.3$.

As previously noted, for simplicity we restrict our analysis to the case with $x_f>0$, but note that the case $x_f<0$ can be treated analogously. Indeed, changing the sign of $x_f$ simply reflects the instanton trajectory, leaving the qualitative dynamics unchanged. Having described the first-passage mechanisms of the feedback OU process, next we analyze the action function $S_T(z_f)$ in order to assess the likelihood of these mechanisms.

Time-Dependent Action

As defined in Eq. (37), the action $S_T(z_f)$ depends both on the first-passage point $z_f$ in Eq. (63) and on the first-passage time T. Moreover, as shown in Eq. (38), the action determines the probability of finding the process at $z_f$ for the first time T. In Fig. 3, we collect a series of plots illustrating the important behavior of this action.

In Fig. 3(a), we plot the action at a fixed first-passage point $x_f=0.3$ as a function of time T, for five values of $\beta$. When $\beta <0$ ($\beta >0$) the action lies above (below) the analytical action $S_T^{\mathrm{OU}}$ of the classical OU process given in Eq. (45) and the black dashed line for $\beta =0$. This implies that, for a given $x_f$, the finite-time slingshot mechanism employed by the feedback OU process with $\beta <0$ is less probable than the accumulation of positive fluctuations occurring for $\beta >0$.

For $\beta <0$, Fig. 3(a) shows the emergence of a global minimum as $T\to \infty$, as seen by the convergence of the colored curves towards the OU action from above. Thus, at large times, all first-passage mechanisms asymptote to the standard OU dynamics, confirming the theoretical results on the long-time behavior of self-interacting diffusions discussed in Section 2.4. On the other hand, at short times, solving Hamilton’s equations for $x_f=0.3$ reveals that the instanton dynamics is nearly ballistic, as seen in Fig. 3(c).

In contrast, when $\beta >0$ the global minimum of the action moves from $T\to \infty$ to a finite time, the value of which decreases as $\beta$ decreases. Therefore, accumulating and storing positive fluctuations via memory feedback is always more favorable than escaping through classical OU dynamics. This shift in the global minimum significantly influences the computation of the quasi-potential and optimal first-passage time, as discussed in Section 4.3.

As seen in Fig. 3(b), for $\beta =0$, classical OU first-passage dynamics are more probable as T increases, since the action decreases and approaches ${lim}_{T\to \infty }{S}_T^{\mathrm{OU}}(x_f)$. This monotonic behavior persists for all negative values of $\beta$. However, for $\beta >0$, the behavior changes: reflecting the emergence of the global minimum shown in Fig. 3(a), wherein the action is minimized at an intermediate time.

A unified means of visualizing the information in Figs. 3(a) and (b) is obtained by constructing a heatmap of the action as a function of $\beta$ and first-passage time T, at fixed $x_f=0.3$, as shown in Fig. 3(d). Here we see that across all times the first-passage dynamics for $\beta <0$, be they ballistic or slingshot, are less probable than for $\beta >0$, where the accumulation of positive fluctuations dominates. Similarly, in Fig. 4 we plot heatmaps of the action as a function of T and first-passage point $x_f$, for three fixed values of the feedback parameter $\beta$. For all values of $\beta$ the figures show that escaping through boundary points farther from the origin is unlikely, and least likely for $\beta <0$.

Fig. 4.

Heatmaps of the action $S_T(z_f)$ as a function of $x_f$ and T for three feedback parameters: $\beta =-0.5$ (a), $\beta =0$ (b), and $\beta =0.5$ (c)

In Section 4.3, we turn to the time-independent properties of the first-passage dynamics, captured by the quasi-potential in Eq. (40). Additionally, we examine the Kramers first-passage rate and the optimal first-passage time introduced in Eq. (41).

Quasi-Potential and Optimal First-Passage Time

The quasi-potential $U(z_f)$ is the large deviation rate function governing the probability of finding the process at a point $x_f$ in the small-noise limit $ϵ\to 0$. It characterizes the effective ‘energy barrier’ that the process must overcome to first-passage through $x_f$. The larger the quasi-potential, the greater the barrier required for first-passage and the longer the mean first-passage time, which is approximated by the inverse of the probability in the Kramers rate formula in Eq. (39).

Fig. 5(a) shows $U(z_f)$ as a function of the first-passage point $x_f$ for five values of $\beta$ straddling the origin. Clearly, for $\beta >0$ the quasi-potential lies below the $\beta =0$ black dashed line corresponding to the OU result $U^{\mathrm{OU}}(x_f)=x_f^2$ given in Eq. (48). This is a consequence of the global minimum of the action $S_T(z_f)$ occurring at finite time shown in Fig. 3(a). Thus, the most likely mechanism of reaching a given first-passage point $x_f$ must have $\beta >0$, and involves the accumulation of positive fluctuations in the memory. Moreover, as $\beta$ increases, this mechanism becomes more likely.

Fig. 5.

(a) Quasi-potential $U(z_f)$ as a function of $x_f$. Curves for $\beta >0$ lie below the OU result $U^{\mathrm{OU}}(x_f)$, indicating that the most likely mechanism is the accumulation of positive fluctuations. For $\beta <0$, curves collapse over $U^{\mathrm{OU}}(x_f)$. (b) Quasi-potential $U(z_f)$ as a function of $\beta$ for several values of $x_f$. For $\beta >0$, the curves decrease monotonically, reinforcing the enhanced likelihood of first-passage via memory accumulation. For $\beta <0$, the quasi-potential plateaus at $U^{\mathrm{OU}}(x_f)$. (c) Heatmap of quasi-potential $U(z_f)$ as a function of $x_f$ and $\beta$

Importantly, this behavior is significantly different from the scenario in which the empirical occupation measure stabilizes before first-passage, as discussed at the end of Section 2.4 and reflected in the $\beta$-independent rate function in Eq. (22). We argue that this difference arises because the quasi-potential in Eq. (40), and shown in Fig. 5(a), is the infimum of the action, rather than its value in the infinite-time limit, which would be a sufficient condition for the stabilization of the occupation measure.

As expected, the quasipotential coincides with the OU result when $\beta <0$, since the action $S_T$ reaches a global minimum as $T\to \infty$, as discussed in Fig. 3. In Fig. 5(b), we plot the quasi-potential as a function of $\beta$ for five first-passage points $x_f$. The plateaus for $\beta <0$ correspond to the OU value of Eq. (48). For $\beta >0$, $U(z_f)$ decreases monotonically with $\beta$, reflecting the shift of the global minimum of the action to finite times. This reduction in the quasi-potential effectively lowers the escape barrier, thereby accelerating first-passage dynamics by reducing the mean first-passage time, which is approximated by taking the inverse of the Kramers rate in Eq. (39).

As previously, a heatmap perspective combines the behavior in a single figure, in this case the quasi-potential as a function of both $\beta$ and $x_f$, as shown in Fig. 5(c). Key here is the enhanced first-passage through a fixed $x_f$ with increasing $\beta$. In summary, Fig. 5 demonstrates that, rather than classical OU dynamics or slingshot effects, memory-accumulated positive fluctuations towards the target is the most likely mechanism underlying the escape process through $x_f$. The transition between classical OU first-passage dynamics and that due to memory-accumulated positive fluctuations is delineated by $\beta =0$.

Finally, in Fig. 6 we plot the optimal first-passage time $\tau$ as a function of $\beta$ for three points $x_f$. This provides further evidence that the global minimum of the action $S_T$ shifts from infinite time for $\beta \le 0$ (with the proviso of the cut off at $T=10$) to a finite time for $\beta \in (0,1)$. Near $\beta =0$ the transition appears abrupt, however, further investigation is needed to fully characterize the nature of this transition. The emergence of a finite optimal first-passage time for $\beta >0$ reflects the role that feedback plays in accelerating the first-passage dynamics through memory-accumulated fluctuations.

Fig. 6.

Optimal first-passage time $\tau$ as a function of the feedback parameter $\beta$ for several values of first-passage point $x_f$. For $\beta \le 0$, $\tau$ diverges and saturates at the numerical cutoff $T=10$. For $\beta >0$, the optimal time becomes finite, confirming the acceleration of first-passage dynamics. As $\beta \to 1$, $\tau$ increases again and eventually reaches the cutoff value

Furthermore, for all $x_f$, the optimal first-passage time initially decreases with $\beta$, reaches a minimum, and then grows again for larger $\beta$, eventually reaching the numerical cutoff $T=10$. This suggests that, although feedback initially accelerates first-passage dynamics, the global minimum of the action gradually shifts to longer times as $\beta$ increases. Recall from Section 2.3 that $\beta =1$ denotes a bifurcation in the deterministic dynamics at which the origin $z_0=(0,0)$ becomes an unstable fixed point, and small fluctuations can rapidly drive the process away from it, in either direction along the x-axis. Thus, it is difficult to predict the behavior of the optimal time as $\beta \to 1$, but the saturation at $T=10$ in Fig. 6 suggests a divergence in the optimal time, possibly due to the contribution of trajectories escaping to the left.

Conclusion

We have investigated the first-passage dynamics of an exemplary non-Markovian stochastic process. The non-Markovianity is treated by modifying a one-dimensional Ornstein–Uhlenbeck process using a time-averaged feedback wherein the drift is influenced by the empirical mean of its own trajectory. Importantly, this model belongs to the broader class of self-interacting diffusions and provides a simple, yet semi-analytically tractable, framework to study the effect of memory feedback on rare events. Beyond its theoretical appeal, such dynamics are relevant in many contexts, including feedback control systems [18] and reinforcement learning algorithms [14], where history-dependent feedback plays a central role in optimizing task execution and controlling stochastic agents.

We used a weak-noise large deviation approach to derive the equations governing the most probable paths that reach a specified position at a given time, or instantons, and characterized the associated action and quasi-potential. We found that the feedback parameter $\beta$ delineates three qualitatively distinct first-passage mechanisms. When $\beta >0$, first-passage is driven by the gradual accumulation of positive fluctuations that shift the effective potential and lead to accelerated dynamics through a reduction of the effective energy barrier at leading order in the noise. The optimal first-passage time, which is the time that minimizes the action, also transitions from being infinite for $\beta \le 0$ to finite for $\beta \in (0,1)$, providing further evidence of accelerated first-passage dynamics. When $\beta <0$, first-passage occurs either through a slingshot-like mechanism or through an almost-ballistic trajectory, both of which are suboptimal compared to classical OU behavior. Our results establish a clear link between feedback strength and first-passage likelihood, showing that memory can be harnessed to fundamentally reshape the statistics of rare events.

In addition to its basic theoretical relevance, our study suggests more broadly that empirical occupation measures could act as versatile memory devices in feedback-controlled systems. In particular, we suggest that one could encode information on transient rare events into a feedback mechanism that influences future dynamics. This feedback, naturally expressible in terms of the empirical occupation measure, is especially responsive on short time scales, where it adapts quickly to new information. On longer time scales, it relaxes towards the invariant distribution, becoming progressively harder to manipulate. This feature makes it particularly suitable for capturing finite-time rare events and using that memory to modulate a system response in a closed-loop control setting, effectively guiding the process to behave in a desired manner.

Our work suggests a number of immediate questions for future research. The model studied here is a continuous-space, continuous-time self-interacting process. It is instructive, however, to draw a parallel with the Elephant Random Walk (ERW) and the Gaussian ERW (GERW) [5, 11], defined in discrete space–time and in continuous space with discrete time respectively. Despite these differences in construction, their fluctuation paths—featuring giant leaps and long excursions—are reminiscent of the first-passage dynamics we observe for $\beta >0$, where beneficial fluctuations are retained through memory feedback. The analogy, however, cannot be pushed too far: the ERW and GERW couple to the empirical current, whereas our model couples to the empirical density; in particular its mean. This raises the intriguing possibility of a broader “universality class” of fluctuation mechanisms in self-interacting processes, extending beyond the specific interaction considered here.

In designing feedback-controlled systems that use self-interaction as a versatile memory device, both theoretical questions and practical implementations are of interest. A compelling direction is to examine the role of finite-time window memory as a feedback mechanism. Rather than considering a full-time average of the form $t^{-1}{\int }_0^{t}K(X_s,X_t)ds$, we suggest introducing a feedback based on a time-dependent finite-time window, such as $\lambda {(t)}^{-1}{\int }_{t-\lambda (t)}^{t}K(X_s,X_t)ds$. This formulation is truly non-Markovian and, in contrast to the model considered here, does not appear to admit a mapping to a finite-dimensional Markov system. Therefore, within the framework developed in this paper, one could also explore alternative forms of feedback, including for example those governed by the empirical variance or standard deviation of the process, rather than the empirical mean. Finally, it is of interest to analyze the effect of a changing environment by incorporating multiplicative noise, explicitly time-dependent noise or a non-autonomous deterministic backbone.

Appendices

Derivation of boundary value problem for mean first-passage time in Eq. (30)

The boundary value problem in (30) can be obtained by adapting the derivation of result 7.1 in [73] by considering a time-dependent generator, such as that in (23). Given the definition of mean first-passage time in (25) and in terms of the survival probability (28), we can write

$$\begin{array}{cc}\hfill {\overline{T}}_s(z_s)& ={\mathbb{E}}_{s,z_s}\left[T_s,(z_s)\right]\hfill \end{array}$$ 64

$$\begin{array}{cc}& ={\int }_s^{+\infty }{f}_{s,t}(z_s)(t-s)dt\hfill \end{array}$$ 65

$$\begin{array}{cc}& =-{\int }_s^{+\infty }\frac{\partial Q_{s,t}(z_s)}{\partial t}(t-s)dt,\hfill \end{array}$$ 66

which, by first integrating by parts and by then using (29), can be re-written as

$$\begin{array}{cc}\hfill {\overline{T}}_s(z_s)& ={\int }_s^{+\infty }{Q}_{s,t}(z_s)dt\hfill \end{array}$$ 67

$$\begin{array}{cc}& ={\int }_s^{+\infty }\left({\int }_{-\infty }^f,{\int }_{-\infty }^{\infty },\rho ,(z,t|z_s,s),,d,\overline{x},,d,x\right)dt.\hfill \end{array}$$ 68

Then, by introducing the semigroup associated to the time-dependent infinitesimal generator (23) and using the initial condition in (27) we find

$$\begin{array}{cc}\hfill {\overline{T}}_s(z_s)& ={\int }_s^{+\infty }\left({\int }_{-\infty }^f,{\int }_{-\infty }^{\infty },e^{L_s^{†}(t-s)},\delta ,(z_s-z),,d,\overline{x},,d,x\right)dt\hfill \end{array}$$ 69

$$\begin{array}{cc}& ={\int }_s^{+\infty }\left({\int }_{-\infty }^f,{\int }_{-\infty }^{\infty },\delta ,(z_s-z),\left(e^{L_s(t-s)},1\right),(z),,d,\overline{x},,d,x\right)dt\hfill \end{array}$$ 70

$$\begin{array}{cc}& ={\int }_s^{+\infty }\left(e^{L_s(t-s)},1\right)(z_s)dt.\hfill \end{array}$$ 71

Applying $L_s$ on both sides, we obtain

$$\begin{array}{c}\hfill L_s{\overline{T}}_s(z_s)={\int }_s^{+\infty }\frac{d}{\mathit{dt}}\left(e^{L_s(t-s)},1\right)(z_s)dt,\end{array}$$ 72

which, after integrating by parts, yields

$$\begin{array}{c}\hfill L_s{\overline{T}}_s(z_s)=-1,\mathrm{for}{\mathrm{z}}_{\mathrm{s}}\in (-\infty ,\mathrm{xf})\times (-\infty ,{\mathrm{x}}_{\mathrm{f}}),\end{array}$$ 73

and ${\overline{T}}_s(z_s)=0$, for $z_s\in x_f\times (-\infty ,\infty )$.