Delay estimation in a two-node acyclic network

Abstract

Linear measures such as cross-correlation have been used successfully to determine time delays from the given processes. Such an analysis often precedes identifying possible causal relationships between the observed processes. The present study investigates the impact of a positively correlated driver whose correlation function decreases monotonically with lag on the delay estimation in a two-node acyclic network with one and two-delays. It is shown that cross-correlation analysis of the given processes can result in spurious identification of multiple delays between the driver and the dependent processes. Subsequently, delay estimation of increment process as opposed to the original process under certain implicit constraints is explored. Short-range and long-range correlated driver processes along with those of their coarse-grained counterparts are considered.

1. Introduction

Estimating delays from the observed processes has been an area of great interest both from theoretical and experimental standpoints. Inferring delays from temporal processes is an inverse problem and can be also be useful in inferring causal relationships between them [1]. The present study investigates a primitive two-node acyclic network comprising of a driver and a dependent process with single and two delays, Fig. 1. We consider the class of drivers (x) whose auto-correlation functions R_x (k) = E(x_nx_n+k) are positive and decreases monotonically as a function of lag (k).

Figure 1.

Two-node acyclic networks with one and two-delays are shown in (a) and (b) respectively. The driver and the dependent processes are represented by (x) and (y).

Classical delay estimation techniques using linear measures such as cross-correlation function are useful when the driver process is uncorrelated. The procedure begins, by estimating the cross-correlation functions R_xy (k) = E(x_ny_n+k) between the driver (x) and the dependent processes (y) as function of lag (k), Fig. 1. A non-zero cross-correlation at a given lag is chosen as the desired delay between x and y. However, drivers need not necessarily be uncorrelated. A classic example is that of a genetic network where an up-stream gene (driver) with auto-regulatory feedback regulates a down-stream gene (dependent) through multiple pathways with distinct delays. In such cases, we show that direct estimation of the delay between x and y from their observed values using measures such as cross-correlation may not be sufficient. Subsequently, we explore delay estimation from the increment processes as opposed to that of the original processes. It is shown that such an approach is highly suitable for correlated drivers under certain constraints.

2. Methods and Results

A. Statistically significant delays

Only positive cross-correlation estimates between the driver and the dependent processes are assessed for statistical significance. The cross-correlation estimate at a given lag is deemed significant if its value is considerably higher than those obtained on the random shuffled counterparts. A brief description of the procedure is enclosed below.

Step 1 Estimate the cross-correlations as a function of the lags R_xy (k), τ = 1…T between the driver and dependent processes x_n and y_n.

Step 2 Generate random shuffled counterparts $x_i^{\ast }$ and $y_i^{\ast },i=1\dots n_s$ of x_n and y_n by resampling without replacement [2]. Estimate the cross-correlation as a function of the delays on the n_s shuffled counterparts $R_{x_i^{\ast }{y}_i^{\ast }}^{\ast }\left(\tau \right),\tau =1\dots {\tau }_{\max },i=1\dots n_s$.

Step 3 Cross-correlation estimate at lag k is statistically significant if $R_{xy}\left(k\right)>R_{x_i^{\ast }{y}_i^{\ast }}^{\ast }\left(k\right),\forall i=1\dots n_s$. This lag (k) is the desired delay between the driver and the dependent processes. Thus a one-side test is sufficient. The number of surrogates was fixed at n_s = 99, this corresponds to a significance level of α⁺ = 1/(99+1) = 0.01 [3-4] for a one-sided test.

In order to estimate statistically significant delays from the increment processes δx_n = x_n+1 - x_n and δy_n = y_n+1 - y_n repeat Steps 1, 2 and 3 for the increment processes.

Prior to a detailed discussion we illustrate the motivation behind the choice of delay estimation on the increment process with a simple example.

Example: Consider a two-node acyclic network with a single delay

$$\begin{array}{cc}\text{Driver}\left(x\right)\hfill & :x_1{x}_2{x}_3..........x_n{x}_{n+1}{x}_{n+2}........................................x_N\hfill \\ \text{Dependent}(y_n=x_{n-\tau })\hfill & :y_1{y}_2{y}_3..........y_n{y}_{n+1}{y}_{n+2}.......y_{n+\tau }{y}_{n+\tau +1}{y}_{n+\tau +2}.....y_N\hfill \\ \text{White noise process}\left(e\right)\hfill & :e_1{e}_2{e}_3..........e_n{e}_{n+1}{e}_{n+2}..........................................e_N\hfill \end{array}$$

Case (i) Uncorrelated Driver

Delay estimation from the given processes

Consider the uncorrelated driver (x) sampled from a white-noise process e with zero-mean and variance R_e (0) (i.e. x_n = e_n,n = 1…N) and the dependent process (y_n = x_n-τ). Cross-correlation estimates as a function of lag k yields

$$\begin{array}{cc}\hfill E(x_{n+k}\cdot y_{n+\tau })=E(x_{n+k}\cdot x_{n+\tau })=E(e_{n+k}\cdot e_{n+\tau })& =R_e\left(0\right),\mathrm{for}k=0\hfill \\ \hfill & =0,\mathrm{for}k\ne 0\hfill \end{array}$$ (1)

A positive cross-correlation estimate exists only for k = 0, which corresponds to delay τ between x_n and y_n.

Delay estimation from the increment processes

Consider the increment processes δx_n = x_n+1 - x_n and δy_n = y_n+1 - y_n.

$$\begin{array}{cc}\hfill E(\delta x_{n+k}\cdot \delta y_{n+\tau })=E(\delta x_{n+k}\cdot \delta x_{n+\tau })=E(e_{n+k}\cdot e_{n+\tau })& =2R_e\left(0\right),\mathrm{for}k=0\hfill \\ \hfill & =2R_e\left(k\right)-R_e(k+1)-R_e(k-1),\mathrm{for}k\ne 0\hfill \end{array}$$ (2)

Unlike (1) cross-correlation of the increment processes persist for delays k = -1, 0, 1. However, from (2) cross-correlation estimate is positive only for k = 0 and negative for k = -1 and 1. From our definition of statistical significance (Sec. A), cross-correlation estimate only at k = 0 is statistically significant.

Since cross-correlation is a linear measure it is possible to identify the delay even for nonlinearly correlated drivers (x) with fast linear de-correlation time comparable to that of white noise process. An example of such a driver is a chaotic logistic map given by the expression x_n+1 = 4.x_n. (1 - x_n). Therefore, in the subsequent discussion the term correlated drivers implicitly refers to drivers whose linear de-correlation time is comparatively larger than those of white noise.

Case (ii) Correlated Driver

Consider a driver process generated as linear combination of samples from a white noise process i.e. x_i = e_i + e_i+1, i = 1…n -1. Let the dependent process be y_n = x_n-τ.

Delay Estimation from the given processes

$$\begin{array}{cc}E(x_n\cdot y_{n+\tau })\hfill & =R_x\left(0\right)=2R_e\left(0\right)>0\hfill \\ E(x_{n+1}\cdot y_{n+\tau })\hfill & =R_e\left(0\right)>0\hfill \end{array}$$ (3)

For the correlated driver, positive cross-correlation (3) persists for delays other than τ. Such correlations are an outcome of the correlated nature of the driver and shall be referred to as correlation leak in the subsequent sections. Correlation leak can be statistically significant (Sec. A), and may imply spurious existence of multiple delays between the driver and the dependent processes.

Delay Estimation from the increment processes

$$\begin{array}{cc}\hfill & E(\delta x_n\cdot \delta y_{n+\tau })=2[R_x\left(0\right)-R_x\left(1\right)]=2R_e\left(0\right)>0\hfill \\ \hfill & E(\delta x_{n+1}\cdot \delta y_{n+\tau })=2.R_e\left(k\right)-R_e(k+2)-R_e(k-2)\mathrm{for}k\ne 0\hfill \end{array}$$ (4)

Cross-correlation analysis of the increment series (4) reveals statistically significant positive correlation (Sec. A), only at k = 0. While cross-correlation persists for lags k = -2, 2, these are negative. Unlike (3), cross-correlation analysis of the increment process can be useful in minimizing contributions due to correlation leak.

Inspired by the above example, cross-correlation analysis of increment processes in conjunction with those of the original process in delay estimation in a two-node acyclic network is explored. As noted earlier, the driver processes is implicitly assumed to be positively correlated with monotonic decreasing auto-correlation function. In this respect, we discuss the results for short-range correlated stationary first-order Gauss-Markov driver process and long-range correlated stationary fractional auto-regressive integrated moving average driver (FARIMA) process [5]. Instances of delay estimation on the coarse-grained counterparts of their increment series are also discussed.

B. Short-range correlated driver

Short-range correlated stationary first-order Gauss-Markov process is given by the expression

$$x_t=\alpha x_{t-1}+e_t$$ (5)

where e_t sampled from a normally distributed white noise with zero-mean and unit variance. Since we consider positively correlated driver process with monotonic decreasing auto-correlation function we consider only processes (5) where 0 <α <1. The corresponding auto-correlation function R_x (k) for 0 <α <1

$$R_x\left(k\right)=E\left(x_n{x}_{n+k}\right)=a^{k}R\left(0\right)>0\forall k$$ (6)

B1. Short-range correlated driver and single-delay

An example of two-node acyclic network with single delay is shown in Fig. 1a. Consider the cases where the dependent node $y_n^o$ which lags the driver node by a delay τ given by

$$y_n^o=\beta x_{n-\tau }\text{such that}\beta >0,\tau >0$$ (7)

In (7), β contributes to the overall variance of the process, hence can be factored out to obtain the normal form y_n,

$$y_n=\frac{y_n^o}{\beta }=x_{n-\tau }$$ (8)

Delay estimation from the original process

Cross-correlation function between the driver x_n (7) and the dependent y_n (8) processes at lag k is given by

$$R_{xy}(\tau -k)=E\left(x_{n+k}{y}_{n+\tau }\right)=E\left(x_{n+k}{x}_n\right)=R_x\left(k\right)$$ (9)

Substituting for the auto-correlation function R_x ()from (6) we get

$$R_{xy}(\tau -k)={\alpha }^k{R}_x\left(0\right)>0\forall k$$ (10)

Since 0 <α <1, we have R_xy(τ-k) < R_xy(τ),∀k > 0. Thus irrespective of the choice of the process parameterα, the driver and the dependent nodes are maximally correlated at lag k = τ which corresponds to the delay between x_n and y_n. However, there is considerable positive correlation leak across lags (τ - k), k ≠ 0. whose magnitude R_xy (τ - k), k ≠ 0 increases as a function of the process parameterα. The correlation leak is especially significant in the limitα → 1. An instance of cross-correlation estimate as a function of lags for the driver and the dependent processes (5 and 8) with parameters (α = 0.9, τ = 10) is shown in Fig. 2a. Statistically significant cross-correlation (Sec. A) is observed at a number of lags in addition to k = τ. This is not a drawback of the estimation procedure but an inherent feature due to the correlated nature of the driver. As noted earlier (3), it is possible to infer spurious existence of multiple delays (directional paths) from the driver to the dependent process.

Figure 2.

Cross-correlation estimates as a function of delay (k) for the original R_xy (k) (a) and increment processes R_δxδy (k) (b) in a two-node acyclic network with a single delay (τ = 10) and Gauss-Markov driver (α = 0.9, N = 4000). Statistically significant delay estimates (n_s = 99, Sec A) are shown by circles.

Delay estimation from the increment process

Consider the increment processed δy_n+1 = y_n+1 - y_n and δx_n+1 = x_n+1 - x_n corresponding to x_n (5) and y_n (8). The corresponding cross-correlation function at lag k is given by

$$R_{\delta x\nabla \delta }(\tau -k)=E\left(\delta x_{n+k}\delta y_{n+\tau }\right)=2R_x\left(k\right)-R_x(k+1)-R_x(k-1)$$ (11)

Substituting for the auto-correlation function R_x ()from (6) we get

$$R_{\delta x\delta y}\left(\tau \right)=2.(1-\alpha )R_x\left(0\right)>0$$ (12)

$$R_{\delta x\delta y}(\tau -k)=-{\alpha }^{(k-1)}(1-{\alpha }^2)<0$$ (13)

From (12 and 13) we note that R_δxδy (τ - k) < R_δxδy (τ). More importantly, we note that R_δxδy (τ) > 0 whereas R_δxδy (τ - k) < 0, ∀ k ≠ 0. An instance of cross-correlation estimates as a function of lags for the increment of the driver and the dependent processes with (α = 0.9, τ = 10) is shown in Fig. 2b. The cross-correlation estimate was statistically significant (Sec. A), only at lag k = τ which corresponds to the delay between the driver and the dependent processes. These results have to be contrasted with those of Fig. 2a, where the correlation leak R_xy (τ - k) resulted in identifying multiple delays between the driver and the dependent processes.

Summary I For a two-node acyclic network with a single delay and Gauss-Markov driver with parameter0 < α < 1, delay estimation using cross-correlation on the original process can result in significant positive correlation at several delays in addition to that of τ, attributed to inherent correlation leak. These in turn may indicate spurious existence of multiple delays (directional paths) between the driver and the dependent processes. However, analysis on the increment processes resulted in positive cross-correlation only at lag corresponding to the delay between the driver and the dependent processes.

B2. Short-range correlated driver and two-delays

An example of two-node acyclic network with two delays and a correlated driver (5) is shown in Fig. 1b. The dependent process is generated as a linear combination of the driver (5) with delays τ₁ and τ₂ as

$$y_n^o={\beta }_1{x}_{n-{\tau }_1}+{\beta }_2{x}_{n-\tau 2}\text{such that}{\beta }_2>{\beta }_1>0,{\tau }_2>{\tau }_1>0$$ (14)

In order to obtain the normal form y_n of $y_n^o$ we follow the steps below

$$y_n^o={\beta }_2(\frac{{\beta }_1}{{\beta }_2}{x}_{n-{\tau }_1}+x_{n-\tau 2})$$

Substituting, $\beta =\frac{{\beta }_1}{{\beta }_2}$ such that 0 < β < 1 in the above expression, we get

$$\begin{array}{cc}\hfill y_n^o& ={\beta }_2(\beta x_{n-{\tau }_1}+x_{n-\tau 2})\text{I}\hfill \\ \hfill y_n& =\frac{y_n^o}{{\beta }_2}=\beta x_{n-{\tau }_1}+x_{n-\tau 2}\text{such that}{\tau }_2>{\tau }_1>0,0<\beta <1\hfill \end{array}$$ (15)

In (15) β₂ affects the overall variance of $y_n^o$, hence can be factored out. In the subsequent discussion we shall only consider the normal form y_n (15).

Delay estimation from the original process

From (15) we have

$$y_{n+{\tau }_1}=\beta x_n+x_{n-\tau },\mathrm{where}\tau ={\tau }_2-{\tau }_1>0$$ (16)

$$y_{n+{\tau }_2}=\beta x_{n+\tau }+x_n,\mathrm{where}\tau ={\tau }_2-{\tau }_1>0$$ (17)

Their corresponding cross-correlation functions with x_n (5) is given by

$$R_{xy}\left({\tau }_1\right)=E\left(x_n{y}_{n+{\tau }_1}\right)=\beta .R_x\left(0\right)+R_x\left(\tau \right)>0,\mathrm{where}\tau ={\tau }_2-{\tau }_1$$ (18)

$$R_{xy}\left({\tau }_2\right)=E\left(x_n{y}_{n+{\tau }_2}\right)=\beta .R_x\left(\tau \right)+R_x\left(0\right)>0,\mathrm{where}\tau ={\tau }_2-{\tau }_1$$ (19)

From (18 and 19) it can be seen that the magnitude of the cross-correlation between the driver and the dependent process is proportional to parameter β.

$$\mathrm{Remark}1R_{xy}\left({\tau }_2\right)>R_{xy}\left({\tau }_1\right)$$ (20)

Subtracting (18) from (19) we get

$$R_{xy}\left({\tau }_2\right)-R_{xy}\left({\tau }_1\right)=(1-\beta )\cdot [R_x\left(0\right)-R_x\left(\tau \right)]$$

Since R_x (m) > R_x(n) for m < n and 0 < β < 1, R_xy (τ₂) > R_xy (τ₁).

In the case of uncorrelated driver, the following inequality holds R_xy (τ₂) > R_xy (τ₁) > R_xy (k) = 0 for k ≠ τ₁,τ₂. Thus ranking the cross-correlation function in descending order is useful in inferring the delays between the driver (5) and the dependent (15) processes. However, such a ranking need not necessarily hold in the case of correlated drivers. As correlation leak around delayτ₂ can be significantly higher than that of R_xy (τ₁). This in turn implies that ranking the cross-correlation can result in spurious identification of delays between the driver and the dependent processes. In the following Remark, we derive a constraint on the process parameters (α and β) in order to preserve the ranking R_xy (τ₂) > R_xy (τ₁) > R_xy (k) for k ≠ τ₁,τ₂.

Remark 2 Constraint on the parameters α and β such that R_xy (τ₂) > R_xy (τ₁) > R_xy (k) for k ≠ τ₁,τ₂.

From (16), we have

$$E\left(x_{n+1}{y}_{n+{\tau }_2}\right)=R_{xy}({\tau }_2-1)=E\left(x_{n+1}{y}_{n+{\tau }_2}\right)=\beta .R_{xy}(\tau -1)+R_x\left(1\right)$$ (21)

In order for the ranking to be preserved we need

$$R_{xy}\left({\tau }_1\right)>R_{xy}({\tau }_2-1)$$ (22)

Since R_xy (τ₂) > R_xy (τ₂ - 1) and R_xy (τ₂) > R_xy (τ₁) from (20) Substituting from (18) and (21) in (22) we get

$$\begin{array}{c}\hfill \beta .R_x\left(0\right)+R_x\left(\tau \right)>\beta .R_x(\tau -1)+R_x\left(1\right)\hfill \\ \hfill \beta >\frac{R_x\left(1\right)-R_x\left(\tau \right)}{(R_x\left(0\right)-R_x(\tau -1))}\hfill \end{array}$$

Substituting for the auto-correlation function R_x ()from (6) we get

$$\beta >\frac{\alpha -{\alpha }^{\tau }}{1-{\alpha }^{\tau -1}}=\alpha i.e.\beta >\alpha$$ (23)

Thus the constraint on the parameters (α and β) so as to preserve the ranking R_xy (τ₂) > R_xy (τ₁) > R_xy (k) is β > α.

Cross-correlation estimates R_xy (k) between the driver and the dependent processes as a function of lag (k) for (β <α) with parameters (β = 0.5,α = 0.7,τ₁ = 5,τ₂ = 11) is shown in Fig. 3a. As expected (23), correlation leak around (τ₂ = 5) results in statistically significant cross-correlation estimates at lags (k = 10 and 11) considerably larger than those at (τ₁ = 5). This in turn disrupts the ranking R_xy (τ₂) > R_xy (τ₁) > R_xy (k). However, for β > α with (β = 0.8,α = 0.7) dominant cross-correlation estimates occur at delays (τ₁ = 5,τ₂ = 11) preserving the ranking R_xy (τ₂) > R_xy (τ₁) > R_xy (k), Fig. 3c. While the ranking is preserved for constraint β > α, cross-correlation estimates at lags other than (k = 5,11) corresponding to delays (τ₁ and τ₂) are rendered statistically significant. As seen earlier (Sec. B), these can indicate spurious existence of multiple delays between the driver and the dependent processes in addition to (τ₁ and τ₂). It is also important to note that the constraint β > α for preserving the rank turns out to be stringent especially in the limit α → 1 (5), i.e. the family of processes from which the delays can be inferred reduces dramatically as α → 1.

Figure 3.

Cross-correlation estimates as a function of delay (k) of the original R_xy (k) (a, c) and increment processes R_δxδy (k) (b, d) in a two-node acyclic network with a two delays (τ₁ = 5,τ₂ = 11) and Gauss-Markov driver (α = 0.7, N = 4000). Cross-correlation estimates R_xy (k) for original processes violating constraint (23) i.e. (β < α, β = 0.5,α = 0.7)is shown in Fig 3a. Those of its increment R_δxδy (k) are shown in Fig. 3b. Cross-correlation estimates of original processes R_xy (k) satisfying constraint (β > α, β = 0.8,α = 0.7) is shown in Figs. 3c. Those of its increment series R_δxδy (k) are shown in Fig. 3d. Statistically significant delay estimates (n_s = 99, Sec. A) are shown by circles.

Delay estimation from the increment process

Cross-correlation between the increment series δx_n+1 = x_n+1 - x_n and δy_n+1 = y_n+1 - y_n delays τ₁ and τ₂ are given by

$$R_{\delta x\delta y}\left({\tau }_1\right)=E\left(\delta x_n\delta y_{n+{\tau }_1}\right)=2\beta .[R_x\left(0\right)-R_x\left(1\right)]+[2R_x\left(\tau \right)-R_x(\tau +1)-R_x(\tau -1)]$$ (24)

$$R_{\delta x\delta y}\left({\tau }_2\right)=E\left(\delta x_n\delta y_{n+{\tau }_2}\right)=2.[R_x\left(0\right)-R_x\left(1\right)]+\beta [2R_x\left(\tau \right)-R_x(\tau +1)-R_x(\tau -1)]$$ (25)

Substituting for the auto-correlation function R_x ()from (6) we get

$$R_{\delta x\delta y}\left({\tau }_1\right)=[2\beta .(1-\alpha )-{\alpha }^{\tau -1}{(1-\alpha )}^2]R_x\left(0\right)$$ (26)

$$R_{\delta x\delta y}\left({\tau }_2\right)=[2.(1-\alpha )-\beta {\alpha }^{\tau -1}{(1-\alpha )}^2]R_x\left(0\right)$$ (27)

It is important to note that the expressions (26) and (27) need not necessarily be positively correlated for every choice of the parameters (α and β). As noted earlier (Sec. B), we are interested in identifying only delays whose cross-correlation functions are positive. Therefore, prior to checking rank preservationR(τ₂) > R(τ₁) > R(k), ∀k ≠ τ₁,τ₂ we impose the constraint for positive cross-correlations at delays τ₁ and τ₂.

Remark 3 Constraint on parameters (α and β) such that R_δxδy (τ₁) and R_δxδy (τ 2) are positively correlated.

Substituting for R_δxδy (τ₁) from (26) and imposing the constraint for positive correlation i.e. R_δxδy (τ₁) > 0 we get

$$\beta >(1∕2){\alpha }^{\tau -1}(1-\alpha )$$ (28)

Subtracting (26) from (27) we get

$$R_{\delta x\delta y}\left({\tau }_2\right)-R_{\delta x\delta y}\left({\tau }_1\right)=(1-\beta ).(1-\alpha )[2.+{\alpha }^{\tau -1}(1-\alpha )]R_x\left(0\right)$$ (29)

From (5) and (15) we know 0 < α < 1 and 0 < β < 1, therefore

$$R_{\delta x\delta y}\left({\tau }_2\right)-R_{\delta x\delta y}\left({\tau }_1\right)>0$$

From (28) and (29) we obtain

$$R_{\delta x\delta y}\left({\tau }_2\right)>R_{\delta x\delta y}\left({\tau }_1\right)>0\mathrm{for}\beta >(1∕2){\alpha }^{\tau -1}(1-\alpha )$$ (30)

While the constraint on the original processes (23) is a function of the parameter (α), the constraint on the increment processes (30) is a function of the parameter (α) as well as the differential delay(τ = τ₂ - τ₁). It is important to note that the constraint on the increment process (30) is not as stringent as that on the original process (23) in general. For instance, cross-correlation analysis of the increment process, Figs. 3b and 3d, preserves the ranking R_xy (τ₂) > R_xy (τ₁) > R_xy (k) for both the instances (β > α) and (β < α) discussed earlier, Figs. 3a and 3c. However, for the special case where the differential delay (τ = τ₂ - τ₁ = 1), the constraint on β for the increment process (30), $(\beta >\frac{1-\alpha }{2})$ can be considerably larger than those on the original process (23) (β > α) especially for (α < 1/3). Thus delay estimation on the original process as opposed to that of the increment process is preferred for (α < 1/3) and (τ = τ₂ - τ₁ = 1). An instance with parameters (α = 0.1, β = 0.3,τ₁ = 10,τ₂ = 11) is shown in Fig. 4a. For these choices of parameters constraint (23) is satisfied whereas constraint (30) is not. Therefore, the delays can be successfully estimated from the original processes, Fig. 4a and not from the increment processes, Fig. 4b. However, cross-correlation estimates of the original processes reveals delays (τ₁ = 9 and τ₂ = 12) as being statistically significant in addition to (τ₁ = 10 and τ₂ = 11), Fig. 4a. For τ > 1, constraint (30) is considerably less stringent than constraint (23) irrespective of the choice of α, encouraging estimation of the delay from increment process as opposed to the original process. An instance (β = 0.05,α = 0.7,τ₁ = 10,τ₁ = 12) where neither of the constraints (23 and 30) is satisfied is shown in Figs. 4c and 4d respectively. In such cases, it is not possible to estimate the delays using the techniques described in the present study.

Figure 4.

Cross-correlation estimates as a function of delay (k) of the original R_xy (k) (a) and increment processes R_δxδy (k) (b) in a two-node acyclic network with a two delays (τ₁ = 10,τ₂ = 11) and Gauss-Markov driver (α = 0.1, N = 4000). Cross-correlation estimates that satisfy constraints (23) and (30) i.e. (β = 0.3,α = 0.1,τ = 1)for the original and increment processes is shown in Figs. 4a and 4b respectively. Cross-correlation estimates for the original and increment processes with parameters (β = 0.05,α = 0.7,τ₁ = 10,τ₁ = 12) is shown in Figs. 4c and 4d respectively. Statistically significant delay estimates (n_s = 99, Sec. A) are show by circles.

Finally, we show in the following remark that the ranking of the cross-correlation R(τ₂) > R(τ₁) > R(k)∀k ≠ τ₁,τ₂ between the driver and the dependent processes is implicitly preserved in the increment series unlike those of the original series (23). Cros-correlation estimates satisfy R_δxδy (τ₂) > R_δxδy (τ₁) > 0 under constraint (30). The only possibility that can disrupt the ranking R_δxδy(τ₂) > R_δxδy (τ₁) > R_δxδy (k), k ≠ τ₁, τ₂ is correlation leak around τ₁ and τ₂. In the following remark we show that correlation leak around τ₁ and τ ₂ are strictly negative. Therefore, the only positive cross-correlations estimates on the increment processes occur at delays τ₁ and τ₂. i.e. R_δxδy(τ₂) > R_δxδy (τ₁) > 0 whereas R_δxδy(k) < 0 for k ≠ τ₁,τ₂.

Remark 4 E(δx_n+kδy_n+τ1) < 0 for any k > 0, τ > 0

$$E\left(\delta x_{n+k}\delta y_{n+{\tau }_1}\right)=\beta .[2.R_x\left(k\right)-R_x(k+1)-R_x(k-1)]+[2R_x(k+\tau )-R_x(k+\tau +1)-R_x(k+\tau -1)]$$

Substituting for the auto-correlation function R_x ()from (5) we get

$$E\left(\delta x_{n+k}\delta y_{n+{\tau }_1}\right)=-{(1-\alpha )}^2(\beta {\alpha }^{k-1}+{\alpha }^{k+\tau -1})R_x\left(0\right)<0$$ (31)

Remark 5 E(δx_n+kδy_n+t2) < 0 for k > 0, τ > 0 Substituting for the auto-correlation function R_x ()from (5) we get For k > 0, 0 < t < k

$$E\left(\delta x_{n+k}\delta y_{n+t_2}\right)=-{(1-\alpha )}^2{\alpha }^{k-1}({\alpha }^{-\tau }\beta +1)R_x\left(0\right)<0$$ (32)

For k > 0, t > k

$$E\left(\delta x_{n+k}\delta y_{n+t_2}\right)=-{(1-\alpha )}^2({\alpha }^{\tau -k-1}\beta +{\alpha }^{k-1})<0$$ (33)

Summary II For a two-node network with two delays and Gauss-Markov driver (0 < α < 1), delay estimation on the increment processes results in significant positive cross-correlation only at the respective delays τ₁ and τ₂ under constraint (30). This should be contrasted against delay estimation on the original processes where significant positive cross-correlations is observed at several delays in addition to that of τ₁ andτ₂. Thus it is possible to identify multiple delays in addition to τ₁ and τ₂ on cross-correlation analysis of the original processes. Constraint (23) imposed on the original processes for preserving the rank R(τ₂) > R(τ₁) > R(k), ∀k ≠ τ₁,τ₂ is in general more stringent than the constraint (30) on the increment processes.

C. Long-range correlated driver with single and two-delays

Gauss-Markov driver process (5) considered in the above discussion is a short-range correlated driver whose correlation function decays exponentially as a function of lag (6). Non-markovian or long-range correlations have been observed in a wide-range of experimental systems [5, 6] and accompanied by auto-correlation functions that decay as a power-law [5, 7] with lag. Identifying delays from the original and increment processes for a two-node acyclic network with a long-range correlated driver is briefly discussed below.

Power-law correlated driver

Auto-correlation function of classical long-range correlated noise exhibit power-law decay at large time scales (k) and follows the generic form [5, 7].

$$R_x\left(k\right)=k^{-\gamma },\text{where the Hurst exponent}\gamma \text{line in the interval}(0.5,1)$$ (34)

The auto-correlation function (34) is positive and decays monotonically as function of the lag k.

C1. Long-range correlated driver and one delay

Consider the driver process (34) and the dependent process (Sec. B1)

$$y_n=x_{n-\tau }$$ (35)

Delay estimation from the original process

Following procedure similar to (Sec. B1) we get

$$E\left(x_{n+k}{y}_{n+\tau }\right)=R_x\left(k\right)>0\forall \mathrm{k}$$ (36)

Also from (34)

$$R_x\left(k\right)<R_x\left(0\right)$$ (37)

As in the case of Gauss-Markov process (9, 10, Sec. B1) positive cross-correlations persist for lags other than delay τ.

Delay estimation from the increment process

Following procedure similar to (Sec. B2) we get

$$E\left(\delta x_n\delta y_{n+\tau }\right)=2[R_x\left(0\right)-R_x\left(1\right)]>0$$ (38)

$$E\left(\delta x_{n+k}\delta y_{n+\tau }\right)=2R_x\left(k\right)-R_x(k+1)-R_x(k-1)$$ (39)

Substituting for R_x (k) from (34) into (39) we get

$$E\left(\delta x_{n+k}\delta y_{n+\tau }\right)=k^{-\gamma }[2-{(1+\frac{1}{k})}^{-\gamma }-{(1+\frac{1}{k})}^{-\gamma }]$$ (40)

Binomial expansion of (40) under the assumptions in (34), i.e. k >> γ and 0 < β < 1 we get

$$E\left(\delta x_{n+k}\delta y_{n+\tau }\right)=-k^{-\gamma }[\frac{\gamma (\gamma +1)}{k^2}+.....]<0$$ (41)

The above expression (41) is negative for ∀k ≠ 0. As in the case of Gauss-Markov driver (12, 13, Sec. B1) E(δx_nδy_n+τ) > 0 whereas E(δx_n+kδy_n+τ) < 0 for ∀ k ≠ 0.

C2. Long-range correlated driver and two delays

Consider the case of two delays, where the driver process x_n satisfies (34) and the dependent process (Sec. B2) satisfies

$$y_n=\beta .x_{n-{\tau }_1}+x_{n-{\tau }_2};0<\beta <1,{\tau }_2>{\tau }_1>0$$ (42)

Delay estimation from the original process

As in the case of the Gauss-Markov process (20) we obtain

$$R_{xy}\left({\tau }_2\right)>R_{xy}\left({\tau }_1\right)$$

Constraint on the parameter (β) (23) in order to preserve the ranking R (τ₂) > R (τ₁) > R (k), ∀k ≠ τ₁, τ₂ is

$$\beta >\frac{R_x\left(1\right)-R_x\left(\tau \right)}{(R_x\left(0\right)-R_x(\tau -1))}$$ (43)

Delay estimation from the increment process

Following procedure similar to (Sec. B2) and from the binomial expansion (41) it is possible to obtain a constraint for R_δxδy (τ₂) > R_δxδy (τ₁) > 0. Following procedure similar to Remarks 4 and 5 and using the binomial expansion (41) it can be shows that E(δx_n+kδy_n+τ1) < 0 and E(δx_n+kδy_n+t2) < 0.

Summary III As in the case of Gauss-Markov process (Summary I and Ii), delay estimation on the increment process of long-range correlated driver can significantly minimize the impact of spurious identification of delays between the driver and the dependent process.

An instance of delay estimation from two-node acyclic network with long-range correlated driver and with one and two-delays is shown in Figs. 5 and 6. Long-range correlated driver process was generated from stationary fractional auto-regressive integrated moving average process FARIMA (0, d, 0) with Gaussian innovations and parameter d = 0.3 [5]. This corresponds to Hurst exponent γ = d + 0.5 (34). Cross-correlation analysis R_xy (k) between FARIMA (0, d, 0) driver and the dependent process (y_n = x_n-τ, τ = 10, N = 4000) along with those of their increment processes R_δxδy (k) is shown in Figs. 5c and 5d respectively. As seen earlier, delay estimation of the increment process minimizes spurious statistically significant delays. Cross-correlation estimates for long-range correlated driver and dependent process y_n = β.x_n-τ1 + x_n-τ2, with parameters (β = 0.5, τ₁ = 5, τ₂ = 11, N = 4000) is shown in Figs. 6c and 6d respectively. The ranking R_δxδy (τ₂) > R_δxδy (τ₁) > R_δxδy (k), k ≠ τ₁, τ₂ is preserved on cross-correlation analysis of the increment process, Fig. 6d. This has to be contrasted to analysis of the original process where the ranking is not preserved, Fig. 6c. Analysis of the original process also reveals statistically significant cross-correlation estimates at several lags in addition to (τ₁ = 5 and τ₂ = 11).

Figure 5.

Cross-correlation estimates as a function of delay (k) for the original R_xy (k) (a) and increment processes R_δxδy (k) (b) in a two-node acyclic network with a single delay (τ = 10), with Gauss-Markov (α = 0.9, N = 4000) (a, b) and FARIMA (0, d, 0) driver (d = 0.3, N = 4000) (c, d). Cross-correlation estimates of the corresponding coarse-grained realizations of the original R_xcyc (k) and increment series R_δxcδyc (k) of the Gauss-Markov (e, f) and FARIMA (0, d, 0) (g, h) driver is shown right below them. Statistically significant delay estimates (n_s = 99, Sec. A) are show by circles.

Figure 6.

Cross-correlation estimates as a function of delay (k) for the original R_xy (k) (a) and increment processes R_δxδy (k) (b) in a two-node acyclic network with two delays (τ₁ = 5,τ₂ = 11, β = 0.5) with Gauss-Markov (α = 0.9, N = 4000) (a, b) and FARIMA (0, d, 0) driver (d = 0.3, N = 4000) (c, d). Cross-correlation estimates of the corresponding coarse-grained realizations of the original R_xcyc (k) and R_δxcδyc (k) increment series corresponding to the Gauss-Markov (e, f) and FARIMA (0, d, 0) (g, h) driver is shown right below them. Statistically significant delay estimates (n_s = 99, Sec. A) are show by circles.

D. Delay estimation from coarse-grained realizations

Coarse-grained realizations are simplified representations of the actual processes. An example is that of a one-dimensional ising spin model where each element is either an up (+1) or a down (-1) spin. In the present study, we generate coarse-grained realizations of the given process about their mean, E(x), given by

$$\begin{array}{cc}\hfill x_c^i& =+1\mathrm{if}{x}_i>E\left(x\right)\hfill \\ \hfill & =-1\mathrm{otherwise}\hfill \end{array}$$ (44)

For stationary zero-mean normally distributed processes, an analytical expression can be derived relating the correlation of the original process R_x (k) to that of its coarse-grained counterpart R_xc (k) [8, 9], given by

$$R_{x_c}\left(k\right)=\frac{2}{\pi }\arcsin \frac{R_x\left(k\right)}{R_x\left(0\right)}$$ (45)

It is important to note that in Sec. B and C, the short-range (5) and the long-range (34) correlated driver were generated as linear combinations of normally distributed variables, hence normally distributed. This in turn implies that coarse-grained realizations about the mean of the driver processes (5 and 34) follow relation (45). Since the short-range and long-range driver processes R_x (k) considered have monotonic decreasing autocorrelation function, those of their coarse-grained counterpart R_xc (k) (45) are also monotonic decreasing. Dependent processes y_n = x_n-τ and y_n = β.x_n-τ1 + x_n-τ2 of the normally distributed driver x_n are linear combinations of normal processes, hence implicitly normal. Thus coarse-grained representation of the driver process and the dependent processes about their means follows relation (45). It should also be noted that the corresponding increment series by definition is the difference of normally distributed processes, hence normal.

In the following discussion, coarse-grained realizations of the original and the increment driver x_n and dependent y_n processes shall be represented by x_nc and y_nc respectively. The coarse-grained realizations of the increment processes (δx_n and δy_n) are represented by (δx_nc and δy_nc). The cross-correlation estimates on the coarse-grained original and increment processes are represented by R_xcyc (k) and R_δxcδyc (k).

We show instances where R_δxcδyc (k) is useful identifying delays whereas unlike R_xcyc (k). This is demonstrated on the two-node acyclic networks with short-range and long-range correlated drivers with one and two-delays. Cross-correlation estimates R_xcyc (k) for the coarse-grained realizations of the Gauss-Markov driver (5) with (α = 0.9, N = 4000) and the dependent process (y_n = x_n-τ,τ = 10), along with those of the ir increment series R_δxcδyc (k) is shown in Figs. 5e and 5f respectively. Cross-correlation estimates R_xy (k) and R_δxδy (k) obtained on x_n and y_n is shown in Figs. 5a and 5b for qualitative comparison. It is important to note that the estimation on the increment series results is minimizing the effect of correlation leak as observed earlier (Summary I). Similar results were obtained in the case of FARIMA (0, d, 0) driver with (d = 0.3, N = 4000), Figs. 5g and 5h. These results conform to earlier observations (Summary I and III), where analysis of the increment processes minimize statistically significant false-positive correlation.

Cross-correlation estimates R_xcyc (k) for the coarse-grained realizations of the Gauss-Markov driver (5) with (α = 0.9, N = 4000) with dependent process (y_n = β.x_n-τ1 + x_n-τ2, τ₁ = 5, τ₂ = 11, N = 4000) is shown in Fig. 6e. Cross-correlation analysis of coarse-grained counterparts of the corresponding increment series R_δxcδyc (k) is shown in Fig. 6f. A similar analysis of the FARIMA (0, d, 0) driver with (d = 0.3, N = 4000) is shown in Figs. 6g and 6h. These results conform to earlier observations (Summary II and III), where analysis of the increment processes minimizes statistically significant correlation leak and preserves the rank ordering R(τ₂) > R(τ₁) > R(k), k ≠ τ₁,τ₂. Therefore, analysis of the increment process can minimize statistically significant false-positive correlations even in the case of coarse-grained counterparts.

3. Discussion

The present study, investigated statistical estimation of delays between the driver and the dependent processes of a two-node acyclic network with one and two delays using linear measures such as cross-correlation function. While delay estimation is straightforward in the case of uncorrelated drivers, correlated drivers can result in significant correlation leak around the actual delay between the driver and dependent process. Such correlation leak can result in spurious identification of statistically significant delays and existence of multiple paths between the driver and dependent process. Cross-correlation analysis of the increment processes was shown to significantly minimize the effect of correlation leak under certain constraints. In the presence of two-delays between the driver and the dependent node, cross-correlation analysis preserved the ranking of the auto-correlation function in addition to identifying the delays. This was demonstrated on short-range correlated Gauss-Markov process whose auto-correlation function decays exponentially and long-range correlated FARIMA (0, d, 0) driver with power-law decaying autocorrelation function. Correlation properties of stationary normal processes are analytically related to correlation of their corresponding coarse-grained counterpart generated about their mean. An instance was shown where cross-correlation estimates on the coarse-grained realizations of the increment series significantly minimized the effect of correlation leak. Thus from the above results cross-correlation analysis of the increment processes can provide insight into the nature of delays not evident from the analysis of the original processes.