Information Geometry Theoretic Measures for Characterizing Neural Information Processing from Simulated EEG Signals

Abstract

In this work, we explore information geometry theoretic measures for characterizing neural information processing from EEG signals simulated by stochastic nonlinear coupled oscillator models for both healthy subjects and Alzheimer’s disease (AD) patients with both eyes-closed and eyes-open conditions. In particular, we employ information rates to quantify the time evolution of probability density functions of simulated EEG signals, and employ causal information rates to quantify one signal’s instantaneous influence on another signal’s information rate. These two measures help us find significant and interesting distinctions between healthy subjects and AD patients when they open or close their eyes. These distinctions may be further related to differences in neural information processing activities of the corresponding brain regions, and to differences in connectivities among these brain regions. Our results show that information rate and causal information rate are superior to their more traditional or established information-theoretic counterparts, i.e., differential entropy and transfer entropy, respectively. Since these novel, information geometry theoretic measures can be applied to experimental EEG signals in a model-free manner, and they are capable of quantifying non-stationary time-varying effects, nonlinearity, and non-Gaussian stochasticity presented in real-world EEG signals, we believe that they can form an important and powerful tool-set for both understanding neural information processing in the brain and the diagnosis of neurological disorders, such as Alzheimer’s disease as presented in this work.

1. Introduction

Identifying quantitative features from neurophysiological signals such as electroencephalography (EEG) is critical for understanding neural information processing in the brain and the diagnosis of neurological disorders such as dementia. Many such features have been proposed and employed to analyze neurological signals, which not only resulted in insightful understanding of the brain neurological dynamics of patients with certain neurological disorders versus healthy control (CTL) groups, but also helped build mathematical models that replicate the neurological signal with these quantitative features [1,2,3,4,5].

An important distinction, or non-stationary time-varying effects of the neurological dynamics, is the switching between eyes-open (EO) and eyes-closed (EC) states, where numerous research studies have been conducted on this distinction between EO and EC states to quantify important features of CTL subjects and patients using different techniques on EEG data such as traditional frequency-domain analysis [6,7], transfer entropy [8], energy landscape analysis [9], and nonlinear manifold learning for functional connectivity analysis [10], while also attempting to relate these features to specific clinical conditions and/or physiological variables, including skin conductance levels [11,12], cerebral blood flow [13], brain network connectivity [14,15,16], brain activities in different regions [17], and performance on the unipedal stance test (UPST) [18]. Clinical physiological studies found that there are distinct mental states related to the EO and EC states. Specifically, there is an “exteroceptive” mental activity state characterized by attention and ocular motor activity during EO, and an “interoceptive” mental activity state characterized by imagination and multisensory activity during EC [19,20]. Ref. [21] suggested that the topological organization of human brain networks dynamically switches corresponding to the information processing modes when the brain is visually connected to or disconnected from the external environment. However, patients with Alzheimer’s disease (AD) show loss of brain responsiveness to environmental stimuli [22,23], which might be due to impaired or loss of connectivities in the brain networks. This suggests that dynamical changes between EO and EC might represent an ideal paradigm to investigate the effect of AD pathophysiology and could be developed as biomarkers for diagnosis purposes. However, sensible quantification of robust features of these dynamical changes between EO and EC of both healthy and AD subjects, solely relying on EEG signals, is nontrivial. Despite the success of many statistical and quantitative measures being applied to neurological signal analysis, the main challenges stem from the non-stationary time-varying dynamics of the human brain with nonlinearity and non-Gaussian stochasticity, which makes most, if not all, of these traditional quantitative measures inadequate, and blindly applying these traditional measures to nonlinear and nonstationary time series/signals may produce spurious results, leading to incorrect interpretation.

In this work, by using simulated EEG signals of both CTL groups and AD patients under both EC and EO conditions and based on our previous works on information geometry [24,25,26], we develop novel and powerful quantitative measures in terms of information rate and causal information rate to quantify the important features of neurological dynamics of brains. We are able to find significant and interesting distinctions between CTL subjects and AD patients when they switch between the eyes-open and eyes-closed status. These quantified distinctions may be further related to differences in neural information processing activities of the corresponding brain regions, and to differences in connectivities among these brain regions, and therefore, they can be further developed as important biomarkers to diagnose neurological disorders, including but not limited to Alzheimer’s disease. It should be noted that these novel and powerful quantitative measures in terms of information rate and causal information rate can be applied to experimental EEG signals in a model-free manner, and they are capable of quantifying non-stationary time-varying effects, nonlinearity, and non-Gaussian stochasticity presented in real-world EEG signals, and hence, they are more robust and reliable than other information-theoretic measures applied to neurological signal analysis in the literature [27,28]. Therefore, we believe that these information geometry theoretic measures can form an important and powerful tool set for the neuroscience community.

The EEG signals have been modeled using many different methodologies in the literature. An EEG model in terms of nonlinear stochastic differential equation (SDE) could be sufficiently flexible in that it usually contains many parameters, whose values can be tuned to match the model’s output with actual EEG signals for different neurophysiological conditions, such as EC and EO, of CTL subjects or AD patients. Moreover, an SDE model of EEG can be solved by a number of numerical techniques to generate simulated EEG signals superior to actual EEG signals in terms of much higher temporal resolution and much larger number of sample paths available. These are the two main reasons why we choose to work with SDE models of EEG signals. Specifically, we employed a model of stochastic coupled Duffing–van der Pol oscillators proposed by Ref. [1], which is flexible enough to represent the EC and EO conditions for both CTL and AD subjects and straightforward enough to be simulated by using typical numerical techniques for solving SDE. Moreover, the model parameters reported in Ref. [1] were fine tuned against real-world experimental EEG signals of CTL and AD patients with both EC and EO conditions, and therefore, quantitative investigations on the model’s output of simulated signals are sufficiently representative for a large population of healthy and AD subjects.

2. Methods

2.1. Stochastic Nonlinear Oscillator Models of EEG Signals

A phenomenological model of the EEG based on a coupled system of Duffing–van der Pol oscillators subject to white noise excitation has been introduced [1] with the following form:

$$\{\begin{array}{c}{\ddot{x}}_1+(k_1+k_2)x_1-k_2{x}_2=-b_1{x}_1^3-b_2{(x_1-x_2)}^3+{ϵ}_1{\dot{x}}_1(1-x_1^2),\hfill \\ {\ddot{x}}_2-k_2{x}_1+k_2{x}_2=b_2{(x_1-x_2)}^3+{ϵ}_2{\dot{x}}_2(1-x_2^2)+\mu dW,\hfill \end{array}$$ (1)

where $x_i,{\dot{x}}_i,{\ddot{x}}_i,i=1,2$ are positions, velocities, and accelerations of the two oscillators, respectively. Parameters $k_i,b_i,{ϵ}_i,i=1,2$ are the linear stiffness, cubic stiffness, and van der Pol damping coefficient of the two oscillators, respectively. Parameter $\mu$ represents the intensity of white noise and $dW$ is a Wiener process representing the additive noise in the stochastic differential system. The physical meanings of these variables and parameters were nicely explained in a schematic figure in Ref. [1].

By using actual EEG signals, Ref. [1] utilized a combination of several different statistical and optimization techniques to fine tune the parameters in the model equations for eyes-closed (EC) and eyes-open (EO) conditions of both healthy control (CTL) subjects and Alzheimer’s disease (AD) patients, and these parameter values for different conditions are summarized in Table 1 and Table 2.

Table 1.

Optimal parameters of the Duffing–van der Pol oscillator for EC and EO of healthy control (CTL) subjects.

Parameter	Eyes-Closed (EC)	Eyes-Open (EO)
$k_1$	7286.5	2427.2
$k_2$	4523.5	499.92
$b_1$	232.05	95.61
$b_2$	10.78	103.36
${ϵ}_1$	33.60	48.89
${ϵ}_2$	0.97	28.75
$\mu$	2.34	1.82

Table 2.

Optimal parameters of the Duffing–van der Pol oscillator for EC and EO of Alzheimers disease (AD) patients.

Parameter	Eyes-Closed (EC)	Eyes-Open (EO)
$k_1$	1742.1	3139.9
$k_2$	1270.8	650.32
$b_1$	771.99	101.1
$b_2$	1.91	81.3
${ϵ}_1$	63.7	56.3
${ϵ}_2$	20.7	19.12
$\mu$	1.78	1.74

The model Equation (1) can be easily rewritten in a more standard form of stochastic differential equation (SDE) as follows:

$$\{\begin{array}{c}{\dot{x}}_1=x_3,\hfill \\ {\dot{x}}_2=x_4,\hfill \\ {\dot{x}}_3=-(k_1+k_2)x_1+k_2{x}_2-b_1{x}_1^3-b_2{(x_1-x_2)}^3+{ϵ}_1{x}_3(1-x_1^2),\hfill \\ {\dot{x}}_4=k_2{x}_1-k_2{x}_2+b_2{(x_1-x_2)}^3+{ϵ}_2{x}_4(1-x_2^2)+\mu dW,\hfill \end{array}$$ (2)

which is more readily suitable for stochastic simulations.

2.2. Initial Conditions (ICs) and Specifications of Stochastic Simulations

For simplicity, we employ the Euler–Maruyama scheme [29] to simulate $2\times {10}^7$ trajectories in total of the model Equation (2); although, other more sophisticated methods for stochastic simulations exist. We simulate such a large number of trajectories, because calculations of information geometry theoretic measures rely on accurate estimation of probability density functions (PDFs) of the model’s variables $x_i\left(t\right)$, which requires a large number of data samples of $x_i\left(t\right)$ at any given time t.

Since nonlinear oscillators’ solution is very sensitive to initial conditions, we start the simulation with a certain initial probability distribution (e.g., a Gaussian distribution) for all $x_1\left(0\right),x_2\left(0\right),x_3\left(0\right),x_4\left(0\right)$, which means that the 20 million $x_i\left(0\right)$$(\forall i=1,2,3,4)$ are randomly drawn from a probability density function (PDF) of the initial distribution. The time-step size $dt$ is set to ${10}^{-6}$ to compensate for the very-high values of stiffness parameters $k_1$ and $k_2$ in Table 1 and Table 2. The total number of simulation time steps is $1\times {10}^7$, making the total time range of simulation $[0,10]$. The $\mathsf{\Delta }t={10}^{-4}$ is the time interval when the probability density functions (PDFs) $p(x_1,t)$ and $p(x_2,t)$ are estimated for calculating information geometry theoretic measures such as information rates and causal information rates, as explained in Section 2.3.

For nonlinear oscillators, different initial conditions can result in dramatically different long-term time evolution. So in order to explore more diverse initial conditions, we simulated the SDE with 6 different initial Gaussian distributions with different means and standard deviations, i.e., $x_1\left(0\right)\sim \mathcal{N}({\mu }_{x_1\left(0\right)},{\sigma }^2)$, $x_2\left(0\right)\sim \mathcal{N}({\mu }_{x_2\left(0\right)},{\sigma }^2)$, $x_3\left(0\right)\sim \mathcal{N}({\mu }_{x_3\left(0\right)},{\sigma }^2)$, $x_4\left(0\right)\sim \mathcal{N}({\mu }_{x_4\left(0\right)},{\sigma }^2)$, where the parameters are summarized alongside other specifications in Table 3.

Table 3.

Initial conditions (IC): $x_1\left(0\right),x_2\left(0\right),x_3\left(0\right),x_4\left(0\right)$ are randomly drawn from Gaussian distributions $\mathcal{N}({\mu }_{x_i\left(0\right)},{\sigma }^2)$ with different ${\mu }_{x_i\left(0\right)}$’s and $\sigma$’s $(i=1,2,3,4)$.

	IC No.1	IC No.2	IC No.3	IC No.4	IC No.5	IC No.6
${\mu }_{x_1\left(0\right)}$	1.0	0.9	0.2	0.1	0.5	0.2
${\mu }_{x_2\left(0\right)}$	0.5	0.1	0.5	0.5	0.9	0.9
${\mu }_{x_3\left(0\right)}$	0	1.0	0.5	0.2	1.0	0.1
${\mu }_{x_4\left(0\right)}$	0	0.5	1.0	1.0	0.8	0.5
$\sigma$	0.1	0.1	0.1	0.5	0.5	0.5
Num. of trajectories	$2\times {10}^7$	$2\times {10}^7$	$2\times {10}^7$	$2\times {10}^7$	$2\times {10}^7$	$2\times {10}^7$
$dt$	${10}^{-6}$	${10}^{-6}$	${10}^{-6}$	${10}^{-6}$	${10}^{-6}$	${10}^{-6}$
$\mathsf{\Delta }t$	${10}^{-4}$	${10}^{-4}$	${10}^{-4}$	${10}^{-4}$	${10}^{-4}$	${10}^{-4}$
Num. of time-steps	$1\times {10}^7$	$1\times {10}^7$	$1\times {10}^7$	$1\times {10}^7$	$1\times {10}^7$	$1\times {10}^7$
Total range of t	$[0,10]$	$[0,10]$	$[0,10]$	$[0,10]$	$[0,10]$	$[0,10]$

For brevity, in this paper, we use the word “initial conditions” or its abbreviation “IC” to refer to the (set of 4) initial Gaussian distributions from which the 20 million $x_i\left(0\right)$$(\forall i=1,2,3,4)$ are randomly drawn. For example, the “IC No.6” in Table 3 (and simply “IC6” elsewhere in this paper) refers to the 6th (set of 4) Gaussian distributions with which we start the simulation, and the specifications of this stimulation are listed in the last column of Table 3.

2.3. Information Geometry Theoretic Measures: Information Rate and Causal Information Rate

When a stochastic differential equation (SDE) model exhibits non-stationary time-varying effects, nonlinearity, and/or non-Gaussian stochasticity, while we are interested in large fluctuations and extreme events in the solutions, simple statistics such as mean and variance might not suffice to compare the solutions of different SDE models (or same model with different parameters). In such cases, quantifying and comparing the time evolution of probability density functions (PDFs) of solutions will provide us with more information [30]. The time evolution of PDFs can be studied and compared through the framework of information geometry [31], wherein PDFs are considered as points on a Riemannian manifold (which is called the statistical manifold), and their time evolution can be considered as a motion on this manifold. Several different metrics can be defined on a probability space to equip it with a manifold structure, including a metric related to the Fisher Information [32], known as the Fisher Information metric [33,34], which we use in this work:

$$g_{\mu \nu }\left(\theta \right)\stackrel{\mathrm{def}}{=}{\int }_X\frac{\partial logp(x;\{\theta \left\}\right)}{\partial {\theta }_{\mu }}\frac{\partial logp(x;\{\theta \left\}\right)}{\partial {\theta }_{\nu }}p(x;\left\{\theta \right\})dx.$$ (3)

Here, $p(x;\{\theta \left\}\right)$ denotes a continuous family of PDFs parameterized by parameters $\left\{\theta \right\}$. If a time-dependent PDF $p(x,t)$ is considered as a continuous family of PDFs parameterized by a single parameter time t, the metric tensor is then reduced to a scalar metric:

$$g\left(t\right)=\int dx\frac{1}{p(x,t)}{\left[\frac{\partial p(x,t)}{\partial t}\right]}^2.$$ (4)

The infinitesimal distance $d\mathcal{L}$ on the manifold is then given by $d{\mathcal{L}}^2=g\left(t\right)d{t}^2$, where $\mathcal{L}$ is called the Information Length and defined as follows:

$$\mathcal{L}\left(t\right)\stackrel{\mathrm{def}}{=}{\int }_0^{t}d{t}_1\sqrt{\int dx\frac{1}{p(x,t_1)}{\left[\frac{\partial p(x,t_1)}{\partial t_1}\right]}^2}.$$ (5)

The Information Length $\mathcal{L}$ represents the dimensionless distance, which measures the total distance traveled on the statistical manifold. The time derivative of $\mathcal{L}$ then represents the speed of motion on this manifold:

$$\mathsf{\Gamma }\left(t\right)\stackrel{\mathrm{def}}{=}\underset{dt\to 0}{lim}\frac{d\mathcal{L}\left(t\right)}{dt}=\sqrt{\int dx\frac{1}{p(x,t)}{\left[\frac{\partial p(x,t)}{\partial t}\right]}^2},$$ (6)

which is referred to as the Information Rate. If multiple variables are involved, such as $x_i\left(t\right)$ where $i=1,2,3,4$ as in the stochastic nonlinear oscillator model Equation (2), we will use subscript in $\mathsf{\Gamma }\left(t\right)$, e.g., ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ to denote the information rate of signal $x_2\left(t\right)$.

The notion of Causal Information Rate was introduced in Ref. [25] to quantify how one signal instantaneously influences another signal’s information rate. As an example, the causal information rate of signal $x_1\left(t\right)$ influencing signal $x_2\left(t\right)$’s information rate is denoted and defined by ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)\stackrel{\mathrm{def}}{=}{\mathsf{\Gamma }}_{x_2}^{*}\left(t\right)-{\mathsf{\Gamma }}_{x_2}\left(t\right)$, where

$${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2=\int d{x}_{2}p(x_2,t){\left[{\partial }_{t}lnp(x_2,t)\right]}^2,$$ (7)

and

$$\begin{array}{cc}\hfill {\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2& \stackrel{\mathrm{def}}{=}\underset{t_{*}\to t^{+}}{lim}\int d{x}_{1}d{x}_{2}p(x_2,t_{*};x_1,t){\left[{\partial }_{t_{*}}lnp(x_2,t_{*}|x_1,t)\right]}^2\hfill \\ \hfill & =\underset{t_{*}\to t^{+}}{lim}\int d{x}_{1}d{x}_{2}p(x_2,t_{*};x_1,t){\left[{\partial }_{t_{*}}lnp(x_2,t_{*};x_1,t)\right]}^2,\hfill \end{array}$$ (8)

where the relation between conditional, joint, and marginal PDFs $p(x_2,t_{*}|x_1,t)=\frac{p(x_2,t_{*};x_1,t)}{p(x_1,t)}$ and the fact ${\partial }_{t_{*}}p(x_1,t)=0$ for $t_{*}\ne t$ are used in the 2nd equal sign above. ${\mathsf{\Gamma }}_{x_2}^{*}$ denotes the (auto) contribution to the information rate from $x_2$ itself, while $x_1$ is given/known and frozen in time. In other words, ${\mathsf{\Gamma }}_{x_2}^{*}$ represents the information rate of $x_2$ when the additional information of $x_1$ (at the same time with $x_2$) becomes available or known. Subtracting ${\mathsf{\Gamma }}_{x_2}$ from ${\mathsf{\Gamma }}_{x_2}^{*}$ following the definition of ${\mathsf{\Gamma }}_{x_1\to x_2}$ then gives us the contribution of (knowing the additional information of) $x_1$ to ${\mathsf{\Gamma }}_{x_2}$, signifying how $x_1$ instantaneously influences the information rate of $x_2$. One can easily verify that if signals $x_1\left(t\right)$ and $x_2\left(t\right)$ are statistically independent such that the equal-time joint PDF can be separated as $p(x_1,t;x_2,t)=p(x_1,t)·p(x_2,t)$, then ${\mathsf{\Gamma }}_{x_2}^{*}\left(t\right)$ will reduce to ${\mathsf{\Gamma }}_{x_2}\left(t\right)$, making the causal information rate ${\mathsf{\Gamma }}_{x_1\to x_2}=0$, which is consistent with the assumption that $x_1\left(t\right)$ and $x_2\left(t\right)$ are statistically independent at the same time t.

For numerical estimation purposes, one can derive simplified equations ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$ = ${\displaystyle 4\int d{x}_2{\left[{\partial }_t\sqrt{p(x_2,t)}\right]}^2}$ and ${\displaystyle {\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2=4\underset{t_{*}\to t^{+}}{lim}\int d{x}_{1}d{x}_2{\left[{\partial }_{t_{*}}\sqrt{p(x_2,t_{*};x_1,t)}\right]}^2}$ to ease the numerical calculations and avoid numerical errors in PDFs (due to finite sample-size estimations using a histogram-based approach) being doubled or enlarged when approximating the integrals in the original Equations (7) and (8) by finite summation. On the other hand, the time derivatives of the square root of PDFs are approximated by using temporally adjacent PDFs with each pair of two adjacent PDFs being separated by $\mathsf{\Delta }t={10}^{-4}$ in time, as mentioned at the end of Section 2.2.

2.4. Shannon Differential Entropy and Transfer Entropy

As a comparison with more traditional and established information-theoretic measures, we also calculate differential entropy and transfer entropy using the numerically estimated PDFs and compare them with information rate and causal information rate, respectively.

The Shannon differential entropy of a signal $x\left(t\right)$ is defined to extend the idea of Shannon discrete entropy as

$$h\left(x\left(t\right)\right)=E[-lnp(x,t)]=-\int p(x,t)lnp(x,t)dx=-\int P\left(dx\left(t\right)\right)ln\frac{P\left(dx\right(t\left)\right)}{\mu \left(dx\right)},$$ (9)

where $\mu \left(dx\right)=dx$ is the Lebesgue measure, and $P\left(dx\right(t\left)\right)=p(x,t)\mu \left(dx\right)=p(x,t)dx$ is the probability measure. In other words, differential entropy is the negative relative entropy (Kullback-Leibler divergence) from the Lebesgue measure (considered as an unnormalized probability measure) to a probability measure P (with density p). In contrast, information rate ${\mathsf{\Gamma }}_x\left(t\right)$ = ${\displaystyle \sqrt{\int dxp(x,t){\left[{\partial }_{t}lnp(x,t)\right]}^2}=\sqrt{\underset{dt\to 0}{lim}\frac{2}{d{t}^2}\int dxp(x,t+dt)ln\frac{p(x,t+dt)}{p(x,t)}}}$ (see Refs. [24,25,26] for detailed derivations) is related to the rate of change in relative entropy of two infinitesimally close PDFs $p(x,t)$ and $p(x,t+dt)$. Therefore, although differential entropy can measure the complexity of a signal $x\left(t\right)$ at time t, it neglects how the signal’s PDF $p(x,t)$ changes instantaneously at that time, which is crucial to quantify how new information can be reflected from the instantaneous entropy production rate of the signal $x\left(t\right)$. This is the theoretical reason why the information rate is a much better and more appropriate measure than differential entropy for characterizing the neural information processing from EEG signals of the brain, and the practical reason for this will be illustrated in terms of numerical results and discussed at the end of Section 3.3.2 and Section 3.3.3.

The transfer entropy (TE) measures the directional flow or transfer of information between two (discrete-time) stochastic processes. The transfer entropy from a signal $x_1\left(t\right)$ to another signal $x_2\left(t\right)$ is the amount of uncertainty reduced in future values of $x_2\left(t\right)$ by knowing the past values of $x_1\left(t\right)$ given past values of $x_2\left(t\right)$. Specifically, if the amount of information is measured using Shannon’s (discrete) entropy $H\left(X_t\right)$ = $-{\sum }_{x}p(x,t){log}_{2}p(x,t)$ of a stochastic process $X_t$ and conditional entropy $H\left(Y_{t_2}\right|X_{t_1})=-{\sum }_{x,y}p(x,t_1;y,t_2){log}_{2}p(y,t_2|x,t_1)$, the transfer entropy from a process $X_t$ to another process $Y_t$ (for discrete-time $t\in \mathbb{Z}$) can be written as follows:

$$\begin{array}{c}\mathrm{(10)}& {\mathrm{TE}}_{X_t\to Y_t}\left(t\right)=H\left(Y_{t+1}\mid Y_{t:t-(k-1)}\right)-H\left(Y_{t+1}\mid Y_{t:t-(k-1)},X_{t:t-(l-1)}\right),\hfill =-{\displaystyle \sum _y}p(y_{t+1},y_{t:t-(k-1)}){log}_{2}p\left(y_{t+1}\right|y_{t:t-(k-1)})\hfill \\ +{\displaystyle \sum _{x,y}}p(y_{t+1},y_{t:t-(k-1)},x_{t:t-(l-1)}){log}_{2}p\left(y_{t+1}\right|y_{t:t-(k-1)},x_{t:t-(l-1)}),\hfill \\ ={\displaystyle \sum _{x,y}}p(y_{t+1},y_{t:t-(k-1)},x_{t:t-(l-1)}){log}_2\frac{p\left(y_{t+1}\right|y_{t:t-(k-1)},x_{t:t-(l-1)})}{p\left(y_{t+1}\right|y_{t:t-(k-1)})},\hfill \\ \mathrm{(11)}& ={\displaystyle \sum _{x,y}}p(y_{t+1},y_{t:t-(k-1)},x_{t:t-(l-1)}){log}_2\frac{p(y_{t+1},y_{t:t-(k-1)},x_{t:t-(l-1)})p\left(y_{t:t-(k-1)}\right)}{p(y_{t:t-(k-1)},x_{t:t-(l-1)})p(y_{t+1},y_{t:t-(k-1)})},\hfill \end{array}$$

which quantifies the amount of reduced uncertainty in future value $Y_{t+1}$ by knowing the past l values of $X_t$ given past k values of $Y_t$, where $Y_{t:t-(k-1)}$ and $X_{t:t-(l-1)}$ are shorthands of past k values $Y_t,Y_{t-1},\dots ,Y_{t-(k-1)}$ and past l values $X_t,X_{t-1},\dots ,X_{t-(l-1)}$, respectively.

In order to properly compare with causal information rate signifying how one signal instantaneously influences another signal’s information rate (at the same/equal-time t), we set $k=l=1$ in calculating the transfer entropy between two signals. Also, since the causal information rate involves partial time derivatives, which have to be numerically estimated using temporally adjacent PDFs separated by $\mathsf{\Delta }t={10}^{-4}$ in time (as mentioned at the end of Section 2.2), the discrete-time $t\in \mathbb{Z}$ in transfer entropy should be changed to $n\mathsf{\Delta }t$ with $n\in \mathbb{Z}$. Therefore, the transfer entropy appropriate for comparing with the causal information rate should be rewritten as follows:

$$\begin{array}{c}\mathrm{(12)}& {\mathrm{TE}}_{x_1\to x_2}\left(t\right)=H\left(x_2(t+\mathsf{\Delta }t)\mid x_2\left(t\right)\right)-H\left(x_2(t+\mathsf{\Delta }t)\mid x_2\left(t\right),x_1\left(t\right)\right),\hfill ={\displaystyle \sum _{x_1,x_2}}p(x_2,t+\mathsf{\Delta }t;x_2,t;x_1,t){log}_2\frac{p(x_2,t+\mathsf{\Delta }t|x_2,t;x_1,t)}{p(x_2,t+\mathsf{\Delta }t|x_2,t)},\hfill \\ \mathrm{(13)}& ={\displaystyle \sum _{x_1,x_2}}p(x_2,t+\mathsf{\Delta }t;x_2,t;x_1,t){log}_2\frac{p(x_2,t+\mathsf{\Delta }t;x_2,t;x_1,t)p(x_2,t)}{p(x_2,t;x_1,t)p(x_2,t+\mathsf{\Delta }t;x_2,t)}.\hfill \end{array}$$

Numerical estimations of the information rate, causal information rate, differential entropy, and transfer entropy are all based on numerical estimation of PDFs using histograms. In particular, in order to sensibly and consistently estimate the causal information rate (e.g., to avoid getting negative values), special caution is required when choosing the binning for histogram estimation of PDFs in calculating ${\displaystyle {\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2=4\int d{x}_2{\left[{\partial }_t\sqrt{p(x_2,t)}\right]}^2}$ and ${\displaystyle {\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2=4\underset{t_{*}\to t^{+}}{lim}\int d{x}_{1}d{x}_2{\left[{\partial }_{t_{*}}\sqrt{p(x_2,t_{*};x_1,t)}\right]}^2}$. The finer details for these numerical estimation techniques are elaborated in Appendix A.

3. Results

We performed simulations with six different Gaussian initial distributions (with different means and standard deviations summarized in Table 3). Initial Conditions No.1 (IC No.1, or simply IC1) through No.3 (IC3) are Gaussian distributions with a narrow width or smaller standard deviation, whereas IC4 through IC6 have a larger width/standard deviation, and therefore, the simulation results of IC4 through IC6 exhibit more diverse time evolution behaviors (e.g., more complex attractors, as explained next), and hence, the corresponding calculation results are more robust or insensitive to the specific mean values ${\mu }_{x_i\left(0\right)}$’s of the initial Gaussian distributions (see Table 3 for more details). Therefore, in the main text here, we focus on these results from initial Gaussian distributions with wider width/larger standard deviation, and we list complete results from all six initial Gaussian distributions in the Appendix B. Specifically, we found that the results from IC4 through IC6 are qualitatively the same or very similar, and therefore, in the main text here, we illustrate and discuss the results from Initial Conditions No.4 (IC4), which is sufficiently representative for IC5 and IC6, and refer to other IC’s (by referencing the relevant sections in Appendix B or explicitly illustrating the results) if needed.

3.1. Sample Trajectories of $X_1\left(T\right)$ and $X_2\left(T\right)$

To give a basic idea of how the simulated trajectories evolve in time, we start by illustrating 50 sample trajectories of $x_1\left(t\right)$ and $x_2\left(t\right)$ from the total $2\times {10}^7$ simulated trajectories for both CTL subjects and AD patients with both EC and EO conditions, which are visualized in Figure 1 and Figure 2. Notice that from Figure 1c, one can see that it takes some time for the trajectories of $x_2\left(t\right)$ to settle down on some complex attractors for EC, which suggests a longer memory associated with EC of CTL. This is more evident as shown in the time evolution of PDF $p(x_2,t)$ in Figure 3c below.

Figure 1.

Fifty Sample trajectories of healthy CTL subjects. Each single trajectory is labeled by a different color.

Figure 2.

Fifty Sample trajectories of AD patients. Each single trajectory is labeled by a different color.

Figure 3.

Time evolution of estimated PDFs of healthy CTL subjects.

3.2. Time Evolution of PDF $P(X_1,T)$ and $P(X_2,T)$

The empirical PDFs $p(x_1,t)$ and $p(x_2,t)$ can better illustrate the overall time evolution of a large number of trajectories, and they serve as a basis for calculations of information geometry theoretic measures such as information rates and causal information rates. These empirical PDFs are estimated using a histogram-based approach with Rice’s rule [35,36], where the number of bins is $n_{\mathrm{bins}}=2\sqrt[3]{n_{\mathrm{samples}}}$, and since we simulated $2\times {10}^7$ sample trajectories in total, the $n_{\mathrm{bins}}$ is rounded to 542. The centers of bins are plotted on the y-axis in sub-figures of Figure 3 and Figure 4, where the function values of $p(x_1,t)$ and $p(x_2,t)$ are color-coded following the color bars.

Figure 4.

Time evolution of estimated PDFs of AD patients.

As mentioned in the previous section, from Figure 3c, one can see more clearly that after around $t\ge 5$, the trajectories settle down on some complex attractors, and the time evolution of $p(x_2,t)$ undergoes only minor changes. Meanwhile, from Figure 3a, one can observe that a similar settling down of $x_1\left(t\right)$ on some complex attractors happens after around $t\ge 7.5$. Therefore, we select only PDFs with $t\ge 7.5$ for statistical analysis of information rates and causal information rates to investigate the stationary properties.

From Figure 3 and Figure 4, one can already observe some qualitative differences between healthy control (CTL) subjects and AD patients. For example, the time evolution patterns of $p(x_1,t)$ and $p(x_2,t)$ are significantly different when CTL subjects open their eyes from eyes-closed (EC) state, whereas for AD patients, these differences are relatively minor. One of the best ways to provide quantitative descriptions of these differences (instead of being limited to qualitative descriptions) is using information geometry theoretic measures such as information rates and causal information rates, whose results are listed in Section 3.3 and Section 3.4, respectively.

As can be seen from Appendix B.2, IC1 through IC3 exhibit much simpler attractors than IC4 through IC6. Since the width/standard deviation of the initial Gaussian distributions of IC1 through IC3 is much smaller, they are more sensitive to the specific mean values ${\mu }_{x_i\left(0\right)}$’s of the initial Gaussian distribution, and one can see that IC3’s time evolution behaviors of $p(x_i,t)$ are somewhat qualitatively different from IC1 and IC2, whereas $p(x_i,t)$’s time evolution behaviors of IC4 through IC6 are all qualitatively the same.

3.3. Information Rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$

Intuitively speaking, the information rate is the (instantaneous) speed of PDF’s motion on the statistical manifold, as each given PDF corresponds to a point on that manifold, and when the time changes, a time-dependent PDF will typically move on a curve on the statistical manifold, whereas a stationary or equilibrium state PDF will remain at the same point on the manifold. Therefore, the information rate is a natural tool to investigate the time evolution of PDF.

Moreover, since the information rate is quantifying instantaneous rate of change in the infinitesimal relative entropy between two adjacent PDFs, it is hypothetically a reflection of neural information processing in the brain, and hence, it may provide important insight into the neural activities in different regions of the brain, as long as the regional EEG signals can be sufficiently collected for calculating the information rates.

3.3.1. Time Evolution

The time evolution of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ are shown in Figure 5a,b for CTL subjects and AD patients, respectively. Since ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ quantify the (infinitesimal) relative entropy production rate instantaneously at time t, they represent the information-theoretic complexities of signals $x_1\left(t\right)$ and $x_2\left(t\right)$ of the coupled oscillators, respectively, and are hypothetical reflections of neural information processing in the corresponding regions in the brain.

Figure 5.

Information rates along time of CTL and AD subjects.

For example, in Figure 5a, there is a clear distinction between eyes-closed (EC) and eyes-open (EO) for CTL subjects: both ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ decrease significantly when healthy subjects open their eyes, which may be interpreted as the neural information processing activities of the corresponding brain regions being “suppressed” by the incoming visual information when eyes are opened from being closed.

Interestingly, when AD patients open their eyes, both ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ are increasing instead of decreasing, as shown in Figure 5b. This might be interpreted as that the incoming visual information received when eyes are opened is in fact “stimulating” the neural information processing activities of the corresponding brain regions, which might be impaired or damaged by the relevant mechanism of Alzheimer’s disease (AD).

In Figure 5a,b, we annotate the mean and standard deviation for ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ after $t\ge 7.5$ in the legend, because as mentioned above, the PDFs of this time range reflect longer-term temporal characteristics, and hence, the corresponding ${\mathsf{\Gamma }}_{x_1}(t\ge 7.5)$ and ${\mathsf{\Gamma }}_{x_2}(t\ge 7.5)$ should reflect more reliable and robust features of neural information processing activities of the corresponding brain regions. Therefore, meaningful statistics will require collecting samples of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ in this time range, for which the results are shown in the section below.

3.3.2. Empirical Probability Distribution (for $T\ge 7.5$)

The statistics of ${\mathsf{\Gamma }}_{x_1}(t\ge 7.5)$ and ${\mathsf{\Gamma }}_{x_2}(t\ge 7.5)$ can be further and better visualized using empirical probability distributions of them, as shown in Figure 6. Again, we use histogram-based density estimation with Rice’s rule, and since the time interval $\mathsf{\Delta }t$ for estimating PDFs and computing ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ is ${10}^{-4}$ (whereas the time-step size $dt$ for simulating the SDE model is ${10}^{-6}$), we collected 24,999 samples of both ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ for $7.5\le t<10$, and hence, the number of bins following Rice’s rule is rounded to 58. Figure 6 confirms the observation in the previous section, while it also better visualizes the sample standard deviation in the shapes of the estimated PDFs, indicating that the PDFs of both ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ are narrowed down when healthy subjects open their eyes but are widened when AD patients do so.

Figure 6.

Empirical probability distributions of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$.

As a comparison, we also calculate more traditional/established information-theoretic measure, namely, the Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$, and estimate their empirical probability distributions in the same manner as we do for information rates, as shown in Figure 7.

Figure 7.

Empirical probability distributions of differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$.

One can see that the empirical distributions of differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$ are not able to make clear distinction between EC and EO conditions, especially for AD patients. This may be better summarized in Table 4, comparing the mean and standard deviation values of information rate vs. differential entropy for the four cases. Therefore, the information rate is a superior measure for quantifying the non-stationary time-varying dynamical changes in EEG signals when switching between EC and EO states and is a better and more reliable reflection of neural information processing in the brain.

Table 4.

Mean and standard deviation values ($\mu \pm \sigma$) of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ & ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ vs. differential entropy $h\left(x_1\left(t\right)\right)$ & $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$.

	CTL EC	CTL EO	AD EC	AD EO
${\mathsf{\Gamma }}_{x_1}\left(t\right)$	744.48 ± 165.91	172.80 ± 22.59	147.95 ± 18.72	451.10 ± 108.99
${\mathsf{\Gamma }}_{x_2}\left(t\right)$	620.95 ± 148.37	179.85 ± 18.51	113.74 ± 27.85	217.84 ± 89.11
$h\left(x_1\left(t\right)\right)$	0.59 ± 0.57	1.05 ± 0.22	1.11 ± 0.19	0.73 ± 0.34
$h\left(x_2\left(t\right)\right)$	−0.05 ± 0.46	0.78 ± 0.20	1.11 ± 0.21	0.93 ± 0.17

3.3.3. Phase Portraits (for $T\ge 7.5$)

In addition to empirical statistics of information rates for $t\ge 7.5$ in terms of estimated probability distributions, one can also visualize the temporal dynamical features of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ combined using phase portraits, as shown in Figure 8. Notice that when healthy subjects open their eyes, the fluctuation ranges of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ shrink by roughly 5-fold, whereas when AD patients open their eyes, the fluctuation ranges are enlarged.

Figure 8.

Phase portraits of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ vs. ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$.

Moreover, when plotting EC and EO of healthy subjects separately in Figure 9a to zoom into the ranges of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ for EO, one can also see that the phase portrait of EO exhibits a fractal-like pattern, whereas the phase portrait of EC exhibits more regular dynamical features, including an overall trend of fluctuating between bottom left and top right, indicating that the ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ are somewhat synchronized, which could be explained by the strong coupling coefficients in Table 1 of healthy subjects. Contrarily, for AD patients, the phase portraits of EC and EO both exhibit fractal-like patterns in Figure 9b.

Figure 9.

Phase portraits of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ vs. ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Same as at the end of Section 3.3.2, as a comparison, we also visualize the phase portraits of Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$ in Figure A52d and Figure A56 in Appendix B.4.3, where one can see that it is hard to distinguish the phase portraits of $h\left(x_1\left(t\right)\right)$ vs. $h\left(x_2\left(t\right)\right)$ of AD EC from those of AD EO, as they are qualitatively the same or very similar. Contrarily, in Figure 8, the fluctuation ranges of phase portraits of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ vs. ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ are significantly enlarged when AD patients open their eyes. Therefore, this reconfirms our claim at the end of Section 3.3.2 that the information rate is a superior measure than differential entropy in quantifying the dynamical changes in EEG signals when switching between EO and EO states and is a better and more reliable reflection of neural information processing in the brain.

3.3.4. Power Spectra (for $T\ge 7.5$)

Another perspective to visualize the dynamical characteristics of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ is by using power spectra, i.e., the absolute values of (fast) Fourier transforms of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$, as shown in Figure 10. Frequency-based analyses will not make much sense if the signals or time series of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ have non-stationary time-varying effects, and this is why we only consider time range $t\ge 7.5$ for ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$, when the time evolution patterns of $p(x_1,t)$ and $p(x_2,t)$ almost stop changing as shown in Figure 3 (and especially in Figure 3a,c).

Figure 10.

Power spectra of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$.

The power spectra of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ also exhibit a clear distinction between EC and EO for CTL and AD subjects. Specifically, the power spectra of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ can be fit by power law for frequencies between ∼100 Hz to ∼1000 Hz (the typical sampling frequency of experimental EEG signals is 1000 Hz, whereas most of brain wave’s/neural oscillations’ frequencies are below 100 Hz). From Figure 11a, one can see that power law fit exponents (quantifying how fast the power density decreases with increasing frequency) of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$’s and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$’s power spectra are largely reduced when healthy subjects open their eyes, which indicates that the strength of noise in ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ decreases significantly when switching from EC to EO. Contrarily, for AD patients as shown in Figure 11b, the power law fit exponents of ${\mathsf{\Gamma }}_{x_1}\left(t\right)$’s and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$’s power spectra increase significantly and slightly, respectively, indicating that the strength of noise in ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ increases when switching from EC to EO.

Figure 11.

Power law fit for power spectra of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

3.4. Causal Information Rates ${\mathsf{\Gamma }}_{x_2\to X_1}\left(T\right),{\mathsf{\Gamma }}_{x_1\to X_2}\left(T\right)$, and Net Causal Information Rates ${\mathsf{\Gamma }}_{x_2\to X_1}\left(T\right)-{\mathsf{\Gamma }}_{x_1\to X_2}\left(T\right)$

The notion of causal information rate was introduced in Ref. [25], which quantifies how one signal instantaneously influences another signal’s information rate. A comparable measure of causality is transfer entropy; however, as shown in Appendix B.6, our calculation results of transfer entropy are too spiky/noisy to reliably quantify causality, and hence, the results are only included in Appendix as a comparison, which we will discuss at the end of this section. Nevertheless, similar to net transfer entropy, one can calculate the net causal information rate, e.g., ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$, signifying the net causality measure from signal $x_2\left(t\right)$ to $x_1\left(t\right)$. Since ${\dot{x}}_2\left(t\right)$ is the only variable that is directly affected by random noise in the stochastic oscillator model Equation (2), we calculate ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ for the net causal information rate of the coupled oscillator’s signal $x_2\left(t\right)$ influencing $x_1\left(t\right)$.

Notice that for stochastic coupled oscillator model Equation (2), causal information rates ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ will reflect how strongly the two oscillators are directionally coupled or causally related. Since signals $x_1\left(t\right)$ and $x_2\left(t\right)$ are the results of neural activities in the corresponding brain regions, the causal information rates can be used to measure connectivities among different regions of the brain.

3.4.1. Time Evolution

Similar to Section 3.3.1, we also visualize the time evolution of causal information rates ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ in Figure 12a,b for CTL subjects and AD patients, respectively.

Figure 12.

Causal information rates along time of CTL and AD subjects.

For both CTL and AD subjects, ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ both decrease when changing from EC to EO, except for AD subjects’ ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)$ increasing on average. On the other hand, the net causal information rate ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ changes differently: when healthy subjects open their eyes, it increases and changes from significantly negative on average to slightly positive on average, whereas for AD patients, it increases from almost zero on average to significantly positive on average without net directional change. A possible interpretation might be that, when healthy subjects open their eyes, the brain region generating the signal $x_2\left(t\right)$ becomes more sensitive to the noise, causing it to influence $x_1\left(t\right)$ more compared to the eyes-closed state.

3.4.2. Empirical Probability Distribution (for $T\ge 7.5$)

Similar to Section 3.3.2, we also estimate the empirical probability distributions of ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)$, ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ to better visualize their statistics in Figure 13a. In particular, we plot the empirical probability distributions of ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ for both healthy and AD subjects with both EC and EO conditions together in Figure 13b, in order to better visualize and compare net causal information rates’ changes when CTL and AD subjects open their eyes. It can be seen that the estimated PDF of ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ shrinks its width in shape when healthy subjects open their eyes. Combining with the observation that the magnitude of sample mean of ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ is close to 0 for healthy subjects with eyes opened, a possible interpretation might be that the directional connectivity between brain regions generating signals $x_1\left(t\right)$ and $x_2\left(t\right)$ is reduced to almost zero, either by incoming visual information received by opened eyes or due to the brain region generating signal $x_2\left(t\right)$ becoming more sensitive to noise when eyes are opened. Contrarily, the estimated PDF of ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ for AD patients qualitatively change in an inverse direction to become widened in shape.

Figure 13.

Empirical probability distributions of causal information rates and net causal information rates $(t\ge 7.5)$.

As mentioned earlier, as a comparison, we also calculate more traditional/established information-theoretic measure of causality, i.e., transfer entropy (TE), and estimate their empirical probability distributions in the same manner as we do for causal information rates, as shown in Figure 14.

Figure 14.

Empirical probability distributions of transfer entropy and net transfer entropy $(t\ge 7.5)$.

One can see that the empirical distributions of transfer entropy ${\mathrm{TE}}_{x_2\to x_1}\left(t\right)$ and ${\mathrm{TE}}_{x_1\to x_2}\left(t\right)$, as well as net transfer entropy ${\mathrm{TE}}_{x_2\to x_1}\left(t\right)-{\mathrm{TE}}_{x_1\to x_2}\left(t\right)$ are not able to make clear distinction between EC and EO conditions, especially for AD patients in terms of net transfer entropy. This may be better summarized in Table 5, comparing the mean and standard deviation values of causal information rate vs. transfer entropy for the four cases.

Table 5.

Mean and standard deviation values ($\mu \pm \sigma$) of causal information rates ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)$, ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ and net causal information rates ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$ vs. transfer entropy ${\mathrm{TE}}_{x_2\to x_1}\left(t\right)$, ${\mathrm{TE}}_{x_1\to x_2}\left(t\right)$ and net transfer entropy ${\mathrm{TE}}_{x_2\to x_1}\left(t\right)-{\mathrm{TE}}_{x_1\to x_2}\left(t\right)$$(t\ge 7.5)$.

	CTL EC	CTL EO	AD EC	AD EO
${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)$	545.40 ± 227.23	494.11 ± 114.95	125.38 ± 52.89	201.58 ± 92.02
${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$	626.65 ± 243.03	489.87 ± 103.08	125.06 ± 52.25	109.07 ± 63.88
${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$	−81.25 ± 84.15	4.24 ± 31.96	0.32 ± 25.96	92.51 ± 80.23
${\mathrm{TE}}_{x_2\to x_1}\left(t\right)$	0.011 ± 0.012	0.015 ± 0.0068	0.0067 ± 0.0043	0.0086 ± 0.0062
${\mathrm{TE}}_{x_1\to x_2}\left(t\right)$	0.013 ± 0.013	0.011 ± 0.0036	0.004 ± 0.0019	0.0046 ± 0.0028
${\mathrm{TE}}_{x_2\to x_1}\left(t\right)-{\mathrm{TE}}_{x_1\to x_2}\left(t\right)$	−0.0018 ± 0.015	0.0038 ± 0.0062	0.0027 ± 0.0049	0.004 ± 0.0068

Moreover, the magnitude of numeric values of transfer entropy and net transfer entropy is ∼${10}^{-2}$ or ∼${10}^{-3}$, which is too close to zero, making it too noise-like or unreliable to quantify causality. Therefore, the causal information rate is a much superior measure than transfer entropy in quantifying causality, and since causal information rate quantifies how one signal instantaneously influences another signal’s information rate (which is a reflection of neural information processing in corresponding brain region), it can be used to measure directional or causal connectivities among different brain regions.

4. Discussion

A major challenge for practical usage of information geometry theoretic measures on real-world experimental EEG signals is that they require a significant amount of data samples to estimate the probability density functions. For example, in this work, we simulated $2\times {10}^7$ trajectories or sample paths of the stochastic nonlinear coupled oscillator models, such that at any time instance, we always have a sufficient amount of data samples to accurately estimate the time-dependent probability density functions with a histogram-based approach. This is usually not possible for experimental EEG signals which often contain only one trajectory for each channel, and one has to use a sliding window-based approach to collect data samples for histogram-based density estimation. This approach implicitly assumes that the EEG signals are stationary within each sliding time window, and hence, one has to balance between the sliding time window’s length and number of available data samples, in order to account for non-stationarity while still having enough data samples to accurately and meaningfully estimate the time-dependent probability densities. And therefore, this approach will not work very well if the EEG signals exhibit severely non-stationary time-varying effects, requiring a very short length of sliding windows, which will contain too few data samples.

An alternative approach to overcome this issue is using kernel density estimation to estimate the probability density functions, which usually requires a much smaller number of data samples while still being able to approximate the true probability distribution with acceptable accuracy. However, this approach typically involves a very high computational cost, limiting its practical use for many cases such as computational resource-limited scenarios. A proposed method to avoid this is using the Koopman operator theoretic framework [37,38] and its numerical techniques applicable to experimental data in a model-free manner, since the Koopman operator is the left-adjoint of the Perron–Frobenious operator evolving the probability density functions in time. This exploration will be left for our future investigation.

5. Conclusions

In this work, we explore information geometry theoretic measures to characterize neural information processing from EEG signals simulated by stochastic nonlinear coupled oscillator models. In particular, we utilize information rates to quantify the time evolution of probability density functions of simulated EEG signals and utilize causal information rates to quantify one signal’s instantaneous influence on another signal’s information rate. The parameters of the stochastic nonlinear coupled oscillator models of EEG were fine tuned for both healthy subjects and AD patients, with both eyes-closed and eyes-open conditions. By using information rates and causal information rates, we find significant and interesting distinctions between healthy subjects and AD patients when they change their eyes’ open/closed status. These distinctions may be further related to differences in neural information processing activities of the corresponding brain regions (for information rates) and to differences in connectivities among these brain regions (for causal information rates).

Compared to more traditional or established information-theoretic measures such as differential entropy and transfer entropy, our results show that information geometry theoretic measures such as information rate and causal information rate are superior to their more traditional counterparts, respectively (information rate vs. differential entropy, and causal information rate vs. transfer entropy). Since information rates and causal information rates can be applied to experimental EEG signals in a model-free manner, and they are capable of quantifying non-stationary time-varying effects, nonlinearity, and non-Gaussian stochasticity presented in real-world EEG signals, we believe that these information geometry theoretic measures can become an important and powerful tool-set for both understanding neural information processing in the brain and diagnosis of neurological disorders such as Alzheimer’s disease in this work.

Abbreviations

The following abbreviations are used in this manuscript:

PDF	probability density function
SDE	stochastic differential equation
IC	Initial Conditions (in terms of initial Gaussian distributions)
CTL	healthy control (subjects)
AD	Alzheimer’s disease
EC	eyes-closed
EO	eyes-open
TE	transfer entropy

Appendix A. Finer Details of Numerical Estimation Techniques

Recall from Equation (7) that the squared information rate is

$${\mathsf{\Gamma }}_x{\left(t\right)}^2=\int dxp(x,t){\left[{\partial }_{t}lnp(x,t)\right]}^2=4\int dx{\left[{\partial }_t\sqrt{p(x,t)}\right]}^2,$$ (A1)

where the partial time derivative and integral can be numerically approximated using discretization, i.e., ${\partial }_t\sqrt{p(x,t)}\approx \frac{1}{\left(\mathsf{\Delta }t\right)}(\sqrt{p(x,t+\mathsf{\Delta }t)}-\sqrt{p(x,t)})$ and ${\displaystyle \int dxf\left(x\right)\approx \sum _i\mathsf{\Delta }{x}_{i}f\left(x_i\right)}$, respectively, where for brevity and if no ambiguity arises, the summation over index i is often omitted and replaced by x itself as ${\displaystyle \sum _x\mathsf{\Delta }xf\left(x\right)}$, and the symbol x serves as both the index of summation (e.g., the x-th interval with length $\mathsf{\Delta }x$) and the actual value x in $f\left(x\right)$.

A common technique to improve the approximation of integral by finite summation is the trapezoidal rule ${\displaystyle \int dxf\left(x\right)\approx \sum _i\mathsf{\Delta }{x}_i\frac{f\left(x_{i-1}\right)+f\left(x_i\right)}{2}}$, which will be abbreviated as ${\sum }_x^{\mathrm{Trapz}.}\mathsf{\Delta }xf\left(x\right)$ to indicate that the summation is following the trapezoidal rule imposing a $1/2$ weight/factor on the first and last summation terms (corresponding to the lower and upper bounds of the integral). Similarly, we use ${\sum }_{x,y}^{\mathrm{Trapz}.2\mathrm{D}}\mathsf{\Delta }x\mathsf{\Delta }yf(x,y)$ to denote a 2D trapezoidal approximation of the double integral ${\displaystyle \int dxdyf(x,y)}$, where different weights ($1/4$ or $1/2$) will be applied to the “corner”/boundary terms of the summation. Meanwhile, to distinguish regular summation from trapezoidal approximation, we use the notation ${\sum }_x^{\mathrm{naive}}\mathsf{\Delta }xf\left(x\right)$ to signify a regular summation as a more naive approximation of the integral.

The PDF $p(x,t)$ is numerically estimated using a histogram with Rice’s rule applied, i.e., the number of bins is $n_{\mathrm{bins}}=⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋$ (with uniform bin width $\mathsf{\Delta }x=\frac{\mathrm{range}\mathrm{of}\mathrm{samples}’\mathrm{values}}{n_{\mathrm{bins}}}$), which is rounded towards zero to avoid overestimating the number of bins needed. And for joint PDF $p(x_i,t_i;x_j,t_j)$, since the bins are distributed in a 2D plane of $(x_i,x_j)$, the number of bins in each dimension is rounded to $⌊\sqrt{⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋}⌋$ (and similarly for 3D joint probability as in the transfer entropy calculation, the number of bins in each dimension is rounded to $⌊\sqrt[3]{⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋}⌋$). Combining all of the above, the information rate’s square will be approximated by

$${\mathsf{\Gamma }}_x{\left(t\right)}^2=4\int dx{\left[{\partial }_t\sqrt{p(x,t)}\right]}^2\approx 4\sum _x^{\mathrm{Trapz}.}\frac{\mathsf{\Delta }x}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x,t+\mathsf{\Delta }t)}-\sqrt{p(x,t)}\right]}^2,$$ (A2)

where the bin width $\mathsf{\Delta }x$ can be moved into the square root and multiplied with the PDF to get the probability (mass) of finding a data sample in the x-th bin, which is estimated as $\frac{n_{\mathrm{sample}}\mathrm{inside}x-\mathrm{th}\mathrm{bin}}{n_{\mathrm{sample}}}$, i.e., the number of data samples inside that bin divided by the number of all data samples (using the relevant functions in MATLAB or Python). The trapezoidal rule imposes a $1/2$ factor on the first and last terms of summation, corresponding to the first and last bins.

For the causal information rate ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)\stackrel{\mathrm{def}}{=}{\mathsf{\Gamma }}_{x_2}^{*}\left(t\right)-{\mathsf{\Gamma }}_{x_2}\left(t\right)$, the ${\mathsf{\Gamma }}_{x_2}^{*}\left(t\right)$ can be estimated by

$$\begin{array}{cc}\hfill {\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2& =\underset{t_{*}\to t^{+}}{lim}\int d{x}_{1}d{x}_{2}p(x_2,t_{*};x_1,t){\left[{\partial }_{t_{*}}lnp(x_2,t_{*};x_1,t)\right]}^2\hfill \\ \hfill & =4\underset{t_{*}\to t^{+}}{lim}\int d{x}_{1}d{x}_2{\left[{\partial }_{t_{*}}\sqrt{p(x_2,t_{*};x_1,t)}\right]}^2\hfill \\ \hfill & \approx 4\sum _{x_2,x_1}\frac{\mathsf{\Delta }{x}_2\mathsf{\Delta }{x}_1}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t;x_1,t)}-\sqrt{p(x_2,t;x_1,t)}\right]}^2,\hfill \end{array}$$ (A3)

where the number of bins in each of the $x_1$ and $x_2$ dimensions is rounded to $⌊\sqrt{⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋}⌋$, and the ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$ can be estimated as above as ${\displaystyle 4\int d{x}_2{\left[{\partial }_t\sqrt{p(x_2,t)}\right]}^2\approx 4\sum _{x_2}\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2}$ using regular or trapezoidal summation. However, here for ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$, the number of bins for $x_2$ must not be chosen as $⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋$ following the 1D Rice’s rule, which is very critical to avoid insensible or inconsistent estimation of ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$, for which the reason is explained below.

Consider the quantity ${\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2-{\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$; theoretically and by definition, the $d{x}_2$ can be pulled outside the integral over $d{x}_1$ to combine the two integrals into one integral as follows:

$$\begin{array}{c}\hfill {\displaystyle {\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2-{\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2=4\underset{t_{*}\to t^{+}}{lim}\int d{x}_2\left\{\int {\left[{\partial }_{t_{*}}\sqrt{p(x_2,t_{*};x_1,t)}\right]}^{2}d{x}_1-{\left[{\partial }_t\sqrt{p(x_2,t)}\right]}^2\right\},}\end{array}$$ (A4)

and the corresponding numerical approximations of integrals should be combined as

$$\begin{array}{cc}\hfill & \approx 4\sum _{x_2}\mathsf{\Delta }{x}_2\left\{\sum _{x_1}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t;x_1,t)}-\sqrt{p(x_2,t;x_1,t)}\right]}^2\frac{\mathsf{\Delta }{x}_1}{{\left[\mathsf{\Delta }t\right]}^2}\right.\hfill \\ \hfill & \left.-\frac{{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2}{{\left[\mathsf{\Delta }t\right]}^2}\right\},\hfill \end{array}$$ (A5)

where the sum over $x_2$ is performed on the same bins for both of the two terms inside the large braces $\left\{·\right\}$ above. On the other hand, if one numerically approximates ${\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2$ and ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$ separately as

$$\begin{array}{cc}\hfill {\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2-{\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2& \approx 4\sum _{x_2,x_1}\frac{\mathsf{\Delta }{x}_2\mathsf{\Delta }{x}_1}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t;x_1,t)}-\sqrt{p(x_2,t;x_1,t)}\right]}^2\hfill \\ \hfill & -4\sum _{x_2}\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2,\hfill \end{array}$$ (A6)

then the sum over $x_2$ in the second term $4{\sum }_{x_2}\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2$ should still be performed on the same bins of $x_2$ for the first term involving the joint PDFs estimated by 2D histograms (i.e., using the square root number of bins $⌊\sqrt{⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋}⌋$ of Rice’s rule, instead of following the 1D Rice’s rule without the square root), even though this second summation term is written as a separate and “independent” term from the first double-summation term. The definition ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)\stackrel{\mathrm{def}}{=}{\mathsf{\Gamma }}_{x_2}^{*}\left(t\right)-{\mathsf{\Gamma }}_{x_2}\left(t\right)$ might result in a misimpression that one can estimate ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2\approx 4{\sum }_{x_2}\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2$ separately by using a Rice’s rule’s binning method containing $⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋$ bins, while estimating ${\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2\approx 4{\sum }_{x_2,x_1}\frac{\mathsf{\Delta }{x}_2\mathsf{\Delta }{x}_1}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t;x_1,t)}-\sqrt{p(x_2,t;x_1,t)}\right]}^2$ using the square root of Rice’s rule’s number of bins $⌊\sqrt{⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋}⌋$. Using different bins for $x_2$ will make it invalid to combine the two summations into one summation over the same $x_2$’s (and hence invalid to combine the two integrals into one integral by pulling out the same $d{x}_2$).

Using $⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋$ bins for $x_2$ will overestimate the value of ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2\approx 4{\sum }_{x_2}\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2$, for example, if there are 1 million samples/data points to estimate the PDFs, then $⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋=200$ for 1D distribution and $⌊\sqrt{⌊2\sqrt[3]{n_{\mathrm{sample}}}⌋}⌋\approx 14$ for 2D joint distribution. Calculating ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2\approx 4{\sum }_{x_2}\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2$ using 200 bins will result in a much larger value than calculating it using 14 bins, which will result in negative values in calculating the causal information rate ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)={\mathsf{\Gamma }}_{x_2}^{*}\left(t\right)-{\mathsf{\Gamma }}_{x_2}\left(t\right)$. When using the same 14 bins of $x_2$ (for estimating the 2D joint PDF of $(x_1,x_2)$) to estimate the 1D PDF in ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$$\approx 4{\sum }_{x_2}\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2$, all the unreasonable negative values disappear, except for only some isolated negative values remained, which is related to estimating ${\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2$ and ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$ using 1D and 2D trapezoidal rules for summations approximating the integrals: if one uses 1D trapezoidal summation for ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$$\approx 4{\sum }_{x_2}^{\mathrm{Trapz}}.$$\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2$, while on the other hand, one blindly and inconsistently uses 2D trapezoidal summation for ${\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2\approx 4{\sum }_{x_2,x_1}^{\mathrm{Trapz}}.2\mathrm{D}$$\frac{\mathsf{\Delta }{x}_2\mathsf{\Delta }{x}_1}{{\left[\mathsf{\Delta }t\right]}^2}[\sqrt{p(x_2,t+\mathsf{\Delta }t;x_1,t)}$$-\sqrt{p(x_2,t;x_1,t)}{]}^2$, this will also result in some negative values in computing ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$$={\mathsf{\Gamma }}_{x_2}^{*}\left(t\right)-{\mathsf{\Gamma }}_{x_2}\left(t\right)$, because the 2D trapezoidal sum will under-estimate the ${\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2$ as compared to the 1D trapezoidal-sum-estimated ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$.

To resolve this inconsistent mixing of 1D and 2D trapezoidal rules, there are two possible methods:

1. Using 2D trapezoidal rule for both ${\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2$ and ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$, that is, ${\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2\approx 4{\sum }_{x_2,x_1}^{\mathrm{Trapz}.2\mathrm{D}}$$\frac{\mathsf{\Delta }{x}_2\mathsf{\Delta }{x}_1}{{\left[\mathsf{\Delta }t\right]}^2}[\sqrt{p(x_2,t+\mathsf{\Delta }t;x_1,t)}-$$\sqrt{p(x_2,t;x_1,t)}{]}^2$, and ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2\approx 4{\sum }_{x_2}^{\mathrm{Trapz}.}$$\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2\approx 4{\sum }_{x_2}^{\mathrm{Trapz}.}\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{[\sqrt{{\sum }_{x_1}^{\mathrm{Trapz}.}\mathsf{\Delta }{x}_{1}p(x_2,t+\mathsf{\Delta }t;x_1,t)}-\sqrt{{\sum }_{x_1}^{\mathrm{Trapz}.}\mathsf{\Delta }{x}_{1}p(x_2,t;x_1,t)}]}^2$. In other words, when calculating ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$, instead of estimating marginal PDF $p(x_2,t+\mathsf{\Delta }t)$ and $p(x_2,t)$ directly by 1D histograms (using the relevant functions in MATLAB or Python), one first estimates the joint PDF $p(x_2,t+\mathsf{\Delta }t;x_1,t)$ and $p(x_2,t;x_1,t)$ by 2D histograms and integrates over $x_1$ by trapezoidal summation on it. This will reduce the value of estimated ${\mathsf{\Gamma }}_{x_2}\left(t\right)$, and integrals over both $x_1$ and $x_2$ are both estimated by trapezoidal summation.

2. Using the 1D trapezoidal rule for both ${\mathsf{\Gamma }}_{x_2}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}^{*}\left(t\right)$, that is, ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2\approx 4{\sum }_{x_2}^{\mathrm{Trapz}.}$$\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t)}-\sqrt{p(x_2,t)}\right]}^2=4{\sum }_{x_2}^{\mathrm{Trapz}.}\frac{\mathsf{\Delta }{x}_2}{{\left[\mathsf{\Delta }t\right]}^2}[\sqrt{{\sum }_{x_1}^{\mathrm{naive}}\mathsf{\Delta }{x}_1·p(x_2,t+\mathsf{\Delta }t;x_1,t)}$$-\sqrt{{\sum }_{x_1}^{\mathrm{naive}}\mathsf{\Delta }{x}_1·p(x_2,t;x_1,t)}{]}^2$, and ${\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2\approx \frac{4}{{\left[\mathsf{\Delta }t\right]}^2}{\sum }_{x_2}^{\mathrm{Trapz}.}\mathsf{\Delta }{x}_2\left\{{\sum }_{x_1}^{\mathrm{naive}}\mathsf{\Delta }{x}_1{\left[\sqrt{p(x_2,t+\mathsf{\Delta }t;x_1,t)}-\sqrt{p(x_2,t;x_1,t)}\right]}^2\right\}$. In this approach, the marginal PDF ${\displaystyle p(x_2,t)=\int p(x_2,t;x_1,t)d{x}_1}$, where the equal sign holds exactly for the regular or naive summation $p(x_2,t)={\sum }_{x_1}^{\mathrm{naive}}\mathsf{\Delta }{x}_1·p(x_2,t;x_1,t)$. This is because the histogram estimation in MATLAB and Python is performed by counting the occurrence of data samples inside each bin, and the probability (mass) is estimated as $\frac{n_{\mathrm{sample}}\mathrm{inside}x-\mathrm{th}\mathrm{bin}}{n_{\mathrm{sample}}}$, and the density is estimated as $\frac{n_{\mathrm{sample}}\mathrm{inside}x-\mathrm{th}\mathrm{bin}}{n_{\mathrm{sample}}·\mathsf{\Delta }x}$, where $\mathsf{\Delta }x$ is the width of the x-th bin (and for 2D histogram, this is replaced by bin area $A_{x_1,x_2}=\mathsf{\Delta }{x}_1·\mathsf{\Delta }{x}_2$), and therefore, summing over $x_1$ is aggregating the 2D bins of $(x_1,x_2)$ and combining or mixing samples with $x_2$-values/coordinates in the same $x_2$-bin (but with $x_1$-values/coordinates in different $x_1$-bins) together. In other words, it is always true that $n_{x_2}={\sum }_{x_1}^{\mathrm{naive}}{n}_{x_1,x_2}$, where $n_{x_2}$ is the number of samples inside the $x_2$-th bin and $n_{x_1,x_2}$ is number of samples inside the $(x_1,x_2)$-th bin in 2D, and hence, for estimated probability (mass), $\frac{n_{x_2}}{n_{\mathrm{sample}}}={\sum }_{x_1}^{\mathrm{naive}}\frac{n_{x_1,x_2}}{n_{\mathrm{sample}}}$, and for estimated PDFs, $\frac{n_{x_2}}{n_{\mathrm{sample}}·\mathsf{\Delta }{x}_2}={\sum }_{x_1}^{\mathrm{naive}}\frac{n_{x_1,x_2}}{n_{\mathrm{sample}}·\mathsf{\Delta }{x}_2·\mathsf{\Delta }{x}_1}·\mathsf{\Delta }{x}_1$, which is why $p(x_2,t)$$={\sum }_{x_1}^{\mathrm{naive}}\mathsf{\Delta }{x}_1·p(x_2,t;x_1,t)$ holds exactly for numerically estimated marginal and joint PDFs using histograms, which is consistent with the theoretical relation between marginal and joint PDFs $p(x_2,t)=\int p(x_2,t;x_1,t)d{x}_1$, and this has been numerically verified using the relevant 1D and 2D histogram functions in MATLAB and Python, i.e., by (naively) summing the estimated joint PDF over $x_1$, and the (naively) summed marginal is exactly the same as the one estimated directly by 1D histogram function. So in this approach, integral over $x_1$ is estimated by naive summation on $x_1$, but integral over $x_2$ is estimated by trapezoidal summation on $x_2$.

The 1st approach will violate the relation between joint and marginal $p(x_2,t)$${\displaystyle =\int p(x_2,t;x_1,t)d{x}_1}$, because as explained in the 2nd approach above, when using MATLAB’s and Python’s 1D and 2D histogram functions, one will always get exactly $p(x_2,t)$$={\sum }_{x_1}^{\mathrm{naive}}\mathsf{\Delta }{x}_1·p(x_2,t;x_1,t)$ and $p(x_2,t+\mathsf{\Delta }t)={\sum }_{x_1}^{\mathrm{naive}}\mathsf{\Delta }{x}_1·p(x_2+\mathsf{\Delta }t,t;x_1,t)$ for naive summation, but not for trapezoidal summation over $x_1$ due to the weights/factors (≠1) imposed on the “corner”/boundary/first/last summation terms, which is used in the 1st approach. However, the 2nd approach puts different importance or weights on the summation over $x_1$ as compared to $x_2$, which might also be problematic, because the original definition is a double integral over $x_1$ and $x_2$ without different weights/factors imposed by different summation methods.

To resolve this, we use the regular or naive summations on both $x_1$ and $x_2$, which avoids the issues in both the 1st and 2nd approaches, and we find that the numerical difference between the 1st and 2nd approaches and our adopted simply naive summations are really negligible, and because in this work, we are performing empirical statistics on the estimated causal information rates and illustrating the qualitative features of the empirical probability distributions of them, we use our simple naive summations over both $x_1$ and $x_2$ when estimating ${\mathsf{\Gamma }}_{x_2}^{*}{\left(t\right)}^2$ and ${\mathsf{\Gamma }}_{x_2}{\left(t\right)}^2$ in causal information rate ${\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)={\mathsf{\Gamma }}_{x_2}^{*}\left(t\right)-{\mathsf{\Gamma }}_{x_2}\left(t\right)$.

Appendix B. Complete Results: All Six Groups of Initial Conditions

For completeness, we list the full results of all figures for all six different initial Gaussian distributions listed in Table 3.

Appendix B.1. Sample Trajectories of $x_1\left(t\right)$ and $x_2\left(t\right)$

Appendix B.1.1. Initial Conditions No.1 (IC1)

Figure A1.

Initial Conditions No.1 (IC1): 50 sample trajectories of healthy CTL subjects. Each single trajectory is labeled by a different color.

Figure A2.

Initial Conditions No.1 (IC1): 50 sample trajectories of AD patients. Each single trajectory is labeled by a different color.

Appendix B.1.2. Initial Conditions No.2 (IC2)

Figure A3.

Initial Conditions No.2 (IC2): 50 sample trajectories of healthy CTL subjects. Each single trajectory is labeled by a different color.

Figure A4.

Initial Conditions No.2 (IC2): 50 sample trajectories of AD patients. Each single trajectory is labeled by a different color.

Appendix B.1.3. Initial Conditions No.3 (IC3)

Figure A5.

Initial Conditions No.3 (IC3): 50 sample trajectories of healthy CTL subjects. Each single trajectory is labeled by a different color.

Figure A6.

Initial Conditions No.3 (IC3): 50 sample trajectories of AD patients. Each single trajectory is labeled by a different color.

Appendix B.1.4. Initial Conditions No.4 (IC4)

Figure A7.

Initial Conditions No.4 (IC4): 50 sample trajectories of healthy CTL subjects. Each single trajectory is labeled by a different color.

Figure A8.

Initial Conditions No.4 (IC4): 50 sample trajectories of AD patients. Each single trajectory is labeled by a different color.

Appendix B.1.5. Initial Conditions No.5 (IC5)

Figure A9.

Initial Conditions No.5 (IC5): 50 sample trajectories of healthy CTL subjects. Each single trajectory is labeled by a different color.

Figure A10.

Initial Conditions No.5 (IC5): 50 sample trajectories of AD patients. Each single trajectory is labeled by a different color.

Appendix B.1.6. Initial Conditions No.6 (IC6)

Figure A11.

Initial Conditions No.6 (IC6): 50 sample trajectories of healthy CTL subjects. Each single trajectory is labeled by a different color.

Figure A12.

Initial Conditions No.6 (IC6): 50 sample trajectories of AD patients. Each single trajectory is labeled by a different color.

Appendix B.2. Time Evolution of PDF $p(x_1,t)$ and $p(x_2,t)$

Appendix B.2.1. Initial Conditions No.1 (IC1)

Figure A13.

Initial Conditions No.1 (IC1): Time evolution of estimated PDFs of healthy CTL subjects.

Figure A14.

Initial Conditions No.1 (IC1): Time evolution of estimated PDFs of AD patients.

Appendix B.2.2. Initial Conditions No.2 (IC2)

Figure A15.

Initial Conditions No.2 (IC2): Time evolution of estimated PDFs of healthy CTL subjects.

Figure A16.

Initial Conditions No.2 (IC2): Time evolution of estimated PDFs of AD patients.

Appendix B.2.3. Initial Conditions No.3 (IC3)

Figure A17.

Initial Conditions No.3 (IC3): Time evolution of estimated PDFs of healthy CTL subjects.

Figure A18.

Initial Conditions No.3 (IC3): Time evolution of estimated PDFs of AD patients.

Appendix B.2.4. Initial Conditions No.4 (IC4)

Figure A19.

Initial Conditions No.4 (IC4): Time evolution of estimated PDFs of healthy CTL subjects.

Figure A20.

Initial Conditions No.4 (IC4): Time evolution of estimated PDFs of AD patients.

Appendix B.2.5. Initial Conditions No.5 (IC5)

Figure A21.

Initial Conditions No.5 (IC5): Time evolution of estimated PDFs of healthy CTL subjects.

Figure A22.

Initial Conditions No.5 (IC5): Time evolution of estimated PDFs of AD patients.

Appendix B.2.6. Initial Conditions No.6 (IC6)

Figure A23.

Initial Conditions No.6 (IC6): Time evolution of estimated PDFs of healthy CTL subjects.

Figure A24.

Initial Conditions No.6 (IC6): Time evolution of estimated PDFs of AD patients.

Appendix B.3. Information Rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$

Appendix B.3.1. Time Evolution: Information Rates

Initial Conditions No.1 (IC1)

Figure A25.

Initial Conditions No.1 (IC1): Information rates along time of CTL and AD subjects.

Initial Conditions No.2 (IC2)

Figure A26.

Initial Conditions No.2 (IC2): Information rates along time of CTL and AD subjects.

Initial Conditions No.3 (IC3)

Figure A27.

Initial Conditions No.3 (IC3): Information rates along time of CTL and AD subjects.

Initial Conditions No.4 (IC4)

Figure A28.

Initial Conditions No.4 (IC4): Information rates along time of CTL and AD subjects.

Initial Conditions No.5 (IC5)

Figure A29.

Initial Conditions No.5 (IC5): Information rates along time of CTL and AD subjects.

Initial Conditions No.6 (IC6)

Figure A30.

Initial Conditions No.6 (IC6): Information rates along time of CTL and AD subjects.

Appendix B.3.2. Empirical Probability Distribution: Information Rates (for $t\ge 7.5$)

Figure A31.

Empirical probability distributions of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$.

Appendix B.3.3. Phase Portraits: Information Rates (for $t\ge 7.5$)

Figure A32.

Phase portraits of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ vs. ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$.

Initial Conditions No.1 (IC1):

Figure A33.

Initial Conditions No.1 (IC1): Phase portraits of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ vs. ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.2 (IC2):

Figure A34.

Initial Conditions No.2 (IC2): Phase portraits of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ vs. ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.3 (IC3):

Figure A35.

Initial Conditions No.3 (IC3): Phase portraits of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ vs. ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.4 (IC4):

Figure A36.

Initial Conditions No.4 (IC4): Phase portraits of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ vs. ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.5 (IC5):

Figure A37.

Initial Conditions No.5 (IC5): Phase portraits of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ vs. ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.6 (IC6):

Figure A38.

Initial Conditions No.6 (IC6): Phase portraits of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ vs. ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Appendix B.3.4. Power Spectra: Information Rates (for $t\ge 7.5$)

Initial Conditions No.1 (IC1):

Figure A39.

Initial Conditions No.1 (IC1): Power law fit for power spectra of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.2 (IC2):

Figure A40.

Initial Conditions No.2 (IC2): Power law fit for power spectra of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.3 (IC3):

Figure A41.

Initial Conditions No.3 (IC3): Power law fit for power spectra of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.4 (IC4):

Figure A42.

Initial Conditions No.4 (IC4): Power law fit for power spectra of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.5 (IC5):

Figure A43.

Initial Conditions No.5 (IC5): Power law fit for power spectra of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.6 (IC6):

Figure A44.

Initial Conditions No.6 (IC6): Power law fit for power spectra of information rates ${\mathsf{\Gamma }}_{x_1}\left(t\right)$ and ${\mathsf{\Gamma }}_{x_2}\left(t\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Appendix B.4. Shannon Differential Entropy of $p(x_1,t)$ and $p(x_2,t)$

Appendix B.4.1. Time Evolution: Shannon Differential Entropy

Initial Conditions No.1 (IC1):

Figure A45.

Initial Conditions No.1 (IC1): Shannon differential entropy along time of CTL and AD subjects.

Initial Conditions No.2 (IC2):

Figure A46.

Initial Conditions No.2 (IC2): Shannon differential entropy along time of CTL and AD subjects.

Initial Conditions No.3 (IC3):

Figure A47.

Initial Conditions No.3 (IC3): Shannon differential entropy along time of CTL and AD subjects.

Initial Conditions No.4 (IC4):

Figure A48.

Initial Conditions No.4 (IC4): Shannon differential entropy along time of CTL and AD subjects.

Initial Conditions No.5 (IC5):

Figure A49.

Initial Conditions No.5 (IC5): Shannon differential entropy along time of CTL and AD subjects.

Initial Conditions No.6 (IC6):

Figure A50.

Initial Conditions No.6 (IC6): Shannon differential entropy along time of CTL and AD subjects.

Appendix B.4.2. Empirical Probability Distribution: Shannon Differential Entropy (for $t\ge 7.5$)

Figure A51.

Empirical probability distributions of Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$.

Appendix B.4.3. Phase Portraits: Shannon Differential Entropy (for $t\ge 7.5$)

Figure A52.

Phase portraits of Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$.

Initial Conditions No.1 (IC1):

Figure A53.

Initial Conditions No.1 (IC1): Phase portraits of Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.2 (IC2):

Figure A54.

Initial Conditions No.2 (IC2): Phase portraits of Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.3 (IC3):

Figure A55.

Initial Conditions No.3 (IC3): Phase portraits of Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.4 (IC4):

Figure A56.

Initial Conditions No.4 (IC4): Phase portraits of Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.5 (IC5):

Figure A57.

Initial Conditions No.5 (IC5): Phase portraits of Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Initial Conditions No.6 (IC6):

Figure A58.

Initial Conditions No.6 (IC6): Phase portraits of Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$ of CTL and AD subjects.

Appendix B.4.4. Power Spectra: Shannon Differential Entropy (for $t\ge 7.5$)

Figure A59.

Power spectra of Shannon differential entropy $h\left(x_1\left(t\right)\right)$ and $h\left(x_2\left(t\right)\right)$$(t\ge 7.5)$.

Appendix B.5. Causal Information Rates ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right),{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$, and Net Causal Information Rates ${\mathsf{\Gamma }}_{x_2\to x_1}\left(t\right)-{\mathsf{\Gamma }}_{x_1\to x_2}\left(t\right)$

Appendix B.5.1. Time Evolution: Causal Information Rates

Initial Conditions No.1 (IC1):

Figure A60.

Initial Conditions No.1 (IC1): Causal information rates along time of CTL and AD subjects.

Initial Conditions No.2 (IC2):

Figure A61.

Initial Conditions No.2 (IC2): Causal information rates along time of CTL and AD subjects.

Initial Conditions No.3 (IC3):

Figure A62.

Initial Conditions No.3 (IC3): Causal information rates along time of CTL and AD subjects.

Initial Conditions No.4 (IC4):

Figure A63.

Initial Conditions No.4 (IC4): Causal information rates along time of CTL and AD subjects.

Initial Conditions No.5 (IC5):

Figure A64.

Initial Conditions No.5 (IC5): Causal information rates along time of CTL and AD subjects.

Initial Conditions No.6 (IC6):

Figure A65.

Initial Conditions No.6 (IC6): Causal information rates along time of CTL and AD subjects.

Appendix B.5.2. Empirical Probability Distribution: Causal Information Rates (for $t\ge 7.5$)

Initial Conditions No.1 (IC1):

Figure A66.

Initial Conditions No.1 (IC1): Empirical probability distributions of causal information rates and net causal information rates $(t\ge 7.5)$.

Initial Conditions No.2 (IC2):

Figure A67.

Initial Conditions No.2 (IC2): Empirical probability distributions of causal information rates and net causal information rates $(t\ge 7.5)$.

Initial Conditions No.3 (IC3):

Figure A68.

Initial Conditions No.3 (IC3): Empirical probability distributions of causal information rates and net causal information rates $(t\ge 7.5)$.

Initial Conditions No.4 (IC4):

Figure A69.

Initial Conditions No.4 (IC4): Empirical probability distributions of causal information rates and net causal information rates $(t\ge 7.5)$.

Initial Conditions No.5 (IC5):

Figure A70.

Initial Conditions No.5 (IC5): Empirical probability distributions of causal information rates and net causal information rates $(t\ge 7.5)$.

Initial Conditions No.6 (IC6):

Figure A71.

Initial Conditions No.6 (IC6): Empirical probability distributions of causal information rates and net causal information rates $(t\ge 7.5)$.

Appendix B.6. Causality Based on Transfer Entropy (TE)

Appendix B.6.1. Time Evolution: Transfer Entropy (TE)

Initial Conditions No.1 (IC1):

Figure A72.

Initial Conditions No.1 (IC1): Transfer Entropy (TE) along time of CTL and AD subjects.

Initial Conditions No.2 (IC2):

Figure A73.

Initial Conditions No.2 (IC2): Transfer Entropy (TE) along time of CTL and AD subjects.

Initial Conditions No.3 (IC3):

Figure A74.

Initial Conditions No.3 (IC3): Transfer Entropy (TE) along time of CTL and AD subjects.

Initial Conditions No.4 (IC4):

Figure A75.

Initial Conditions No.4 (IC4): Transfer Entropy (TE) along time of CTL and AD subjects.

Initial Conditions No.5 (IC5):

Figure A76.

Initial Conditions No.5 (IC5): Transfer Entropy (TE) along time of CTL and AD subjects.

Initial Conditions No.6 (IC6):

Figure A77.

Initial Conditions No.6 (IC6): Transfer Entropy (TE) along time of CTL and AD subjects.

Appendix B.6.2. Empirical Probability Distribution: Transfer Entropy (TE) (for $t\ge 7.5$)

Initial Conditions No.1 (IC1):

Figure A78.

Initial Conditions No.1 (IC1): Empirical probability distributions of Transfer Entropy (TE) and net Transfer Entropy $(t\ge 7.5)$.

Initial Conditions No.2 (IC2):

Figure A79.

Initial Conditions No.2 (IC2): Empirical probability distributions of Transfer Entropy (TE) and net Transfer Entropy $(t\ge 7.5)$.

Initial Conditions No.3 (IC3):

Figure A80.

Initial Conditions No.3 (IC3): Empirical probability distributions of Transfer Entropy (TE) and net Transfer Entropy $(t\ge 7.5)$.

Initial Conditions No.4 (IC4):

Figure A81.

Initial Conditions No.4 (IC4): Empirical probability distributions of Transfer Entropy (TE) and net Transfer Entropy $(t\ge 7.5)$.

Initial Conditions No.5 (IC5):

Figure A82.

Initial Conditions No.5 (IC5): Empirical probability distributions of Transfer Entropy (TE) and net Transfer Entropy $(t\ge 7.5)$.

Initial Conditions No.6 (IC6):

Figure A83.

Initial Conditions No.6 (IC6): Empirical probability distributions of Transfer Entropy (TE) and net Transfer Entropy $(t\ge 7.5)$.

Author Contributions

Conceptualization, J.-C.H., E.-j.K. and F.H.; Methodology, J.-C.H. and E.-j.K.; Software, J.-C.H.; Validation, J.-C.H.; Formal analysis, J.-C.H.; Investigation, J.-C.H., E.-j.K. and F.H.; Resources, E.-j.K. and F.H.; Writing—original draft, J.-C.H.; Writing—review & editing, J.-C.H., E.-j.K. and F.H.; Visualization, J.-C.H.; Supervision, E.-j.K. and F.H.; Project administration, J.-C.H., E.-j.K. and F.H.; Funding acquisition, E.-j.K. and F.H. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

The stochastic simulation and calculation scripts will be made publicly available in an open repository, which is likely to be updated under https://github.com/jia-chenhua?tab=repositories or https://gitlab.com/jia-chen.hua (both accessed on 17 February 2024).

Conflicts of Interest

The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Funding Statement

This work was supported by the EPSRC Grant (EP/W036770/1 (https://gow.epsrc.ukri.org/NGBOViewGrant.aspx?GrantRef=EP/W036770/1, accessed on 17 February 2024)).