Nonlinear system identification of neural systems from neurophysiological signals

Abstract

The human nervous system is one of the most complicated systems in nature. Complex nonlinear behaviours have been shown from the single neuron level to the system level. For decades, linear connectivity analysis methods, such as correlation, coherence and Granger causality, have been extensively used to assess the neural connectivities and input-output interconnections in neural systems. Recent studies indicate that these linear methods can only capture a small amount of neural activities and functional relationships, and therefore cannot describe neural behaviours in a precise or complete way. In this review, we highlight recent advances in nonlinear system identification of neural systems, corresponding time and frequency domain analysis, and novel neural connectivity measures based on nonlinear system identification techniques. We argue that nonlinear modelling and analysis are necessary to study neuronal processing and signal transfer in neural systems quantitatively. These approaches can hopefully provide new insights to advance our understanding of neurophysiological mechanisms underlying neural functions. These nonlinear approaches also have the potential to produce sensitive biomarkers to facilitate the development of precision diagnostic tools for evaluating neurological disorders and the effects of targeted intervention.

Introduction

The human nervous system is a complicated network comprised of more than 10 billion neurons, with trillions of synapses connecting them. Neuronal information processing is complex at different levels, from the microscopic pre- and post-synaptic cellular interactions to the macroscopic interactions between large populations of neurons in, for instance, sensory processing and motor response (Stanley, 2005). The behaviour of a single neuron is highly nonlinear, showing a step-like ‘none-or-all’ firing response (Koch and Segev, 2000), while the behaviour of neurons in a population could be relatively similar. Therefore, the nonlinear response of each individual neuron may be smoothed out by the distribution of membrane thresholds across the population, known as the pool effect (Rosenbaum et al., 2010). This effect typically occurs in a mono-synaptic neural system such as the cortico-spinal tract where the supraspinal motor command is linearly transferred to the motor output due to the pool effect of motor units (Negro and Farina, 2011). However, multi-synaptic neural systems, such as the somatosensory system, have been reported highly nonlinear, showing harmonic responses to periodic stimuli (Langdon et al., 2011; Yang et al., 2016a; Ren et al., 2019; Yang et al., 2020b). Cross-frequency coupling in the corticothalamic interactions has also been reported when characterising essential tremor (He et al., 2016). Nonlinear behaviours in neural systems are thought to be associated with various neural functions, including neuronal encoding, neural processing of synaptic inputs, communication between different neuronal populations and functional integration (Friston, 2001; Yang et al., 2018; Fell and Axmacher, 2011; French and Korenberg, 1989).

Various functional and effective connectivity measures have been developed (Pereda et al., 2005) to characterise such linear and nonlinear functional integration in neural networks, from large-scale neurophysiological signals. These signals, such as electroencephalogram (EEG) and magnetoencephalogram (MEG) from the brain, electromyogram (EMG) from muscles, measure neural activities from macro-scale neuronal populations. While functional connectivity measures, e.g. correlation, coherence, mutual information, only quantify the undirected statistical dependencies among signals from different areas, effective connectivity attempts to quantify the directed causal influences of one neural system over another, either at a synaptic or population level (Friston, 2011). Mostly, effective connectivity measures are based on models of neural interactions or coupling (although there exist model-free measures like transfer entropy (Dimitrov et al., 2011)) and is often time-dependent (dynamic). Therefore, effective connectivity has a strong link with dynamic modelling, also known as system identification in control systems theory (Ljung, 1999; Pintelon and Schoukens, 2012; Schoukens and Ljung, 2019), and corresponding model-based causality analysis.

Nonlinear system identification techniques have been formally applied to study neuronal information processing and neural systems since the 1970s. Some pioneering work includes: the nonlinear dynamic modelling of the retinal neuron chains in receptive-field responses (Marmarelis and Naka, 1973a,b), the identification of nonlinear synaptic interactions (Brillinger et al., 1976), the identification of neural systems using stimulus-response and white-noise approach (Marmarelis and Marmarelis, 1978; Marmarelis and Naka, 1972; Marmarelis, 2012), the development of nonlinear systems analytic approach based on functional power (or Volterra/Wiener) series to study central nervous system function and hippocampal formation (Sclabassi et al., 1988b,a), and nonlinear identification of stretch reflex dynamics (Kearney and Hunter, 1988). Until now, many linear and nonlinear system identification methods have then been proposed and developed in the neuroscience context. Nevertheless, recent studies indicate that linear methods can only capture a small amount of neural activities and functional relationships, and therefore cannot describe neural behaviours in a precise or complete way (Vlaar et al., 2016, 2017). Nonlinear approaches provide us with useful tools to explore the nonlinear nature of neural systems (Friston, 2001). In this review article, we highlight the need and recent advances in nonlinear system identification of neural systems, as well as novel neural connectivity analysis methods based on nonlinear system identification techniques. A diagram that summarises the linear and nonlinear functional and effective connectivity measures and their links with system identification is provided in Figure 1.

Figure 1:

Overview of the linear and nonlinear functional and effective connectivity (causality) measures and their links with system identification methods. The linear functional connectivity, linear system identification and linear causality measures are first reviewed in the ‘Linear connectivity and system identification’ section. The nonlinear and nonlinear time-varying system identification approaches (in both time and frequency domains) are then investigated in the ‘Nonlinear system identification of neural systems’ section. The recently proposed nonlinear function connectivity measures and nonlinear causality measures (based on nonlinear system identification) are introduced in the ‘Nonlinear neural connectivity analysis’ section. The abbreviations in the diagram are defined as: autoregressive (AR), autoregressive with exogenous input (ARX), frequency response function (FRF), nonlinear autoregressive (moving average) model with exogenous inputs (NAR(MA)X), artificial neural networks (ANN), deep neural networks (DNN), generalised frequency response function (GFRF), output frequency response function (OFRF), time-varying (TV), Granger causality analysis (GCA), partial directed coherence (PDC), directed transfer function (DTF), spectral Granger causality and directed coherence (DCOH), Granger causality (GC), dynamic causal modelling (DCM), nonlinear partial directed coherence (NPDC).

Nonlinearity in the neuronal level and neural systems

At a single neuron level, the action potential spike is the principal basis of information encoding, which allows signal transmission across different neuronal populations (French and Korenberg, 1989). The spike timing is thought to be associated with the coding scheme in neural systems (Fetz, 1997). The nonlinear nature of the neuronal process of synaptic input influences the temporal firing behaviour of individual neurons. Different types of neurons have their own repertoire of ion channels that are responsible for their characteristic nonlinear firing patterns and associated neural functions. For example, persistent inward currents mediated by their voltage-gated sodium and calcium channels are an important source of the nonlinear behaviour of spinal motoneurons. They are instrumental in generating the sustained force outputs required for postural control (Hounsgaard et al., 1988). Activation of the L-type calcium channels in nigral dopaminergic neurons results in intrinsic bursting behaviour (Sinha et al., 2020b), exhibiting low-dimensional determinism and likely encodes meaningful information in the awake state of the brain Jeong et al. (2012). The nonlinearity of the neuronal transfer function mediated by its component ion channels can generate various types of nonlinear output patterns such as harmonic, subharmonic and/or intermodulation of input patterns.

Despite plenty of knowledge of the nonlinear behaviour of a single neuron, the input-output relation at the neural system level is yet to be understood entirely. The system-level neural response is a composite output of collective neuronal activities from a large number of neurons. In a neuronal population, the pool effect can reduce the nonlinearity generated from each individual neuron, by smoothing the neuronal dynamics from a scale of milliseconds (spikes) to 10 milliseconds (local field potentials) or to 100 milliseconds (large-scale neurophysiological activities or signals such as EEG) (Ros et al., 2014). Such effects have been previously demonstrated through both a computational model and an in vivo study in the human motor system, where the motor command can be transmitted linearly via the mono-synaptic corticospinal tract when more than five motoneurons are activated (Negro and Farina, 2011). However, a small amount of nonlinearity may still be present (Yang et al., 2016b). A recent study simulated nonlinear neuronal dynamics on a large-scale neural network that captured the inter-regional connections of neocortex in the macaque. The authors applied information-theoretic measures to identify functional networks and characterised structure-function relations at multiple temporal scales (Honey et al., 2007). The nonlinearity in each neuronal population can cumulatively increase if the system involves multiple synaptic connections (Sinha et al., 2020a). A recent study in hemiparetic stroke shows that the nonlinearity in the motor system increases due to an increased usage of multi-synaptic indirect motor pathways, e.g. cortico-reticulospinal tract (Li et al., 2019), following damage to the mono-synaptic corticospinal tract (Yang et al., 2020a).

Assessing the input-output relation in neural systems, e.g. sensory, motor and cognitive processes, is essential to a better understanding of the nervous system. For instance, it could help to gain a better insight into the normal and pathological neural functions. It is well known that a linear system generates only iso-frequency interactions between an input and the output, e.g. the coupling of neuronal oscillations at a specific frequency band (Fries, 2015). For decades, correlation and coherence measures have been used to identify the linear interaction in neural systems. More recently, various studies indicate the input-output neural interactions can cross different frequency components or bands, which is named cross-frequency coupling (Jensen and Colgin, 2007; Canolty and Knight, 2010; Hyafil et al., 2015; Aru et al., 2015) and is a distinctive feature of a nonlinear system. In the following sections, we review both the linear and nonlinear approaches for identifying neural systems and associating neural connectivity, especially from a system identification perspective.

Linear connectivity and system identification

The nervous system is a highly cooperative network composed of different groups of neurons. Neural connectivity, i.e., the synchronization of neural activity across these groups, is crucial to the coordination among distant, but functionally related, neuronal groups (Varela et al., 2001). Linear neural connectivity can be assessed by determining the signal correlation or causality between the recorded neural signals. This section reviews commonly used linear connectivity, system identification methods and their interconnections in studying neural systems.

Correlation and coherence

The most widely used measure of interdependence between two time series in the time domain is the cross-correlation function (Pereda et al., 2005), which measures the linear correlation between two signals or stochastic processes X and Y with discrete observations x(t) and y(t), at t = 1, 2, …, N, as a function of their delay time:

$$C_{xy}(\tau )=\frac{1}{N-\tau }\sum _{k=1}^{N-\tau }x(k+\tau )y(k)$$ (1)

where N is the number of samples and τ the time lag between two signals. This function ranges from −1 (complete linear inverse correlation) to 1 (complete linear direct correlation). The value of τ that maximizes this function can be used to estimate the linearly related delay between signals. The well-known Pearson correlation coefficient is equal to C_xy(τ) when τ = 0. The linear dependence between two signals in the frequency domain is usually measured by the spectral coherence. The coherence between two signals at frequency f is defined as:

$$C_{xy}(f)=\frac{{\left|S_{xy}(f)\right|}^2}{S_{xx}(f)S_{yy}(f)}$$ (2)

where S_xy(f) is the cross-spectral density between x and y, and S_xx(f) and S_yy(f) the auto-spectral density of x and y respectively. The cross-spectral and auto-spectral densities are the Fourier transforms of the cross-correlation and auto-correlation functions of the two signals. Values of coherence are always between 0 and 1. The correlation and coherence measures have been widely applied to EEG, MEG or EMG signals to characterise the neuronal interactions, from the firing of cortical neuron spike trains to complicated neural systems (for reviews see (Pereda et al., 2005; Bastos and Schoffelen, 2016)).

System identification and causality

Unlike functional connectivity, effective connectivity emphasises on the directional causal influences between neural areas or signals. Here, we first introduce the classical Granger causality and its link with the time-domain linear system identification, i.e. regression models of bivariate time series. The frequency-domain causality measures can then be linked with the frequency response function of linear systems.

Considering two signals or variables X and Y, the interactions of the signals can be described by bivariate linear autoregressive with exogenous input (ARX) models jointly,

$$\begin{gathered} x(t)=\sum _{k=1}^q{a}_{11,k}x(t-k)+\sum _{k=1}^p{a}_{12,k}y(t-k)+e_x(t) \\ y(t)=\sum _{k=1}^q{a}_{21,k}x(t-k)+\sum _{k=1}^p{a}_{22,k}y(t-k)+e_y(t) \end{gathered}$$ (3)

where p and q are the model order of y and x regressors; e_x(t) and e_y(t) are the uncorrelated model prediction errors over time. A linear causal influence from X to Y defined by Granger can be expressed as a log ratio of the prediction error variances of the corresponding restricted (AR) and unrestricted (ARX) models:

$$F_{X\to Y}=\ln \frac{\mathit{var}\left(y\mid y^{-}\right)}{\mathit{var}\left(y\mid y^{-},x^{-}\right)}=\ln \frac{{\Sigma }_{22}^{y^{-}}}{{\Sigma }_{22}}$$ (4)

where x⁻ and y⁻ denotes contributions from lagged input and output terms, respectively; y⁻₂₂ denotes the variance of e_y when there are only regression terms of Y. The linear ARX models (3) can be re-written in matrix form and mapped to the frequency domain by Fourier transformation:

$$\left(\begin{array}{cc}{A}_{11}(f)& A_{12}(f)\\ A_{21}(f)& A_{22}(f)\end{array}\right)\left(\begin{array}{c}X(f)\\ Y(f)\end{array}\right)=\left(\begin{array}{c}{E}_x(f)\\ E_y(f)\end{array}\right)$$ (5)

where the components of the coefficient matrix A(f) are $A_{lm}(f)={\delta }_{lm}-{\sum }_{k=1}^{p(orq)}{a}_{lm,k}{e}^{-j2\pi kf/f_s}$ with f_s the sampling frequency and δ_lm the Kronecker delta function. We can re-write the above equation by inverting the coefficient matrix G(f) = A⁻¹(f) and moving the so-called transfer function matrix G(f) to the right-hand-side the equation. Different frequency-domain Granger causality measures, such as partial directed coherence (PDC), directed transfer function (DTF), spectral Granger causality and directed coherence (DCOH) (Baccalá and Sameshima, 2001; Chicharro, 2011; Gourévitch et al., 2006), can then be expressed as a function of the elements of either the coefficient matrix A(f) or the transfer function matrix G(f) (Baccala and Sameshima, 2001; Chicharro, 2011). By dividing both sides of (5) with the corresponding diagonal elements in the coefficient matrix A, the off-diagonal elements in the transformed coefficient matrix are actually related to the negative frequency response functions (FRFs) of linear ARX systems, if one signal is treated as the input while the other is treated as the output. For instance,

$$\frac{A_{21}(f)}{A_{21}(f)}=-H_{X\to Y}(f)$$ (6)

the FRF, H_X→Y (f), describes the input-output relationship, i.e., with input X and output Y, of the (noise-free) ‘system’ in the frequency domain. It is also known as the ‘transfer function’ in linear system theory. Frequency-domain Granger causality measure, e.g. PDC, can be expressed as a function of the FRFs of the corresponding linear ARX and AR models:

$$PD{C}_{X\to Y}(f)=\frac{-H_{X\to Y}(f)}{\sqrt{{\left|A_{11}(f)/A_{21}(f)\right|}^2+{\left|H_{X\to Y}(f)\right|}^2}}$$ (7)

Establishing such a link between the causality measures and linear system identification, in both time and frequency domains, is crucial to the further development of nonlinear model-based causality measures via nonlinear system identification, which will be investigated in the ‘Nonlinear system identification of neural systems’ and the ‘Nonlinear neural connectivity analysis’ sections.

Limitation of linear approaches on identifying neural system

Linear connectivity and system identification allow the assessment of communication between neuronal populations at the same oscillatory frequency band or similar neuronal firing patterns. The applications of linear approaches have been thoroughly reviewed previously (Sakkalis, 2011; Blinowska, 2011). However, it is not clear how much information is missing when using the linear approach since the behaviour of various neural systems can be highly nonlinear (Friston, 2001; Yang et al., 2018). When one uses a linear measure to investigate a neural system, the nonlinear neural interaction is ignored, especially between the neuronal populations which have very different mean firing rates such as the central nervous system and the periphery. A recent study reported that in the human somatosensory system over 80% of the cortical response to wrist joint sensory input comes from nonlinear interactions, where a linear model explains only 10% of the cortical response (Vlaar et al., 2016). Therefore, nonlinear connectivity and modelling approaches are needed to investigate neural systems in a complete way.

Nonlinear system identification of neural systems

It is often impossible to derive a mechanistic model of a neural system, due to the complexity of the underlying biological process and many unobservable state variables. In this section, we focus on the generic nonlinear model representations of a single-input and single-output (SISO) neural dynamic system, its identification process in the time domain, and corresponding frequency-domain analysis. We first investigate the identification of nonlinear time-invariant systems, and then time-varying nonlinear systems.

Time-domain nonlinear system identification

Volterra series

The Volterra series model is a direct generalisation of the linear convolution integral and provides an intuitive representation for a nonlinear input-output system. The output y(t) of a SISO nonlinear system can be expressed as a Volterra functional of the input signal u(t):

$$\begin{gathered} y(t)=\sum _{n=1}^M{y}^{(n)}(t)+e_y(t) \\ {y}^{(n)}(t)={\int }_{-\infty }^{+\infty }\cdots {\int }_{-\infty }^{+\infty }{h}_n\left({\tau }_1,\cdots ,{\tau }_n\right)\prod _{i=1}^{n}u\left(t-{\tau }_i\right)d{\tau }_i \end{gathered}$$ (8)

where y⁽ⁿ⁾(t) is the nth-order output and M is the maximum order of the system’s nonlinearity; h_n(τ₁, · · ·, τ_n) is the nth-order impulse response function or Volterra kernel, which describes nonlinear interactions among n copies of input and generalises the linear convolution integral to deal with nonlinear systems. Neurobiologically, Volterra series can be directly interpreted as the effective connectivity - ‘the influence that one neural system exerts over another, either at a synaptic or population level’ (Friston, 1994). The first-order kernel describes the linear ‘driving’ efficacy or linear synchronous interactions, and the second- or higher-order kernels represent the ‘modulatory’ influence or asynchronous interactions (Friston, 2001). The Fourier transform of the first-order kernel is the FRF (or transfer function) that describes the interactions in the same frequencies, while the frequency-domain counterparts of the higher-order kernels are the GFRFs (to be discussed in the ‘Frequency-domain nonlinear system analysis: nonlinear frequency response functions’ section) which quantify the cross-frequency interactions.

Practically, to deal with a large number of Volterra series coefficients, a regularization strategy is often employed in the estimation procedure (Birpoutsoukis et al., 2017). Volterra model has been widely used in physiological systems, including neural systems, modelling. Some recent examples include the study of nonlinear interactions in the hippocampal-cortical neurons (Song et al., 2007), in the spectrotemporal receptive fields of the primary auditory cortex (Pienkowski and Eggermont, 2010), in the sensory mechanoreceptor system (Vlaar et al., 2016), in the human somatosensory system (i.e. the cortical response to the wrist joint sensory input) indicating the dominance of nonlinear response (Vlaar et al., 2017), in multiple-input and multiple-output (MIMO) spiking neural circuits (Lazar and Slutskiy, 2015) and hippocampal memory prostheses (Song et al., 2016). The Volterra model also has a strong theoretical link with the NARMAX model (Billings, 2013) and the dynamic causal modelling (Friston et al., 2003) to be discussed next.

NARMAX model

Although Volterra series can provide an intuitive representation for nonlinear systems, there are several critical limitations including i) it cannot represent severely nonlinear systems; ii) the order of the Volterra series expansion can be very high in order to achieve a good approximation accuracy; however iii) the estimation of high order Volterra kernel requires a large number of data and can be computationally very expensive. Nonlinear Autoregressive Moving Average Model with Exogenous Inputs (NARMAX) model (Leontaritis and Billings, 1985; Billings, 2013) has therefore been developed as an alternative to the Volterra series. NARMAX model normally contains a much smaller number of terms due to the inclusion of output delay terms, and its identification process is computationally more efficient. Similar to the Volterra series, a polynomial Nonlinear Autoregressive Model with Exogenous Inputs, NARX (the simplest NARMAX) model, can be expressed as a summation of a series of output terms with different orders of nonlinearity:

$$y^{(n)}(t)=\sum _{p=0}^n\sum _{k_1,k_{p+q}=1}^K{c}_{p,q}\left(k_1,\dots ,k_{p+q}\right)\times \prod _{i=1}^{p}y\left(t-k_i\right)\prod _{i=p+1}^{p+q}x\left(t-k_i\right)$$ (9)

where p + q = n, k_i = 1, …, K, and ${\sum }_{k_1,k_{p+q}=1}^K\equiv {\sum }_{k_1}^K\cdots {\sum }_{k_{p+q}=1}^K$. The number of model terms depends on the order of input and output (q and p) and the maximum lags (K). The NARX model structure and parameters are typically identified based on the forward regression with the orthogonal least squares (FROLS) method (Chen et al., 1989). In cases where the system under study is stochastic with unknown coloured noise, noise moving average (MA) models should be employed to form a full NARMAX model. The identified model can be statistically validated using nonlinear correlation tests (Billings and Voon, 1983; He et al., 2014a).

A wide range of nonlinear systems can be represented by NARMAX method, including systems with exotic nonlinear behaviours such as subharmonics, bifurcations, and chaos, as observed in the human nervous system (Breakspear, 2017). Until now, NARMAX methodology has been employed to develop dynamic models for nonlinear sensory processing circuit from spiking neuron data (Florescu and Coca, 2018) as an improvement to the previous Volterra model-based studies (Lazar and Slutskiy, 2015), to investigate the somatosensory afferent pathways from muscles to the brain (Tian et al., 2018; Gu et al., 2020); as well as to study the corticothalamic nonlinear interactions during tremor active and resting states (He et al., 2016). Apart from efficient time-domain predictive modelling, NARMAX also provides an essential base for the nonlinear frequency-domain analysis, nonlinear time-varying modelling, and nonlinear causality analysis to be discussed in the following sections.

Dynamic causal modelling

Most of the effective connectivity models, e.g. linear and nonlinear autoregressive models, are directly identified from functional neurophysiological signals. However, sometimes it would be more accurate and meaningful to identify the causal interactions of the underlying neuronal activities at the level of neuronal dynamics (Stephan et al., 2007). The aim of dynamic causal modelling (DCM) (Friston et al., 2003, 2019) is to infer such connectivity among brain regions (or sources) under different experimental factors or inputs. A DCM comprises typically two parts: a neuronal part that describes dynamics among neuronal sources and a measurement part that describes how the source dynamics generate measurable observations, e.g. EEG or MEG (Kiebel et al., 2008; David et al., 2006). Therefore, DCM can be naturally expressed as a nonlinear state-space model with hidden states denoting unobserved neuronal dynamics and the observation equation (e.g. the lead-field) assumed linearly in the states. The effective connectivity among those sources can be identified via Bayesian model selection and Bayesian inference of the neuronal model parameters. One strength of DCM is its biophysical and neuronal interpretation of how the neurophysiological signals are generated from the underlying neuronal system, through the generative or forward (state-space) modelling. Due to the complexity and computational cost of Bayesian model selection, DCM is more suitable to investigate the connectivity among pre-defined regions of interest, rather than exploratory analysis of relatively large brain or neural networks (Stephan et al., 2010). Compared to the hypothesis-driven DCM, the NARMAX or Volterra models are more flexible in terms of model structure identification and their direct frequency-domain mapping (to be discussed) is a powerful tool to study the nonlinear cross-frequency interactions between neurological regions.

Other black-box neural nonlinear system identification methods

Apart from the aforementioned three important generic nonlinear model representations, other black-box modelling approaches have also been applied in the ‘neural system identification’ context. For example, artificial neural networks (ANNs), e.g. recurrent, multilayer perceptron, fuzzy, probabilistic neural networks, have often been used as alternatives to classical system identification models. ANNs have been applied to predict neural responses in visual cortex (Lehky et al., 1992; Lau et al., 2002), and to improve the prediction of synaptic motor neuron responses (Meruelo et al., 2016). More recently, deep neural networks (DNNs), such as convolutional neural network (CNN) or recurrent neural network (RNN), are employed to model sensory neural responses, to understand neural computations and to learn feature spaces for neural system identification (McIntosh et al., 2016; Batty et al., 2016; Klindt et al., 2017; Kell and McDermott, 2019; Keshishian et al., 2020). Nevertheless, in the current neuroscience literature, ANNs and DNNs are applied more towards automatic feature extraction and classification problems rather than traditional ‘system identification’. For instance, automatic detection and diagnosis of neurological disorders via a combination of ANN with other nonlinear feature extraction techniques such as approximate entropy and wavelet (Srinivasan et al., 2007; Ocak, 2009; Guo et al., 2010; Subasi, 2006), or direct implementation of DNNs (Zhou et al., 2018; Emami et al., 2019). Nonparametric Bayesian approaches like Gaussian process (GP) is closely related to ANN. GP has recently been used for system identification purpose (Kocijan et al., 2005; Gregorčič and Lightbody, 2008, 2009) and applied to analyse neurophysiological signals (Wu et al., 2015), such as the use of GP modelling for EEG-based seizure detection and prediction (Faul et al., 2007) and heteroscedastic modelling of noisy high-dimensional MEG data (Fyshe et al., 2012). Compared with ANN, GP can be applied to model datasets with small sample size and it has a relatively small number of hyperparameters. Additionally, due to its Bayesian nature, GP can incorporate prior knowledge and specifications into the modelling and can directly capture the model uncertainty. Another well-known system identification paradigm is the nonlinear state-space model (Verdult, 2002; Schön et al., 2011). Its strength in dynamic (latent) state estimation and sequential inference process makes it a suitable candidate in the identification of certain neural systems. The state-space models have been applied to infer neural spiking activity induced by an implicit stimulus observed through point processes (Smith and Brown, 2003), to perform optimal decoding given multi-neuronal spike train data and tracking nonstationary neuron tuning properties (for a review, see (Paninski et al., 2010)), and to perform source localization from neurophysiological signals like MEG and EEG (Fukushima et al., 2012; Barton et al., 2008). All of those black-box modelling approaches are usually flexible and accurate in quantifying complex and long-range nonlinear interactions. In comparison, the advantages of NARMAX and Volterra models are their modelling simplicity, interpretability of nonlinear interactions in the time-domain (e.g. the order of nonlinearity, phase delay), and frequency-domain mapping and analysis (e.g. energy transfer, intermodulations).

Frequency-domain nonlinear system analysis: nonlinear frequency response functions

Many nonlinear effects, such as harmonics, intermodulations and energy transfer, can only be accurately and intuitively characterised in the frequency domain. Thus, it is important to ‘map’ the identified time-domain nonlinear models to the frequency domain for further analysis. A multidimensional Fourier transform of the nth-order Volterra kernel in (8) yields the so-called nth-order generalised frequency response function (GFRF), H_n(f₁, · · ·, f_n), which is a natural extension of the linear frequency response function to the nonlinear case (Billings, 2013). The output spectrum Y (f) of a nonlinear system can then be expressed as a function of the input spectrum X(f) and GFRF, known as the output frequency response function (OFRF) (Lang and Billings, 1996; Billings, 2013):

$$Y(f)=\sum _{n=1}^M\left(\frac{1}{\sqrt{n}}{\int }_{f_1+\cdots +f_n=f}{H}_n(f_1,\cdots ,f_n)\prod _{i=1}^{n}X(f_i)df\right)$$ (10)

Compared with the Volterra series, the GFRFs can be more efficiently computed from the identified time-domain NARMAX model (9) and corresponding model parameters (Billings and Tsang, 1989). As shown in Figure 2, the peaks in 1st-order GFRF indicate the well-known ‘resonance frequencies’ of the linear part of the system; and the peaks (or ridges) in the 2nd-order GFRF would indicate nonlinear harmonics (f₁ + f₂ when f₁ = f₂) or inter-modulation effects (f₁ ± f₂ when f₁ ≠ f₂) in the output spectrum, and so on. Since $Y(f)={\sum }_{n=1}^m{Y}_n(f)$, the nth OFRF Y_n(f) represents the nth-order (linear or nonlinear) contribution from the input to the output spectrum. Practically, by comparing the OFRF with the spectrum of the output signal, obtained from a classical nonparametric estimation such as fast Fourier transform, one can also ‘validate’ the accuracy of the time-domain modelling in addition to the aforementioned NARMAX model validation (He et al., 2016).

Figure 2:

The GFRFs of an exemplar nonlinear system. (A) The linear 1st-order GFRF, H₁(f), shows a ‘resonance’ peak at f = 0.9Hz; (B) and (C) the 3-D and contour plots of the 2nd-order GFRF, H₂(f₁, f₂). It shows a peak at f₁ = 0.9Hz, f₂ = 0.9Hz, which indicates harmonics at 2f = f₁ + f₂ = 1.8Hz can be introduced in the output spectrum if input contains a 0.9Hz component.

NARMAX-based frequency-domain analysis method has been applied to quantify the dynamic characteristics of nonlinear sensory processing circuit models from spiking neuron data (Florescu and Coca, 2018), the cross-frequency interactions in the corticothalamic loops with respect to tremor (He et al., 2016), and the characterisation of epileptic brain states (He et al., 2014a). More details will be discussed in the ‘Neurological and clinical applications’ section.

Time-varying nonlinear system identification

Many neurological subsystems are inherently nonstationary, since the brain is a dissipative and adaptive dynamical system (Kaplan et al., 2005; Lehnertz et al., 2017). Modelling and identification approaches of nonstationary processes have been well developed for linear systems, i.e. linear time-varying (LTV) systems. They are primarily based on adaptive recursive methods, such as recursive least squares, least mean squares, and the Kalman filter (Ljung and Gunnarsson, 1990), or based on a finite basis function expansion of the time-varying coefficients (Zou et al., 2003; Tsatsanis and Giannakis, 1993; Niedzwiecki and Klaput, 2002; Zheng et al., 2001). The identification of a nonlinear time-varying system is more sophisticated. The primary difficulty is how to effectively and simultaneously resolve the nonlinear model structure selection and the time-varying coefficient estimation. Approaches based on time-varying Volterra series combining artificial neural networks (Iatrou et al., 1999), principal dynamic modes (Zhong et al., 2007), or sliding-window strategy (Li et al., 2016), have been proposed. However, the model structure selection is still an unsolved issue, and the identification and frequency-domain analysis are computationally costly.

A better strategy is to extend the basis function expansion approach, originally proposed for LTV identification, to nonlinear time-varying cases (Billings and Wei, 2005). The time-varying (TV) parameters in TV-NARX models are first expanded using multi-wavelet basis functions, and TV nonlinear model is transformed into an expanded time-invariant model; the challenging TV model selection and parameter estimation problem can then be solved using the computational efficient FROLS algorithm. To accommodate the stochastic perturbations or additive coloured noise, this procedure can also be extended to more general TV-NARMAX models by introducing a modified extended least squares (ELS) algorithm (He et al., 2015). Several modifications to the TV-NARX model has recently been proposed using different basis functions or model selection procedure (Li et al., 2018; Guo et al., 2019). The corresponding frequency-domain analysis for nonlinear time-varying systems has also been developed based on the identified time-domain TV-NARX or TV-NARMAX model and the TV-GFRF concepts (He et al., 2013, 2015). By fitting TV-NARX models to two fragments of intracranial EEG recordings measured from epileptic patients, the corresponding frequency-domain analysis (i.e. TV-GFRF and TV-OFRF) shows the nonlinear energy transfer effect – the underlying neural system transfers the energy from lower frequencies to higher frequencies as the seizure spreading from the left to the right brain regions over time (He et al., 2013, 2014b).

An overview of the NARMAX model-based system identification framework, including both time-invariant (NARMAX) and time-varying (TV-NARX) modelling along with corresponding frequency-domain analysis to neurophysiological signal analysis, is summarised in Figure 3.

Figure 3:

Analysing neurophysiological signals uses nonlinear time-invariant and time-varying system identification and corresponding frequency-domain analysis methods. The upper part of the diagram illustrates the nonlinear time-invariant modelling: first a NARMAX model is identified from the input and output neurophysiological signals (e.g. EEG, EMG, MEG, LFP); this time-domain model is then mapped to the frequency-domain with GFRFs (i.e. H₁(f), H₂(f), …), and the OFRF (Y (f) = Y₁(f) + Y₂(f), …) can be computed from the input spectrum and GFRFs. The lower part of the diagram shows the nonlinear time-varying system identification using a TV-NARX model, and the identified time-varying model can then be mapped to the time-frequency domain with (averaged) TV-GFRFs (i.e. H₁(f, t), H₂(f, t), …). The TV-OFRF can therefore be computed from a combination of the input time-frequency spectrum and the TV-GFRFs.

Nonlinear neural connectivity analysis

The communication between different neuronal populations which have very different firing behaviours can result in nonlinear neural connectivity, showing neural coupling across two or more different frequencies. To quantitatively study such a ‘cross-frequency coupling’, this section reviews recent advances in nonlinear neural functional and effective connectivity analysis.

High-order spectrum and nonlinear coherence

The power spectra and coherence discussed in the ‘Correlation and coherence’ section are Fourier transforms of the auto- and cross-correlations of signals, hence they are only linear frequency-domain measures. Practically these measures cannot detect certain nonlinear effects such as quadratic moments in or between signals that have a zero mean (Billings, 2013). Higher-order spectral analysis has been developed to detect nonlinear coupling between spectral components (Nikias and Mendel, 1993). For example, the most widely used bispectral analysis is the frequency-domain mapping of the third-order statistics. It can be used to quantify the quadratic nonlinear interactions, i.e. quadratic phase coupling. The bispectrum or bicoherence (and the bivariate cross-bispectrum or cross-bicoherence) analysis is well-known in engineering signal processing, whereas it has only relatively recently been applied to study the nonlinear interactions in neurophysiological signals (Sigl and Chamoun, 1994; Chella et al., 2014, 2016). For example, bispectral measures were used to detect long-range nonlinear coupling and synchronization in healthy subjects from human EEG (Isler et al., 2008; Chella et al., 2014), to characterise and predict epileptic seizures (Chua et al., 2009), and to study the nonlinear interactions between different frequency components related to Parkinson’s disease and tremor (Marceglia et al., 2006; Wang et al., 2014; He et al., 2016).

However, bispectrum or bicoherence cannot detect nonlinearity beyond second order, such as the higher-order harmonics and intermodulation effects, or the subharmonic coupling. A generalised nonlinear coherence analysis framework has therefore been proposed, based on two different nonlinear mappings from the input to the output of an ‘open-loop’ system in the frequency domain (Yang et al., 2016a):

1) n:m Mapping: to measure harmonic or subharmonic coupling related to individual input frequencies. $Y^m\left(f_Y\right)=H(n:m)X^n\left(f_X\right)$, $X^n\equiv \underset{n}{\underbrace{XX\cdots X}}$, where the output frequencies (f_Y ) are related to the input frequencies (f_X) by the ratio n/m (n and m are co-prime positive integers), and H(n : m) is the n:m mapping function. The n:m mapping can generate cross-frequency (e.g. harmonic m = 1 or subharmonic m > 1) coupling between the input and the output.

2) Integer Multiplication Mapping: to quantify intermodulation coupling among multiple (≥ 2) input frequencies.

$$Y\left(f_Y\right)=H\left(f_1,f_2,\cdots ,f_N;a_1,a_2,\dots ,a_N;f_Y\right)M\prod _{i=1}^N{X}^{a_i}\left(f_i\right)$$ (11)

where f_Y = a₁f₁ + a₂f₂ + · · · + a_Nf_N. The M is the corresponding multinomial coefficient, and H(f₁, f₂, · · ·, f_N; a₁, a₂, · · ·, a_N; f_Y ) indicates amplitude scaling and phase shift from the input to the output.

According to these two different types of nonlinear mapping, Yang and colleagues proposed two basic metrics for quantifying nonlinear coherence: (i) n:m coherence and (ii) multi-spectral coherence (Yang et al., 2016a).

n:m coherence

The n:m coherence is a generalized coherence measure for quantifying nonlinear coherence between two frequency components of the input X(f) and the output Y (f) (Yang et al., 2016a):

$$C_{nm}\left(f_X,f_Y\right)=\frac{\left|S_{XY}\left(f_X,f_Y\right)\right|}{\sqrt{S_{XX}^n\left(f_X\right)S_{YY}^m\left(f_Y\right)}}$$ (12)

where n : m = f_Y : f_X. S_XY (f_X, f_Y ) =< Xⁿ(f_X)(Y^m(f_Y ))* > is the n:m cross-spectrum, with < · > represents the averaging over repetitions. $S_{XX}^n\left(f_X\right)=<X^n\left(f_X\right){\left(X^n\left(f_X\right)\right)}^{*}>=<|\underset{n}{\underbrace{X\left(f_X\right)X\left(f_X\right)\cdots X\left(f_X\right)}}{|}^2>$ is the nth-order auto-spectra. According to Cauchy-Schwarz-inequality, we have:

$$\left|<X^n(f_X){(Y^m(f_Y))}^{*}>\right|\le {(<{\left|X^n(f_X)\right|}^2>)}^{1/2}\cdot {(<{\left|Y^n(f_Y)\right|}^2>)}^{1/2}$$ (13)

Thus, n:m coherence is bounded by 0 and 1, where one indicates that two signals are perfectly coupled for the given frequency pair (f_X, f_Y ).

A simplified version of n:m coherence that considers only the phase relation between the input and the output is known as n:m phase synchronization index (Wacker and Witte, 2010). The n:m coherence and n:m phase synchronization index has been widely applied to neuroscience research to investigate nonlinear functional connectivity in different brain regions (Scheffer-Teixeira and Tort, 2016; Darvas et al., 2009a), as well as the nonlinear connectivity between the brain and muscles (Yang et al., 2016b).

Multi-spectral coherence

Multi-spectral coherence measures the multi-frequency nonlinear coupling generated by the integer multiplication mapping (Yang et al., 2016a). It is defined as:

$$C_{MS}(f_1,\cdots ,f_N;a_1,\cdots ,a_N;f_Y)=\frac{\left|S_{XY}(f_1,\cdots ,f_N;a_1,\cdots ,a_N;f_Y)\right|}{\sqrt{({\prod }_{i=1}^n{S}_{XX}^{\left|a_i\right|}(f_i))S_{YY}(f_Y)}}$$ (14)

where f_Y = a₁f₁ + · · · + a_Nf_N; S_XY (f₁, · · ·, f_N; a₁, · · ·, a_N; f_Y ) is the high-order cross-spectrum between X and Y, and equal to $<{\prod }_{i=1}^n{X}^{\left|a_i\right|}\left(f_i\right))Y^{*}\left(f_Y\right)>$. Here ‘*’ indicates the complex conjugate. When there are only two input frequencies, the multi-spectral coherence is degraded to the bicoherence (Elgar and Guza, 1988). The multi-spectral coherence or bicoherence has been applied to study the nonlinear behaviours in visual (Shils et al., 1996), auditory (Chandran, 2012) and somatosensory systems (Yang et al., 2016a), which are thought to be associated with neural coding and functional integration of various sensory inputs (Gundlach and Müller, 2013).

A simplified version of multi-spectral coherence that considers only the phase relation between the input and the output is known as multi-spectral phase coherence (Yang et al., 2016c). Similarly, there is a degraded measure, named bi-phase locking value (Darvas et al., 2009b), for the case involving only two input frequencies. The advantage of multi-spectral phase coherence or bi-phase locking value is that it allows precise estimation of time delay in the nervous system based on the relative phase relationship between the input and output (Yang et al., 2016c; Tian et al., 2020). The multi-spectral phase coherence or bi-phase locking value has been previously used to determine neural transmission delays in the human visual system (Shils et al., 1996) and the stretch reflex loops (Yang et al., 2016b).

Nonlinear causality analysis: system identification based approaches

Time-domain analysis

In terms of effective connectivity, classical linear Granger causality analysis (GCA) (as discussed in the ‘System identification and causality’ section) may provide misleading results when used to analyse EEG/MEG or EMG signals, as the possible nonlinear interactions within a neural system are not modelled explicitly by simply using linear regression models. The Granger causality definition has been extended to nonlinear cases in the time domain, based on nonparametric methods (Diks and Panchenko, 2006; Hiemstra and Jones, 1994), radial basis functions (Ancona et al., 2004), kernel methods (Marinazzo et al., 2008), local linear models (Chen et al., 2004). DCM (Friston et al., 2003) (see the ‘Dynamic causal modelling’ section) was developed to accommodate both linear and nonlinear causal effects using a dynamic state-space model, and the effective connectivity among hidden states (unobserved neuronal dynamics) can be identified via Bayesian inference. Information-theoretical effective connectivity measures have also been proposed, such as the bivariate transfer entropy (TE) (Liu and Aviyente, 2012; Shovon et al., 2017) and phase transfer entropy (PTE). TE is a model-free measure, which compares two conditional probabilities using the Kullback-Leibler divergence - the amount of uncertainty in the future of target signal Y conditioned only on the target’s past and the future of Y conditioned on the past of both its own Y and the source X, in a conceptually similar way as the GCA. A more recent work (Harmah et al., 2019) generalised the TE method by using multivariate transfer entropy, which can overcome the problems of inferring spurious or redundant causality and missing synergistic interactions between multiple sources and target.

Another strategy to implement nonlinear granger causality under a system identification framework is to use NARX models (Faes et al., 2008; Zhao et al., 2013), by calculating the relative predictability improvement obtained from the NARX model compared to the corresponding NAR model. More importantly, compared to other nonlinear causality measures (e.g. nonparametric or information-theoretic measures), the advantage of using NARX-based causal inference (Zhao et al., 2013) is that one can easily separate the linear and nonlinear causal influence, for example from an input X to an output Y. After fitting a polynomial NARX model with the form (9), the linear causality can still be calculated from (4) based on the linear part of the NARX model, while the nonlinear causal influence from X to Y can be defined as:

$$F_{X\to Y}^n=\ln \frac{var\left(Y\mid Y_l^{-},Y_n^{-},X_l^{-}\right)}{var\left(Y\mid Y_l^{-},Y_n^{-},X_l^{-},X_n^{-},{(XY)}_n^{-}\right)}or\ln \frac{var\left(Y\mid X_l^{-}\right)}{var\left(X_l^{-},X_n^{-},{(XY)}_n^{-}\right)}$$ (15)

Here $Y_n^{-}$, $X_n^{-}$ and ${(XY)}_n^{-}$ denote the sets of all nonlinear delayed terms of Y, X and nonlinear product terms XY. This nonlinear causality measure can also be generalised to nonlinear time-varying systems, by computing similar linear and nonlinear causality indices based on the identified TV-NARX models (as described in the ‘Time-varying nonlinear system identification’ section), as proposed in (Zhao et al., 2013; Li et al., 2012).

Frequency-domain analysis

In the frequency domain, linear Granger causality measures, such as PDC, DTF and spectral Granger causality, can all be expressed as a function of the elements in the coefficient matrix or its inverse the transfer function matrix of the corresponding linear ARX models (3). By identifying the link between the PDC and the FRFs of the corresponding linear ARX models (as described in the the ‘System identification and causality’ section), a new nonlinear PDC (NPDC) measure has been proposed (He et al., 2014a) by generalising the spectrum decomposition with respect to a nonlinear NARX model in a similar way as the linear case. The NPDC from X to Y can then be expressed as a direct generalization of the linear PDC:

$$NPD{C}_{X\to Y}(f)=\frac{-H_{X\to Y}(f)}{\sqrt{{\left|H_y^e(f)/H_x^e(f)\right|}^2+{\left|H_{X\to Y}(f)\right|}^2}}$$ (16)

Here, the H_X→Y (f) is the ‘nonlinear FRF’ which replaces the FRF in the linear PDC (7), and $H_y^e(f)$ and $H_x^e(f)$ are the error-driven GFRFs correspond to the restricted NAR models with respect to Y and X as discussed in (He et al., 2014a). The NPDC measures both linear and nonlinear causal influences from X to Y. The linear causal effects can be interpreted as a special case of (16) by only considering the 1st-order nonlinear FRFs of NARX (i.e. H_1,X→Y (f)) and NAR (i.e. $H_{1,x}^e(f)$ and $H_{1,y}^e(f)$) models.

This new NPDC measure has recently been applied to predict epileptic seizures from EEG data (Zhang et al., 2020) by advancing the construction of functional brain networks, nonlinear feature selection and classification. This new nonlinear causality measure helps to provide better prediction accuracy compared to other standard graph theory or nonlinear classification based methods. A nonlinear generalization of Geweke’s spectral Granger causality has also been proposed (He et al., 2014b) using the NARX methodology.

Neurological and clinical applications

Movement, sensation and cognition arise from the cumulative activity of neurons within neural circuits and across distant, macroscale networks in the nervous system. Although the behaviour of an individual neuron has been investigated and well understood for decades, the mechanisms underlying neural communications between macroscale neural networks are still yet to better understand. Newly developed nonlinear system identification approaches allow us to investigate neural communications from large-scale neural activities measured by EEG, MEG and EMG, with the most recent application examples discussed below.

Nonlinear cortical response to somatosensory inputs

The human somatosensory system is highly nonlinear (Yang et al., 2018). Previous studies applied periodic sinusoid tactile stimulations to the index finger and measured the cortical response, where they found harmonic and subharmonic patterns in the response (Langdon et al., 2011; Tobimatsu et al., 1999). Several recent studies used sum-of-sinusoid stimulations to the wrist joint and found not only harmonics and subharmonics but also intermodulation patterns (Vlaar et al., 2016; Yang et al., 2016a). The majority of intermodulation responses presented the second-order nonlinearity, which is the sum or the difference between input frequencies (Yang et al., 2020b). These findings indicate that the nonlinearities in the somatosensory system allow the functional integration of input signals at different frequencies, and they can be transmitted in different somatosensory ascending pathways.

Yang and colleagues recently built a hierarchical neural network based on known neuroanatomical connections and corresponding transmission delays in neural pathways to model the cortical response to somatosensory input (Tian et al., 2018). The proposed computational model contains a neural layer at the thalamus that integrates the inputs from different ascending pathways, including Ia and II afferents. The computational model well captured the majority of the cortical response to the given somatosensory inputs, indicating the functional integration of different somatosensory input signals may occur at the thalamus and is then transmitted to the cortex via the thalamocortical radiation.

Tremor: nonlinearity in the thalamocortical loop

Essential tremor is a common neurological movement disorder widely considered to have a centrally-driven origin. There is both neurophysiological and clinical evidence of thalamic involvement in the central oscillatory network generating essential tremor (Hua and Lenz, 2005; Marsden et al., 2000; Deuschl et al., 2011). Local field potential (LFP) recordings of thalamic ventralis intermedius (Vim) nucleus show a strong linear correlation with the contralateral EMG during tremor (Marsden et al., 2000). Some studies using EEG and MEG suggest that the sensorimotor cortex is also part of the central tremor-related oscillatory network, with significant coupling in some cases with the contralateral tremorgenic EMG (Hellwig et al., 2001; Raethjen and Deuschl, 2012; Buijink et al., 2015). Despite a well-established reciprocal anatomical connection between the thalamus and cortex, the functional association between the two structures during ‘tremor-on’ periods had not been extensively defined.

He and co-authors (He et al., 2016) investigated the functional connectivity among cortical EEG, thalamic (Vim) LFPs and contralateral EMG signals over both ‘tremor-on’ and ‘tremor-off’ states, using linear coherence and nonlinear bispectral analysis methods. In addition to expected strong coherence between EMG and thalamic LFP, nonlinear interactions (i.e. quadratic phase coupling) at different frequencies, i.e. low frequency during tremor on and higher frequency during tremor off, in LFPs have been reported. More importantly, by using the NARX-based nonlinear system identification and frequency-domain analysis (as described in the ‘Frequency-domain nonlinear system analysis: nonlinear frequency response functions’ section), two distinct and non-overlapping frequency ‘channels’ of communication between thalamic Vim and the ipsilateral motor cortex were identified, which robustly defined the ‘tremor-on’ versus ‘tremor-off’ states. Longer corticothalamic nonlinear phase lags in the tremor active state were also uncovered, suggesting the possible involvement of an indirect multi-synaptic loop. This work demonstrates, for the first time, the importance of cross-frequency nonlinear interactions between the cortex and the thalamus in characterising the essential tremor.

Nonlinear analysis for determining motor impairment in stroke

After a stroke, damage to the brain increases the reliance on indirect motor pathways resulting in motor impairments and changes in neural connectivity between the brain and muscles. A hallmark of impairments post-stroke is a loss of independent joint control that leads to abnormal co-activation between shoulder, arm and hand muscles, known as the upper limb synergy (Roh et al., 2013). The upper limb synergy is thought to be caused by progressive recruitment of indirect motor pathways via the brainstem following a stroke-induced loss of corticospinal projections (McPherson et al., 2018). Thus, a neural connectivity measure that quantifies the recruitment of these indirect motor pathways would be crucial to evaluate post-stroke motor impairments. Recent model-based simulation and clinical studies indicate that the increased usage of indirect motor pathways enhances nonlinear distortion of motor command transmissions, which leads to stronger nonlinear interaction between the brain and muscles (Sinha et al., 2020a; Yang et al., 2020a). The ratio of nonlinear interaction over linear interaction, known as the nonlinear-over-linear index (N-L Index), has been reported to be associated with the relative ratio of the recruitment of indirect versus direct motor pathways (Yang et al., 2020a). This new measure may facilitate the future determination of the effect of new therapeutic interventions that aim to optimise the usage of motor pathways, and thus facilitate the stroke recovery.

Epilepsy

It has been widely recognised that epileptic seizures are highly nonlinear phenomena, due to low dimensional chaos during epileptic seizure or transitions between ordered and disordered stats (Iasemidis and Sackellares, 1996). Currently, the treatment mainly relies on long-term medication with antiepileptic drugs or neurosurgery, which can cause cognitive or other neurological deficits. New treatment strategies such as on-demand therapies during the pre-seizure (preictal) state or electrical stimulation are therefore needed. A vital part of this new on-demand strategy is the accurate and timely detection of the preictal state, even seconds before seizure onset (Lehnertz, 2008). A range of univariate, bivariate and multivariate linear and nonlinear measures have been developed for the characterisation and detection or prediction of epileptic brain states and achieving a better understanding of the spatial and temporal dynamics of the epileptic process (Carney et al., 2011; Lehnertz, 2008). There is a comprehensive review of using different parametric and nonparametric nonlinear features (in time, frequency and time-frequency domains) for the automated epilepsy stage detection and classification (Acharya et al., 2013).

Given the current challenges in epilepsy detection and diagnostics (Elger and Hoppe, 2018; Lehnertz, 2008), e.g. to improve the understanding of brain dynamics and mechanisms during the seizure-free interval and seizure initiation and termination, there is a great need to develop more accurate nonlinear methods to improve the detectability of directional interactions in underlying functional and anatomical networks. Developing new nonlinear system identification and nonlinear causality measures are therefore crucial. A nonlinear causality measure, partial transfer entropy (Papana et al., 2012), has been applied to analyse the EEG of epileptic patients during preictal, ictal and postictal states. It can provide better detection of causality strength variations compared to linear PDC. An adaptive nonlinear Granger causality measure was also proposed (Sysoeva et al., 2014) and applied to LFP data (intracranial EEG in cortex and thalamus) in rats. It was reported to provide more sensitive detection of changes in the dynamics of network properties compared to linear Granger causality. The recently proposed nonlinear frequency-domain causality measure NPDC (He et al., 2014a) (as reviewed in the ‘Frequency-domain analysis’ section) has been applied to analyse EEG recordings of two bipolar channels from a patient with childhood absence epilepsy. It shows this nonlinear measure can detect extra frequency-domain causal interactions compared to standard linear PDC.

Discussion

The complexity and nonlinearity of neural systems require advanced system identification techniques to understand their properties and mechanisms better. This review investigated the links between connectivity analysis and system identification, as well as recent progress of nonlinear system identification of neural systems. With the state-of-the-art examples of clinical applications, we argued that nonlinear dynamic modelling and corresponding connectivity analysis allows new insights into the underlying neural functions and neuropathological mechanisms of the abnormality caused by various neurological disorders. These novel approaches may well facilitate the development of new precision diagnostic tools and brain-computer interface (BCI) techniques (Nicolas-Alonso and Gomez-Gil, 2012; Chaudhary et al., 2016; Abiri et al., 2019), and therefore improve the diagnosis and treatment of neurological disorders as well as restore communication and movement for people with motor disabilities.

Compared to the linear system identification and iso-frequency connectivity analysis, nonlinear dynamic modelling and cross-frequency analysis are much more complicated. Such complexity brings challenges but also research opportunities. Potential future work includes: i) further developing multivariate system identification techniques and corresponding multivariate nonlinear frequency-domain analysis and causality analysis measures. Most existing nonlinear system identification based (time or frequency-domain) analysis or causality analysis are primarily bivariate, which limits nonlinear analysis to the only pre-defined local brain or neural regions. New multivariate system identification (e.g. multivariate nonlinear regression modelling) or inference approaches would generalise the existing nonlinear connectivity analysis to larger neural networks, although developing efficient model selection and reducing the computational cost would be challenging tasks; ii) many neuronal systems or interactions are in nature nonstationary and nonlinear, how to automatically distinguish the nonlinearity and time-varying effects (nonstationarity) via novel system identification technique is still an open and important research topic, although significant progress has been made so far (as reviewed in the ‘Time-varying nonlinear system identification’ section); iii) machine learning and deep learning techniques have recently been applied to Granger causality analysis (Tank et al., 2018; Chivukula et al., 2018; Peng, 2020), an interesting future work is to further explore and combine the advantages of deep learning, e.g. accurate quantification of complex and long-range nonlinear interactions, and nonlinear system identification approaches to study the nonlinear causal interactions in complex neural networks; iv) using nonlinear system identification techniques to extract new nonlinear features for the BCI; v) apart from those clinical applications described in the ‘Neurological and clinical applications’ section, the importance of nonlinearity in understanding and characterising other important neurological disorders, e.g. Parkinson’s disease and Alzheimer’s disease (Stam, 2005; Stam et al., 1995; Blackburn et al., 2018; Gunawardena et al., 2020), has been reported recently. Therefore nonlinear system identification approaches will have great potential in developing new diagnostic tools for those primary neurological disorders that affect a large population worldwide.