Specification of initial Kalman recursions of symmetric nonlinear state-space model

Abstract

A new class of nonlinear Time Series model referred to as Symmetric Nonlinear State-Space Model (SNSSM) was successfully developed using Kalman filter methodology. Some vital properties of the SNSSM such as optimal Kalman gain and optimal filter state covariance were derived. We finally initialized the filter which enabled us obtained the initial Kalman recursions under stationarity and nonstationarity assumptions. Under the former, the mean and variance were obtained unconditionally using Kronecker products and vec operator. But under the later, the mean and variance/covariance of the system were conditionally obtained using a well-known marginal and conditional property of multivariate normal distribution. It is expected that the former will be better than the later if the system is stationary, otherwise the later will be better.

Mathematics; Statistics; Time series analysis; Nonlinear models; State-space model; Kalman filter; Kalman gain; Kalman recursion; Vec operator; Kronecker products; System update.

1. Introduction

A State-Space Model (SSM) consists of a transition/state equation and a measurement equation. The transition equation formulates the dynamics of the state variables and the measurement equation relates the observed variables to the unobserved transition vector. The state vector can contain trend, seasonal, cyclical and regression components together with an error term also known as innovation. However, the stochastic behavior of the state variable, its association to the data and the covariance structure of the errors depend on parameters that are almost always unknown; (Sascha, 2009). The word "Kalman" was from the original author/inventor of the filter, Rudolf Kalman (1930–2016). So this is nothing but a name given to filters in this concept. Basically, all filters share a common attribute: to give opportunity for something to pass through, at the same time seize the opportunity for something to pass through. A Kalman filter also acts as a filter, but its operation is somewhat more complex and harder to understand! Rhudy et al. (2017). Basically, a Kalman filter takes in information which is known to have some innovations/uncertainty. The aim of the filter is to allow perfect information to pass through and at the same time disallows the imperfect information (noise or error) to pass through. Technically, Kalman filter refers to a class of recursive equations that gives a clear way to update the information in a system of equations known as State-Space model. Kalman filter and State-Space model formulation together, provide a very powerful tool for the recursive treatment of a dynamic systems, Amold et al. (2008). The purpose of filtering is to update our knowledge of the state vector as soon as a new observation becomes available, Raphael (2016) and Robert et al. (2016).

Researchers from different fields across the world are contributing immensely to the development of the State-Space/Kalman filter models both theoretically and emphatically. Theoretically for example, Hamilton (1994) gave a State-Space representation of a linear dynamic system. The wisdom behind this representation is to capture all the dynamics of the unobserved measurement vector ${\mathbf{Y}}_t$ in terms of unobserved state vector, say ${\mathbf{X}}_t$. He imposed some restrictions on the parameters of the measurement vector that would ensure the stability of the process. He further proposed a general form of linear State-Space model with a constant parameter; and he derived an optimal forecast of the system via a well-established result for normal variables; all the needed components of the linear Kalman filter algorithm have been derived. The major limitation of the Hamilton's work on the State-Space modeling and Kalman filtering is the assumption of linearity, the frame work was designed to handle a linear or approximately linear system, as such, it cannot handle any nonlinear system.

Extended Kalman Filter (EKF), Particle Filter (PF), and Unscented Kalman Filter (UKF), are the most commonly used techniques to estimate the state of a nonlinear dynamic system (Gao et al., 2018) and (Hu et al., 2020). The EKF is an approximate technique that utilized the first-order Taylor series expansion to linearize the nonlinear dynamic system so that the classical Kalman filter can be applied. It was however observed that if the nonlinear dynamic model to linearize has a high degree of nonlinearity, the linearization usually results to a very large estimation error which make the filtering solution of the EKF unsuitable as well as unreliable (Del Moral, 1996) and (Hu et al., 2020). To overcome the limitations of the EKF, Particle Filter (PF) was proposed.

The PF (which is sometimes called Sequential Monte Carlo (SMC)) is a recursive Bayesian optimal filtering technique that approximates the posteriori distribution of a state of dynamic system (haven large amount of particles) based on Monte Carlos algorithm (Gao et al., 2018) and (Anons, 2019). However, computational complexity of the PF as a result of the large number of particles (samples) required in the process of resampling serves as its major drawback. To handle the limitations of both EKF and PF, the Uncented Kalman Filter (UKF) was proposed.

The UKF is a member of a set of filters called Linear Regression Filters also known as Sigma-Point Filters (Terejanu, 2019). This method forms a linear regression between n sample (sigma) points drawn from prior distribution of a random variable to linearize a nonlinear function of the random variable. The UKF is regarded as a derivative-free alternative to the EKF because it uses a deterministic sampling approach to linearize the nonlinear dynamic system (Wan and van der Merwe, 2019). Comparing to EKF, the estimation accuracy and convergence rate of the UKF is higher. Comparing to PF, the implementation of the UKF is simpler and its computation is easier (Gao et al., 2018). It was however reported that the implementation of the UKF requires that system models are exactly pre-defined; therefore if the said models involve uncertainties, the estimation of UKF will be deteriorated (Hu et al., 2020).

Similarly, Antti (2001), proposed a State-Space model that is nonlinear in nature; the model is called Switching Nonlinear State-State model (Switching Nonlinear SSM) which is a combination of the SSM and Hidden Markov Model (HHM). As we all know, the state vector is continuous in the SSM where as it is discrete in the HMM. In this case, the author designed the SSM part to model the short-term changes/dynamics of the data, while the HMM describes the long-term dynamics of the data and controls the SSM. He further described the development of a switching Nonlinear SSM and a Bayesian learning algorithm for its parameters. It uses multilayer perception networks to model the nonlinear function of the SSM. In the same vain; the author stated that “computational complexity of the Nonlinear SSM algorithm sets serious limitation for the Bayesian switching SSM for most practical uses and its learning algorithm is very slow”. In addition to the above stated limitations by the author, the model frame work is more general! It did not take into consideration the nature of series/data (such as symmetry or asymmetry); and avoiding this fact of great importance would lead to drawing a very wrong inference in the system under study.

A class of robust Kalman like filters was also developed by many researchers such as Zorzi (2017), Levy and Nikoukhah (2013), Levy and Zorzi (2016), Zorzi and Levy (2015), among other. These filters were designed to avoid large errors usually encountered in the implementation of the existing filters. The robust Kalman like filters are performing according to risk sensitive approach. The sensitivity to large errors was tuned by what is known as risk sensitivity parameter. In addition to the risk sensitive approach, a divergence-based minimax approaches were also been proposed to perform the robust filtering, where the robust filter is obtained by minimizing the mean square error according to least favorable model in a bounded continuum (ball) on the Tau-divergence family. The gain matrix of these kind of filters is updated according to the risk sensitivity.

In a quest for Kalman filter like algorithm that will model and predict Time Series characterized by a Regime-Switching behavior, Raphael (2016), proposed another nonlinear State-Space model known as Modified State-Space Model (MSSM). The MSSM allows for the introduction of nonlinear function: Logistic Smooth Transition Autoregressive (LSTAR) model in the state equation of the CSSM and this transformed the CSSM from linear to nonlinear model. The basic limitation of the Raphael's work is the asymmetric behavior of its state equation (which is inappropriate for modelling many financial series such as exchange rates). We therefore addressed this challenge by developing a nonlinear Time Series model with symmetric state equation known as Symmetric Nonlinear State-Space Model (SNSSM). Hence, the aim of this paper is to specify the initial equations of the developed system which would enable us obtain the Kalman recursions of the SNSSM under stationary and nonstationary situations.

2. Methodology

In this section, we would sequentially demonstrate the procedure upon which we followed to develop the SNSSM. To enable the reader(s) grasp the idea effectively, we introduced the existing and classical methods given in section 2.1 below. The developed methodology: SNSSM is presented in section 2.2; finally, the initial equations of the SNSSM were proposed/specify in section 3.

2.1. Classical state-space models (CSSM)

As stated in section 1 above, the State-Space model is a system of two equations as given in (1(1) and (2)(2):

$$Y_{t+1}=H{X}_{t+1}+V_{t+1}$$ (1)

$$X_{t+1}=\psi X_t+{\omega }_{t+1}$$ (2)

Eq. (1) called measurement (observation) equation, describes the relation between the observed Time Series, $Y_{t+1}$ and the (possibly unobserved) state $X_{t+1}$. Eq. (2) called the State (transition) equation, describes the evolution of the state variables as being driven by the stochastic process of innovations ${\omega }_{t+1}$.

The terms $V_{t+1}$ and ${\omega }_{t+1}$ are the measurement and the process noise respectively. Usually one assumes normal innovations, such that $V_{t+1}\sim N\left(0,{\sigma }_V^2\right)$ and ${\omega }_{t+1}\sim N\left(0,{\sigma }_{\omega }^2\right)$. Similarly, these error terms $V_{t+1}$ and ${\omega }_{t+1}$ are assumed to be serially independent and independent of each other at all time periods as well as uncorrelated with the initial state. The role of $V_{t+1}$ in (1) is to account for any uncertainty in the measurement of the output (i. e. it tells us how much or little we can trust the equation). The parameter H is an unknown that links the unobservable variables and regression effects of the state equation with the observation equation, $\psi$ is an unknown parameter that determines how the observation and state equations evolve (change) in time (Yu, 2015).

Moreover, one can decides to look at (1(1) and (2)(2) in multivariate perception which can subsequently be written as

$$\underset{\left(m\times 1\right)}{{\mathbf{Y}}_{t+1}}=\underset{\left(m\times n\right)\left(n\times 1\right)}{\mathbf{H}{\mathbf{X}}_{t+1}}+\underset{\left(m\times 1\right)}{{\mathbf{V}}_{t+1}}$$ (3)

$$\underset{\left(n\times 1\right)}{{\mathbf{X}}_{t+1}}=\underset{\left(n\times n\right)\left(n\times 1\right)}{\mathbf{\psi }{\mathbf{X}}_t}+\underset{\left(n\times 1\right)}{{\mathit{\Omega }}_{t+1}}$$ (4)

In this case, $E\left({\mathbf{V}}_{t+1}{\mathbf{V}}_{t+1}^{\prime }\right)=\underset{m\times m}{\mathbf{L}}$ and $E\left({\mathit{\Omega }}_{t+1}{\mathit{\Omega }}_{t+1}^{\text{'}}\right)=\underset{n\times n}{\mathbf{Z}}$

However, the matrices ${\mathbf{V}}_{t+1}$ and ${\mathit{\Omega }}_{t+1}$ are not really implemented/included in evaluations of (3(3) and (4)(4) because they are assumed to be random innovations with zero mean, but instead are always used in determination of any information about the observation and state error covariance matrices $\mathbf{L}$ and $\mathbf{Z}$. The system matrices $\mathbf{H}$ and $\mathbf{\psi }$ are in general vary with time, but would not change with respect to states/transitions. In most cases, they are regarded as constants. $\mathbf{\psi }$ Contains the coefficients of the transition terms in (4) and $\mathbf{H}$ performs similar task in (3) (Rhudy et al., 2017).

As stated earlier, the MSSM of Raphael (2016) was developed with the aid of LSTAR model: a family of Smooth Transition Autoregressive (STAR) model. The STAR model is a nonlinear Time Series model that allows for regime-switching behavior through a continuous transition function. Alenka et al. (2014), Terasvirla (1994) and Zhou (2010). In this context, the data-generating process to be modeled is viewed as a linear process that switches between numbers of regimes according to some rules. It has been assumed that there is a continuum of switches, that is, there is a smooth transition from one extreme regime to the other. It consists of three stages: specification, estimation and evaluation; Iquebal (2016), and Usman et al. (2018), The STAR model of order p is given as,

$$x_{t+1}={\mathbf{\phi }}_j^{\prime }{\mathbf{x}}_t\left[1-G\left(X_{t-d};\gamma ,c\right)\right]+{\mathbf{\phi }}_j^{\prime }{\mathbf{x}}_{t}G\left(X_{t-d};\gamma ,c\right)+{\omega }_{t+1}$$ (5)

where

${\mathbf{x}}_t={\left(1,x_{t-1},x_{t-2},\mathrm{...},x_{t-p}\right)}^{\prime }$, ${\mathbf{\phi }}_j={\left({\phi }_{j0},{\phi }_{j1},\mathrm{...},{\phi }_{jp}\right)}^{\prime }$ and $J=\left\{\mathrm{1,2}\right\}$, $G\left(X_{t-d};\gamma ,c\right)$ is known as transition function which allows the model to switch between different regimes smoothly. It is bounded between zero and one, i.e. 0 ≤$G\left(X_{t-d};\gamma ,c\right)$≤ 1. This property also makes Eq. (5) capable to explain the two extreme states as well as a continuum of states lie in-between these extreme states/regimes. The variable $X_{t-d}$ is known as transition variable and d is delay parameter. $\gamma >0$, is a smoothness parameter for the transition function $G\left(X_{t-d};\gamma ,c\right)$. c is a location or threshold parameter and it represents the point of transition between the two extreme regimes. Transition function, $G\left(X_{t-d};\gamma ,c\right)$ causes the nonlinear dynamics in the model, and can have different functional choices. For each choice of transition function, we get different regime switching behavior; Yaya and Shittu (2016). The most common choices are logistic and exponential forms as given in Eqs. (6) and (7) respectively.

$$G\left(X_{t-d};\gamma ,c\right)=\frac{1}{1+\exp [-\gamma \left(X_{t-d}-c\right)]}$$ (6)

$$G\left(X_{t-d};\gamma ,c\right)=1-\exp [-\gamma {\left(X_{t-d}-c\right)}^2]$$ (7)

Note: If (6) is considered as $G\left(X_{t-d};\gamma ,c\right)$ in (5), then (5) is called Logistic Smooth Transition Autoregressive (LSTAR) model. Similarly, if (7) is considered as $G\left(X_{t-d};\gamma ,c\right)$ in (5), then (5) is called Exponential Smooth Transition Autoregressive (ESTAR) model.

Comparing between the two transition functions: (6(6) and (7)(7), the logistic is changing monotonically with $X_{t-d}$, while the exponential is changing symmetrically at $C$ with $X_{t-d}$. To visualize the asymmetric and symmetric features of the two transition functions: logistic and exponential, see Figure 1 below.

Figure 1.

Logistic and exponential transition functions with varying values of gamma ($\gamma$).

2.2. Symmetric nonlinear state-space model (SNSSM)

In order to handle the shortcomings of the MSSM highlighted earlier, we modified our system of equations given in (1(1) and (2)(2) by replacing the state equation: (2) of the system with the STAR model given in (5) and considering (7) as a transition function assuming that $p=2$ (for crystal clear). Hence, the new state equation is given as:

$$\begin{array}{ll}{X}_{t+1}\hfill & ={\psi }_1{X}_t\left(1-\left(1-e^{-\gamma \left(X_t-C\right){}^2}\right)\right)+{\psi }_2{X}_t\left(1-e^{-\gamma \left(X_t-C\right){}^2}\right)+{\omega }_{t+1}\hfill \\ \hfill & ={\psi }_1{X}_t{e}^{-\gamma \left(X_t-C\right){}^2}+{\psi }_2{X}_t\left(1-e^{-\gamma \left(X_t-C\right){}^2}\right)+{\omega }_{t+1}\hfill \end{array}$$ (8)

Note that when $\gamma \to \infty ,$ then Eq. (8) becomes Eq. (2) which is linear; the same situation happens when $\gamma \to 0$.

Differentiating Eq. (8) partially with respect to the previous state $X_t$, we have

$$\frac{\partial X_{t+1}}{\partial X_t}={\psi }_1{X}_t\left(-2\left(\gamma \left(X_t-C\right)\right)\right)e^{-\gamma \left(X_t-C\right){}^2}+{\psi }_1{e}^{-\gamma \left(X_t-C\right){}^2}+{\psi }_2{X}_t\left(2\gamma \left(X_t-C\right)e^{-\gamma \left(X_t-C\right){}^2}\right)+{\psi }_2\left(1-e^{-\gamma \left(X_t-C\right){}^2}\right)$$ (9)

which gives

$$=-2{\psi }_1\gamma X_t\left(X_t-C\right)e^{-\gamma \left(X_t-C\right){}^2}+{\psi }_1{e}^{-\gamma \left(X_t-C\right){}^2}+2{\psi }_2\gamma X_t\left(X_t-C\right)e^{-\gamma \left(X_t-C\right){}^2}+{\psi }_2\left(1-e^{-\gamma \left(X_t-C\right){}^2}\right)$$ (10)

expanding and equation to zero, we have

$$\left({\psi }_1-{\psi }_2\right)\left(1-G\left(X_t\right)\right)\left(-2\gamma X_t^2+2\gamma C{X}_t+1\right)+{\psi }_2=0$$ (11)

simplifying further we have

$$-2\gamma {\left(X_t-\frac{C}{2}\right)}^2=\frac{-{\psi }_2}{\left({\psi }_1-{\psi }_2\right)\left(1-G\left(X_t\right)\right)}-\frac{\left(2+\gamma C^2\right)}{2}$$ (12)

taking the L. C. M. of the R. H. S. of (12) and simplifying further gives

$${\hat{X}}_t=\frac{-C\pm \sqrt{\frac{2{\psi }_2+\left(\gamma C^2+2\right)\left({\psi }_1-{\psi }_2\right)\left(1-G\left(X_t\right)\right)}{\gamma \left({\psi }_1-{\psi }_2\right)\left(1-G\left(X_t\right)\right)}}}{2}$$ (13)

Note that (13) is an estimate of the previous state with the following regularity conditions:

$C\ge 0,0<\gamma <\infty ,{\psi }_1>0,{\psi }_2>0,{\psi }_1>{\psi }_2,0<G\left(X_t\right)<1\text{and}-\infty <X_{t-d}<\infty$. Recall that $G\left(X_t\right)=G\left(X_{t-d};\gamma ,c\right)=1-{\ell }^{-\gamma {\left(X_{t-d}-c\right)}^2}$ as in (7).

One would ask whether (13) inherited the symmetrical feature of the exponential transition function given in (7) or not? To answer this, we need to visualize (13) to see if it is really symmetric; even though it will give two graphs because of the presence of $[\pm ]$ signs, the two graphs are given in Figure 2 below:

Figure 2.

Positive and negative estimate of predicted state with varying values of gamma ($\gamma$).

Figure 2 clearly show that the symmetrical properties of Eq. (7) were inherited by (13); hence our system of equation: Eqs. (1) and (2) is now nonlinear as well as symmetric. The system is therefore capable of handling/modeling any symmetric nonlinear series such as exchange rate.

Hence, the Kalman filter algorithms of the proposed system (SNSSM) was developed using an established marginal and conditional property of multivariate normal distribution. This procedure was also adopted by Hamilton (1994), Zivot (2006) and Tsay (2010). Thus, the algorithms (also known as Kalman recursions) are given in the predictions and updating equations below:

• •Predictions:

$$\begin{array}{l}{\mathbf{X}}_{t+1|t}=\mathbf{\psi }{\mathbf{X}}_{t|t}\\ {\mathbf{F}}_{t+1|t}=\mathbf{\psi }{\mathbf{F}}_{t|t}{\mathbf{\psi }}^{\prime }+\mathbf{Z}\end{array}$$ (14)

where ${\mathbf{X}}_{t|t}$ is as given in (13), ${\mathbf{F}}_{t|t}$ is the error covariance matrix of the previuos state and $\mathbf{Z}$ is as previously defined.

• •Updating:

$$\begin{array}{l}{\mathbf{X}}_{t+1|t+1}={\mathbf{X}}_{t+1|t}+\mathbf{H}{\mathbf{F}}_{t+1|t}{\mathbf{Q}}_{t+1|t}^{-1}\left({\mathbf{Y}}_{t+1}-{\mathbf{Y}}_{t+1|t}\right)\\ {\mathbf{F}}_{t+1|t+1}={\mathbf{F}}_{t+1|t}-\mathbf{H}{\mathbf{F}}_{t+1|t}{\mathbf{Q}}_{t+1|t}^{-1}{\mathbf{H}}^{\prime }{\mathbf{F}}_{t+1|t}^{\prime }\end{array}$$ (15)

where ${\mathbf{Y}}_{t+1|t}=\mathbf{H}{\mathbf{X}}_{t+1|t}$, ${\mathbf{Q}}_{t+1|t}=\mathbf{H}{\mathbf{F}}_{t+1|t}{\mathbf{H}}^{\prime }+\mathbf{L}$ and $\mathbf{L}$ is as previously defined.

Note: the initialization process of the above algorithms is discussed in section 3; and that is the overall objective of this paper.

3. Specifying the initial Kalman recursions of the developed SNSSM

Initialization of the Kalman filter usually involves two possible situations: It may be that the system under study is stationary and it may also be that the system under study is nonstationary. So, in Kalman filter theory, if the series under study is stationary (which is very rare in real life), the mean and variance are usually obtained unconditionally. Similarly, if the said system is nonstationary, the mean and variance/covariance are usually obtained conditionally. Details of these two cases were given below:

3.1. Case I: the system is assumed to be stationary

Consider our system of Eqs. (3) and (4); if the system is covariance stationary, then the state vector ${\mathbf{X}}_t$ is also covariance stationary. Hence, the mean and variance/covariance will be determined unconditionally using Kronecker products and vec operator. The vec operator is a technique of transforming a matrix to a column vector by stacking the columns of the original matrix (say $\mathbf{D}$) on top of the other sequentially1. For details on this see for example Knut et al. (2005), Helmut (2005), Caswell and Daalen (2016), Casarin et al. (2017), David (2008) among others.

Now, the initialization is started by considering the state Eq. (4) and assuming that the initial value of the state vector ${\mathbf{X}}_t$ is chosen from normal distribution (note: it is customary at the initial stage to let $t=0$ with mean ${\mathbf{\mu }}_0={\hat{\mathbf{X}}}_0$ and covariance ${\mathbf{F}}_0$; and we can recall that for this process to be stationary, there is need for all the eigenvalues of $\mathbf{\psi }$ to lie within the unit circle.

Now, we can write the unconditional mean of (4) as

$$E\left({\mathbf{X}}_0\right)={\hat{\mathbf{X}}}_0=0$$ (16)

And the unconditional variance is given as

$${\mathbf{F}}_0=E\left({\mathbf{X}}_0{\mathbf{X}}_0^{\prime }\right)$$ (17)

As an illustration, the unconditional variance of ${\mathbf{X}}_t$ given in (18) can be calculated from (4). To achieve this; we postmultiplied the state equation by its transpose, taking the expectation and equating all cross products to zero, that is

$$\begin{array}{ll}E\left({\mathbf{X}}_{t+1}{\mathbf{X}}_{t+1}^{\text{'}}\right)& =E\left(\left(\mathbf{\psi }{\mathbf{X}}_t+{\mathit{\Omega }}_{t+1}\right){\left(\mathbf{\psi }{\mathbf{X}}_t+{\mathit{\Omega }}_{t+1}\right)}^{\text{'}}\right)\\ & =\mathbf{\psi }E\left({\mathbf{X}}_t{\mathbf{X}}_t^{\text{'}}\right){\mathbf{\psi }}^{\text{'}}+E\left({\mathit{\Omega }}_{t+1}{\mathit{\Omega }}_{t+1}^{\text{'}}\right)\end{array}$$ (18)

Remember that if the state vector ${\mathbf{X}}_t$ is stationary, then $E\left({\mathbf{X}}_{t+1}{\mathbf{X}}_{t+1}^{\prime }\right)=E\left({\mathbf{X}}_t{\mathbf{X}}_t^{\prime }\right)={\mathbf{F}}_t$; and for $t=0$, Eq. (18) becomes

$${\mathbf{F}}_0=\mathbf{\psi }{\mathbf{F}}_0{\mathbf{\psi }}^{\prime }+\mathbf{Z}$$ (19)

Applying the vec operator to (20), (19), we have

$$vec\left({\mathbf{F}}_0\right)=\left({\left({\mathbf{\psi }}^{\prime }\right)}^{\prime }\otimes \mathbf{\psi }\right)\cdot vec\left({\mathbf{F}}_0\right)+vec\left(\mathbf{Z}\right)$$

which becomes

$$vec\left({\mathbf{F}}_0\right)-\left(\mathbf{\psi }\otimes \mathbf{\psi }\right)\cdot vec\left({\mathbf{F}}_0\right)=vec\left(\mathbf{Z}\right)$$

which is

$$vec\left({\mathbf{F}}_0\right)\left(\mathbf{I}-\mathbf{\psi }\otimes \mathbf{\psi }\right)=vec\left(\mathbf{Z}\right)$$

therefore

$$vec\left({\mathbf{F}}_0\right)={\left(\mathbf{I}-\mathbf{\psi }\otimes \mathbf{\psi }\right)}^{-1}\cdot vec\left(\mathbf{Z}\right)$$ (20)

where

$vec\left({\mathbf{F}}_0\right)$ is an $n^2\times 1$ column vector, $\mathbf{I}$ is an $n^2\times n^2$ identity matrix, $\left(\mathbf{\psi }\otimes \mathbf{\psi }\right)$ is an $n^2\times n^2$ matrix2 and $vec\left(\mathbf{Z}\right)$ is an $n^2\times 1$ column vector.

In the case of two variables (i. e. $n=2$), the unconditional variance equation for the initial Kalman recursions of the SNSSM representation given in Eq. (20) can be written out in full as

$$Vec\left(\begin{array}{cc}{f}_{11}^0& f_{12}^0\\ f_{21}^0& f_{22}^0\end{array}\right)={\left[\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)-\left(\left(\begin{array}{cc}{\phi }_{11}& {\phi }_{12}\\ {\phi }_{21}& {\phi }_{22}\end{array}\right)\otimes \left(\begin{array}{cc}{\phi }_{11}& {\phi }_{12}\\ {\phi }_{21}& {\phi }_{22}\end{array}\right)\right)\right]}^{-1}.Vec\left(\begin{array}{cc}{z}_{11}& z_{12}\\ z_{21}& z_{22}\end{array}\right)$$ (21)

Simplifying further, we have

$${\left(\begin{array}{c}{f}_{11}^0\\ f_{21}^0\\ f_{12}^0\\ f_{22}^0\end{array}\right)=\left(\begin{array}{cccc}1-{\phi }_{11}^2& -{\phi }_{11}{\phi }_{12}& -{\phi }_{12}{\phi }_{11}& -{\phi }_{12}^2\\ -{\phi }_{11}{\phi }_{21}& 1-{\phi }_{11}{\phi }_{22}& -{\phi }_{12}^2& -{\phi }_{12}{\phi }_{22}\\ -{\phi }_{21}{\phi }_{11}& -{\phi }_{12}^2& 1-{\phi }_{22}{\phi }_{11}& -{\phi }_{22}{\phi }_{12}\\ -{\phi }_{21}^2& -{\phi }_{21}{\phi }_{22}& -{\phi }_{22}{\phi }_{12}& 1-{\phi }_{22}^2\end{array}\right)}^{-1}.\left(\begin{array}{c}{z}_{11}\\ z_{21}\\ z_{12}\\ z_{22}\end{array}\right)$$ (22)

where

$f_{11}^0$ is the unconditional variance of the state $x_{11}^0$; $f_{22}^0$ is the unconditional variance of the state $x_{22}^0$ and $f_{21}^0$ is the unconditional covariance between the states $x_{11}^0$ and $x_{22}^0$. Hence, in order to handle the inversion problem of the $4\times 4$ matrix associated with Eq. (22), we need to partition the matrix and apply the principle of inverse of a partitioned matrix (matrix inversion lemma) accordingly.

Matrix inversion lemma is a tool that is found very useful in Time Series Analysis, Estimation Theory, Signal Processing and Control Engineering. Applying the lemma, the inverse of any partitioned matrix can be found provided that the elements (submatrices) on the main diagonal of the partitioned matrix are invertible square matrices, and the other elements (submatrices) may or may not be square (Dan, 2006).

Now, from (22) let the $4\times 4$ matrix be

$$\mathbf{\kappa }=\left(\begin{array}{cc}{\mathbf{\kappa }}_{11}& {\mathbf{\kappa }}_{12}\\ {\mathbf{\kappa }}_{21}& {\mathbf{\kappa }}_{22}\end{array}\right)$$ (23)

where

${\mathbf{\kappa }}_{11}=\left(\begin{array}{cc}1-{\phi }_{11}^2& -{\phi }_{11}{\phi }_{12}\\ -{\phi }_{11}{\phi }_{21}& 1-{\phi }_{11}{\phi }_{22}\end{array}\right)$, ${\mathbf{\kappa }}_{12}={\mathbf{\kappa }}_{21}^{\prime }=\left(\begin{array}{cc}-{\phi }_{12}{\phi }_{11}& -{\phi }_{12}^2\\ -{\phi }_{12}^2& -{\phi }_{12}{\phi }_{22}\end{array}\right)$ and ${\mathbf{\kappa }}_{22}=\left(\begin{array}{cc}1-{\phi }_{22}{\phi }_{11}& -{\phi }_{22}{\phi }_{12}\\ -{\phi }_{22}{\phi }_{12}& 1-{\phi }_{22}^2\end{array}\right)$

Some unique properties of $\mathbf{\kappa }$ are

• 1.${\left({\mathbf{\kappa }}_{11}-{\mathbf{\kappa }}_{12}{\mathbf{\kappa }}_{22}^{-1}{\mathbf{\kappa }}_{21}\right)}^{-1}={\mathbf{\kappa }}_{11}^{-1}+{\mathbf{\kappa }}_{11}^{-1}{\mathbf{\kappa }}_{12}{\left({\mathbf{\kappa }}_{22}-{\mathbf{\kappa }}_{21}{\mathbf{\kappa }}_{11}^{-1}{\mathbf{\kappa }}_{12}\right)}^{-1}{\mathbf{\kappa }}_{21}{\mathbf{\kappa }}_{11}^{-1}.$
• 2.$\left|\mathbf{\kappa }\right|=\left|{\mathbf{\kappa }}_{11}\right|.\left|{\mathbf{\kappa }}_{22}-{\mathbf{\kappa }}_{21}{\mathbf{\kappa }}_{11}^{-1}{\mathbf{\kappa }}_{12}\right|=\left|{\mathbf{\kappa }}_{22}\right|.\left|{\mathbf{\kappa }}_{11}-{\mathbf{\kappa }}_{12}{\mathbf{\kappa }}_{22}^{-1}{\mathbf{\kappa }}_{21}\right|.$
• 3.${\mathbf{\kappa }}^{-1}$ can be written as

$${\mathbf{\kappa }}^{-1}={\left(\begin{array}{cc}{\mathbf{\kappa }}_{11}& {\mathbf{\kappa }}_{12}\\ {\mathbf{\kappa }}_{21}& {\mathbf{\kappa }}_{22}\end{array}\right)}^{-1}=\left(\begin{array}{cc}{\mathbf{\kappa }}_{11}^{-1}+{\mathbf{\kappa }}_{11}^{-1}{\mathbf{\kappa }}_{12}{\mathbf{\theta }}^{-1}{\mathbf{\kappa }}_{21}{\mathbf{\kappa }}_{11}^{-1}& -{\mathbf{\kappa }}_{11}^{-1}{\mathbf{\kappa }}_{12}{\mathbf{\theta }}^{-1}\\ -{\mathbf{\theta }}^{-1}{\mathbf{\kappa }}_{21}{\mathbf{\kappa }}_{11}^{-1}& {\mathbf{\theta }}^{-1}\end{array}\right)$$ (24)

assuming that ${\mathbf{\kappa }}_{11}^{-1}$ exists. And $\mathbf{\theta }={\mathbf{\kappa }}_{22}-{\mathbf{\kappa }}_{21}{\mathbf{\kappa }}_{11}^{-1}{\mathbf{\kappa }}_{12}$ alternatively

$${\mathbf{\kappa }}^{-1}={\left(\begin{array}{cc}{\mathbf{\kappa }}_{11}& {\mathbf{\kappa }}_{12}\\ {\mathbf{\kappa }}_{21}& {\mathbf{\kappa }}_{22}\end{array}\right)}^{-1}=\left(\begin{array}{cc}{\mathbf{\Theta }}^{-1}& -{\mathbf{\Theta }}^{-1}{\mathbf{\kappa }}_{12}{\mathbf{\kappa }}_{22}^{-1}\\ -{\mathbf{\kappa }}_{22}^{-1}{\mathbf{\kappa }}_{21}{\mathbf{\Theta }}^{-1}& {\mathbf{\kappa }}_{22}^{-1}+{\mathbf{\kappa }}_{22}^{-1}{\mathbf{\kappa }}_{21}{\mathbf{\Theta }}^{-1}{\mathbf{\kappa }}_{12}{\mathbf{\kappa }}_{22}^{-1}\end{array}\right)$$ (25)

assuming that ${\mathbf{\kappa }}_{22}^{-1}$ exists. And $\mathbf{\Theta }={\mathbf{\kappa }}_{11}-{\mathbf{\kappa }}_{12}{\mathbf{\kappa }}_{22}^{-1}{\mathbf{\kappa }}_{21}$

Substituting either Eq. (24) or (25) in Eq. (22), the unconditional variance equation for the initial Kalman recursions of the SNSSM representation given in Eq. (22) can easily be found. For the details of the above lemma, see (Knut et al., 2005) (Tsay, 2014), (Dan, 2006) and (Knut et al., 2008).

3.2. Case II: the system is assumed to be nonstationary

Recalling our system of equations: (3(3) and (4)(4); and assuming that the system is nonstationary. Under this situation, the mean and variance/covariance of the initial state $\left({\mathbf{X}}_0\right)$ were usually obtained conditionally. To achieve this, we used a well-known marginal and conditional property of the multivariate normal distribution; [see, Rencher (2002), for more details] even though we can achieve same from the prediction and updating equations: (14(14) and (15)(15) by letting $t=1$, but for better understanding, we choose to follow this path.

From (3(3) and (4)(4); let ${\mathbf{X}}_{t+1}$ and ${\mathbf{Y}}_{t+1}$ denote $\left(n\times 1\right)$ and $\left(m\times 1\right)$ subvectors respectively whose joint normal distribution is given as

$$\mathbf{J}=\left(\begin{array}{c}{\mathbf{X}}_{t+1}\\ {\mathbf{Y}}_{t+1}\end{array}\right)\sim {\mathbf{N}}_{n+m}\left(\left(\begin{array}{c}{\mathbf{\mu }}_X\\ {\mathbf{\mu }}_Y\end{array}\right),\left(\begin{array}{cc}{\mathbf{\tau }}_{XX}& {\mathbf{\tau }}_{XY}\\ {\mathbf{\tau }}_{YX}& {\mathbf{\tau }}_{YY}\end{array}\right)\right)$$ (26)

Then the conditional distribution of ${\mathbf{X}}_{t+1}$ given ${\mathbf{Y}}_{t+1}$ is multivariate normal with mean $\mathbf{\mu }$ and variance-covariance matrix $\mathbf{\tau }$; i. e. ${\mathbf{X}}_{t+1}|{\mathbf{Y}}_{t+1}\sim \mathbf{N}\left(\mathbf{\mu },\mathbf{\tau }\right)$;where

$$\mathbf{\mu }=E\left({\mathbf{X}}_{t+1}|{\mathbf{Y}}_{t+1}\right)={\mathbf{\mu }}_{X|Y}={\mathbf{\mu }}_X+{\mathbf{\tau }}_{XY}{\mathbf{\tau }}_{YY}^{-1}\left({\mathbf{Y}}_{t+1}-{\mathbf{\mu }}_Y\right)$$ (27)

$$\mathbf{\tau }=\mathrm{var}\left({\mathbf{X}}_{t+1}|{\mathbf{Y}}_{t+1}\right)={\mathbf{\tau }}_{XX}-{\mathbf{\tau }}_{XY}{\mathbf{\tau }}_{YY}^{-1}{\mathbf{\tau }}_{YX}$$ (28)

In the theory of the State-Space model with Gaussianity assumption, the system innovations ${\mathit{\Omega }}_{t+1}$ and ${\mathbf{V}}_{t+1}$ are normally distributed and the initial state vector ${\mathbf{X}}_0$ is also normally distributed. We can therefore write the state equation at $t=0$ as

$${\mathbf{X}}_1=\mathbf{\psi }{\mathbf{X}}_0+{\mathit{\Omega }}_1$$ (29)

Since ${\mathbf{X}}_0\sim N\left({\mathbf{\mu }}_0,{\mathbf{F}}_0\right)$, ${\mathit{\Omega }}_1\sim N\left(0,\mathbf{Z}\right)$ also ${\mathbf{X}}_0$ and ${\mathit{\Omega }}_1$ are independent, so

$$E\left({\mathbf{X}}_1\right)={\mathbf{\mu }}_{1|0}=\mathbf{\psi }{\hat{\mathbf{X}}}_0$$ (30)

Similarly,

$${\mathbf{\tau }}_{XX}=\mathrm{var}\left({\mathbf{X}}_1\right)={\mathbf{F}}_{1|0}=E\left[\left({\mathbf{X}}_1-E\left({\mathbf{X}}_1\right)\right){\left({\mathbf{X}}_1-E\left({\mathbf{X}}_1\right)\right)}^{\prime }\right]$$

$${\mathbf{F}}_{1|0}=E\left[\left({\mathbf{X}}_1-{\mathbf{\mu }}_{1|0}\right){\left({\mathbf{X}}_1-{\mathbf{\mu }}_{1|0}\right)}^{\prime }\right]$$

using the facts from (29(29) and (30)(30), we have

$${\mathbf{F}}_{1|0}=E\left[\left(\mathbf{\psi }{\mathbf{X}}_0+{\mathit{\Omega }}_1-\mathbf{\psi }{\hat{\mathbf{X}}}_0\right){\left(\mathbf{\psi }{\mathbf{X}}_0+{\mathit{\Omega }}_1-\mathbf{\psi }{\hat{\mathbf{X}}}_0\right)}^{\text{'}}\right]$$

that is

$${\mathbf{F}}_{1|0}=\mathbf{\psi }E\left(\left({\mathbf{X}}_0-{\hat{\mathbf{X}}}_0\right){\left({\mathbf{X}}_0-{\hat{\mathbf{X}}}_0\right)}^{\text{'}}\right){\mathbf{\psi }}^{\text{'}}+E\left({\mathit{\Omega }}_1{\mathit{\Omega }}_1^{\text{'}}\right)$$

therefore

$${\mathbf{\tau }}_{XX}={\mathbf{F}}_{1|0}=\mathbf{\psi }{\mathbf{F}}_0{\mathbf{\psi }}^{\prime }+\mathbf{Z}$$ (31)

It follows that ${\mathbf{X}}_1\sim N\left({\mathbf{\mu }}_{1|0},{\mathbf{F}}_{1|0}\right)$.

Note that (30(30) and (31)(31) are the prediction equation and prediction error of the state ${\mathbf{X}}_1$(at $t=0$) respectively.

And for the measurement equation at $t=0$, we have

$${\mathbf{Y}}_1=\mathbf{H}{\mathbf{X}}_1+{\mathbf{V}}_1$$ (32)

Since ${\mathbf{X}}_1\sim N\left({\mathbf{\mu }}_{1|0},{\mathbf{F}}_{1|0}\right)$, ${\mathbf{V}}_1\sim N\left(0,\mathbf{L}\right)$, ${\mathbf{X}}_1$ and ${\mathbf{V}}_1$ are independent, so ${\mathbf{Y}}_1$ is also distributed normal with mean and variance given below

$$E\left({\mathbf{Y}}_1\right)={\mathbf{\mu }}_{1|0}^y=\mathbf{H}E\left({\mathbf{X}}_1\right)=\mathbf{H}{\mathbf{\mu }}_{1|0}$$ (33)

and

$${\mathbf{\tau }}_{YY}=\mathrm{var}\left({\mathbf{X}}_1\right)=E\left[\left({\mathbf{Y}}_1-E\left({\mathbf{Y}}_1\right)\right){\left({\mathbf{Y}}_1-E\left({\mathbf{Y}}_1\right)\right)}^{\prime }\right]$$

using the facts from (32) and (33), we have

$${\mathbf{\tau }}_{YY}=E\left[\left(\mathbf{H}{\mathbf{X}}_1+{\mathbf{V}}_1-\mathbf{H}{\mathbf{\mu }}_{1|0}\right){\left(\mathbf{H}{\mathbf{X}}_1+{\mathbf{V}}_1-\mathbf{H}{\mathbf{\mu }}_{1|0}\right)}^{\prime }\right]$$

$${\mathbf{\tau }}_{YY}=\mathbf{H}E\left[\left({\mathbf{X}}_1-{\mathbf{\mu }}_{1|0}\right){\left({\mathbf{X}}_1-{\mathbf{\mu }}_{1|0}\right)}^{\prime }\right]{\mathbf{H}}^{\prime }+E\left({\mathbf{V}}_1{\mathbf{V}}_1^{\prime }\right)$$

therefore

$${\mathbf{\tau }}_{YY}=\mathbf{H}{\mathbf{F}}_{1|0}{\mathbf{H}}^{\prime }+\mathbf{L}$$ (34)

We can notice that (33(33) and (34)(34) are the prediction equation and error dispersion for the measurement equation ${\mathbf{Y}}_1$(at $t=0$) respectively.

Now, the next step is to find the covariance of ${\mathbf{X}}_1$ and ${\mathbf{Y}}_1$ to enable us update $\mathbf{J}$: (26) as well as (27(27) and (28)(28) i. e. the updating equations at $t=0$.Recall that

$$\begin{array}{l}\\ \begin{array}{ll}\mathrm{cov}\left({\mathbf{X}}_1,{\mathbf{Y}}_1\right)={\mathbf{\tau }}_{XY}\hfill & =E\left[\left({\mathbf{X}}_1-E\left({\mathbf{X}}_1\right)\right){\left({\mathbf{Y}}_1-E\left({\mathbf{Y}}_1\right)\right)}^{\prime }\right]\hfill \\ \hfill & =E\left[\left({\mathbf{X}}_1-{\mathbf{\mu }}_{1|0}\right){\left({\mathbf{Y}}_1-{\mathbf{\mu }}_{1|0}^y\right)}^{\prime }\right]\hfill \\ \hfill & =E\left[\left({\mathbf{X}}_1-{\mathbf{\mu }}_{1|0}\right){\left(\mathbf{H}{\mathbf{X}}_1+{\mathbf{V}}_1-\mathbf{H}{\mathbf{\mu }}_{1|0}\right)}^{\prime }\right]\hfill \\ \hfill & =E\left[\left({\mathbf{X}}_1-{\mathbf{\mu }}_{1|0}\right){\left({\mathbf{X}}_1-{\mathbf{\mu }}_{1|0}\right)}^{\prime }\right]{\mathbf{H}}^{\prime }+E\left({\mathbf{V}}_1^{\prime }\right)\hfill \end{array}\end{array}$$

therefore

$${\mathbf{\tau }}_{XY}={\mathbf{F}}_{1|0}{\mathbf{H}}^{\prime }$$ (35)

Hence, the update of $\mathbf{J}$ at $t=0$ is

$$\mathbf{J}=\left(\begin{array}{c}{\mathbf{X}}_1\\ {\mathbf{Y}}_1\end{array}\right)\sim {\mathbf{N}}_{n+m}\left(\left(\begin{array}{c}{\mathbf{\mu }}_{1|0}\\ \mathbf{H}{\mathbf{\mu }}_{1|0}\end{array}\right),\left(\begin{array}{cc}{\mathbf{F}}_{1|0}& {\mathbf{F}}_{1|0}{\mathbf{H}}^{\prime }\\ \mathbf{H}{\mathbf{F}}_{1|0}& \mathbf{H}{\mathbf{F}}_{1|0}{\mathbf{H}}^{\prime }+\mathbf{L}\end{array}\right)\right)$$ (36)

And the conditional distribution of ${\mathbf{X}}_1$ given ${\mathbf{Y}}_1$ is multivariate normal: ${\mathbf{X}}_1|{\mathbf{Y}}_1\sim \mathbf{N}\left({\mathbf{\mu }}_1,{\mathbf{F}}_1\right)$, where ${\mathbf{\mu }}_1$ and ${\mathbf{F}}_1$ are the update of (27(27) and (28)(28) at $t=0$ and are given below

$$\begin{array}{l}{\mathbf{\mu }}_1=E\left({\mathbf{X}}_1|{\mathbf{Y}}_1\right)={\mathbf{\mu }}_{1|0}+{\mathbf{F}}_{1|0}{\mathbf{H}}^{\text{'}}{\left(\mathbf{H}{\mathbf{F}}_{1|0}{\mathbf{H}}^{\text{'}}+\mathbf{L}\right)}^{-1}\left({\mathbf{Y}}_1-\mathbf{H}{\mathbf{\mu }}_{1|0}\right)\\ ={\mathbf{\mu }}_{1|0}+{\mathbf{K}}_1\left({\mathbf{Y}}_1-\mathbf{H}{\mathbf{\mu }}_{1|0}\right)\\ \end{array}$$ (37)

and

$$\begin{array}{ll}{\mathbf{F}}_1=\mathrm{var}\left({\mathbf{X}}_1|{\mathbf{Y}}_1\right)\hfill & ={\mathbf{F}}_{1|0}-{\mathbf{F}}_{1|0}{\mathbf{H}}^{\prime }{\left(\mathbf{H}{\mathbf{F}}_{1|0}{\mathbf{H}}^{\prime }+\mathbf{L}\right)}^{-1}\mathbf{H}{\mathbf{F}}_{1|0}\hfill \\ \hfill & ={\mathbf{F}}_{1|0}-{\mathbf{K}}_1\mathbf{H}{\mathbf{F}}_{1|0}\hfill \\ \hfill & =\left(\mathbf{I}-{\mathbf{K}}_1\mathbf{H}\right){\mathbf{F}}_{1|0}\hfill \end{array}$$ (38)

Note that (37(37) and (38)(38) are the updating equations of the Kalman filter at $t=0$. Similarly, the prediction equations: (30(30) and (31)(31) together with the updating equations: (37(37) and (38)(38) gives one complete Kalman filter's iteration also known as the Kalman recursion. Repeating the same process at $t=\mathrm{1,2},\mathrm{...},T$(where T is the number of observations) yields the Kalman recursions in Kalman filter literature.

To illustrate how the algorithms provided above will work sequentially, it is necessary to provide a flow chart for the algorithms of the system (SNSSM). The flow chart shows/demonstrate the relationship between case I and II of section 3, and how they (the cases) collectively relate with section 2.2. Similarly, the flow chart provided the criteria that will enable the Kalman recursions (iterations) commence. The flow chart is given in Figure 3.).

Figure 3.

Flow Chart for the Kalman Recursions of Symmetric Nonlinear State-Space Model.

In summary, it is customary to initializes/starts the filter with some arbitrary values (a prior information) say ${\mathbf{\mu }}_{0|0}$ and ${\mathbf{F}}_{0|0}$, use it to predicts ${\mathbf{Y}}_{1|0}$ and ${\mathbf{Q}}_{1|0}$; whenever the observation ${\mathbf{Y}}_1$ becomes available, it will be use in the updating equations and compute ${\mathbf{\mu }}_{1|1}$ and ${\mathbf{F}}_{1|1}$ which at the same time considered as a prior for the subsequent observation. This process completes one Kalman recursion. It is very important to note that the effect of initial prior ${\mathbf{\mu }}_{0|0}$ and ${\mathbf{F}}_{0|0}$ is decreasing with the increase of time $t$; (Yu, 2015), and (Tsay, 2010).

Declarations

Author contribution statement

M. Tasi'u: Conceived and designed the experiments; Wrote the paper.

H. G. Dikko: Performed the experiments.

O. I. Shittu: Analyzed and interpreted the data.

I. A. Fulatan: Contributed reagents, materials, analysis tools or data.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.