An MCMC computational approach for a continuous time state-dependent regime switching diffusion process

ABSTRACT

State-dependent regime switching diffusion processes or hybrid switching diffusion (HSD) processes are hard to simulate with classical methods which leads us to adopt a Markov chain Monte Carlo (MCMC) Bayesian approach very convenient to estimate complicated models such as the HSD one. In the HSD, the diffusion component is dependent on the switching discrete hidden regimes and the transition rates of the regime switching are dependent on the diffusion observations. Since in reality phenomena are only observed in discrete times, data imputation is called for to create more observations so as to have good approximations for the density of the diffusion process. Three categories of entities will be computed in a Bayesian context: The latent imputed observations, the regime switching states, and the parameters of the models. The latent imputed data is updated at random time intervals in block using a Metropolis Hastings algorithm. The switching states are computed by an adaptation of a forward filtering backward smoothing algorithm to the HSD model. The parameters are estimated after prior specifications and conditional posterior densities formulation using Gibbs sampler or Metropolis Hastings algorithm.

1. Introduction

Understanding phenomena dynamic such as disease progression is very challenging because it incorporates behavioural switching in continuous time; which requires us to apply a methodology that is suitable when data is observed at irregular times. Indeed, one should appeal to a rich class of continuous time models. Those models are usually taken to be diffusion processes such as the Ornstein–Uhlenbeck models [13]. Even though the usefulness of diffusion processes and the many advantages they can offer, they cannot handle all the dynamics caused by some complex phenomena. In many cases, phenomena change behaviour or evolution depending on intermediate indicators or variables. For example, in economics, the states can be the economic situations (growth, stability, crisis)and the developed variable can be options pricing or any economic factors that are changing attitude due to different economics situations. Also, in ecology [7], animal movement has been shown to change behaviour depending on the state of whether the animal is resting at a nest, foraging, or heading for a nest. Our modelling can be well present in disease progression such as HIV progression [23], where an observed bio-marker called CD4 counts decline is varying from one HIV disease stage to another. Thus, we recourse for diffusion processes but with the extension of regime switching. By adding the regime switching characteristics, the regime switching model (RSM) will be combining two components: a continuous diffusion observation component and an unobserved component most of the time discrete and supposed to be Markovian. Consequently, RSM is usually supposed to be Markovian and hence called Markov regime switching models. Their underlying idea is the switching mechanism who is supposed to be latent (unobserved or unknown). The simplest model supposes a two-dimensional Markov processes, where the first component is continuous and real valued and depending on the second, and the second has discrete values and acts monotonously from the first process. Such models have attracted a lot of attention in diverse fields or sciences such as in population dynamics [28], in pattern recognition [17], or in ecology [5,6]. However, in our case, we will go a little further and direct our attention toward a more advanced switching diffusion process that supposes the dependence of the rate of the switching or of the transition on the observations or covariates. A fundamental theoretical description and practical significance related to this modelling is provided in [4]. A recent example with the Bayesian estimation methodology is presented in [7], where animal movement is an adaptive movement since it depends and follows the habitat (resting or foraging). Those kinds of models are called hybrid switching diffusion (HSD) models and highlight the coexistence of continuous dynamics and discrete events: one component describes the continuous dynamics, whereas the other is a switching process representing discrete events with the switching part depending on the continuous dynamics. The exception to RSM is that the rates of transition for the discrete states in an HSD process depend on the continuous dynamics [20,27].

In this paper, we will provide a Markov chain Monte Carlo (MCMC) method to estimate the parameters of a HSD model. While we could find classical methods to compute the parameters of an RSM as in [22,24] where a maximum likelihood and expectation–maximization procedures are adopted, likelihood inference is very challenging for HSD. In fact, the diffusion component is non linear most of the time and most of the diffusion phenomena of practical interest are non-linear in their nature [32]. Furthermore, the transition rates of the regime switching are dependent on the diffusion observations [4,7]. Moreover, the observations are available only at scattered discrete times or are subject to measurement errors [21]. This leads us to impute data between successive observations to find an efficient approximation to the unavailable transition density of the HSD using an Euler approximation in the case of HSD as in [33]. The method of imputation aims at augmenting the likelihood with the imputed data. In the Bayesian context, the procedure alternates between updating the imputed data and updating the parameters. Unfortunately, the update of the imputed data can suffer from poor mixing such as the single site update of [16], or has a long mixing times of MCMC algorithm such as the block update of [14] if the number of imputation data is so large. The challenge of our modelling is that we have to augment the likelihood with the hidden states beside the imputed data and the parameters. Also, unlike regular state independent Markov switching diffusion process, our model supposes the dependence of the transition rate matrix on the diffusion process. Furthermore, instead of fixed times imputation, we opt for the algorithm of random time points update as described in [34] and applied to HSD in a Bayesian context in [7]. These will avoid any discretization error, where the times and the imputed data proposed are accepted or rejected using a Metropolis Hastings algorithm (MHA). Thus, we will appeal to the Bayesian inference through MCMC methods to sample the posterior distribution for the parameters, the latent imputed data and the hidden states for the uni-dimensional regime switching process since the discrete states of the switches are supposed unknown in this model. We will extend the Bayesian inference and data imputation of [19, Ch. 7] for non-linear diffusion to uni-dimensional HSD using the random time intervals for the imputation as in [7].

This paper is organized as follows: In Section 2, we describe the model considered. Section 3 provides the Bayesian inference for the latent imputed data, the hidden states, and the parameters including the parameters of transition rate matrix. In Section 4, we see more details on how priors are chosen and how posteriors densities are computed with an illustration by an important non-linear HSD process, and an application to disease progression. Finally, we give a discussion and a conclusion.

2. Model description and notations

We consider a continuous time HSP of the following form:

$$\begin{array}{rl}\mathrm{d}X\left(t\right)& =\mu \left(X\right(t),S(t\left)\right)\mathrm{d}t+\sigma \left(X\right(t),S(t\left)\right)\mathrm{d}{W}_t.\\ X\left(t\right)& =X_0\mathrm{a}\mathrm{n}\mathrm{d}S\left(t\right)=S_0,\end{array}$$ (1)

with $P\left(S\right(t+\delta t)=j|S(t)=i,X(s),S(s),s\le t)=q_{ij}\left(X\right(t\left)\right)\delta t+o\left(\delta t\right),i\ne j$, $\mu \left(\right)$ and $\sigma (\cdot )$ are appropriate real valued functions satisfying certain regularity conditions. W is a uni-dimensional standard Brownian motion. $S(\cdot )$ is a switching process taking discrete values in $\{1,\dots ,a\}$ with a is an integer and with the dynamics of $S(\cdot )$ depending on $X\left(t\right)$. Specifically, the spontaneous transition rate matrix $Q(x,t)=\left(q_{kl}\right(x,t){)}_{1\le k,l\le a}$ for the switching process has the following properties:

1. $q_{kl}(x,t)\ge 0$, for $k\ne l$, for any x and time t.

2. $\sum _l{q}_{kl}(x,t)=0$, for any x and time t.

Yin and Zhu [45] has summarized the difference between continuous state-dependent switching diffusion process and Markovian switching models in term of properties of solutions of the process and numerical procedures.

We suppose we have N individuals with $n_i$ the number of observation times for each individual i such that the trajectory for each individual i is given by $x_i=(x_{i1},\dots ,x_{i{n}_i})$. We consider $t_{ij}$ the times of observations for individual i, and the latent switching states will be denoted $s_{ij}$ for individual i at time $t_{ij}$. Time points are usually different between subjects, also $n_i$ may differ between individuals. Θ is the set of the parameters in the model, while θ is the set of all parameters except the parameters related to the transition rate matrix. Hence, by applying the Markov property and the conditional probability and Bayes rules, the likelihood could be written as

$$\begin{array}{rl}L(X,S;\mathrm{\Theta })& \propto \prod _{i=1}^N\left(P\right(x_{i1},s_{i1},\mathrm{\Theta }\left)\prod _{j=2}^{n_i}P\right(x_{ij},s_{ij}|x_{ij-1},s_{ij-1},\mathrm{\Theta })\\ & \propto \prod _{i=1}^N\left(P\right(x_{i1},s_{i1},\mathrm{\Theta }\left)\prod _{j=2}^{n_i}P\right(x_{ij}|x_{ij-1},s_{ij},s_{ij-1},\mathrm{\Theta }\left)P\right(s_{ij}|x_{ij-1},s_{ij-1},\mathrm{\Theta })\\ & \propto \prod _{i=1}^N\left(P\right(x_{i1},s_{i1},\mathrm{\Theta }\left)\prod _{j=2}^{n_i}P\right(x_{ij}|x_{ij-1},s_{ij-1},\mathrm{\Theta }\left)P\right(s_{ij}|x_{ij-1},s_{ij-1},\mathrm{\Theta })\\ & \mathrm{A}\mathrm{p}\mathrm{p}\mathrm{l}\mathrm{y}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{e}\mathrm{x}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{a}\mathrm{l}\mathrm{m}\mathrm{a}\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{x}\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{s}\mathrm{w}\mathrm{i}\mathrm{t}\mathrm{c}\mathrm{h}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{g}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{s}\\ L(X,S;\mathrm{\Theta })& \propto \prod _{i=1}^N\left(P\right(x_{i1},s_{i1},\mathrm{\Theta }\left)\prod _{j=2}^{n_i}P\right(x_{ij}|x_{ij-1},s_{ij-1},\mathrm{\Theta })\\ & \times [\exp (Q\left(x_{ij-1}\right)\mathrm{\Delta }{t}_{ij}){]}_{s_{ij-1},s_{ij}}\end{array}$$

$P(s_{ij}|x_{ij-1},s_{ij-1},\mathrm{\Theta })$ is the element $(s_{ij-1},s_{ij})$ of the transition probability matrix evaluated at the diffusion location $x_{ij-1}$. Since the transition matrix is obtained from the exponential of the generator Q evaluated at $x_{ij-1}$, and we obtain the elements $\left[\exp \right(Q\left(x_{ij-1}\right)\mathrm{\Delta }{t}_{ij}){]}_{s_{ij-1},s_{ij}}$, with $\mathrm{\Delta }{t}_{ij}=t_{ij}-t_{ij-1}$. This exponential approximation supposes constant $x_t$ for a short period of time $\mathrm{\Delta }{t}_{ij}$.

3. Bayesian inference

Direct estimation of HSD processes is very difficult because the transition density of the diffusion component is most of the time unavailable in a closed form and we need to approximate it. Thus, to overcome the issue of low frequency data and consequently the density approximation can be used, we need to introduce intermediate data between successive observations. Thus, we appeal to Bayesian data imputation [15,16,35]. Hence an $m_{ij}-1$ missing observations are imputed between successive observations $x_{ij}$ and $x_{ij-1}$ to obtain for each individual i a vector: $X_i=(x_{i1},X_{i1}^2,\dots ,X_{i1}^{m_{i1}},x_{i2},X_{i2}^2\dots ,x_{i{n}_{i-1}},X_{i{n}_{i-1}}^2,\dots ,X_{i{n}_{i-1}}^{m_{i{n}_i}},x_{i{n}_i})$, with $X_{ij}^1=x_{ij}$ and $X_{ij}^k$ will have corresponding $S_{ij}^k$ at the time point $t_{ij}^k$ (the kth imputation time after observation $x_{ij}$). Since we have three categories of variables to estimate: The parameters Θ, the latent switching state S and the imputed (latent) data X, our imputed joint posterior density is formulated as follows:

$$\begin{array}{rl}P(\mathrm{\Theta },X,S|x)& \propto P\left(\mathrm{\Theta }\right)\prod _{i=1}^N\left(P\right(X_{i1}^1,S_{i1}^1|\mathrm{\Theta })\\ & \times \prod _{j=1}^{n_i-1}P(X_{ij+1}^1|X_{ij}^{m_{ij}},S_{ij}^{m_{ij}},\mathrm{\Theta })[\exp (Q\left(X_{ij}^{m_{ij}}\right)\mathrm{\Delta }{t}_{ij+1}^1){]}_{S_{ij}^{m_{ij}},S_{ij+1}^1}\\ & \times \prod _{j=1}^{n_i-1}\prod _{k=1}^{m_{ij-1}}P(X_{ij}^{k+1}|X_{ij}^k,S_{ij}^k,\mathrm{\Theta })[\exp (Q\left(X_{ij}^k\right)\mathrm{\Delta }{t}_{ij}^{k+1}){]}_{S_{ij}^k,S_{ij}^{k+1}}.\end{array}$$

To evaluate the conditional posterior, we use a numerical Euler approximation [25] for the small time $t_{ij}^k$ (the kth imputation time between observation $x_{ij}$ and $x_{ij+1}$, with the exception that $\mathrm{\Delta }{t}_{ij+1}^1=t_{ij+1}^1-t_{ij}^{m_{ij}}$ and $t_{ij}^1$ is the time at the observation $x_{ij}$), and we have ${\pi }^{\mathrm{E}\mathrm{u}\mathrm{l}\mathrm{e}\mathrm{r}}(X_{ij}^{k+1}|X_{ij}^k,S,\mathrm{\Theta })\approx \phi (X_{ij}^{k+1}|X_{ij}^k+\mu (X_{ij}^k,S_{ij}^k,\mathrm{\Theta })\mathrm{\Delta }{t}_{ij}^{k+1},\sigma (X_{ij}^k,S_{ij}^k,\mathrm{\Theta }\left)\mathrm{\Delta }{t}_{ij}^{k+1}\right)$, with $S_{ij}^k$ the regime switching state value for individual i at time imputation $t_{ij}^k$, with $\mathrm{\Delta }{t}_{ij}^{k+1}=t_{ij}^{k+1}-t_{ij}^k$ and $\phi (z|\nu ,\mathrm{\Lambda })$ is a normal distribution with mean ν and variance Λ. $P\left(\mathrm{\Theta }\right)$ is the prior distribution for the set of parameters Θ. The MCMC algorithm will generate a Markov chain targeting the augmented posterior $P(\mathrm{\Theta },X,S|x)$ under some mild regularity conditions. In fact, the data augmentation procedure of [40] inference may proceed by alternating between the simulation of the parameters conditional on the augmented data, and the simulation of the augmented data given the observed data and the current state of the model parameters. A discussion of convergence issues related to non linear diffusion processes with latent data is provided in [21]. Such issues include the update choice for the imputed data and that the $m_{ij}$ should not be so large in order to avoid the slow mixing of the targeted chain in the MCMC algorithm. Hence, we perform our simulation by alternating between drawing from the following conditional posteriors given all other quantities (denoted by ·):

1. Draw the latent(imputed) observations from $P(X|\cdot )$

2. Draw the latent switching states from $P(S|\cdot )$

3. Draw the parameters from $P(\mathrm{\Theta }|\cdot )$.

3.1. Sampling the latent data

While it is possible to update each latent(imputed) data separately, we will adopt a block update for the whole sequence of imputed data for each individual observation $X_i$. Since it is difficult to come with an analytical form for the posterior distribution of the imputed data, an MHA [19, Ch. 7] is utilized to draw the new imputed data $X_i^{\mathrm{n}\mathrm{e}\mathrm{w}}$ from an old $X_i^{\mathrm{o}\mathrm{l}\mathrm{d}}$, for $i=1,\dots ,n$. Though for the imputation of auxiliary data, we opt for the algorithm of random time points update as described in [34] and applied to HSD in a Bayesian context in [7]. These will avoid any discretization error.

Suppose that the transition rates are all bounded above and let the times will be generated from a Poisson process with parameter $\kappa >\underset{i,x,t}{max}\{-Q_{ii}(x,t\left)\right\}$, and by the way we can use algorithm (1) to generate the times for each individual $i=1,\dots ,N$.

So by introducing $m_{ij}-1$ new observations between two successive observations $x_{ij-1}$ and $x_{ij}$, the MHA acceptance ratio will be

$$\zeta (X_i^{\mathrm{n}\mathrm{e}\mathrm{w}},T_i^{\mathrm{n}\mathrm{e}\mathrm{w}};X_i^{\mathrm{o}\mathrm{l}\mathrm{d}},T_i^{\mathrm{o}\mathrm{l}\mathrm{d}})=1\wedge \frac{P(X_i^{\mathrm{n}\mathrm{e}\mathrm{w}}|D,S,\mathrm{\Theta })\psi (X_i^{\mathrm{o}\mathrm{l}\mathrm{d}}|X_i^{\mathrm{n}\mathrm{e}\mathrm{w}},D,S,\mathrm{\Theta })}{P(X_i^{\mathrm{o}\mathrm{l}\mathrm{d}}|D,S,\mathrm{\Theta })\psi (X_i^{\mathrm{n}\mathrm{e}\mathrm{w}}|X_i^{\mathrm{o}\mathrm{l}\mathrm{d}},D,S,\mathrm{\Theta })}.$$

Where P denote the posterior density, ψ the proposal density, and D the observed, not the imputed data. Due to the Markov property in HSD, we have

$$\begin{array}{rl}& \frac{P(X_i^{\mathrm{n}\mathrm{e}\mathrm{w}}|D,S,\mathrm{\Theta })}{P(X_i^{\mathrm{o}\mathrm{l}\mathrm{d}}|D,S,\mathrm{\Theta })}\\ & =\prod _{j=1}^{n_i-1}\frac{P(X_{ij+1}^1|X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},m_{ij}},S,\mathrm{\Theta })}{P(X_{ij+1}^1|X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},m_{ij}},S,\mathrm{\Theta })}\prod _{j=1}^{n_i-1}\prod _{k=1}^{m_{ij-1}}\frac{P(X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k+1}|X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k},S,\mathrm{\Theta })}{P(X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},k+1}|X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},k},S,\mathrm{\Theta })}\\ & =\prod _{j=1}^{n_i-1}\frac{P(\mathrm{\Delta }{t}_{ij+1}^{\mathrm{n}\mathrm{e}\mathrm{w},1},X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},m_{ij}},X_{ij+1}^1)}{P(\mathrm{\Delta }{t}_{ij+1}^{\mathrm{o}\mathrm{l}\mathrm{d},1},X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},m_{ij}},X_{ij+1}^1)}\prod _{j=1}^{n_i-1}\prod _{k=1}^{m_{ij-1}}\frac{P(\mathrm{\Delta }{t}_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k+1},X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k},X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k+1})}{P(\mathrm{\Delta }{t}_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},k+1},X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},k},X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},k+1})}\end{array}$$

$\mathrm{\Delta }{t}_{ij}^{k+1}=t_{ij}^{k+1}-t_{ij}^k$ is very small to permit an Euler approximation for P by

$$P^{\mathrm{E}\mathrm{u}\mathrm{l}\mathrm{e}\mathrm{r}}(X_{ij}^{k+1}|X_{ij}^k,S,\mathrm{\Theta })\approx \phi (X_{ij}^{k+1}|X_{ij}^k+\mu (X_{ij}^k,S_{ij}^k,\mathrm{\Theta })\mathrm{\Delta }{t}_{ij}^{k+1},\sigma (X_{ij}^k,S_{ij}^k,\mathrm{\Theta }\left)\mathrm{\Delta }{t}_{ij}^{k+1}\right),$$

with $S_{ij}^k$ the regime switching state value for individual i at time imputation $t_{ij}^k$, and $\phi (z|\nu ,\mathrm{\Lambda })$ is normal distribution with mean ν and variance Λ.

While to the choice of the proposal density ψ, one could choose an Euler proposal or a double-sided Euler proposal, but due to the dependency between $X^{\mathrm{n}\mathrm{e}\mathrm{w}}$ and $X^{\mathrm{o}\mathrm{l}\mathrm{d}}$, we will adopt the modified bridge proposal ${\psi }_{MB}$ of [14]; supposed to overcome the issue of dependence between successive draws; as adopted in the Bayesian context by [11]. Note that $X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},1}=X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},1}=x_{ij}$ and $S_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},1}=S_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},1}$, and the proposed path will be accepted with probability:

$$\begin{array}{rl}& \zeta (X_i^{\mathrm{n}\mathrm{e}\mathrm{w}},T_i^{\mathrm{n}\mathrm{e}\mathrm{w}};X_i^{\mathrm{o}\mathrm{l}\mathrm{d}},T_i^{\mathrm{o}\mathrm{l}\mathrm{d}})=1\wedge \left(\prod _{j=1}^{n_i-1}\frac{P(X_{ij+1}^1|X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},m_{ij}},S,\mathrm{\Theta })}{P(X_{ij+1}^1|X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},m_{ij}},S,\mathrm{\Theta })}\right)\\ & \times \left(\prod _{j=1}^{n_i-1}\prod _{k=1}^{m_{ij-1}}\frac{P(X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k+1}|X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k},S,\mathrm{\Theta })}{P(X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},k+1}|X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},k},S,\mathrm{\Theta })}\right)\times \frac{{\psi }_{MB}(X_i^{\mathrm{o}\mathrm{l}\mathrm{d}}|X_{i1},X_{i{n}_i},S,\mathrm{\Theta })}{{\psi }_{MB}(X_i^{\mathrm{n}\mathrm{e}\mathrm{w}}|X_{i1},X_{i{n}_i},S,\mathrm{\Theta })},\end{array}$$ (2)

where

$$\frac{{\psi }_{MB}(X_i^{\mathrm{o}\mathrm{l}\mathrm{d}}|X_{i1},X_{i{n}_i},S,\mathrm{\Theta })}{{\psi }_{MB}(X_i^{\mathrm{n}\mathrm{e}\mathrm{w}}|X_{i1},X_{i{n}_i},S,\mathrm{\Theta })}=\prod _{j=1}^{n_i-1}\prod _{k=1}^{m_{ij-1}}\frac{{\psi }_{MB}(X_{ij}^{\mathrm{o}\mathrm{l}\mathrm{d},k+1}|X_{ij}^1,X_{ij+1}^1,S,\mathrm{\Theta })}{{\psi }_{MB}(X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k+1}|X_{ij}^1,X_{ij+1}^1,S,\mathrm{\Theta })}$$

with

$$\begin{array}{rl}& {\psi }_{MB}(X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k+1}|X_{ij}^1,X_{ij+1}^1,S,\mathrm{\Theta })\\ & =\phi (X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k+1}|X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k}+\frac{X_{ij}^{m_{ij}}-X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k}}{t_{ij+1}^{\mathrm{n}\mathrm{e}\mathrm{w},1}-t_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k}}\mathrm{\Delta }{t}_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k+1},\dots \\ & \frac{t_{ij+1}^{\mathrm{n}\mathrm{e}\mathrm{w},1}-t_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k+1}}{t_{ij+1}^{\mathrm{n}\mathrm{e}\mathrm{w},1}-t_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k}}\sigma (X_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k},S_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k},\mathrm{\Theta })\mathrm{\Delta }{t}_{ij}^{\mathrm{n}\mathrm{e}\mathrm{w},k+1}).\end{array}$$

3.2. Sampling the switching hidden states

Instead of simulating each hidden state separately, Chib [9] developed a method called block update of the hidden states in which we simulate the full latent data for each individual i, $i=1,2,\dots ,N$. Its algorithm was adapted to a multivariate autoregressive hidden Markov model in [30]. We will adapt this algorithm to the HSD process, with the modification that our model suppose observations with different lengths and non-equi-distant intervals. For clarity of representation and for good understanding of this so called forward filtering backward smoothing (FFBS) algorithm, we stack the imputed observations with the observations in one vector of size $N_i$ (The staked data contains both the observation data $x_i^{″}$ and the imputed terms $X_{ij}^k$ to give the new stacked vector $\widetilde{X_i}$) ${\tilde{X}}_{i.}=({\tilde{X}}_{i1},\dots ,{\tilde{X}}_{i{N}_i})$, with the corresponding switching hidden vector ${\tilde{S}}_{i.}=({\tilde{S}}_{i1},\dots ,{\tilde{S}}_{i{N}_i})$, and the times ${\tilde{t}}_{i.}=({\tilde{t}}_{i1},\dots ,{\tilde{t}}_{i{N}_i})$. The times contains both the original as well as the new times staked in a new vector and we use the FFBS to sample the whole sequence of the hidden states. A similar procedure is described in [34]. Let denote ${\tilde{X}}_i^{-j}=({\tilde{X}}_{i1},\dots ,{\tilde{X}}_{ij})$, ${\tilde{X}}_i^j=({\tilde{X}}_{ij},\dots ,{\tilde{X}}_{i{N}_i})$, ${\tilde{S}}_i^{-j}=({\tilde{S}}_{i1},\dots ,{\tilde{S}}_{ij})$, and ${\tilde{S}}_i^j=({\tilde{S}}_{ij},\dots ,{\tilde{S}}_{i{N}_i})$. Now, we write the joint conditional distribution for the hidden states as

$$P({\tilde{S}}_{i.}|{\tilde{X}}_{i.},\mathrm{\Theta })=\prod _{j=1}^{N_i}P({\tilde{S}}_{ij}|{\tilde{X}}_{i.},{\tilde{S}}_i^{j+1}).$$

Hence the states computation is based on the term $P({\tilde{S}}_{ij}|{\tilde{X}}_{i.},{\tilde{S}}_i^{j+1})$ which will be evaluated in the backward pass after running the forward filtering as detailled in algorithm (2).

Again, let's remind that in the HSD model, the transition rates are related to the observed diffusion component.

3.3. Sampling from $P(\theta |\tilde{X},\tilde{S})$

Given the estimation of $\tilde{X}$ and $\tilde{S}$ in the previous subsection, the conditional posterior of θ is proportional to the prior $P\left(\mathrm{\Theta }\right)$ multiplied by the likelihood, which gives

$$\begin{array}{rl}& P(\theta |\tilde{X},\tilde{S})\propto P\left(\theta \right)P(X_{i1}^1,S_{i1}^1|\mathrm{\Theta })\\ & \times \prod _{i=1}^N\prod _{j=1}^{N_i-1}P({\tilde{X}}_{ij+1}|{\tilde{X}}_{ij},{\tilde{S}}_{ij},\theta )[\exp (Q\left({\tilde{X}}_{ij}\right){\mathrm{\Delta }\tilde{t}}_{ij+1}){]}_{{\tilde{S}}_{ij},{\tilde{S}}_{ij+1}}\end{array}$$

Usually, we do not come with a standard distribution to sample from by using Gibbs sampler, thus we will appeal to an MHA. After using proportionality and dropping the terms of the transition rate matrix since they are independent of θ. Hence, we come with an MHA [19, Ch. 7] with an acceptance probability ratio for every new proposal ${\theta }^{\ast }$ as

$$\zeta (\theta ,{\theta }^{\ast })=1\wedge \frac{P\left({\theta }^{\ast }\right)}{P\left(\theta \right)}\times \left(\frac{\prod _{i=1}^N\prod _{j=1}^{N_i-1}P({\tilde{X}}_{ij+1}|{\tilde{X}}_{ij},{\tilde{S}}_{ij},{\theta }^{\ast })}{\prod _{i=1}^N\prod _{j=1}^{N_i-1}P({\tilde{X}}_{ij+1}|{\tilde{X}}_{ij},{\tilde{S}}_{ij},\theta )}\right)\times \frac{\psi (\theta |{\theta }^{\ast },\tilde{X},\tilde{S})}{\psi ({\theta }^{\ast }|\theta ,\tilde{X},\tilde{S})}.$$

As before, using Euler approximation we have $P({\tilde{X}}_{ij+1}|{\tilde{X}}_{ij},{\tilde{S}}_{ij},\theta )=P^{\mathrm{E}\mathrm{u}\mathrm{l}\mathrm{e}\mathrm{r}}({\tilde{X}}_{ij+1}|{\tilde{X}}_{ij},S,\theta )\approx \phi ({\tilde{X}}_{ij+1}|{\tilde{X}}_{ij+1}+\mu ({\tilde{X}}_{ij+1},{\tilde{S}}_{ij},\theta )\mathrm{\Delta }{\tilde{t}}_{ij},\sigma ({\tilde{X}}_{ij},{\tilde{S}}_{ij},\theta \left)\mathrm{\Delta }{\tilde{t}}_{ij+1}\right)$, ψ is a proposal density to draw a new ${\mathrm{\Theta }}^{\ast }$, it depends on the form of the likelihood and the hypothesis of the HSD. Some times, we could use simply a random walk proposal which is independent of $\tilde{X}$ and $\tilde{S}$ and it is only related to the old draw ${\mathrm{\Theta }}^{\left(\mathrm{o}\mathrm{l}\mathrm{d}\right)}$, and we simply propose from a gaussian distribution for some ${\theta }_j^{\mathrm{n}\mathrm{e}\mathrm{w}}\sim \mathcal{N}({\theta }_j^{\mathrm{o}\mathrm{l}\mathrm{d}},ϵ)$. ε is the random walk step that can be adjusted to improve convergence. For positive values, we could propose from the log-normal distribution: $\log {\theta }_j^{\mathrm{n}\mathrm{e}\mathrm{w}}\sim \mathcal{L}\mathcal{N}(\log {\theta }_j^{\mathrm{o}\mathrm{l}\mathrm{d}},ϵ)$.

Finally, and since we have a HSD, the parameters will be dependent on the switching process $\tilde{S}$, so for every given parameter ${\theta }_j$, we will have to estimate $({\theta }_j^1,\dots ,{\theta }_j^k,\dots ,{\theta }_j^a)$, and the posterior for ${\theta }_j^k$ is

$$P({\theta }_j^k|\tilde{X},\tilde{S})\propto P\left(\theta \right)\times \prod _{i=1}^N\prod _{j=1,{\tilde{S}}_{ij}=k}^{N_i-1}P({\tilde{X}}_{ij+1}|{\tilde{X}}_{ij},{\tilde{S}}_{ij},\theta ).$$ (3)

4. Numerical example with application to disease progression

We will consider here an example of a non-linear HSD process and give more details on how we can estimate each parameter of the model. In fact, a non-linear drift and a non-linear volatility would allow the HSD process to dispose of elements to represent clearly any complex problem and to be more flexible. Considering the non-linear process of [2] with an extra addition of the regime switching and we get in our case $\mathrm{d}{\tilde{X}}_{ij}=[{\alpha }_{0,k}+{\alpha }_{1,k}{\tilde{X}}_{ij}+{\alpha }_{2,k}{\tilde{X}}_{ij}^2+{\alpha }_{3,k}/{\tilde{X}}_{ij}]\mathrm{d}{\tilde{t}}_{ij+1}+\sigma {{\tilde{X}}_{ij}}^{{\eta }_k}\mathrm{d}W$, for the hidden switching state $k\in \{1,\dots ,a\}$. This general form with the regime switching gives many explanations for majority of phenomena, where a negative ${\alpha }_{2,k}$ guarantees ergodicity and second order stationarity for a volatility function ${\sigma }_k\left({\tilde{X}}_{ij}\right)=\sigma {{\tilde{X}}_{ij}}^{{\eta }_k}$. Than the first order Euler approximation of this model for a state ${\tilde{S}}_{ij}=k$:

$${\tilde{X}}_{ij+1}-{\tilde{X}}_{ij}=\left[{\alpha }_{0,k}+{\alpha }_{1,k}{\tilde{X}}_{ij}+{\alpha }_{2,k}{\tilde{X}}_{ij}^2+\frac{{\alpha }_{3,k}}{{\tilde{X}}_{ij}}\right]\mathrm{\Delta }{\tilde{t}}_{ij+1}+\sigma {{\tilde{X}}_{ij}}^{{\eta }_k}\sqrt{\mathrm{\Delta }{\tilde{t}}_{ij+1}}ϵ,ϵ\sim \mathcal{N}(0,1).$$ (4)

4.1. Computation of ${\alpha }_k$ and ${\sigma }^2$

To simulate our parameters ${\alpha }_k$ and ${\sigma }^2$ using the MCMC algorithm, first let mention that when the posterior density is not known we have to use an MHA as described in parameters update (3), otherwise if we come up with a known posterior density to draw from it directly using Gibbs sampler as it is the case here. For this reason, let's pose $Y_{ij}=({\tilde{X}}_{ij+1}-{\tilde{X}}_{ij})/{{\tilde{X}}_{ij}}^{{\eta }_k}$, ${\mathrm{\text{β}}}_k=({\alpha }_{0,k},{\alpha }_{1,k},{\alpha }_{2,k},{\alpha }_{0,k})$ and

$$y_{ij}=\left(\frac{\sqrt{\mathrm{\Delta }{\tilde{t}}_{ij+1}}}{{{\tilde{X}}_{ij}}^{{\eta }_k}},\frac{{\tilde{X}}_{ij}\sqrt{\mathrm{\Delta }{\tilde{t}}_{ij+1}}}{{{\tilde{X}}_{ij}}^{{\eta }_k}},\frac{{\tilde{X}}_{ij}^2\sqrt{\mathrm{\Delta }{\tilde{t}}_{ij+1}}}{{{\tilde{X}}_{ij}}^{{\eta }_k}},\frac{\left(1|{\tilde{X}}_{ij}\right)\sqrt{\mathrm{\Delta }{\tilde{t}}_{ij+1}}}{{{\tilde{X}}_{ij}}^{{\eta }_k}}\right).$$

In matrix form, the Euler discretization may be represented as in [38]: $Y=y{\mathrm{\text{β}}}_k+ϵ$, with $ϵ\sim \mathcal{N}(0,{\sigma }^2)$, which is the formulation for a regression model. Consequently, the parameters of the HSD can be easily computed using the Bayesian approach for regime switching regression model. Hence for each ${\mathrm{\text{β}}}_k,k=1,\dots ,a$, the posterior is proportional to the prior multiplied by the likelihood

$$P({\mathrm{\text{β}}}_k|\tilde{X},\tilde{S})\propto P\left(\mathrm{\text{β}}\right)\times \prod _{i=1}^N\prod _{j=1,{\tilde{S}}_{ij}=k}^{N_i-1}P({\tilde{X}}_{ij+1}|{\tilde{X}}_{ij},{\tilde{S}}_{ij},\mathrm{\Theta }).$$

As in [18, p:251], by supposing a normal and an inverse gamma prior respectively for ${\mathrm{\text{β}}}_k$ and ${\sigma }^2$: Consequently with a conjugate normal prior for ${\mathrm{\text{β}}}_k\sim {\mathcal{N}}_4(b_{0k},{\sigma }^2{B}_{0K})$, we have ${\mathrm{\text{β}}}_k|\cdot \sim {\mathcal{N}}_4(b_k,B_K)$, where $B_k=(B_{0K}^{-1}+y^{\prime }y{)}^{-1}$ and $b_K=B_k(B_{0K}^{-1}{b}_{0k}+y_k^{\mathrm{\prime }}{Y}_k)$; $Y_k$ and $y_k$ corresponds only to the observations where ${\tilde{S}}_{ij}=k$.

Similarly, under an inverted conjugate prior for ${\sigma }^2\sim \mathcal{I}\mathcal{G}(c_0,C_0)$, the posterior density for ${\sigma }^2$ given the observations, the imputed data, the other parameters and the switching states is ${\sigma }^2|.\sim \mathcal{I}\mathcal{G}(c,C)$, where $c=(c_0+M/2)$, and

$$C=\left(C_0+\frac{1}{2}\sum _{k=1}^a{b}_{0k}^{\mathrm{\prime }}{B}_{0K}^{-1}{b}_{0k}+\frac{1}{2}\sum _{k=1}^a(y_k^{\mathrm{\prime }}{y}_k-b_k^{\mathrm{\prime }}{B}_K^{-1}{b}_k)\right),$$

M is the number for all individuals observations.

Ridge regression: One of the problem that can be faced here is the effects of multi-collinearity on the Bayesian regression estimation. Consequently, we recourse to the Bayesian Ridge Regression approach as in [43] who supposes the same priors as earlier for ${\mathrm{\text{β}}}_k$ and ${\sigma }^2$ and we get again similar posteriors as before, the only difference is that in the ridge regression we have $B_{0K}=Diagonal(R_1,R_2,R_3,R_4)$. Moreover, to overcome the issue of choosing fixed value for $R_j$; we suppose the prior $P(R_1,R_2,R_3,R_4)\sim \prod _{j=1}^4{R}_j^{A/2-1}\exp (-(b/2\left)R_j\right)$. Hence, we alternate between updating the following posteriors, for each $k=1,\dots ,a$:

$$\begin{array}{rl}{R}_{jj}& \sim Gamma\left(\frac{A+M_k}{2},\frac{b+{\mathrm{\text{β}}}_j^2}{2}\right),M_k:\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{o}\mathrm{b}\mathrm{s}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}{\tilde{S}}_{ij}=k.\\ B_{0K}& =\mathrm{D}\mathrm{i}\mathrm{a}\mathrm{g}(R_1,R_2,R_3,R_4).\\ {\mathrm{\text{β}}}_k& \sim {\mathcal{N}}_4(b_k,B_K).\end{array}$$ (5)

$$\begin{array}{rl}{\sigma }^2& \sim \mathcal{I}\mathcal{G}(c,C).\end{array}$$ (6)

4.2. Sampling the posterior dependent parameters on the transition rate matrix Q

In this example, we will suppose that the intensities of the transition rate matrix are dependent on the observed diffusion process through the relation of Gompertz model as in [44]. The consideration of the Gompertz model comes from the fact that this function has been used longer in insurance, in biology such as tumour evolution or bacteria growth and in many other fields [41]. Also the Gompertz model has interpretable parameters, and we have $q_{kl}\left({\tilde{X}}_{ij}\right)={\lambda }_{kl}\exp (-{\gamma }_{kl}{\tilde{X}}_{ij})$, ${\lambda }_{kl},{\gamma }_{kl}>0$, for $k\ne l\in \{1,\dots ,a\},i=1,\dots ,N$; and $j=1,\dots ,N_i$. We will suppose prior independence between the parameters of Θ. While the λ's and the γ's can be updated in block as in [29], here we adopt an approach similar to [37], where each element is updated conditional on the other since they are correlated. Hence by supposing a Gamma prior $\mathcal{G}(0.01,0.01)$ for the ${\lambda }_{kl}$, the posterior of ${\lambda }_{kl}$ will be

$$P({\lambda }_{kl}|\tilde{X},\tilde{S})\propto P\left({\lambda }_{kl}\right)\prod _{i=1}^N\prod _{j=1}^{N_i-1}[\exp (Q\left({\tilde{X}}_{ij}\right){\mathrm{\Delta }\tilde{t}}_{ij+1}){]}_{{\tilde{S}}_{ij},{\tilde{S}}_{ij+1}}.$$

As we can see, all the other parameters especially θ are omitted due to the Bayes rule. Consequently, we obtain a non-standard posterior and we have to adopt a random walk MHA to draw a new ${\lambda }_{kl}^{\mathrm{n}\mathrm{e}\mathrm{w}}$ (the new MCMC iteration) from an old value ${\lambda }_{kl}^{\mathrm{o}\mathrm{l}\mathrm{d}}$ (the previous MCMC iteration) with an acceptance probability:

$$\zeta ({\lambda }_{kl}^{\mathrm{n}\mathrm{e}\mathrm{w}},{\lambda }_{kl}^{\mathrm{o}\mathrm{l}\mathrm{d}})=1\wedge \left(\frac{P\left({\lambda }_{kl}^{\mathrm{n}\mathrm{e}\mathrm{w}}\right)\prod _{i=1}^N\prod _{j=1}^{N_i-1}[\exp (Q\left({\tilde{X}}_{ij}\right){\mathrm{\Delta }\tilde{t}}_{ij+1}){]}_{{\tilde{S}}_{ij},{\tilde{S}}_{ij+1}}^{\mathrm{n}\mathrm{e}\mathrm{w}}}{P\left({\lambda }_{kl}^{\mathrm{o}\mathrm{l}\mathrm{d}}\right)\prod _{i=1}^N\prod _{j=1}^{N_i-1}[\exp (Q\left({\tilde{X}}_{ij}\right){\mathrm{\Delta }\tilde{t}}_{ij+1}){]}_{{\tilde{S}}_{ij},{\tilde{S}}_{ij+1}}^{\mathrm{o}\mathrm{l}\mathrm{d}}}\right)\times \frac{{\lambda }_{kl}^{\mathrm{n}\mathrm{e}\mathrm{w}}}{{\lambda }_{kl}^{\mathrm{o}\mathrm{l}\mathrm{d}}},$$ (7)

where we propose the ${\lambda }_{kl}^{\mathrm{n}\mathrm{e}\mathrm{w}}$ from a log-normal distribution $\mathcal{L}\mathcal{N}({\lambda }_{kl}^{\mathrm{o}\mathrm{l}\mathrm{d}},{ϵ}_{\lambda })$ so as to keep operating on real positive values with ${ϵ}_{\lambda }$ the random walk step. Similarly, a no close form is obtained for the posterior of ${\gamma }_{kl}$. So, we consider again a random walk MHA. With a Gamma prior $\mathcal{G}(0.01,0.01)$ on ${\gamma }_{kl}$ and a random walk proposal $\mathcal{L}\mathcal{N}({\gamma }_{kl}^{\mathrm{o}\mathrm{l}\mathrm{d}},{ϵ}_{\gamma })$, our MHA acceptance probability is:

$$\zeta ({\gamma }_{kl}^{\mathrm{n}\mathrm{e}\mathrm{w}},{\gamma }_{kl}^{\mathrm{o}\mathrm{l}\mathrm{d}})=1\wedge \left(\frac{P\left({\gamma }_{kl}^{\mathrm{n}\mathrm{e}\mathrm{w}}\right)\prod _{i=1}^N\prod _{j=1}^{N_i-1}[\exp (Q\left({\tilde{X}}_{ij}\right){\mathrm{\Delta }\tilde{t}}_{ij+1}){]}_{{\tilde{S}}_{ij},{\tilde{S}}_{ij+1}}^{\mathrm{n}\mathrm{e}\mathrm{w}}}{P\left({\gamma }_{kl}^{\mathrm{o}\mathrm{l}\mathrm{d}}\right)\prod _{i=1}^N\prod _{j=1}^{N_i-1}{\left[\exp \left(Q\right({\tilde{X}}_{ij}\left){\mathrm{\Delta }\tilde{t}}_{ij+1}\right)\right]}_{{\tilde{S}}_{ij},{\tilde{S}}_{ij+1}}^{\mathrm{o}\mathrm{l}\mathrm{d}}}\right)\times \frac{{\gamma }_{kl}^{\mathrm{n}\mathrm{e}\mathrm{w}}}{{\gamma }_{kl}^{\mathrm{o}\mathrm{l}\mathrm{d}}}.$$ (8)

4.3. Computation of ${\eta }_k$

To compute ${\eta }_k$, the posterior is proportional to the prior multiplied by the likelihood. By supposing a Gamma prior $\mathcal{G}(0.01,0.01)$, we come with a non-standard posterior, and we call for random walk MHA with the acceptance probability:

$$\zeta ({\eta }_k^{\mathrm{n}\mathrm{e}\mathrm{w}},{\eta }_k^{\mathrm{o}\mathrm{l}\mathrm{d}})=1\wedge \left(\frac{P\left({\eta }_k^{\mathrm{n}\mathrm{e}\mathrm{w}}\right)\prod _{i=1}^N\prod _{j=1,{\tilde{S}}_{ij}=k}^{N_i-1}P({\tilde{X}}_{ij+1}|{\tilde{X}}_{ij},{\tilde{S}}_{ij},{\eta }_k^{\mathrm{n}\mathrm{e}\mathrm{w}},{\mathrm{\Theta }}_{-{\eta }_k})}{P\left({\eta }_k^{\mathrm{o}\mathrm{l}\mathrm{d}}\right)\prod _{i=1}^N\prod _{j=1,{\tilde{S}}_{ij}=k}^{N_i-1}P({\tilde{X}}_{ij+1}|{\tilde{X}}_{ij},{\tilde{S}}_{ij},{\eta }_k^{\mathrm{n}\mathrm{e}\mathrm{w}},{\mathrm{\Theta }}_{-{\eta }_k})}\right)\times \frac{{\eta }_k^{\mathrm{n}\mathrm{e}\mathrm{w}}}{{\eta }_k^{\mathrm{o}\mathrm{l}\mathrm{d}}}.$$ (9)

${\mathrm{\Theta }}_{-{\eta }_k}$ represent all the parameters except ${\eta }_k$. Each ${\eta }_k$ must be positive. As before we propose from a log-normal distribution $\mathcal{L}\mathcal{N}({\eta }_k^{\mathrm{o}\mathrm{l}\mathrm{d}},{ϵ}_{\eta })$ with step ${ϵ}_{\eta }$ chosen concisely.

4.4. Numerical implementation and simulation

To assess the accuracy of our finding, we will simulate observations for N = 50 individuals with follow up size between 16 and 20 for every individual. We suppose we have a = 3 regime switching states. the parameters of the non-linear HSD are ${\alpha }_0=(0.3,0.6,0.9)$, ${\alpha }_1=(0.02,0.04,0.06)$, ${\alpha }_2=(-0.08,-0.06,-0.04)$, ${\alpha }_3=(0.01,0.05,0.09)$, $\eta =(0.4,0.5,0.6)$, ${\sigma }^2=0.04$, and

$$\lambda =\left(\begin{array}{ccc}0& 0.2& 0.4\\ 2& 0& 3\\ 0.3& 0.9& 0\end{array}\right),\gamma =\left(\begin{array}{ccc}0& 0.2& 0.04\\ 0.01& 0& 4\\ 1& 0.8& 0\end{array}\right).$$

Algorithm (3) provides the data generation process for the HSD model, that works with the help of the Euler approximation. This algorithm is applied for every individual i for $i=1,\dots ,50$.

To assess the efficiency of our methods, we will see how our MCMC algorithm can estimate the true values (values used to generate the simulated data). Before providing the MCMC algorithm, we should point out that we do not opt for the usual regularly spaced points imputation procedure that can drive the Bayesian estimation to break down if the amount of imputation is large [21]. In fact, it has been shown dependence between the unknown parameters in the diffusion and the missing data while adopting this imputation. This can result in slow rates of convergence of naive sampling or could conduct to identifiability problem as in the single update of [16] or the block update of [14]. Thus we call for random time imputation that allow exact estimation as in [7]. After the generation of the simulated data and to check the accuracy of the estimation, we ran the MCMC algorithm (see algorithm (4)) for a large number of iterations. During this ran there is a burnings period (where the algorithm has not converged yet). The burnings period varies depending on the volatility complexity, the number of parameters in the model, or the number of imputation as well as the latent data. After the burnings period, the MCMC converges and the inference is based on the last iterations of the MCMC algorithm. Our algorithm proceeds for 8000 iterations (number of iterations found in our case to be convenient for the MCMC to converge so as we can draw inference from the MCMC output) as follow:

Table 1 gives the posterior statistics of our algorithm such as the posterior mean, the standard deviation and the $95\mathrm{\%}$ credible intervals, where estimations are of good approximation to the true values even though that this complicated process has a large number of parameters. Consequently, parameter estimations have been rendered very simple. Moreover, the convergence issues have been checked for every parameter of the model by the graphical inspection of the trace-plot, the kernel density, and the autocorrelation function plot. It is revealed that we get a good mixing of the MCMC chain (Figure 1), a perfect density shape (Figure 2) and autocorrelations that decay immediately after a few lag (Figure 3). Hence convergence for this MCMC method is achieved. In fact, the MCMC methods show that they are appropriate especially that we have opted for random time imputation. Our algorithm works well if we choose different values for this switching one factor model, or ran a bootstrap simulation. Hence, the algorithm was efficient in approximating the true values.

Table 1. MCMC estimation for the parameters of the model (8000 iterations).

Posterior computations
Parameters	True value	Mean	Standard deviation	Credible interval (95%)
${\alpha }_{0,1}$	0.3	0.3026	0.0011	$(0.3004,0.3048)$
${\alpha }_{0,2}$	0.6	0.6028	0.0201	$(0.5636,0.6426)$
${\alpha }_{0,3}$	0.9	0.9048	0.0047	$(0.8956,0.9140)$
${\alpha }_{1,1}$	0.02	0.0211	$3.5015\times {10}^{-04}$	$(0.0204,0.0218)$
${\alpha }_{1,2}$	0.04	0.0401	0.0036	$(0.0331,0.0472)$
${\alpha }_{1,3}$	0.06	0.0602	$6.0245\times {10}^{-04}$	$(0.0590,0.0614)$
${\alpha }_{2,1}$	−0.08	−0.0722	$6.2750\times {10}^{-04}$	$(-0.0734,-0.0710)$
${\alpha }_{2,2}$	−0.06	−0.0593	0.0038	$(-0.0671,-0.0522)$
${\alpha }_{2,3}$	−0.04	−0.0395	$5.5752\times {10}^{-04}$	$(-0.0406,-0.0384)$
${\alpha }_{3,1}$	0.01	0.0101	$3.4369\times {10}^{-04}$	$(0.0094,0.0107)$
${\alpha }_{3,2}$	0.05	0.0503	0.0037	$(0.0431,0.0577)$
${\alpha }_{3,3}$	0.09	0.0901	$7.1083\times {10}^{-04}$	$(0.0887,0.0915)$
${\lambda }_{12}$	0.2	0.1839	0.0040	$(0.1911,0.2070)$
${\lambda }_{13}$	1	0.9108	0.0195	$(0.9413,1.0181)$
${\lambda }_{21}$	0.3	0.2790	0.0059	$(0.2895,0.3126)$
${\lambda }_{23}$	2	1.8189	0.0392	$(1.8738,2.0267)$
${\lambda }_{31}$	0.4	0.3708	0.0080	$(0.3846,0.4161)$
${\lambda }_{32}$	7	6.5094	0.1432	$(6.7576,7.3138)$
${\gamma }_{12}$	0.2	0.1827	0.0040	$(0.1902,0.2056)$
${\gamma }_{13}$	0.04	0.03681	$7.7337\times {10}^{-04}$	$(0.0381,0.0411)$
${\gamma }_{21}$	0.3	0.2899	0.0063	$(0.3021,0.3268)$
${\gamma }_{23}$	0.06	0.0538	0.0012	$(0.0558,0.0604)$
${\gamma }_{31}$	0.4	0.3841	0.0082	$(0.3948,0.4264)$
${\gamma }_{32}$	0.08	0.0742	0.0016	$(0.0778,0.0842)$
${\eta }_1$	0.3	0.2909	0.0201	$(0.2510,0.3298)$
${\eta }_2$	0.4	0.3788	0.0430	$(0.3098,0.4694)$
${\eta }_3$	0.5	0.5324	0.0199	$(0.4933,0.5716)$
${\sigma }^2$	0.04	0.0377	0.0023	$(0.0333,0.0425)$

Figure 1.

8000 MCMC iteration plots for the parameters of the model.

Figure 2.

Posterior density plots for the parameters of model.

Figure 3.

Autocorrelation sample plots for the parameters of model.

4.5. Application to disease progression

Many models have been proposed to model disease progression through markers observations, among them [46] proposed a deterministic differential equation model (DEM) to model markers in disease such as Alzheimer's disease, Huntington's disease, or Parkinson's disease. Taking the stochastic version of this DEM by adding the regime switching, we get a process similar to the previous regime switching one factor model with the following expression:

$$\mathrm{d}{\tilde{X}}_{ij}=[{\alpha }_{0,k}+{\alpha }_{1,k}{\tilde{X}}_{ij}+{\alpha }_{2,k}{\tilde{X}}_{ij}^2]\mathrm{d}{\tilde{t}}_{ij+1}+\sigma \mathrm{d}W\mathrm{f}\mathrm{o}\mathrm{r}k=1,2,3.$$

We applied this process to model marker observations from a slow developed disease: COPD (Chronic Obstructive Pulmonary Disease). Doctors use Gold stages to address the stage of the COPD (the severity of the disease). Knowing how severe the COPD is in a patient helps to choose the best treatment. The idea is to unravel the stages of the disease using one marker or a combination of many markers. In fact, there are many clinical markers for COPD disease among them: Chest hyperinflation, low body mass index, the use of accessory muscles of respiration, and prolonged expiration. Since, we are interested in one-dimensional diffusion process, we take into consideration as a marker the FEV1 (how much air one can exhales from his lungs in one second, measured in Liter). We will see how the FEV1 observations allow to estimate the parameters of the HSD process. We have supposed that our model has three hidden states; a = 3 (Mild, Moderate, and severe) [39]. We extracted our FEV1 observations using data from the Danish Lung Cancer Screening Trial (DLCST) [36] where 2052 current or ex-smokers aged 50–70 years having FEV1 measured annually for 5 years (2005–2009). We have for each subject, five FEV1 measurements with the date of each measurement. While having more than 10 marker observations by patient would give more parameters precision and identification, disposing here of only 5 marker observations by patient is found to be sufficient for giving accurate results. From the database, it can be seen that the values of the FEV1 marker decrease with the severity of the COPD disease (from values that are greater than 3 L in mild stages to approximately less than 1.5 L for severe stages).

We ran our MCMC algorithm to fit the HSD model and make the inference after the burnings period (inference is based on the last 3000 samples after the burnings). In fact, starting from good initial values for the parameters would help in reducing the time of the burnings period and hence accelerate the MCMC convergence. Such initial values can be obtained from other estimation methods such as the maximum likelihood computation.

Also, while the α's and σ are computed here using Gibbs sampler via a ridge regression approach, in the general case, when the posterior is not a known one we call for the MHA such as for the parameter of the transition rate matrix: the λ's and γ's that have been computed via MHA.

Furthermore, this MCMC uses the random time imputation mechanism. Hence, we have seen that between successive observations, it could happen no imputation or 1 imputation, or 5, or more than 10 time points.

Finally, Table 2 gives the posterior computations for each one of the parameters of the model. It shows how the marker can lead to the estimation of the parameters depending on the hidden states. We could see that most of the parameters (the linear effect parameters and the quadratic effect parameters) have negative values, which goes with the attitudes of the FEV1 marker to decrease with the time. Moreover, and while here we fitted a model with three hidden states one can consider the cases where a = 4 or 5, and uses the Bayesian information criterion (BIC) to choose the best model. Also, we could take other forms of stochastic differential equations and choose which equation fits well the data using the BIC.

Table 2. MCMC output for fitting a HSD Process via FEV1 marker in COPD disease progression.

Posterior computations
Parameters	Mean	Standard deviation	Credible interval (95%)
${\alpha }_{0,1}$	0.0030	0.0084	$(-0.0128,0.0172)$
${\alpha }_{0,2}$	0.0169	0.0231	$(-0.0300,0.0618)$
${\alpha }_{0,3}$	0.0164	0.0127	$(-0.0113,0.0402)$
${\alpha }_{1,1}$	−0.0539	0.0209	$(-0.0952,-0.0136)$
${\alpha }_{1,2}$	−0.0226	0.0656	$(-0.1545,0.1105)$
${\alpha }_{1,3}$	−0.0240	0.0395	$(-0.1037,0.0497)$
${\alpha }_{2,1}$	−0.0264	0.0242	$(-0.0730,0.0202)$
${\alpha }_{2,2}$	−0.0094	0.0744	$(-0.1557,0.1452)$
${\alpha }_{2,3}$	−0.0120	0.0450	$(-0.0991,0.0761)$
${\lambda }_{12}$	0.2657	0.0058	$(0.2538,0.2782)$
${\lambda }_{13}$	0.7661	0.0369	$(0.7224,0.8706)$
${\lambda }_{21}$	0.0458	$7.55\times {10}^{-4}$	$(0.0435,0.0470)$
${\lambda }_{23}$	0.6294	0.0114	$(0.6152,0.6584)$
${\lambda }_{31}$	0.0708	0.0011	$(0.0688,0.0733)$
${\lambda }_{32}$	0.2223	0.0034	$(0.2144,0.2263)$
${\gamma }_{12}$	0.0122	$4.79\times {10}^{-4}$	$(0.0113,0.0131)$
${\gamma }_{13}$	0.312	0.012	$(0.2889,0.3369)$
${\gamma }_{21}$	2.0406	0.081	$(1.8855,2.2007)$
${\gamma }_{23}$	0.0087	$3.59\times {10}^{-4}$	$(0.0081,0.0095)$
${\gamma }_{31}$	4.592	0.187	$(4.2384,4.9777)$
${\gamma }_{32}$	3.122	0.126	$(2.8812,3.3773)$
${\sigma }^2$	0.0086	$7.78\times {10}^{-4}$	$(0.0073,0.0103)$

5. Discussion

Bayesian approach is very efficient in simulating complicated models such as the non-linear diffusion processes. In fact, one can incorporate any prior information or knowledge in the likelihood through the prior specification and this is possible because the posterior of the parameters is proportional to prior multiplied by the likelihood. Also, and while MCMC algorithm can converge even when starting in dispersed initial values for the parameters, we can take use of classical method inference on data to get a good starting value such as maximum likelihood or expectation–maximization algorithms.

Another issue that should be pointed here too is that this model uses observation intervals that are non-equi-distant, though we get accurate estimate using The Euler discretization; and why not should we try to improve this accuracy in the future by using the Milstein discretization [42]. Moreover, the number of observations imputed was chosen using the random time imputation which gives exact simulation. With this way, for small intervals we do not impute any data while for large intervals we could impute data.

Other ideas that can attract attention, is the use of the random walk MHA; that could have many problems such as the moving step. Indeed, a bad choice for the moving step can lead to bad mixing or create high correlated draws. Thus, we should some times avoid the random walk MHA and find good proposal density for every new draw for each parameter or adopting more efficient algorithm such as the accept–reject MHA [10]. Hopefully here the ridge regression has allowed to sample many parameters through Gibbs sampler.

Finally, even that the HSD model here adopts a homoscedastic ${\sigma }^2$, it could be easily extended to ${\sigma }_k^2$ depending on the hidden states $k=1,\dots ,a$ or we can take the stochastic variance as in the case of ARCH and GARCH model [1,8,12,31].

6. Conclusion

This work provides a Bayesian approach for the simulation of a state-dependent switching diffusion process; one of the processes usually hard to handle in a classical framework. We used Euler discretization to overcome the issue of dispersed observations as it is the case for most diffusion models. We have adapted the random time data imputation to the HSD model. We have let the transition rate matrix to depend on the diffusion observations. Hence, we have lead to the estimation of three categories of variables: The imputed data, the hidden switching states, and the parameters of the diffusion process; the sampling of the hidden state has been realized by the FFBS algorithm adapted to the HSD. Overall, even though the complexity of the switching one factor model, the MCMC algorithm has shown its efficiency to estimate the parameters accurately.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

El Houcine Hibbah  http://orcid.org/0000-0001-9319-3670

Hamid El Maroufy  http://orcid.org/0000-0001-6113-7543

Christiane Fuchs  http://orcid.org/0000-0003-3565-8315

Taib Ziad  http://orcid.org/0000-0002-5329-9793