Variability and singularity arising from a Piecewise-Deterministic Markov Process applied to model poor patient compliance in the multi-IV case

ABSTRACT

We propose a Piecewise-Deterministic Markov Process (PDMP) to model the drug concentration in the case of multiple intravenous-bolus (multi-IV) doses and poor patient adherence situation: the scheduled time and doses of drug administration are not respected by the patient, the drug administration considers switching regime with random drug intake times. We study the randomness of drug concentration and derive probability results on the stochastic dynamics using the PDMP theory, focusing on two aspects of practical relevance: the variability of the concentration and the regularity of its stationary probability distribution. The main result show as the regularity of the concentration is governed by a parameter, which quantifies in a precise way the situations where drug intake times are too scarce concerning the elimination rate. Our approach is novel for the study of the regularity of the stationary distribution in PDMP models. This article extends the results given in [J. Lévy-Véhel and P.E. Lévy-Véhel, Variability and singularity arising from poor compliance in a pharmacodynamical model I: The multi-IV case, J. Pharmacokinet. Pharmacodyn. 40 (2013), pp. 15–39], by considering more realistic irregular dosing schedules. The computations permit precise assessment of the effect of various significant parameters such as the mean rate of intake, the elimination rate, and the mean dose. They quantify how much poor adherence will affect the regimen. Our results help to understand the consequences of poor adherence.

1. Introduction

The poor adherence to medical treatment is a problem of the utmost importance that has a critical impact on the efficacy of the therapy, particularly in the case of chronic diseases, [18]. The problem of non-adherence has to be put within the known pharmacokinetic context. The analysis of multiple dosing drug concentrations, with common deterministic models, is usually based on an assumption of patients full adherence, i.e. drugs are administrated at a fixed dosage, with equal or unequal (but fixed) dosing intervals, [9,17]. However, the drug concentration–time curve is often influenced by the random drug input, generated by patients with poor adherence behavior, inducing erratic therapeutical outcomes, [10]. Hence, the well known and predominately used deterministic models are not adequate to handle variable adherence situations. Based on the complex patterns of the non-adherence phenomenon and its intrinsic random properties these deterministic models should be replaced by a stochastic model that allows us to take in to account the fact that the drug doses and the time of dosing are random.

In the seminal work, [13], the authors attacked the problem of mathematically modeling poor adherence using a probabilistic frame. They considered random instants of drug intake and studied the mean and variance of the concentration, conditioned on the time elapsed since the last intake. [13,14], showed that the drug concentration-time evolution is heavily influenced by the random drug input generated by patients with poor adherence behavior. In [12], it is considered the case of multiple intravenous-bolus (multi-IV) dosing using the simplest possible law to model random drug intake, i.e. a homogeneous Poisson distribution. This assumption allows us to perform explicit computations using the well-developed machinery on Poisson processes, and to obtain precise results describing various aspects of the concentration distribution that are important for assessing the efficacy of the regimen. The case of multiple oral doses is studied in [7].

In this work, non-adherence to treatment refers to a particular situation in which the patient does not respect the indications on the administration multiple intravenous doses, specifically is considered the case where the injection times of the drug and the dose they are random, including random changes in the medication regimen over time. These regimen changes, that consider dosages and shooting times with different probability distributions in each regime, generate a stochastic dynamic that can be studied using Piecewise-Deterministic Markov Process (PDMP) theory. An advantage of this broader context is that it allows us to consider medication regimen switching, for instance, thus making it possible to model simultaneously the time deviations from scheduled doses with a regime and the missing doses with a different regime.

The PDMP well knows and was first introduced in the literature by [1,2]. PDMPs form a family of Markov processes involving a deterministic motion punctuated by random jumps. The motion of the PDMP depends on three local characteristics, namely the jump rate, the flow and the probability transition that determine the location of the process at the jump time. These processes have been heavily studied both from a theoretical and from an applied perspective in various domains such as communication networks with the control of congestion TCP/IP [4,9], neurobiology for the Hodgkin-Huxley model of neuronal activity [16], reliability [3], biologic population models [6,15].

We propose a PDMP to model the drug concentration in the case of multiple intravenous doses. In this model, we consider that the dose administration regimen is modeled by a non-homogeneous Poisson process whose jump rate is controlled by the mean of a Markov chain. Thus, we have investigated the probability distribution of drug concentration in the context of multiple dosing and poor adherence. We have focused on two aspects of practical relevance: the variability of the concentration and the regularity of its probability distribution.

We give explicit equations determining the means and variance of concentration and also its characteristic function, showing how poor adherence will increase the variability of the concentration as compared to the full adherence case. Besides, it may also have an impact on the regularity of the concentration probability distribution, resulting in a high probability of having too small a concentration of drugs. An important result of this work is the regularity of the stationary distribution of this PDMP model, we quantify this regularity in a precise way, showing the exact role played by each parameter, the proof of this result is obtained applying the Levinson Theorem [11], this approach is novel to study the regularity of the stationary distribution in PDMP models.

The remaining of this work is organized as follows: In Section 2, we present the PDMP model for the drug concentration taking to account a drug dosing stochastic regimen, we give the characteristic function of the concentration (Section 2.1) and study its variability (Section 2.2). We study the distribution of limit concentration (Section 2.3) and study its variability (Section 2.4) and regularity (Section 2.5). We study the particular case of two regimens in Section 2.6. In Section 3, we present some simulation of our PDMP model for the particular case of two regimens analyzing the variability and regularity of the concentration. In particular, we compare the variability in our poor adherence models to the ones in the cases of full adherence to a single patient. Finally, in Section 4, a discussion and the conclusions of this paper are presented. The proofs are deferred to Appendix 1.

2. The model setting

In our model we consider a drug dosing stochastic regimen defined as follows.

Let us consider $(J_n{)}_{n\in \mathbb{N}}$ a time-homogeneous irreducible Markov chain taking values in the state space $K=\{1,\dots ,k\}$ with initial law ${\alpha }_i=\mathbb{P}(J_0=i)$ for all $i\in K$ and transition probability matrix $Q=(q_{ij}{)}_{i,j\in K}$, i.e.

$$\mathbb{P}(J_{n+1}=j|J_n=i)=q_{ij}.$$

We denote by $(T_n{)}_{n\in \mathbb{N}}$ the sequence of the random time doses and $(S_n{)}_{n\in \mathbb{N}}$ the interdose interval times; i.e. $S_n=T_{n+1}-T_n$.

We consider that the doses administration regimen is modeled by mean of the Markov process $(J_n{)}_{n\in \mathbb{N}}$ considering the following assumptions:

Assumption 2.1

1. The patient takes a dose $D_{J_n}\in \{D_i,i\in K\}$ at the time $T_n$, where the doses $D_i$ are all different and not nulls.

2. The interdose interval time $S_n$ is a random variable with an exponential distribution of parameter ${\lambda }_{J_n}\in \{{\lambda }_i,i\in K\}$, where the jump rate ${\lambda }_i$ of the state i is a strictly positive constant.

We consider that these doses translate into immediate (i.e. for each time $T_n$) increases of the concentration by the value $d_i=D_i/V_d$ if $J_n=i$, where $V_d$ is the apparent volume of distribution. After that, the effect of the dose taken at time $T_n$ decrease exponentially fast with an exponential rate of elimination $k_e$. Here, we consider that kinetics of first-order are involved.

We define $({\nu }_t{)}_{t\in \mathbb{R}}$ by

$${\nu }_t=\sum _{n\ge 0}{J}_{n}1{\mathrm{l}}_{[T_n,T_{n+1}[}(t),$$

where $1{\mathrm{l}}_{[a,b]}$ denotes the indicator function of interval $[a,b]$. Note that $({\nu }_t{)}_{t\in \mathbb{R}}$ is an irreducible Markov process with the same initial law that your immersed Markov chain $(J_n{)}_{n\in \mathbb{N}}$. Here, $J_n$ is the state taken by ${\nu }_t$ on $[T_n,T_{n+1}[$.

The matrix generator $A=(a_{ij}{)}_{i,j\in K}$ of the process $({\nu }_t{)}_{t\in \mathbb{R}}$ is given by $a_{ij}={\lambda }_i{q}_{ij}$ for $i\ne j$ and $a_{ii}=-{\lambda }_i(1-q_{ii})$. The generator A is stable and conservative; i.e. $\sum _{j\in K}{a}_{ij}=0$ and $a_{ij}\ge 0$ for $i\ne j$.

We denote by $(C_t{)}_{t\in {\mathbb{R}}_{+}}$ the drug concentration stochastic process which take values on ${\mathbb{R}}_{+}^{\ast }=]0,\mathrm{\infty }[$, we suppose that $\mathbb{P}(C_0=x)=1$. Between the jumps, the dynamical evolution of the continuous-time process $(C_t)$ is modeled by the flow $\varphi (t,x)$ defined on ${\mathbb{R}}_{+}\times {\mathbb{R}}_{+}^{\ast }$ by the following first order dynamical systems

$$\begin{array}{rl}\frac{\mathrm{d}}{\mathrm{d}t}\varphi (t,x)& =-k_e\varphi (t,x),\\ \varphi (0,x)& =x.\end{array}$$ (1)

Thus, the sample path of the stochastic process $(C_t{)}_{t\in {\mathbb{R}}_{+}}$ with values in ${\mathbb{R}}_{+}^{\ast }$ starting from a fixed point x is defined in the following way, as we illustrate in Figure 1.

Figure 1.

Sample path of the PDMP process $(C_t,{\nu }_t)$.

First, say that ${\nu }_t=J_0$ for $t<T_1=S_0$, where $S_0$ stands for the first jump time of ${\nu }_t$, which has an exponential distribution of parameter ${\lambda }_{J_0}$, and ${\nu }_{T_1}=J_1$. Now, we define $d_{J_1}=D_{J_1}/V_d$. Then the sample path $(C_t)$ up to the first jump time is now defined as follows:

$$\begin{array}{rl}{C}_t& =\varphi (t,x)=x{e}^{-k_{e}t}\mathrm{if}0\le t<T_1,\\ C_{T_1}& =\varphi (T_1,x)+d_{J_1}=x{e}^{-k_e{T}_1}+d_{J_1}.\end{array}$$

The process now restarts from $x_1=C_{T_1}$ according to the same recipe. Thus, we define $S_1$ a random variable with exponential distribution of parameter ${\lambda }_{J_1}$, so we take $T_2=T_1+S_1$ and $d_{J_2}=D_{J_2}/V_d$ where ${\nu }_{T_2}=J_2$. Then, the sample path $C_t$ up to the second jump time, starting from $x_1$ at time $T_1$, is defined as

$$\begin{array}{rl}{C}_t& =\varphi (t-T_1,x_1)=x_0{e}^{-k_{e}t}+d_{J_1}{e}^{-k_e(t-T_1)}\mathrm{if}{T}_1\le t<T_2,\\ C_{T_2}& =\varphi (T_2-T_1,x_1)+d_{J_2}=x{e}^{-k_e{T}_2}+d_{J_1}{e}^{-k_e(T_2-T_1)}+d_{J_2};\end{array}$$

and so on. Finally, for all $n\in \mathbb{N}$, and for $i=1,\dots ,n$ we take $d_{J_i}=D_{J_i}/V_d$, $S_i$ a random variable with exponential distribution of parameter ${\lambda }_{J_i}$, $T_{i+1}=T_i+S_i$ and ${\nu }_{T_{i+1}}=J_{i+1}$. Then, we have

$$C_t=x{e}^{-k_{e}t}+\sum _{i\ge 1}{d}_{J_i}{e}^{-k_e(t-T_i)}1{\mathrm{l}}_{(t\ge T_i)}.$$ (2)

From the independence of $C_0$ and ${\nu }_0$, we have that $(C_t,{\nu }_t{)}_{t\in {\mathbb{R}}_{+}}$ is a piecewise-deterministic Markov process (PDMP).

We denote by $\mathcal{M}$ the set of measurable real-valued functions on $E={\mathbb{R}}_{+}^{\ast }\times K$ and by ${\mathcal{M}}_0$ the set of bounded measurable real valued function on E.

From [2], we have that the domain $\mathcal{D}(\mathcal{U})$ of the infinitesimal generator $\mathcal{U}$ of $(C_t,{\nu }_t{)}_{t\in {\mathbb{R}}_{+}}$ consists of those functions $f\in \mathcal{M}$ differentiable in the first argument. For $f\in \mathcal{D}(\mathcal{U})$ the infinitesimal generator $\mathcal{U}$ is given, for $(x,i)\in E$, by

$$\mathcal{U}f(x,i)=-k_{e}x\frac{\mathrm{d}}{\mathrm{d}x}f(x,i)+{\lambda }_i\sum _{j\in K}{q}_{ij}(f(x+d_j,j)-f(x,i)).$$ (3)

We denote by $P_{ij}(x,B,t)$ the transition probability of $(C_t,{\nu }_t)$; i.e.

$$P_{ij}(x,B,t)=\mathbb{P}(C_t\in B,{\nu }_t=j|C_0=x,{\nu }_0=i).$$

It is defined for all $x\in {\mathbb{R}}_{+}^{\ast }$, $i,j\in K$, $t\in {\mathbb{R}}_{+}$ and $B\in \mathcal{B}$, with $\mathcal{B}$ being the Borel σ-field of ${\mathbb{R}}_{+}^{\ast }$. The transition probability permits to give an expression for the probability distribution of $(C_t,{\nu }_t)$ in the following way:

$$\mathbb{P}(C_t\in B,{\nu }_t=j)=\sum _{i\in K}{\alpha }_i{P}_{ij}(x,B,t).$$ (4)

For $t\in R_{+}$ fix, define an operator ${\mathcal{P}}_t:{\mathcal{M}}_0\to {\mathcal{M}}_0$ by the following conditional expectation given the starting point $(x,i)$

$${\mathcal{P}}_{t}f(x,i)={\mathbb{E}}_{(x,i)}[f(C_t,{\nu }_t)]=\sum _{j\in K}{\int }_{{\mathbb{R}}_{+}^{\ast }}f(y,j)P_{ij}(x,\mathrm{d}y,t).$$

Here, $({\mathcal{P}}_t;t\in {\mathbb{R}}_{+})$ is the semigroup associated with the infinitesimal generator $\mathcal{U}$.

From [2], we recall that for t fixed, $(x,i)\in E$ and $f\in \mathcal{D}(\mathcal{U})$, $z(t,x,i)={\mathcal{P}}_{t}f(x,i)$ is the unique solution of the following partial differential equation (EDP):

$$\begin{array}{rl}\frac{\mathrm{\partial }z}{\mathrm{\partial }t}(t,x,i)& =\mathcal{U}z(t,x,i),\\ z(0,x,i)& =f(x,i).\end{array}$$ (5)

2.1. The characteristic function of the concentration

Let us study the characteristic function of $C_t$ exploiting the connection between the expectations of certain function of $C_t$ and the system of Equations (5) associated with its extended generator.

Theorem 2.1

The characteristic function ${\phi }_{\theta }(t,x,i)$ of $C_t$, given the starting point $(x,i)$, is the unique solution of the following system

$$\begin{array}{rl}\frac{\mathrm{\partial }{\phi }_{\theta }}{\mathrm{\partial }t}(t,x,i)& =-k_{e}x\frac{\mathrm{\partial }{\phi }_{\theta }}{\mathrm{\partial }x}(t,x,i)+{\lambda }_i\sum _{j\in K}{q}_{ij}(e^{\mathrm{i}\theta d_j{e}^{-k_{e}t}}{\phi }_{\theta }(t,x,j)-{\phi }_{\theta }(t,x,i)),\\ {\phi }_{\theta }(0,x,i)& =e^{\mathrm{i}\theta x}.\end{array}$$ (6)

In Appendix 1 we give the proof of Theorem 2.1 using that the characteristic function ${\phi }_{\theta }$ of $C_t$, starting from a fixed point $(x,i)$, is the unique solution of the system (5) for $f(x)=e^{\mathrm{i}\theta x}$.

2.2. Variability of the concentration

First, we will give the mean of the concentration.

Proposition 2.2

The expectation $m(t,x,i)={\mathbb{E}}_{(x,i)}[C_t]$ of $C_t$, given the starting point $(x,i)$, is given by

$$m(t,x,i)=x{e}^{-k_{e}t}+\sum _{\nu ,j\in K}{\lambda }_{\nu }{q}_{\nu j}{d}_j{\int }_0^t{e}^{-k_e(t-s)}{P}_{i\nu }(s)\mathrm{d}s.$$ (7)

In Appendix 1 we give the proof of this fact using that $m(t,x,i)$ is the unique solution of the system (5), for $f(y,\nu )=y$.

Remark 2.1

We recall that the transition matrix $P(t)=(P_{i\nu }(t){)}_{i,\nu \in K}$ satisfies the following differential equation:

$$\frac{\mathrm{d}}{\mathrm{d}t}P(t)=P(t)A=AP(t).$$ (8)

Then, $P(t)=e^{At}$.

Now we will study the variation of the concentration. From Theorem 2.1 we can deduce that $Var(t,x,i)=Var(t,0,i)$; i.e. the variance of concentration do not dependen on x. In what follows, we will denote this variance as $Var(t,i)$

Proposition 2.3

The variance $Var(t,i)$ of $C_t$, given the initial state i, is given by

$$\begin{array}{rl}Var(t,i)& =\sum _{\nu ,j\in K}{\lambda }_{\nu }{q}_{\nu j}{d}_j^2{\int }_0^t{e}^{-2k_e(t-s)}{P}_{i\nu }(s)\mathrm{d}s+2\sum _{\nu ,j,{\nu }^{\prime },j^{\prime }\in K}{\lambda }_{\nu }{q}_{\nu j}{d}_j{\lambda }_{{\nu }^{\prime }}{q}_{{\nu }^{\prime }{j}^{\prime }}{d}_{j^{\prime }}\\ & \times {\int }_0^t{\int }_0^{t-s}{e}^{-k_e(t-s)}{P}_{i\nu }(s)e^{-k_e(t-s-\tau )}{P}_{j{\nu }^{\prime }}(\tau )\mathrm{d}\tau \mathrm{d}s\\ & -{(\sum _{\nu ,j\in K}{\lambda }_{\nu }{q}_{\nu j}{d}_j{\int }_0^t{e}^{-k_e(t-s)}{P}_{i\nu }(s)\mathrm{d}s)}^2.\end{array}$$ (9)

The proof is given in Appendix 1.

Thus, from Propositions 2.2 and 2.3 we have that the mean and variance of $C_t$, given the starting point $C_0=x$, are given by

$$\begin{array}{rl}m(t,x)& =\sum _{i\in K}{\alpha }_{i}m(t,x,i),\end{array}$$ (10)

$$\begin{array}{rl}Var(t)& =\sum _{i\in K}{\alpha }_{i}Var(t,i)+\sum _{i\in K}{\alpha }_i(m_2(t,0,i)-m(t,0){)}^2,\end{array}$$ (11)

where

$$\begin{array}{rl}{m}_2(t,x,i)& =x^2{e}^{-2k_{e}t}+\sum _{\nu ,j\in K}{\lambda }_{\nu }{q}_{\nu j}{d}_j^2{\int }_0^t{e}^{-2k_e(t-s)}{P}_{i\nu }(s)\mathrm{d}s\\ & +2\sum _{\nu ,j\in K}{\lambda }_{\nu }{q}_{\nu j}{d}_j{\int }_0^t{e}^{-k_e(t-s)}{P}_{i\nu }(s)m(t-s,x{e}^{-k_{e}s},j)\mathrm{d}s.\end{array}$$ (12)

2.3. Distribution of the limit concentration

We will be interested in the stationary behavior of the concentration, i.e. the distribution function of the limit $C:=\underset{t\to \mathrm{\infty }}{lim}C(t)$.

This following result is of great importance since it allows us to characterize the distribution of the limit concentration, providing a tool to analyze its variability and regularity.

Theorem 2.4

The random variables $C(t)$ converge in distribution, when t tends to infinity, to a well defined random variable C whose characteristic function is

$$\phi (\theta )={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{\mathrm{i}\theta x}F(\mathrm{d}x)=\sum _{j\in K}\phi (\theta ,j),$$ (13)

with $(\phi (\theta ,j){)}_{j\in K}$ satisfying

$$\begin{array}{rl}& -k_e\theta \frac{\mathrm{d}}{\mathrm{d}\theta }\phi (\theta ,j)+\sum _{i\in K}{\lambda }_i{q}_{ij}{e}^{\mathrm{i}\theta d_j}\phi (\theta ,i)-{\lambda }_j\phi (\theta ,j)=0,\\ & \phi (0,j)={\pi }_j.\end{array}$$ (14)

For the proof of this theorem, we refer the reader to Appendix 1.

2.4. Variability of the limit concentration

From Theorem 2.4 we have the following result.

Proposition 2.5

We denote by $m_j=\int x\pi (\mathrm{d}x,j)$ the mean of the limit concentration C in the state $\nu =j$ and $m=\sum _{j\in K}{m}_j$ the mean of C and V ar its variance. Then,

$$\begin{array}{rl}m& =\frac{1}{k_e}\sum _{j\in K}{\pi }_j{\lambda }_j{d}_j,\\ m_j& =\frac{1}{k_e}{\pi }_j{\lambda }_j{d}_j+\frac{1}{k_e}(\sum _{i\in K}{\lambda }_i{q}_{ij}{m}_i-{\lambda }_j{m}_j),\\ Var& =\frac{1}{2k_e}\sum _{j\in K}{\pi }_j{\lambda }_j{d}_j^2+\frac{1}{k_e}\sum _{i,j\in K}{\lambda }_i{q}_{ij}{d}_j{m}_i-m^2.\end{array}$$ (15)

2.5. The regularity of the limit concentration

We study the regularity of the stationary measure F of the drug concentration C, from the asymptotic behavior of $\phi (\theta )$, given in Theorem 2.4, when θ tends to infinity.

Theorem 2.6

The characteristic function ϕ satisfies

$$|\phi (\theta )|\sim a|\theta {|}^{-{\mu }^{\ast }},\mathrm{when}\theta \to +\mathrm{\infty },$$ (16)

where a is a positive constant and ${\mu }^{\ast }=\underset{\{i\in K\}}{min}{\lambda }_i/k_e$.

The proof of this result is obtained applying the Levinson Theorem [11] which allows studying the asymptotic behavior of the solutions of systems of differential equations. To our knowledge, this approach is novel to study the regularity of the stationary distribution in PDMP models. From Theorem 2.6 we have the following corollary for the regularity of F.

Corollary 2.7

The stationary distribution function F of concentration C satisfies:

1. F is in $L^2$ if and only if ${\mu }^{\ast }>\frac{1}{2}$.

2. For ${\mu }^{\ast }<1$ and $0<\epsilon <<1$, ${\int }_1^{\mathrm{\infty }}{\theta }^{{\mu }^{\ast }-\epsilon }|\phi (\theta )|\mathrm{d}\theta <\mathrm{\infty }$ thus $F\in Lip({\mu }^{\ast }-\epsilon )$.

3. For ${\mu }^{\ast }<\frac{1}{2}$, $1/T{\int }_{-T}^T|\phi (\theta ){|}^2\mathrm{d}\theta =\mathcal{O}(T^{-2{\mu }^{\ast }})$ thus $F\in Lip({\mu }^{\ast })$.

4. The Tauberian theorem implies that $F(\epsilon )\sim |\epsilon {|}^{{\mu }^{\ast }}$, when $\epsilon \to 0$. Then, F is not differentiable at 0 when ${\mu }^{\ast }<1$ and it has a finite non vanishing derivative at 0 exactly when ${\mu }^{\ast }=1$.

5. From Proposition 3 in [12], we have for any x>0

   $$F(x+\epsilon )-F(x)=\mathcal{O}(\epsilon ),$$

   when $\epsilon \to 0^{+}$. This implies that 0 is the only possibly singular point of F.

2.6. Particular case: k = 2

In this section we will interest in analyzing some particular cases of our model.

A first interesting particular case is when we consider that λ is constant ( ${\lambda }_i=\lambda$ for all $i\in K$) and $(J_n{)}_{n\in \mathbb{N}}$ is a sequence of i.i.d. random variables; i.e.

$$\mathbb{P}(J_{n+1}=j|J_n=i)=\mathbb{P}(J_{n+1}=j)={\alpha }_j.$$

This is equivalent to considerer that the drug intake times are model by a homogeneous Poisson process and that the sequence of doses $(D_n{)}_{n\in \mathbb{N}}$ is a sequence of i.i.d. In this case, we obtain that $(C_t{)}_{t\ge 0}$ is a PDMP. This particular case was studied in detail in the previous article [12] within the context of Poisson processes, which is why we will not delve into this case.

We will study in detail the case k = 2, i.e. when we consider only two different doses and two different values for λ. Our purpose is to contrast the results obtained in case k = 2 with those obtained in [12] for the case when λ is constant. Thus, we can analyze the influence of the Markov dependence structure.

First, we study the transition matrix $P(t)=(P_{ij}(t){)}_{ij\in K}$ of jump Markov process $({\nu }_t)$. From Remark 2.1 we have that $P(t)=e^{At}$ with A the matrix generator of $({\nu }_t)$, then is sufficient diagonalize the matrix A to find an analytic expression for $P(t)$. In this case the eigenvalues of A are ${\gamma }_1=tr(A)<0$ and ${\gamma }_2=det(A)=0$ and we can verify that two respective eigenvectors are $v_1=(-a_1,a_2{)}^t$ and $v_2=(1,1{)}^t$, where $a_1,a_2$ are the diagonal elements of matrix A. Then, we can show that

$$P(t)=\left[\begin{array}{cc}{\pi }_1& {\pi }_2\\ {\pi }_1& {\pi }_2\end{array}\right]+\left[\begin{array}{cc}{\pi }_2& -{\pi }_2\\ -{\pi }_1& {\pi }_1\end{array}\right]e^{tr(A)t},$$ (17)

where ${\pi }_1=a_2/tr(A)$, ${\pi }_2=a_1/tr(A)$ is the stationary measure of $({\nu }_t)$. So it follows that the steady state is reached at an exponential rate; i.e.

$$|P_{ij}(t)-{\pi }_j|=\mathcal{O}(e^{tr(A)t}).$$ (18)

Considering ψ the Laplace transform of distribution F of concentration $C_t$ we have from the proof of Theorem 2.6 (see Equations (A13), (A15) and (A16)) that

$$\psi (s)=\sum _{i,j=1}^2{\alpha }_i{\mathrm{\Gamma }}_{ij}(s)s^{-{\mu }_j},$$ (19)

where ${\mu }_j={\lambda }_j/k_e$ and the matrix $\mathrm{\Gamma }(s)=({\mathrm{\Gamma }}_{ij}(s){)}_{i,j\in \{1,2\}}$ is defined by $\mathrm{\Gamma }(s)=e^{G(s)}$ with the matrix $G(s)=(g_{ij}(s){)}_{i,j\in \{1,2\}}$ given by

$$g_{ij}(s)={\int }_0^s\frac{{\mu }_i{q}_{ij}{e}^{-d_{j}u}}{u^{1+({\mu }_i-{\mu }_j)}}\mathrm{d}u.$$ (20)

Thus, we can estimate the distribution F and the characteristic function ϕ of the limit concentration C from Equations (19) and (20).

On the other hand, from Equations (7), (10) and (17), one can write the mean $m(t,x)$, given the starting point x, as

$$m(t,x)=x{e}^{-k_{e}t}+m(1-e^{-k_{e}t})+\frac{1}{tr(A)}\frac{(e^{tr(A)t}-e^{-k_{e}t})}{tr(A)+ke}\sum _{i,j,\nu \in K}{\alpha }_i{a}_{i\nu }{\lambda }_{\nu }{q}_{\nu j}{d}_j,$$ (21)

where m is the stationary mean given in (15), i.e.

$$m={\pi }_1\frac{{\lambda }_1}{k_e}\frac{D_1}{V_d}+{\pi }_2\frac{{\lambda }_2}{k_e}\frac{D_2}{V_d}$$

When the process is starts from the stationary law π of $({\nu }_t)$ then we obtain the following mean $m(t,x)$

$$m(t,x)=x{e}^{-k_{e}t}+m(1-e^{-k_{e}t}).$$ (22)

So, in this case we have

$$|m(t,x)-m|=\mathcal{O}(e^{At}).$$ (23)

Analogously, we can verify that the variance reached the stationary variance at an exponential rate; i.e.

$$|Var(t)-Var|=\mathcal{O}(e^{tr(A)t}).$$ (24)

3. Simulation

In this section, we present some simulations of our PDMP model. We consider the case of a single subject with perfectly-known PK parameters but taking into account poor compliance behavior. We will analyze the variability and regularity of concentration, for this, we assume that the subject follows a random non-compliance regimen of intake drug, which follows a PDMP model with two states, k = 2. Here, the first state represents a regular state of intake drug and sometimes the subject change randomly, according to a Markov chain, to a second state which corresponds to a different dose and a different mean drug intake time.

3.1. Analyzing the variability of concentration

The purpose of this part is to show the behavior of the concentration-time curve and the variability concerning its respective mean curve. We choose the following parameters.

The parameters $k_e$, $V_d$, $D_1=10$ and ${\lambda }_1=\frac{1}{12}$ have been chosen from the pharmacokinetics study given in [8].

Figure 2 shows a sample path of the process $(C_t,{\nu }_t)$, here we can see the schematic representation of the dynamic of PDMP model.

Figure 2.

Sample path of the PDMP process $(C_t,{\nu }_t)$.

In this example, the Markov jumps process $({\nu }_t)$ is irreducible and regular, then there exists a unique stationary measure π. The infinitesimal generator A and the stationary measure π are given by

$$A=\left[\begin{array}{cc}-0.0167& 0.0167\\ 0.0875& -0.0875\end{array}\right],\pi =[0.84,0.16].$$ (25)

From Equation (15) we obtain that the stationary mean and the stationary standard deviation are:

$$m=0.5476\mathrm{mg}/\mathrm{L}\mathrm{and}\sigma =0.2043\mathrm{mg}/\mathrm{L}.$$

We consider the mean $m(t,x)$ given in Equation (22) and the stationary standard deviation σ in order to construct two confidence bands given by $m(t,x)\pm \sigma$ and $m(t,x)\pm 2\sigma$, we approach the variance $Var(t)$ by the stationary variance ${\sigma }^2$ due to $Var(t)$ converge to ${\sigma }^2$ with a exponential rate when t tends to infinity, as is shown in Equation (24).

In Figure 3 we can see the behavior of a sample path of concentration $C_t$ around the mean $m(t,x)$, when we consider the initial probability α or the stationary initial probability π. Comparing the concentration sample paths in Figure 3 we can see that the stationary regime is quickly attained. One can see that the sample path tends to overcome the mean curve, this is because in the second state the subject takes a higher dose with a shorter mean intake time.

Figure 3.

Sample path of $C_t$ considering initial probability α (left) and stationary initial probability π (right). Here, the smooth solid line is the mean $m(t,x)$, the dashed lines correspond to the confidence bands $m(t,x)\pm \sigma$ and the dotted-dashed lines to the confidence bands $m(t,x)\pm 2\sigma$.

It is of interest to compare the behavior of the stochastic concentration $C_t$ to the ones in the case of full compliance. Of course, in the frame of full compliance, there is no randomness involved, in this case, the patient takes a fixed-dose $D_d$ at regularly spaced times $t_0,t_1,\dots ,t_n,\dots$, with $t_i=i/{\lambda }_d$, that is, the patient takes drugs every $1/{\lambda }_d$ units of times for some positive rate ${\lambda }_d$. These doses translate into immediate (i.e. at each time $t_i$) increases of the concentration by the value $D_d/V_d$. after that the effect of the dose taken at $t_i$ on the overall concentration decreases exponentially fast, with exponential speed $k_e$. In this case, the drug concentration at time t, denoted $C_d(t)$, may also be expressed as follows:

$$C_d(t)=x{e}^{-k_{e}t}+\frac{D_d}{V_d}\sum _{i\ge 1}{e}^{-k_e(t-t_i)}1{\mathrm{l}}_{(t\ge t_i)}.$$

Of course, in the frame of full adherence, there is no randomness involved, and one cannot define proper mean and variance. However, since the concentration varies in time, it makes sense to average it overall values of t, and define the variance correspondingly. In other words, we define the mean

$$E_d=\underset{T\to \mathrm{\infty }}{lim}\frac{1}{T}{\int }_0^T{C}_d(t)\mathrm{d}t,$$

as the average of concentration over all values of t, and the variance

$$Va{r}_d=\underset{T\to \mathrm{\infty }}{lim}\frac{1}{T}{\int }_0^T{(C_d(t)-E_d)}^2\mathrm{d}t,$$

as the mean square distance between $C_d(t)$ and $E_d$.

Note that $E_d$ is closely related to the usual PK metric AUC. As for $Va{r}_d$, it represents the time-averaged squared deviation from the long-term average and quantifies the variability within a cycle in the steady-state. In that sense, it is analogous to the ‘Peak trough fluctuation’ parameter considered in the PK literature. Simple computations lead to:

$$E_d=\frac{{\lambda }_d}{k_e}\frac{D_d}{V_d}.$$ (26)

To compare the stochastic model, for k = 2 states, with the deterministic model, we will choose the constants ${\lambda }_d$ and $D_d$ in the following way:

$${\lambda }_d={\pi }_1{\lambda }_1+{\pi }_2{\lambda }_2,D_d=\frac{{\pi }_1{\lambda }_1}{{\lambda }_d}{D}_1+\frac{{\pi }_2{\lambda }_2}{{\lambda }_d}{D}_2.$$

In this manner, we can assure that the mean of $C_d(t)$ is the same that the stationary mean m of $C_t$. However, the quadratic variation of $C_d(t)$ is given by

$$Va{r}_d=\frac{{\mu }_d}{2}{\left(\frac{D_d}{V_d}\right)}^2\left(\frac{e^{1/{\mu }_d}-2{\mu }_d{e}^{1/{\mu }_d}+2{\mu }_d+1}{e^{1/{\mu }_d}-1}\right),$$ (27)

where ${\mu }_d={\lambda }_d/k_e$.

We note the following facts: for a fixed mean m, the variance of $C_d$ tends to infinity at speed $1/12{\mu }_d^2$ when ${\mu }_d\to 0$. The variance tends to 0 at speed $1/2{\mu }_d$ when ${\mu }_d\to \mathrm{\infty }$.

These formulas quantify the obvious fact that everything else being fixed, the variability of the concentration is a decreasing function of the number of takes per unit time.

Taking the values of parameters given in Table 1, we obtain a quadratic variation $Va{r}_d=0.0015$, this implies a standard deviation ${\sigma }_d=0.0383$ mg/L, which is less than the standard deviation σ corresponding to PDMP model.

Table 1. Numerical values of PDMP parameters.

$k_e=0.022{\mathrm{h}}^{-1}$	$V_d=83$ L	$D=[10,15]$ mg/L	$\lambda =[{\displaystyle \frac{1}{12}},{\displaystyle \frac{1}{8}}]{\mathrm{h}}^{-1}$
x = 10 mg/L	$d=[{\displaystyle \frac{10}{83}},{\displaystyle \frac{15}{83}}]$ mg/L	$\alpha =[1,0]$	$Q=\left[\begin{array}{cc}0.8& 0.2\\ 0.7& 0.3\end{array}\right]$

From formula (9), we see that, in a similar way as was observed in the deterministic model, the variability of the concentration is a decreasing function of the expected number of takes per unit time: as is intuitively clear, increasing the mean frequency of intakes while keeping constant the average quantity of administrated drug diminishes the negative impact of poor compliance in terms of the probability of departing significantly from the mean concentration. However, the same

In Figure 4 we can note that the random effect of non-compliance produces a high variability in the drug concentration.

Figure 4.

Path of $C_d(t)$ in the full compliance cases (left). Sample path of $C(t)$ in the non-compliance case (right). Here, the solid line is the mean $m(t,x)$, the dashed lines correspond to the confidence bands $m(t,x)\pm \sigma$ and the dotted-dashed lines to the confidence bands $m(t,x)\pm 2\sigma$.

Remark 3.1

From Equation (10) we can see that the only parameter that does not affect the variance of concentration is the initial concentration x.

Considering a constant rate ${\lambda }_d$ and a matrix transition Q such that the two states of the Markov chain $({\nu }_t)$ are independent and the stationary measure π is the same as that given in Equation (25); i.e. taking $q_{11}={\pi }_1{\lambda }_1/({\pi }_1{\lambda }_1+{\pi }_2{\lambda }_2)$ and $q_{22}={\pi }_2{\lambda }_2/({\pi }_1{\lambda }_1+{\pi }_2{\lambda }_2)$ we obtain the same stationary mean m = 0.5476 mg/L, and a slightly smaller stationary standard deviation ${\sigma }_1=0.1888$ mg/L. We can conclude that the dependence structure provides by the Markov chain produce more uncertainties that in the case of independent states, see [12].

3.2. Analyzing the regularity of concentration

We will show how is affected the concentration regularity for the regularity parameter ${\mu }^{\ast }=min\{{\lambda }_1/k_e,{\lambda }_2/k_e\}$. To comparing numerically various models, we will always use in the sequel the parameters in Table 1 except for parameter ${\lambda }_2$, which will be varied to obtain different values of regularity parameter ${\mu }^{\ast }$. First, we compare the concentration sample path for the two following cases: the first case for $\lambda =[\frac{1}{12},\frac{1}{8}]$, (${\mu }^{\ast }=3.78$) and the second case for $\lambda =[\frac{1}{12},\frac{1}{96}]$, (${\mu }^{\ast }=0.47$).

In the first case,${\mu }^{\ast }=3.78$, the patient takes a dose $D_1=10$ mg in the first medication regime with a mean interdose interval time of 12 hours and in the second medication regime takes a dose $D_2=15$ mg with a mean interdose interval time of 8 hours. If we suppose that the patient should take the drug every 12 hours, then the second medication regime can be interpreted as the patient decrease the mean interdose interval to 8 hours o equivalently advances the following dosage, these produce a fast increase in the concentration, as is illustrated in Figure 5 (left).

Figure 5.

Sample path of concentration $(C_t)$ for ${\lambda }_2=1/8$ and $\mu \ast =3.78$ (left). Sample path of concentration $(C_t)$ for ${\lambda }_2=1/96$ and $\mu \ast =0.47$ (right).

In the second case, ${\mu }^{\ast }=0.47$; the patient takes the dose $D_2=15$ mg with a mean interdose interval of 96 hours. This medication regime can be interpreted as the patient increase the interdose interval to 96 hours or delayed the following dosage. This delayed produces a considerable decreasing in the concentration as is illustrated in Figure 5 (right).

Now, we explore the regularity properties of the distribution of limit concentration C. We show, from Theorem 2.6, that ${\mu }^{\ast }$ controls the regularity of the concentration. In the long term, the cumulative probability distribution of drug concentration display two types of behavior: when ${\mu }^{\ast }$ is larger that one the distribution is regular; while, when it is smaller than one, the distribution is singular only at the origin. In this manner, the amounts ${\mu }^{\ast }$ to quantifying in a precise way the situations where the moments of intakes are too scarce concerning the elimination rate. Thus, when ${\mu }^{\ast }<1$ there is a high probability of having too small concentrations of drugs.

To illustrate this fact, we plot on Figure 6 probability density estimates of $C(T)$, with T a large enough fixed time, for several values of ${\mu }^{\ast }$. For each value of ${\mu }^{\ast }$, the probability density estimate was obtained by simulating N = 50, 000 independent sample path of concentration until time T and estimating the concentration probability density, from these N samples of $C(T)$, using kernel density estimation method with a Gaussian kernel. The time T was chosen large enough so that the steady-state has been reached, we have took T = 10, 000 h. In this figure, we can see the singularity of the probability density of C at the origin for the cases ${\mu }^{\ast }<1$ manifests itself through the sharp spike at the origin.

Figure 6.

Probability densities of C for divers values of ${\mu }^{\ast }$.

We can also see, in Figure 6, the bifurcation phenomenon produced by the two different states present in the medication regimen, this evidence the fact that these densities are a mixture of two conditioned probability densities, one for each state. We detail this bifurcation phenomenon, in Figure 7, for the second case where $\lambda =[\frac{1}{12},\frac{1}{96}]$ and ${\mu }^{\ast }=0.47$.

Figure 7.

Probability density of C and your respective conditioned probability densities.

Note that the conditioned density given the state 1 (case ${\lambda }_1=\frac{1}{12}$) is regular, instead, the conditioned density given the state 2 (case ${\lambda }_2=\frac{1}{96}$) has a singularity at the origin. The singularity at the origin of the probability density of limit concentration C is produced then by the second state because in this state the moments of intakes are too scarce concerning the elimination rate.

Considering the case when the states are independent and the interdose interval rate is constant, i.e. equal to ${\lambda }_d=\sum _{i\in K}{\pi }_i{\lambda }_i$. We obtain that the regularity parameter is ${\mu }_d={\lambda }_d/k_e=$, which is greater than ${\mu }^{\ast }=\underset{i\in K}{min}\{{\lambda }_i/k_e\}$. This implies that the PDMP model can capture better the irregularity of the drug concentration probability distribution.

4. Discussion and conclusions

The Piecewise-Deterministic Markov Process (PDMP) framework proposed a model of drug concentration that allows us to deal with general drug intake schedules with regime switching. In this context, we have shown how poor adherence increases the variability of the concentration compared to the full adherence case. Additionally, from Theorem 2.6, we show that the parameter ${\mu }^{\ast }$ controls the regularity of the concentration, which indicates whether the concentration varies smoothly or not. In the long term, the probability distribution of drug concentration display two types of behavior: when ${\mu }^{\ast }$ is larger that one, the distribution is regular; while, when it is smaller than one, the distribution is singular only at the origin. In this matter, the parameter ${\mu }^{\ast }$ quantifies in a precise way the situations where drug intake times are too scarce concerning the elimination rate. Thus, when ${\mu }^{\ast }<1$ there is a high probability of having too small concentrations of drugs.

We have focused this discussion making a detail study of the case where the patient follows a random non-compliance regimen of intake drug, which follows a PDMP model with two states, k = 2. We compare the results obtained in the case k = 2 with those given by [12] where they consider that the interdose interval rate λ is constant, the drug intake times are modeled by a homogeneous Poisson process, and the sequence of doses $(D_n{)}_{n\in \mathbb{N}}$ is a sequence of i.i.d. random variables. Thus, we can analize the influence of the Markov dependence structure in the model. We conclude that the dependence structure provided by the Markov chain may cause more variability than in the case of independent states. Moreover, in the long term, the regularity parameter of the drug concentration probability distribution for the PDMP model is lower than that obtained for the model with independent states.

Appendix. The Proofs.

Proof of Theorem 2.1 —

Taking $f(x)=e^{\mathrm{i}\theta x}$ we have that the characteristic function ${\phi }_{\theta }$ of $(C_t)$, starting from a fixed point $(x,i)$, is the unique solution of the system (5). We will rewrite the term $\mathcal{U}{\phi }_{\theta }$. First, from (2), we note that the process $C_t$ starting from a point $C_0$ can be written as :

$$C_t=C_0{e}^{-k_{e}t}+\sum _n{d}_n{e}^{-k_e(t-T_n)}1{\mathrm{l}}_{(t\ge T_n)}.$$ (A1)

Since the sequence of random variables $(d_n,T_n{)}_{n\in \mathbb{N}}$ is independent of the starting point $C_0$ we have, for all $B\in \mathcal{B}$ and $a\in {\mathbb{R}}_{+}^{\ast }$

$$\begin{array}{rl}{\mathcal{P}}_{jk}(x+a,B,t)& =\mathbb{P}(C_t\in B,{\nu }_t=k|C_0=x+a,{\nu }_0=j)\\ & =\mathbb{P}(C_t+a{e}^{-k_{e}t}\in B,{\nu }_t=k|C_0=x,{\nu }_0=j)\\ & ={\mathcal{P}}_{jk}(x,B-\{a{e}^{-k_{e}t}\},t),\end{array}$$ (A2)

where we denote by $B-\{a{e}^{-k_{e}t}\}=\{y\in {\mathbb{R}}_{+}^{\ast }:y+a{e}^{-k_{e}t}\in B\}$. Then, we can deduce that for any bounded function $f:{\mathbb{R}}_{+}^{\ast }\to \mathbb{R}$

$${\mathbb{E}}_{(x+a,j)}[f(C_t)]={\mathbb{E}}_{(x,j)}[f(C_t+a{e}^{-k_{e}t})].$$

In particular, we have

$$\begin{array}{rl}{\phi }_{\theta }(t,x+a,j)& ={\mathbb{E}}_{(x+a,j)}\left[e^{\mathrm{i}\theta C_t}\right]\\ & ={\mathbb{E}}_{(x,j)}\left[e^{\mathrm{i}\theta (C_t+a{e}^{-k_{e}t})}\right]\\ & =e^{\mathrm{i}\theta a{e}^{-k_{e}t}}{\phi }_{\theta }(t,x,j).\end{array}$$ (A3)

Now, from (3) and (A3) we can rewrite $\mathcal{U}{\phi }_{\theta }$ as

$$\mathcal{U}{\phi }_{\theta }(t,x,i)=-k_{e}x\frac{\mathrm{\partial }{\phi }_{\theta }}{\mathrm{\partial }x}(t,x,i)+{\lambda }_i\sum _{j\in K}{q}_{ij}(e^{\mathrm{i}\theta d_j{e}^{-k_{e}t}}{\phi }_{\theta }(t,x,j)-{\phi }_{\theta }(t,x,i)).$$ (A4)

Thus, we can rewrite the system (5) for ${\phi }_{\theta }$ as in (6).

Proof of Proposition 2.2 —

We have that $m(t,x,i)$ is the unique solution of the system (5) for $f(y,\nu )=y$. Using that for $f\in \mathcal{D}(\mathcal{U})$ we have $\mathcal{U}({\mathcal{P}}_{t}f)={\mathcal{P}}_t(\mathcal{U}f)$, then we can rewrite (5) as

$$\begin{array}{rl}\frac{\mathrm{\partial }m}{\mathrm{\partial }t}(t,x,i)& =-k_{e}m(t,x,i)+\sum _{\nu ,j\in K}{\lambda }_{\nu }{q}_{\nu j}{d}_j{P}_{i\nu }(t),\\ m(0,x,i)& =x,\end{array}$$ (A5)

where $P_{i\nu }(t)$ is the transition probability of ${\nu }_t$, i.e. $P_{i\nu }(t)=\mathbb{P}({\nu }_t=\nu |{\nu }_0=i)$. Now, applying the variation parameter method to the ordinary differential Equation (A5) we obtain the expression for the expectation $m(t,x,i)$ given in Equation (7).

Proof of Proposition 2.3 —

First, we verify that the variance of concentration given the starting point $(x,i)$ is non depend of x, i.e. $Var(t,x,i)=Var(t,i)$. From Equation (A3) we can deduce that $Var(t,x,i)$ satisfies, for all $a\in \mathbb{R}$,

$$Var(t,x+a,i)=Var(t,x,i).$$

Thus we have that $Var(t,x,i)=Var(t,0,i)$, with $Var(t,i):=Var(t,0,i)$.

We denote by $m_2(t,x,i)={\mathbb{E}}_{(x,i)}[C_t^2]$ the second-order moment of $C_t$, given the starting point $(x,i)$. Then $m_2(t,x,i)$ is the unique solution of the system (5) for $f(y,\nu )=y^2$. From Equation (A3) we can deduce that for all $a\in \mathbb{R}$

$$m_2(t,x+a,i)=m_2(t,x,i)+2a{e}^{-k_{e}t}m(t,x,i)+a^2{e}^{-2k_{e}t}.$$

Thus, we can rewrite (5) as

$$\begin{array}{rl}\frac{\mathrm{\partial }{m}_2}{\mathrm{\partial }t}(t,x,i)& =-k_e\frac{\mathrm{\partial }{m}_2}{\mathrm{\partial }x}(t,x,i)+{\lambda }_i\sum _{j\in K}{q}_{ij}(m_2(t,x,j)-m_2(t,x,i))\\ & +{\lambda }_i\sum _{j\in K}{q}_{ij}(2d_j{e}^{-k_{e}t}m(t,x,j)+d_j^2{e}^{-2k_{e}t}),\\ m_2(0,x,i)& =x^2.\end{array}$$ (A6)

In order to find the solution of this EDP we apply the characteristic method obtaining the following differential equation system:

$$\mathrm{d}t=\frac{\mathrm{d}x}{k_{e}x}=\frac{\mathrm{d}{m}_2}{\mathrm{\Delta }(m_2)+g},$$ (A7)

where

$$\begin{array}{rl}\mathrm{\Delta }(m_2)(t,x,i)& ={\lambda }_i\sum _{j\in K}{q}_{ij}(m_2(t,x,j)-m_2(t,x,i)),\\ g(t,x,i)& ={\lambda }_i\sum _{j\in K}{q}_{ij}(2d_j{e}^{-k_{e}t}m(t,x,j)+d_j^2{e}^{-2k_{e}t}).\end{array}$$ (A8)

We consider the first differential equation $\mathrm{d}t=\mathrm{d}x/k_{e}x$ in (A7), then we have that

$$x_t=x{e}^{k_{e}t}.$$ (A9)

Now, we consider the second differential equation in (A7); i.e. $(\mathrm{d}/\mathrm{d}t)m_2=\mathrm{\Delta }(m_2)+g$. Applying the variation parameter method, from the boundary condition of EDP (A6) and the relation (A9) we obtain (12). Finally, using the expression (7) of the expectation $m(t,x,i)$ we obtain that the variance $Var(t,i)$ is given by (9).

Proof of Theorem 2.4 —

The adjoint operator of infinitesimal generator $\mathcal{U}$ is given by

$${\mathcal{U}}^{\ast }\mathrm{\Pi }(x,j)=\frac{\mathrm{d}}{\mathrm{d}x}(k_{e}x\mathrm{\Pi }(x,j))+\sum _{i\in K}{\lambda }_i{q}_{ij}\mathrm{\Pi }(x-d_j,i)-{\lambda }_j\mathrm{\Pi }(x,j).$$ (A10)

So the stationary distribution Π of $(C_t,{\nu }_t)$ satisfies

$${\mathcal{U}}^{\ast }\mathrm{\Pi }(x,j)=0,\mathrm{for}\mathrm{all}(x,j)\in E.$$ (A11)

We define $F(x)=\sum _{j\in K}\mathrm{\Pi }(x,j)$, F is the stationary distribution of concentration $C_t$.

Let us ${\pi }_j=\int \mathrm{\Pi }(\mathrm{d}x,j)$, we have that $\pi =({\pi }_j{)}_{j\in K}$ is the stationary distribution of Markov chain $({\nu }_t)$ satisfying

$$\sum _{i\in K}{\pi }_i{\lambda }_i{q}_{ij}={\lambda }_j{\pi }_j.$$ (A12)

or equivalently $\pi A=0$, where A is the infinitesimal generator of $({\nu }_t)$.

Taking $\phi (\theta ,j)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{\mathrm{i}\theta x}\mathrm{\Pi }(\mathrm{d}x,j)$, from Equations (A10), (A11) and (A12), we have that the characteristic function of the limit concentration C in the state $\nu =j$, satisfies (14). Thus, the result follows.

Proof of Theorem 2.6 —

Let us ψ the Laplace transform of the stationary measure F given by

$$\psi (s)={\int }_0^{\mathrm{\infty }}{e}^{-sx}F(\mathrm{d}x).$$ (A13)

Since the random variable C is non-negative then we have the following relation

$$\phi (\theta )={\int }_0^{\mathrm{\infty }}{e}^{\mathrm{i}\theta x}F(\mathrm{d}x)=\psi (-\mathrm{i}\theta ),\mathrm{\forall }\theta \in \mathbb{R}.$$ (A14)

Thus, we can study the asymptotic behavior of the characteristic function $\phi (\theta )$ when θ tends to infinity by means of the asymptotic behavior of $\psi (s)$ when s tends to infinity.

To characterize the Laplace transform ψ of F we consider the following functions

$$\psi (s,j)={\int }_0^{\mathrm{\infty }}{e}^{-sx}\mathrm{\Pi }(\mathrm{d}x,j).$$ (A15)

From Equations (A10) and (A11) we have that $\psi (s,j)$ satisfies the differential equation

$$k_{e}s\psi {}^{\prime }(s,j)=\sum _{i\in K}{\lambda }_i{q}_{ij}{e}^{-s{d}_j}\psi (s,i)-{\lambda }_j\psi (s,j).$$ (A16)

Taking the change of variable $s=e^t$ and $\mathrm{y}(t,j)=\psi (e^t,j)$ we obtain

$$k_e\mathrm{y}{}^{\prime }(t,j)=\sum _{i\in K}{\lambda }_i{q}_{ij}{e}^{-e^t{d}_j}\mathrm{y}(t,i)-{\lambda }_j\mathrm{y}(t,j),\text{ for }j\in K.$$ (A17)

Without loss of generality for the asymptotic study, we will suppose that the elimination constant $k_e=1$.

Let us $\mathrm{y}(t)=(\mathrm{y}(t,j){)}_{j\in K}$ the $1\times k$ vector with $\mathrm{y}(x,j)$ satisfying the system of differential Equation (A17). This system can be written in the following way

$$\mathrm{y}{}^{\prime }(t)=\mathrm{y}(t)(-\mathrm{\Lambda }+R(t)),$$ (A18)

where R is defined by

$$R(t)=\mathrm{\Lambda }Q\exp \{-D{e}^t\},$$ (A19)

with $\mathrm{\Lambda }=\mathrm{diag}({\lambda }_i:i\in K)$, $D=\mathrm{diag}(d_i:i\in K)$ and Q the transition probability matrix of the Markov chain $({\nu }_t{)}_{t\in {\mathbb{R}}_{+}}$.

A matrix solution for (A18) is a $k\times k$ matrix $Y(t)$ whose rows are solution ${\mathrm{y}}_i(t)$, $i\in K$, of (A18). Then $Y(t)$ satisfies the corresponding matrix differential equation

$$Y^{\prime }(t)=Y(t)(-\mathrm{\Lambda }+R(t)).$$ (A20)

In the case where the matrix solution $Y(t)$ is non-singular for all t in an interval $\mathcal{I}$, $Y(t)$ is said to be a fundamental matrix of (A18) on $\mathcal{I}$. One the basic properties of a fundamental matrix $Y(t)$ is that any solution of (A18) can be expressed as

$$\mathrm{y}(t)=aY(t),$$ (A21)

where a is a constant vector.

We are interested in the representation of fundamental solutions $Y(t)$ of the perturbed system (A20) in the vicinity of $t=+\mathrm{\infty }$. Since a fundamental solution of the unperturbed equation $X^{\prime }(t)=-X(t)\mathrm{\Lambda }$ is $X(t)=\exp \{-\mathrm{\Lambda }t\}$, one may hope that an asymptotic representation of a fundamental solution of (A20) be given by

$$Y(t)=(I+W(t))\exp \{-\mathrm{\Lambda }t\},$$ (A22)

with $W(t)\to 0$ as $t\to \mathrm{\infty }$ and I the identity $k\times k$ matrix.

The asymptotic theory of systems of linear differential equations provided asymptotic integration under various assumptions implying that $R(t)$ is small in some sense as $t\to \mathrm{\infty }$. In the Levinson's Theorem, see [5,11], the condition on $R(t)$ which arises naturally is

$${\int }_{\delta }^{\mathrm{\infty }}|R(t)|\mathrm{d}t<\mathrm{\infty },$$ (A23)

by which we mean that each entry in $R(t)$ has an absolutely convergent integral on $[\delta ,\mathrm{\infty })$ for some $\delta \ge 0$. The Levinson's theorem states that under condition (A23), the system (A20) has solutions given by (A22) as $t\to \mathrm{\infty }$, with W a diagonal matrix $W(t)=\mathrm{diag}(w_i(t):i\in K)$ such that $w_i(t)\to 0$ as $t\to \mathrm{\infty }$.

Now, we will apply Levinson's theorem to show our result. For this, we should first verify condition (A23) for the system (A20), i.e

$${\int }_0^{\mathrm{\infty }}|R_{ij}(t)|\mathrm{d}t={\lambda }_i{q}_{ij}{\int }_0^{\mathrm{\infty }}{e}^{-d_j{e}^t}\mathrm{d}t={\lambda }_i{q}_{ij}{\int }_1^{\mathrm{\infty }}\frac{e^{-d_{j}s}}{s}\mathrm{d}s<\mathrm{\infty }.$$ (A24)

Thus, the Levinson's theorem implies that the system (A20) has solutions given by (A22). From (A21) and (A22) we obtain

$$\mathrm{y}(t,j)=a_j(1+w_j(t))e^{-{\lambda }_{j}t},$$ (A25)

when t tends to infinity. So

$$\psi (s,j)=a_j(1+w_j(\log s))s^{-{\lambda }_j},$$ (A26)

when s tends to infinity. Thus, for s enough large

$$\psi (s)\sim \sum _{j\in K}{a}_j{s}^{-{\lambda }_j}\sim a{s}^{-{\lambda }^{\ast }}$$ (A27)

with ${\lambda }^{\ast }=\underset{\{j\in K\}}{min}{\lambda }_j$ and a a positive constant.

In general, if $k_e\ne 1$ we replace in the system (A20) ${\lambda }_j$ by ${\mu }_j={\lambda }_j/k_e$. Thus, from (A14) and (A27) we obtain (16).

Funding Statement

This work was supported by Comisión Nacional de Investigación Científica y Tecnológica [MATHAMSUD-CONICYT. 18-MATH-07,PCI-CONICYT. REDI170457].

Disclosure statement

No potential conflict of interest was reported by the authors.