Computational methods for birth-death processes

Abstract

Many important stochastic counting models can be written as general birth-death processes (BDPs). BDPs are continuous-time Markov chains on the non-negative integers in which only jumps to adjacent states are allowed. BDPs can be used to easily parameterize a rich variety of probability distributions on the non-negative integers, and straightforward conditions guarantee that these distributions are proper. BDPs also provide a mechanistic interpretation – birth and death of actual particles or organisms – that has proven useful in evolution, ecology, physics, and chemistry. Although the theoretical properties of general BDPs are well understood, traditionally statistical work on BDPs has been limited to the simple linear (Kendall) process. Aside from a few simple cases, it remains impossible to find analytic expressions for the likelihood of a discretely-observed BDP, and computational difficulties have hindered development of tools for statistical inference. But the gap between BDP theory and practical methods for estimation has narrowed in recent years. There are now robust methods for evaluating likelihoods for realizations of BDPs: finite-time transition, first passage, equilibrium probabilities, and distributions of summary statistics that arise commonly in applications. Recent work has also exploited the connection between continuously- and discretely-observed BDPs to derive EM algorithms for maximum likelihood estimation. Likelihood-based inference for previously intractable BDPs is much easier than previously thought and regression approaches analogous to Poisson regression are straightforward to derive. In this review, we outline the basic mathematical theory for BDPs and demonstrate new tools for statistical inference using data from BDPs.

Graphical abstract

Realization of a birth-death process X(t).

INTRODUCTION

Birth-death processes (BDPs) are a flexible class of continuous-time Markov chains that model the number of “particles” in a system, where each particle can “give birth” to another particle or “die” (Feller, 1971; Karlin & Taylor, 1975). The rate of births and deaths at any given time depends on how many extant particles there are. When there are k particles, a birth occurs with instantaneous rate λ_k and a death with instantaneous rate μ_k. In the classical “simple linear” BDP, λ_k = kλ and μ_k = kμ so that per-particle birth and death rates remain constant. In a “general” BDP, λ_k and μ_k can be any function of k but are time-homogeneous (Kendall, 1948, 1949). Table 1 gives examples of well-known BDPs and their birth and death rates. Figure 1 shows an example realization from a BDP.

Table 1.

Some well-known BDPs with birth and death rates λ_k and μ_k. The SI and SIS models refers to the susceptible-infectious(-susceptible) process in epidemiology in which there are k infectious individuals in a finite population of size N. The Moran/Ehrenfest process models the change in the numbers of particles of two types, where transitions between types occur at a rate proportional to the number of potential contacts between members each type in a finite population of size N.

Model	λk	μk
Poisson	λ	0
Yule/Pure birth	kλ	0
Survival/Pure death	0	kμ
Kendall	kλ	kμ
Kendall + immigration	kλ +α	kμ
M/M/1 queue	λ	μ
M/M/c queue	λ	min(k, c)μ
M/M/∞ queue	λ	kμ
SI/Logistic	k(N – k)λ	0
SIS	k(N – k)λ	kμ
Moran/Ehrenfest	k(N – k)λ	k(N – k)μ

Figure 1.

Stochastic simulation of a BDP starting at X(0) = 1 on the interval 0 < t < 2.

The usefulness of BDPs lies in the fact that “particle” can refer to a member of any discrete potentially interacting system in which one only keeps track of the number of objects in existence. BDPs are popular modeling tools in evolution, population biology, genetics, and ecology (Novozhilov, Karev, & Koonin, 2006). For example, if we interpret the particles as species in a macro-evolutionary setting, BDPs can be used to study speciation and extinction over evolutionary timescales (Nee, 2006; Nee, May, & Harvey, 1994). BDPs can also be used to study infectious disease dynamics in a finite population, where the number of individuals infected is the quantity of interest (Andersson & Britton, 2000; N. T. J. Bailey, 1964). In molecular evolution, BDPs can model inserted and deleted nucleotides in a DNA or RNA sequence as part of a probabilistic alignment method (Holmes & Bruno, 2001; Thorne, Kishino, & Felsenstein, 1991), mobile/transposable genetic elements (Rosenberg, Tsolaki, & Tanaka, 2003), gene families (Demuth, De Bie, Stajich, Cristianini, & Hahn, 2006), or even whole chromosomes (Mayrose, Barker, & Otto, 2010). BDPs can model populations of organisms in a resource-limited environment (Renshaw, 1993, 2011; Tan & Piantadosi, 1991). In finite populations, BDPs are commonly used to model quantities of interest in an evolutionary setting, such as mutations, allele frequencies, selection, or coalescence (Fudenberg, Imhof, Nowak, & Taylor, 2004; Kingman, 1982; Krone & Neuhauser, 1997; McFarland, Mirny, & Korolev, 2014; P. A. P. Moran, 1958; Nowak, Michor, & Iwasa, 2003; Nowak, Sasaki, Taylor, & Fudenberg, 2004).

Branching processes are popular tools used in biological modeling that share some of these properties with BDPs (Guttorp, 1991; Kimmel & Axelrod, 2016). Branching processes are specified by assumptions about what happens to individual particles, which typically behave independently: they can die, give birth to or be replaced by offspring of the same or different types. But the nature of this process is individualistic – rules about how individual particles behave in isolation give rise to models for population-level dynamics. In contrast, BDPs are specified at the population level. The conditional rate of particle birth and death given the numbers in existence can depend arbitrarily on the current size of the population.

Many important models in queuing theory can be written as general BDPs (Norris, 1998; Renshaw, 2011; Ross, 1995). In basic Markovian queues, customers arrive into a queue or buffer as a Poisson process with rate λ, and waiting customers are served (removed from the queue) with per-customer service rate μ. In the M/M/∞ queue, also known as the immigration-death process, there are infinitely many servers, so the arrival and service (birth and death) rates are λ_k = λ and μ_k = kμ for k > 0. In the M/M/1 queue, also known as the immigration-emigration model, there is only a single server, so the rates are λ_k = λ and μ_k = μ. In the M/M/c queue, there are exactly c servers, so μ_k = min{c, k}μ.

BDPs can also be useful for defining arbitrary probability distributions on the non-negative integers. Crawford and Zelterman (2015) demonstrate that any sum of exchangeable Bernoulli random variables can be exactly represented as a pure-birth BDP. In fact, M. J. Faddy (1997) shows that one can define a pure birth process (a BDP with death rates μ_k = 0 for all k) whose transition probabilities reproduce any discrete distribution on the counting numbers. Klar, Parthasarathy, and Henze (2010) establish a correspondence between several power law distributions and the long-time limit of specially constructed BDPs, providing a time-dependent interpretation that may be useful for modelling mechanistic processes that give rise to power law outcomes. Sometimes this power law behavior is motivated by a mechanistic model: researchers have developed models and statistical methodology for estimating BDP parameters for power law behavior in protein domains (Karev, Wolf, Rzhetsky, Berezovskaya, & Koonin, 2002) and gene and protein family sizes (Reed & Hughes, 2004). Crawford and Suchard (2012) define a BDP to mimic a process of frameshift-aware insertions and deletions in DNA sequences. Lee, Weiss, and Suchard (2011) set the birth and death rates of a BDP to exhibit over-dispersion relative to the Poisson distribution, and Crawford, Weiss, and Suchard (2015) define a BDP to model rounding in counts of sex partners to multiples of 5, 10, 25, or 50 in self-reported counts of sex partners in a public health study.

There is a rich history of theoretical research into the properties of BDPs. Kendall (1948, 1949) introduce the process with constant per-particle birth and death rates and finds the transition probabilities by a generating function argument. In their groundbreaking series of papers, Karlin and McGregor (1957b) analyze properties of BDPs, including stationary distributions, moments, transition probabilities, recurrence and passage times, and other quantities of interest (Karlin & McGregor, 1957a, 1957b). They also explore in-depth applications of this theory to BDPs whose rates depend linearly on k (Karlin & McGregor, 1958a), and queuing processes (Karlin & McGregor, 1958b).

Beyond the work of Karlin and McGregor (1957b), many authors have discovered extensions and deeper interpretations for the theoretical properties of BDPs. For example, the theory of BDPs is intimately related to properties of continued fractions (Guillemin & Pinchon, 1999). Flajolet and Guillemin (2000) elucidate the relationship between sample trajectories (or state paths) of a BDP and lattice path combinatorics via continued fractions and develop expressions for a variety of recurrence and passage time variables in terms of continued fractions. Lenin and Parthasarathy (2000) and Parthasarathy, Lenin, Schoutens, and Van Assche (1998) discuss further some well-known continued fractions whose connection to BDPs previously went unappreciated.

The study of BDPs has benefited from wide interest in the theoretical properties of this class of processes. But their usefulness as flexible tools for statistical inference has been under-appreciated. In this review, we outline basic properties of BDPs and show how to perform principled statistical inference using data from continuous and discrete observation of BDPs. First, we present the basic time-evolution equations of general BDPs, derive the transition probabilities for the Kendall process (Feller, 1971; Kendall, 1948), and describe the analytic theory developed by Karlin and McGregor (1957a, 1957b) for general BDPs. Then we outline a computational strategy for evaluating BDP transition probabilities using a continued fraction representation of their Laplace transform, which allows routine computation of likelihoods for discretely observed processes (Crawford & Suchard, 2012). We describe a generic class of EM algorithms for maximum likelihood (or maximum a posteriori ) inference for discretely observed BDPs (Crawford, Minin, & Suchard, 2014). Finally, we derive the distribution of integral summary statistics of BDPs that arise often in applications.

BACKGROUND

A BDP is a continuous-time Markov chain X(t) counting the number of particles in a system at time t, taking values on the non-negative integers ℕ. To construct a general BDP in a formal way, we must define the rules according to which the number of particles evolves. We do this by specifying the behavior of the process for a very short time dt, when there are k particles in the system. If dt is very small, the probability of an event during (t, t+dt) that occurs with rate r is approximately rdt. Therefore, the probability of a birth in the interval (t, t + dt), given X(t) = k, is

$$Pr(X(t+dt)=k+1\mid X(t)=k)={\lambda }_{k}dt+o(dt).$$ (1)

Intuitively, this means that the probability of more than one birth event in a small time dt is negligibly small. The probability of a death in (t, t + dt) is likewise

$$Pr(X(t+dt)=k-1\mid X(t)=k)={\mu }_{k}dt+o(dt),$$ (2)

where k ≥ 1. Together, these assumptions imply that the probability of no births or deaths occurring during (t, t + dt) is

$$Pr(X(t+dt)=k\mid X(t)=k)=1-({\lambda }_k+{\mu }_k)dt+o(dt).$$ (3)

TRANSITION PROBABILITIES

Let P_ab(t) = Pr(X(t) = b | X(0) = a) be the transition probability from state X(0) = a to X(t) = b. We can use the above expressions to form a differential equation describing the change in transition probabilities over time. Suppose that X(0) = a. At the current time t, we want to know the probability that in the next dt units of time, the process will reach state b. We look into the future by writing the probabilities of three types of events that can take the process to state b: birth from b − 1, death from b + 1, or no change from b:

$$P_{ab}(t+dt)={\lambda }_{b-1}{P}_{a,b-1}(t)dt+{\mu }_{b+1}{P}_{a,b+1}(t)dt+(1-{\lambda }_b-{\mu }_b)P_{ab}(t)dt+o(dt).$$ (4)

Subtracting P_ab(t) from both sides, dividing by dt, and sending dt to zero, we obtain the Kolmogorov forward equations:

$$\frac{\mathrm{d}{P}_{ab}(t)}{\mathrm{d}t}={\lambda }_{b-1}{P}_{a,b-1}(t)+{\mu }_{b+1}{P}_{a,b+1}(t)-({\lambda }_b+{\mu }_b)P_{ab}(t),$$ (5)

where P_ab(0) = 1 if a = b and zero otherwise. In this article, we always assume μ₀ = λ₋₁ = 0; this keeps the process on the non-negative integers. Letting P(t) = {P_ab(t)} in matrix form, (5) becomes

$$\frac{\mathrm{d}\mathbf{P}(t)}{\mathrm{d}t}=\mathbf{P}(t)\mathbf{A},$$ (6)

where A is the infinitesimal generator matrix with entries A = {a_ij}, a_i,i−1 = μ_i, a_ii = −(λ_i+μ_i), and a_n,n+1 = λ_i. In the matrix case, the initial condition becomes P(0) = I. This infinite sequence of coupled ordinary differential equations can be difficult or impossible to solve for many general BDPs (Novozhilov et al., 2006; Renshaw, 2011).

KENDALL PROCESS

In the simple linear BDP, also known as the Kendall process where λ_k = kλ and μ_k = kμ, it is possible to solve for these transition probabilities explicitly by finding a generating function solution to the forward equations (N. T. J. Bailey, 1964; Lange, 2010a). To illustrate, let $G_a(s,t)={\sum }_{k=0}^{\infty }{s}^k{P}_{ak}(t)$. Let b = k in (5), multiply both sides by s^k, and sum on k to obtain

$$\begin{gathered} \frac{\partial G_a(s,t)}{\partial t}=\sum _{k=0}^{\infty }{s}^k\frac{\mathrm{d}{P}_{ak}(t)}{\mathrm{d}t} \\ =\lambda s^2\sum _{k=1}^{\infty }(k-1)s^{k-2}{P}_{a,k-1}(t)+\mu \sum _{k=0}^{\infty }(k+1)s^k{P}_{a,k+1}(t)-(\lambda +\mu )s\sum _{k=0}^{\infty }k{s}^{k-1}{P}_{ak}(t) \\ =(\lambda s-\mu )(s-1)\frac{\partial G_a(s,t)}{\partial s}, \end{gathered}$$ (7)

with the initial condition G_a(s, 0) = s^a. The solution is

$$G_a(s,t)={\left(\frac{\mu (1-s)-(\mu -\lambda s)e^{-(\lambda -\mu )t}}{\lambda (1-s)+(\mu -\lambda s)e^{-(\lambda -\mu )t}}\right)}^a.$$ (8)

Inverting and finding the bth coefficient of the power series G_a(s, t), we find the transition probabilities

$$P_{ab}(t)=\sum _{j=0}^{min(a,b)}\left(\begin{array}{c}a\\ j\end{array}\right)\left(\begin{array}{c}a+b-j-1\\ a-1\end{array}\right){\alpha }^{a-j}{\beta }^{b-j}{(1-\alpha -\beta )}^j,$$ (9)

where

$$\alpha (t)=\frac{\mu (e^{(\lambda -\mu )t)}-1)}{\lambda e^{(\lambda -\mu )t}-\mu }\text{and}\beta (t)=\frac{\lambda (e^{(\lambda -\mu )t)}-1)}{\lambda e^{(\lambda -\mu )t}-\mu }.$$ (10)

As in the Bienaymé-Galton-Watson branching process, the Kendall process gives rise to a linear-fractional distribution (Athreya & Ney, 2004) when starting from one individual (a = 1). Sagitov (2013) recently developed a multi-dimensional linear-fractional distribution to characterize the multi-type Bienaymé-Galton-Watson branching process with countably many types. This generalization may also be applicable for some multivariate extensions of the Kendall process.

GENERAL BDPS

The problem becomes much more complicated for general BDPs. Karlin and McGregor (1957b) present the definitive treatment of the existence of transition probabilities and other properties of BDPs. They obtain the following integral form for the transition probabilities:

$$P_{ab}(t)={\omega }_b{\int }_0^{\infty }{e}^{-xt}{Q}_a(x)Q_b(x)\mathrm{d}\psi (x),$$ (11)

where ω₀ = 1 and ω_k = (λ₀ · · · λ_k−1)/(μ₁ · · · μ_k) for k ≥ 1. Here, Q_k(x), k = 0, 1, 2, … is a system of orthogonal polynomials and ψ(x) is an orthogonalizing spectral measure that are specific to a particular set of birth and death rates.

This integral representation is intuitively satisfying because the time-dependency of P_ab(t) is contained entirely in the exponential term, and P_ab(t) depends on Q_a(x) and Q_b(x) in a simple way. In addition, we have the obvious corollary that

$$\frac{P_{ab}(t)}{P_{ba}(t)}=\frac{{\omega }_b}{{\omega }_a}.$$ (12)

Beyond these simple results related to the interpretation of (11), the formalism developed by Karlin and McGregor (1957b) makes possible deep analytic insight into the behavior of general BDPs, including recurrence times and first passage times. Notably, a similar spectral representation for the transition probabilities of time-inhomogeneous linear BDPs has been derived recently (Ohkubo, 2014).

EQUILIBRIUM PROBABILITIES AND EXPLOSION

Equilibrium solutions are straightforward to obtain (Renshaw, 2011). Setting the left-hand side of the Kolmogorov forward equations (5) to zero and replacing the finite-time transition probabilities P_ab(t) with the equilibrium probabilities π_b, we find that

$${\mu }_{b+1}{\pi }_{b+1}-{\lambda }_b{\pi }_b={\mu }_b{\pi }_b-{\lambda }_{b-1}{\pi }_{b-1}.$$ (13)

Since this is the case for every b, it is true for b = 0 in particular, and μ₀ = λ₋₁ = 0, so both sides of (13) are zero for every b by induction. This gives the detailed balance condition for continuous-time Markov chains,

$${\mu }_k{\pi }_k={\lambda }_{k-1}{\pi }_{k-1}\text{for}k=1,2,\mathrm{....}$$ (14)

Therefore every general BDP is a reversible Markov chain. Iterating the recurrence (14), we find that

$${\pi }_k=\frac{{\lambda }_0{\lambda }_1\cdots {\lambda }_{k-1}}{{\mu }_1{\mu }_2\cdots {\mu }_k}{\pi }_0,$$ (15)

where we have chosen π₀ so that Σ_k π_k = 1. Note that π_k ∝ ω_k for every k.

The birth and death rates for a general BDP may be such that the process “runs away” to infinity in finite time. This is known as explosive growth. Formally, suppose the process begins at X(0) = 0 and there are no absorbing states. Renshaw (2011) shows that the expected first passage time to infinity τ^∞ is

$$\mathbb{E}({\tau }^{\infty }\mid X(0)=0)=\sum _{j=0}^{\infty }{\pi }_j\sum _{n=j}^{\infty }\frac{1}{{\mu }_n{\pi }_n},$$ (16)

where π₁ = 1 and

$${\pi }_n=\prod _{k=1}^n\frac{{\lambda }_{k-1}}{{\mu }_k},$$ (17)

for i > 1. When (16) diverges, the process is non-explosive, and the first passage time from 0 to any finite state j is almost surely finite. When (16) is finite, the first passage time to infinity is finite with non-zero probability.

One result of special interest to us gives the conditions under which a BDP with a given generator A is unique: Karlin and McGregor (1957a) show that there is only one transition probability matrix P(t) that satisfies (6) if and only if

$$\sum _{k=0}^{\infty }\left({\omega }_k+\frac{1}{{\lambda }_k{\omega }_k}\right)=\infty .$$ (18)

This property assures that probability is conserved on the non-negative integers. We will always assume this is the case in what follows.

Despite the elegant representation (11) for the transition probabilities, it can be very difficult to find the polynomials {Q_k(x)} (Novozhilov et al., 2006; Renshaw, 2011). Settings in which the expression (11) leads to a tractable analytic representations are rare, even for linear processes. As outlined in Crawford and Suchard (2012), Ismail, Letessier, and Valent (1988) give the polynomials Q_k(x) and measure for the birth-death-immigration-emigration process with λ_k = kλ + ν and μ_k = kμ + γ, but a closed-form expression for transition probabilities remains out of reach. In addition, the problem of finding these polynomials and measure ψ(x) is a fundamentally analytical task, and is generally not amenable to computational solution. In other words, one cannot simply compute P_ab(t) using a computer for an arbitrary set of birth and death rates {λ_k} and {μ_k} using the formula (11) alone.

Since analytic derivation of ψ(x) is so complicated, Renshaw (2011, page 111) writes of the need for an alternative approach to solving the forward system in order to find transition probabilities for general BDPs:

“A worthwhile and potentially rewarding challenge would be to develop a simplified and user-friendly version of this technique which would work over a wide range of stochastic processes.”

The next section is devoted to this task.

TRANSITION PROBABILITIES FOR GENERAL BDPS

We now outline a method, first presented in Crawford and Suchard (2012) and based on work by Murphy and O’donohoe (1975), for numerically computing the transition probabilities for a general BDP with arbitrary birth and death rates. To proceed, denote the Laplace transform of P_ab(t) as

$$f_{ab}(s)=\mathcal{L}[P_{ab}(t)](s)={\int }_0^{\infty }{e}^{-st}{P}_{ab}(t)\mathrm{d}t.$$ (19)

Now, applying the Laplace transform to (5) with a = 0, we have

$$\begin{gathered} s{f}_{00}(s)-P_{00}(0)={\mu }_1{f}_{01}(s)-{\lambda }_0{f}_{00}(s),\text{and} \\ s{f}_{0b}(s)-P_{0b}(0)={\lambda }_{n-1}{f}_{0,b-1}(s)+{\mu }_{b+1}{f}_{0,b+1}(s)-({\lambda }_b+{\mu }_b)f_{0b}(s) \end{gathered}$$ (20)

for b ≥ 1. Recalling that P₀₀(0) = 1 and P_0b(0) = 0 for b ≥ 1, we rearrange (20) to find

$$\begin{gathered} f_{00}(s)=\frac{1}{s+{\lambda }_0-{\mu }_1\left(\frac{f_{01}(s)}{f_{00}(s)}\right)},\text{and} \\ \frac{f_{0b}(s)}{f_{0,b-1}(s)}=\frac{{\lambda }_{b-1}}{s+{\mu }_b+{\lambda }_b-{\mu }_{b+1}\left(\frac{f_{0,b+1}(s)}{f_{0b}(s)}\right)}. \end{gathered}$$ (21)

By combining these recurrence relations, we obtain the generalized continued fraction

$$f_{00}(s)=\frac{1}{s+{\lambda }_0-\frac{{\lambda }_0{\mu }_1}{s+{\lambda }_1+{\mu }_1-\frac{{\lambda }_1{\mu }_2}{s+{\lambda }_2+{\mu }_2-\cdots }}},$$ (22)

that is an exact expression for the Laplace transform of the transition probability P₀₀(t) (Bordes & Roehner, 1983; Flajolet & Guillemin, 2000; Guillemin & Pinchon, 1999; Karlin & McGregor, 1957b). Now define a₁ = 1, a_n = −λ_n−2μ_n−1, b₁ = s+λ₀ and b_n = s+λ_n−1+μ_n−1 for n ≥ 2. Then (22) becomes

$$f_{00}(s)=\frac{a_1}{b_1+}\frac{a_2}{b_2+}\frac{a_3}{b_3+}\cdots$$ (23)

in more concise notation. We denote the kth convergent of the Laplace transform f₀₀(s) by

$$f_{00}^{(k)}(s)=\frac{a_1}{b_1+}\frac{a_2}{b_2+}\cdots \frac{a_k}{b_k}=\frac{A_k(s)}{B_k(s)}.$$ (24)

The main result of Crawford and Suchard (2012) is the following theorem giving continued fraction expressions for the Laplace transform of the transition probability in a general birth-death process.

Theorem 1

The Laplace transform of the transition probability P_ab(t) is given by

$$\begin{gathered} f_{ab}(s)= \\ \{\begin{array}{ll}\left(\prod _{j=b+1}^a{\mu }_j\right)\frac{B_b(s)}{B_{a+1}(s)+}\frac{B_a(s)a_{a+2}}{b_{a+2}+}\frac{a_{a+3}}{b_{a+3}+}\cdots \hfill & \mathit{for}b\le a,\hfill \\ \left(\prod _{j=a}^{b-1}{\lambda }_j\right)\frac{B_a(s)}{B_{b+1}(s)+}\frac{B_b(s)a_{b+2}}{b_{b+2}+}\frac{a_{b+3}}{b_{b+3}+}\cdots \hfill & \mathit{for}a\le b,\hfill \end{array} \end{gathered}$$ (25)

where a_n, b_n, and B_n are as defined above.

The proof of this theorem relies on elementary manipulation of the continued fraction recurrences (21).

COMPUTATION OF TRANSITION PROBABILITIES

The Laplace transforms (25) usually cannot be inverted analytically to obtain time-domain transition probabilities. However, the representation (25) has several desirable properties for computational inversion. First, infinite continued fraction representations often converge much faster than their corresponding power series (Bankier & Leighton, 1942; Wall, 1948). Second, highly efficient numerical methods exist for evaluating continued fractions to a specified depth (Blanch, 1964; Lange, 2010b; Lorentzen & Waadeland, 1992; Wallis, 1972). Third, stable numerical inversion of Laplace transforms is well established, and methods exist for bounding the discretization and approximation error (Abate & Whitt, 1999; Craviotto, Jones, & Thron, 1993; Cuyt, Petersen, Verdonk, Waadeland, & Jones, 2008). For these reasons, Crawford and Suchard (2012) obtain time-domain transition probabilities P_ab(t) from (25) by numerically inverting the Laplace transforms. Error bounds for numerical Laplace inversion are derived in Crawford and Suchard (2012) and Crawford et al. (2014).

Alternatively, when the state space of a BDP is finite, numerical computation of transition probabilities can sometimes be accomplished by matrix methods. For a BDP that takes values on {0, …, N}, consider the N + 1 × N + 1 stochastic transition rate matrix Q whose elements are

$$Q=\left(\begin{array}{cccccc}-{\lambda }_0& {\lambda }_0& & & & \\ {\mu }_1& -({\lambda }_1+{\mu }_1)& {\lambda }_1& & & \\ & {\mu }_2& -({\lambda }_2+{\mu }_2)& {\lambda }_2& & \\ & & \ddots & \ddots & \ddots & \\ & & & {\mu }_N& -({\lambda }_N+{\mu }_N)& {\lambda }_N\end{array}\right).$$ (26)

When the eigendecomposition Q = UDU⁻¹ exists, where U is orthogonal and D is diagonal, the matrix of transition probabilities P(t) satisfies

$$\frac{\mathrm{d}P}{\mathrm{d}t}=PQ$$ (27)

with P(0) = I. Then the time-domain transition probabilities can obtained by matrix exponentiation,

$$P(t)=exp[Qt]=U{e}^{tD}{U}^{-1}.$$ (28)

When the state space is large or infinite, it is sometimes possible to truncate the state space at a suitably large index N and compute transition probabilities using (28). When the decomposition exists and can be found in a numerically stable way, Crawford, Stutz, and Lange (2016) give coupling bounds for finding an appropriate truncation index to control the truncation error.

FIRST PASSAGE TIMES

Now consider the time of first arrival of a BDP X(t) into an arbitrary set S of taboo states, and suppose X(0) = i ∈ ℕ \ S. This first passage time is defined formally as

$${\tau }_i=inf\{t:X(t)\in S\mid X(0)=i\}.$$ (29)

To find the relationship between first passage times and the expressions for transition probabilities discussed above, construct a new process Y (t) identical to X(t) except that λ_j = μ_j = 0 for every j ∈ S, so every state in S is absorbing. Then for this modified process, with P_ij(t) = Pr(Y (t) = j | Y (0) = i),

$$Pr({\tau }_i<t)=\sum _{j\in S}{P}_{ij}(t).$$ (30)

The intuitive reason for this equality is the absorbing nature of the states in S: if Y reaches an absorbing state j ∈ S at any time before t, we must have Y (t) = j. Furthermore, Y cannot visit more than one state in S, so the absorption events are mutually exclusive and the probability of absorption is simply the sum of the individual absorption probabilities. Therefore the cumulative distribution function of the first passage time into S is given by the sum of the transition probabilities from i to every taboo state in S for the modified process Y (t).

LIKELIHOODS

One factor hindering more widespread adoption of BDPs by applied researchers is the difficulty in performing statistical estimation of the unknown parameters in a BDP using real-world data (Doss, Suchard, Holmes, Kato-Maeda, & Minin, 2013; Holmes & Bruno, 2001). Typically efforts in estimation for BDPs have been limited to continuous observation of the process (Anscombe, 1953; Darwin, 1956; P. Moran, 1951, 1953; Reynolds, 1973; Wolff, 1965). In addition, much work to date has focused on the simple linear BDP because it is analytically tractable (Dauxois, 2004; Keiding, 1975; Rosenberg et al., 2003; Thorne et al., 1991), though important progress has been made in analysis of nonlinear processes (Karev et al., 2002; Klar et al., 2010; Reed & Hughes, 2004). In practice researchers often observe data from BDPs only at discrete times through longitudinal sampling. In addition, the simple linear BDP may be unappealing because it fails to capture more complicated dynamics of population growth and decay that arise when particles do not behave independently. To learn from discretely-observed general BDPs, we will need more advanced statistical tools.

LIKELIHOOD FOR THE CONTINUOUSLY-OBSERVED PROCESS

In a discretely-observed general BDP, the likelihood cannot be written in closed form, making analytic maximum likelihood estimation impossible. However, the likelihood of a continuously-observed BDP is straightforward to express (Keiding, 1975; Reynolds, 1973). To develop the likelihood for continuously-observed data from a general BDP, we note the following important fact: the exponentially distributed waiting time of a continuous-time Markov process in a certain state is independent of the destination of the next jump (Lange, 2010a). Recall that the waiting time W for the first event to occur from state k is exponentially distributed with rate λ_k +μ_k. If the waiting time in the current state k is W = τ, and the next change is a birth,

$$Pr(W=\tau ,\text{birth}\mid X(0)=k)=Pr(W=\tau \mid X(0)=k)Pr(\text{birth}\mid X(0)=k)=({\lambda }_k+{\mu }_k)e^{-({\lambda }_k+{\mu }_k)\tau }\left(\frac{{\lambda }_k}{{\lambda }_k+{\mu }_k}\right)={\lambda }_k{e}^{-({\lambda }_k+{\mu }_k)\tau }.$$ (31)

Likewise, the probability of a waiting time W = τ followed by a death is

$$Pr(W=\tau ,\text{death}\mid X(0)=k)={\mu }_k{e}^{-({\lambda }_k+{\mu }_k)\tau }.$$ (32)

Since we can only observe the process for a finite time t, the last observation will be the waiting time in some state k from the time of the jump to k to the end of observation. Using the same reasoning,

$$Pr(W\ge \tau \mid \text{no}\text{births}\text{or}\text{deaths},X(0)=k)=e^{-({\lambda }_k+{\mu }_k)\tau }.$$ (33)

To write the likelihood of a continuously-observed BDP from time 0 to t, we introduce some notation to ease our presentation. Suppose we observe n jumps in the time interval (0, t), and label the jumps i = 1, …, n. Let W_i be the waiting time in the current state just before the ith jump. Define the indicator B_i = 1 if the ith jump is a birth, and B_i = 0 if the ith jump is a death. Let t₁, …, t_n be the times of the n jumps, with t₀ = 0 and t_n < t. Then the likelihood of a sequence of observations Y = {X(τ ), 0 < τ < t} is

$$\begin{gathered} L=\prod _{i=1}^n\left[Pr\left(W_i=t_i-t_{i-1}\mid X(t_{i-1})\right)\times Pr{\left(\text{birth}\mid X(t_{i-1})\right)}^{B_i}Pr{\left(\text{death}\mid X(t_{i-1})\right)}^{1-B_i}\right] \\ \times Pr\left(W_{n+1}=t-t_n\mid \text{no}\text{births}\text{or}\text{deaths},X(t_n)\right) \\ =\prod _{i=1}^n({\lambda }_{X(t_{i-1})}+{\mu }_{X(t_{i-1})})exp\left[-({\lambda }_{X(t_{i-1})}+{\mu }_{X(t_{i-1})})(t_i-t_{i-1})\right] \\ \times \left(\frac{{\lambda }_{X(t_{i-1})}^{B_i}{\mu }_{X(t_{i-1})}^{1-B_i}}{{\lambda }_{X(t_{i-1})}+{\mu }_{X(t_{i-1})}}\right)\times exp\left[-({\lambda }_{X(t_n)}+{\mu }_{X(t_n)})(t-t_n)\right] \\ =\prod _{i=1}^n{\lambda }_{X(t_{i-1})}^{B_i}{\mu }_{X(t_{i-1})}^{1-B_i}exp\left[-({\lambda }_{X(t_{i-1})}+{\mu }_{X(t_{i-1})})(t_i-t_{i-1})\right] \\ \times exp\left[-({\lambda }_{X(t_n)}+{\mu }_{X(t_n)})(t-t_n)\right], \end{gathered}$$ (34)

where X(t_i−1) is the state just before the ith jump. This cumbersome notation can be eliminated if we instead keep track of the total waiting time in each state and the number of births and deaths from each state. Define 𝟙{E} to be the indicator of an event E, and let

$$T_k=\sum _{i=1}^n(t_i-t_{i-1})𝟙\{X(t_{i-1})=k\}$$ (35)

be the total time spent in state k over all visits to k. Then let

$$U_k=\sum _{i=1}^n𝟙\{X(t_{i-1})=k,B_i=1\}$$ (36)

be the number of up steps (births) from state k, and let

$$D_k=\sum _{i=1}^n𝟙\{X(t_{i-1})=k,B_i=0\}$$ (37)

be the number of down steps (deaths) from state k. Then we can re-write the likelihood (34) in much simpler and more transparent form as

$$L=\sum _{k=0}^{\infty }{\lambda }_k^{U_k}{\mu }_k^{D_k}exp[-({\lambda }_k+{\mu }_k)T_k].$$ (38)

Of course, in a BDP observed continuously for a finite time (for which (18) holds), there are only finitely many jumps observed, so the product above is not really infinite in practice.

Equation (38) also reveals that the likelihood for a continuously-observed BDP is a member of the exponential family, where {U_k}, {D_k}, and {T_k} for k = 0, 1, … are the sufficient statistics of the continuously-observed BDP likelihood. In other words, one only needs to know the total number of up and down steps from, and time spent in, each state k visited by the process in order to compute the likelihood.

EXAMPLE: CONTINUOUSLY-OBSERVED KENDALL PROCESS

Maximum likelihood estimation for continuously-observed BDPs is often straightforward. Consider the simple linear BDP with birth rate λ_k = kλ and death rate μ_k = kμ. The likelihood (38) of a single observation, up to a normalizing constant, becomes

$$L\propto {\lambda }^U{\mu }^{D}exp\left[-(\lambda +\mu ){\int }_0^{t}X(\tau )\mathrm{d}\tau \right],$$ (39)

where U = Σ_k U_k is the total number of up steps (births), D = Σ_k D_k is the total number of down steps (deaths) during the interval (0, t), and

$${\int }_0^{t}X(\tau )\mathrm{d}\tau =\sum _{k=0}^{\infty }k{T}_k$$ (40)

is the “total particle time” or total time lived by every particle that existed during the interval (0, t). Maximizing (39) with respect to the unknown parameters λ and μ, we obtain the maximum likelihood estimators

$$\widehat{\lambda }=\frac{U}{{\int }_0^{t}X(\tau )\mathrm{d}\tau }\text{and}\widehat{\mu }=\frac{D}{{\int }_0^{t}X(\tau )\mathrm{d}\tau },$$ (41)

first given by Reynolds (1973). Although the estimators provided by (41) involve an integral over the state path of the process, the integrand is simply a step function that is fully observed over (0, t).

LIKELIHOOD FOR THE DISCRETELY-OBSERVED PROCESS

Suppose now that the process X(τ ) is observed only discretely, once at time 0 and again at time t, without loss of generality owing to the Markov assumption. Let us label the state of the BDP at these times as X(0) = a and X(t) = b. Then given that X(0) = a, the probability that X(t) = b is the transition probability P_ab(t). Above we outlined a method for numerically computing this probability for any general BDP. If we regard the transition probability P_ab(t) as a function of some unknown parameters θ which control the birth and death rates, writing P_ab(t|θ), then we have the likelihood of our observation,

$$L(\theta )=P_{ab}(t\mid \theta ).$$ (42)

In principle, we could numerically maximize the likelihood for discrete observations to find an estimate of θ. However, as the number of parameters increases, naïve numerical optimization often suffers from poor convergence (Doss et al., 2013). The difficulty in writing or computing the likelihood for discrete observations from BDPs has limited the usefulness of BDPs in applications.

In contrast to the appealing analytic characterization (38) of the continuously-observed process likelihood, the discretely-observed process is hard to characterize. To bridge this gap, it is helpful to view computation of the likelihood in the discretely-observed process as a missing data problem. When a BDP is observed discretely, we do not know the sufficient statistics ${\{U_k,D_k,T_k\}}_{k=0}^{\infty }$. This perspective suggests that we exploit analytic information about these statistics, conditional on the start and end states of the observed process.

EM ALGORITHMS FOR MAXIMUM LIKELIHOOD ESTIMATION

In this section, we review the estimation machinery developed by Crawford et al. (2014) for maximum likelihood or maximum a posteriori estimation in BDPs. When a BDP is discretely sampled, U_k, D_k, and T_k are unobserved for every k; we cannot maximize the likelihood without knowing these statistics. We therefore appeal to the expectation-maximization (EM) algorithm for iterative maximum likelihood estimation with missing data (Dempster, Laird, & Rubin, 1977). When the incomplete data likelihood is intractable but the complete data likelihood has a simple form, the EM algorithm operates by replacing each missing datum by a conditional expectation as follows. If X is the complete (unobserved data), Y represents the incomplete (observed) data, and (θ|X) is the complete data log-likelihood, we form a surrogate function Q as the expectation of the complete data likelihood, conditional on the observed data Y and the current (mth) parameter iterate:

$$Q\left(\theta \mid {\theta }^{(m)}\right)=\mathbb{E}\left(\ell (\theta \mid X)\mid Y=y,{\theta }^{(m)}\right).$$ (43)

This is the E-step of the EM algorithm, and it accomplishes a minorization of (θ|X) at θ^(m). The M-step maximizes (or takes a step toward the maximum of) Q. By alternating these steps — minorizing ℓ by Q, then finding a θ that increases Q — the EM algorithm drives succeeding iterates toward the MLE.

Taking the expectation of the logarithm of (38), conditional on the observed data Y = (X(0) = a, X(t) = b, t) and the current parameter estimate θ^(m), we write the surrogate function for the BDP as follows:

$$\begin{gathered} Q\left(\theta \mid {\theta }^{(m)}\right)=\mathbb{E}\left[\ell (\theta )\mid Y,{\theta }^{(m)}\right] \\ =\sum _{k=0}^{\infty }\mathbb{E}(U_k\mid Y)log\left[{\lambda }_k(\theta )\right]+\mathbb{E}(D_k\mid Y)log\left[{\mu }_k(\theta )\right]-\mathbb{E}(T_k\mid Y)\left[{\lambda }_k(\theta )+{\mu }_k(\theta )\right], \end{gathered}$$ (44)

In the above equation and many that follow, we omit the dependence of the conditional expectations on θ^(m) from the mth iterate for visual clarity.

To calculate the conditional expectations necessary for the E-step of the EM algorithm, we appeal to the following integral expressions

$$\mathbb{E}(U_k\mid Y)=\frac{{\int }_0^t{P}_{ak}(\tau ){\lambda }_k{P}_{k+1,b}(t-\tau )\mathrm{d}\tau }{P_{ab}(t)},$$ (45a)

$$\mathbb{E}(D_k\mid Y)=\frac{{\int }_0^t{P}_{ak}(\tau ){\mu }_k{P}_{k-1,b}(t-\tau )\mathrm{d}\tau }{P_{ab}(t)},\text{and}$$ (45b)

$$\mathbb{E}(T_k\mid Y)=\frac{{\int }_0^t{P}_{ak}(\tau )P_{kb}(t-\tau )\mathrm{d}\tau }{P_{ab}(t)}.$$ (45c)

These expressions have appeared repeatedly in literature on inference for discretely-observed continuous-time Markov chains (Bladt & Sorensen, 2005; Hobolth & Jensen, 2005; Holmes & Rubin, 2002; Lange, 1995; Metzner, Dittmer, Jahnke, & Schütte, 2007). When the process takes only finitely many states, matrix solutions are possible using the uniformization method (Neuts, 1995). Hobolth and Stone (2009) develop efficient Monte Carlo methods using simulation conditioned on the start and end points of the discrete observation Y. Finally, Doss et al. (2013) study a linear BDP on an infinite state space and derive the expectations analytically using a generating function argument. None of the exact methods is a general approach for arbitrary BDPs on an infinite state space. The Monte Carlo approaches, while not reliant on a particular parameterization of the process, can suffer from poor performance when observed realizations occur with low probability. The lack of a reliable method for computing the E-step of the EM algorithm for discretely-observed BDPs has hindered progress on statistical inference for these processes.

An alternative approach introduced by Crawford et al. (2014) takes advantage of the Laplace transforms f_ab(s) of the transition probabilities (25). The numerators in (45) are time-domain convolutions of transition probabilities. The functional form of these expressions suggests using the Laplace convolution property to obtain

$$\mathbb{E}(U_k\mid Y)={\lambda }_k\frac{{\mathcal{L}}^{-1}\left[f_{ak}(s)f_{k+1,b}(s)\right](t)}{P_{ab}(t)},$$ (46a)

$$\mathbb{E}(D_k\mid Y)={\mu }_k\frac{{\mathcal{L}}^{-1}\left[f_{ak}(s)f_{k-1,b}(s)\right](t)}{P_{ab}(t)},\text{and}$$ (46b)

$$\mathbb{E}(T_k\mid Y)=\frac{{\mathcal{L}}^{-1}\left[f_{ak}(s)f_{kb}(s)\right](t)}{P_{ab}(t)},$$ (46c)

where ℒ⁻¹[·] denotes inverse Laplace transformation. These expressions are formally equivalent to (45), but they offer substantial computational time savings over numerical integration of (45), and make possible efficient computation of conditional expectations for EM algorithms for any BDP (Crawford et al., 2014).

We now show how to complete the M-step for several BDP models. The first two, variations on the simple linear (Kendall) process, were given in Crawford et al. (2014). The others are novel, yet remarkably easy to derive and implement computationally. In each case, we describe the surrogate likelihood function Q(θ|θ^(m)) and give the M-step updates for each unknown parameter.

EXAMPLE: DISCRETELY-OBSERVED KENDALL PROCESS

In the simple linear BDP, births and deaths happen at constant per-particle rates, so λ_k = kλ and μ_k = kμ. The unknown is θ = (λ, μ). The surrogate function Q becomes

$$Q(\theta )=\sum _{k=0}^{\infty }\mathbb{E}(U_k\mid Y)log[k\lambda ]+\mathbb{E}(D_k\mid Y)log[k\mu ]-\mathbb{E}(T_k\mid Y)k(\lambda +\mu ).$$ (47)

Maximizing (47) with respect to the θ yields the updates:

$${\lambda }^{(m+1)}=\frac{\mathbb{E}(U\mid Y)}{\mathbb{E}(T_{\text{particle}}\mid Y)}\text{and}$$ (48a)

$${\mu }^{(m+1)}=\frac{\mathbb{E}(D\mid Y)}{\mathbb{E}(T_{\text{particle}}\mid Y)},$$ (48b)

where

$$T_{\text{particle}}={\int }_0^{t}X(\tau )\mathrm{d}\tau ,$$ (49)

and we have again suppressed the dependence of the conditional expectations on θ^(m) for typographic clarity. These expressions are identical in form to the estimators given in (41), but are instead iterative updates in the EM algorithm.

EXAMPLE: LINEAR BDP WITH IMMIGRATION

The linear BDP with immigration is similar to the simple linear BDP, but there is a source of new arrivals whose rate is constant and does not depend on the number of particles already in existence. This yields the birth and death rates λ_k = kλ+ν and μ_k = kμ. The log-likelihood becomes

$$\ell (\theta )=\sum _{k=0}^{\infty }{U}_{k}log(k\lambda +\nu )+D_{k}log(\mu )-T_k[k(\lambda +\mu )+\nu ].$$ (50)

Unfortunately, it is difficult to maximize the resulting surrogate function analytically. But since each term in the sum is a concave function of the unknown parameters, we can separate them in a second minorizing function H such that for all θ, H(θ|θ^(m)) ≤ ℓ(θ) and H(θ^(m)|θ^(m)) = ℓ(θ^(m)). To accomplish the minorization, note that

$$log(k\lambda +\nu )\ge \frac{k{\lambda }^{(m)}}{k{\lambda }^{(m)}+{\nu }^{(m)}}log\left[\frac{k{\lambda }^{(m)}+{\nu }^{(m)}}{{\lambda }^{(m)}}\lambda \right]+\frac{{\nu }^{(m)}}{k{\lambda }^{(m)}+{\nu }^{(m)}}log\left[\frac{k{\lambda }^{(m)}+{\nu }^{(m)}}{{\nu }^{(m)}}\nu \right].$$ (51)

We form a minorizing log-likelihood function H as follows:

$$\begin{gathered} H\left(\theta \mid {\theta }^{(m)}\right)\propto \sum _{k=0}^{\infty }{U}_k\left[p_k^{(m)}log(\lambda )+(1-p_k^{(m)})log(\nu )\right]+D_{k}log(\mu ) \\ -\left[k(\lambda +\mu )+\nu \right]T_k, \end{gathered}$$ (52)

where

$$p_k^{(m)}=\frac{k{\lambda }^{(m)}}{k{\lambda }^{(m)}+{\nu }^{(m)}}.$$ (53)

Exploiting this surrogate function and maximizing with respect to the unknown sufficient statistics gives the updates

$${\lambda }^{(m+1)}=\frac{{\displaystyle \sum _{k=0}^{\infty }}{p}_k\mathbb{E}(U_k\mid Y)}{\mathbb{E}(T_{\text{particle}}\mid Y)},\text{and}$$ (54a)

$${\nu }^{(m+1)}=\frac{{\displaystyle \sum _{k=0}^{\infty }}(1-p_k)\mathbb{E}(U_k\mid Y)}{t}.$$ (54b)

The update for μ is the same as (48b).

EXAMPLE: PURE-BIRTH AND GENERALIZED POISSON PROCESSES

Recall that the Poisson process with arrival rate λ is a BDP with λ_k = λ, μ_k = 0 for all k. Many researchers have found that real-world count data are sometimes over- or under-dispersed relative to the Poisson distribution. Statisticians seeking a more flexible distribution for count outcomes that can accommodate over- and under-dispersion have arrived at several alternative distributions. A notable example that fits neatly into the BDP framework is the general pure-birth process with arbitrary birth rates λ_k, k = 0, 1, …, and μ_k = 0 for all k. This class of processes has an appealing property: it can recover any discrete probability distribution on the counting numbers by appropriately setting the birth rates (M. Faddy & Bosch, 2001; M. J. Faddy, 1997). Crawford and Zelterman (2015) show that any such pure-birth process can be represented as a sum of exchangeable Bernoulli random variables, a result that connects BDPs with phenomenological models often used for dependent outcomes in toxicology and epidemiology. Renshaw (2011, page 65) gives an analytic form for these transition probabilities

$$P_{ab}(t)=\left(\prod _{k=a}^{b-1}{\lambda }_k\right)\sum _{k=a}^b{\left(\prod _{\ell \ne k}({\lambda }_{\ell }-{\lambda }_k)\right)}^{-1}exp\left[-{\lambda }_{k}t\right]$$ (55)

for 0 ≤ a ≤ b and t > 0 provided that λ_i ≠ λ_j for all i and j. While (55) has an appealing form, it depends on none of the birth rates being equal. Another potentially serious drawback is that it can be numerically troublesome to compute; the summands may be alternating in sign and the product of small differences in the denominators can lead to serious roundoff error. In many scenarios, especially when some observed counts are large and some λ_k’s are nearly or exactly equal, (55) provides an unappealing way to compute the likelihood. Exactly equal λ_k may arise, for example, when entertaining a Bayesian non-parametric prior. Fortunately, the EM framework does not require use of (55). We now provide an example of a pure birth process intended to generalize the Poisson distribution to accommodate over- and under-dispersion.

M. J. Faddy (1997) describes a class of pure-birth BDPs with λ_k = λ(γ +k)^c and μ_k = 0, where c = 0 corresponds to a Poisson process with rate λ, c > 0 results in overdispersion relative to Poisson, and c < 0 results in underdispersion. The log-likelihood for the continuously-observed process beginning at X(0) = a and ending at X(t) = b is

$$\ell (\theta )=\sum _{k=a}^b𝟙\{k<b\}[log(\lambda )+clog(\gamma +k)]-\lambda {(\gamma +k)}^c{T}_k.$$ (56)

Letting θ = (λ, γ, c), the surrogate function is

$$Q(\theta )=\sum _{k=a}^b𝟙\{k<b\}[log(\lambda )+clog(\gamma +k)]-\lambda {(\gamma +k)}^c\mathbb{E}(T_k\mid \mathbf{Y}).$$ (57)

The update for λ is given by

$${\lambda }^{(m+1)}=\frac{b-a-1}{{\displaystyle \sum _{k=a}^b}{(\gamma +k)}^c\mathbb{E}(T_k\mid \mathbf{Y})},$$ (58)

but the updates for γ and c are not available in closed form. However, Lange (1995) shows that one step of a gradient ascent algorithm suffices to preserve the ascent property of the EM algorithm. Therefore a Newton-Raphson update can be derived, and

$$\left(\begin{array}{c}{\gamma }^{(m+1)}\\ c^{(m+1)}\end{array}\right)={\mathbf{d}}^{2}Q{({\gamma }^{(m)},c^{(m)})}^{-1}\nabla Q({\gamma }^{(m)},c^{(m)}),$$ (59)

where ∇Q and d²Q are the gradient and Hessian of Q with respect to γ and c respectively.

EXAMPLE: MORAN MODEL

The Moran process models genetic drift in a finite population by keeping track of the number of alleles of a certain type at a biallelic locus in a haploid population of constant size N < ∞. Call the two alleles A and B, and suppose we wish to keep track of the number of A carriers in the population. In the Moran model with selection, carriers of A have fitness α, and carriers of B have fitness β. For the sake of identifiability in a statistical setting, we specify β = 1 and let α denote the relative fitness of A carriers over B carriers. Furthermore, A mutates to B in one generation with probability u, and vice versa with probability v. When an existing individual dies, a new allele is drawn at random. The birth and death rates are

$$\begin{gathered} {\lambda }_n=\frac{N-n}{N}\left[\alpha \frac{n}{N}(1-u)+\frac{N-n}{N}v\right],\text{and} \\ {\mu }_n=\frac{n}{N}\left[\frac{N-n}{N}(1-v)+\alpha \frac{n}{N}u\right] \end{gathered}$$ (60)

for n = 0, … , N. Forming the surrogate function from (44), we see that maximizing the log-likelihood with respect to the unknowns α, u, and v is difficult. However, we can again construct a minorizing function to separate the parameters in the logarithm terms. We minorize the birth rate as

$$\begin{gathered} log({\lambda }_n)\propto log[n\alpha (1-u)+(N-n)v] \\ \ge p_n^{(m)}log\left(\frac{n{\alpha }^{(m)}(1-u^{(m)})+(N-n)v^{(m)}}{{\alpha }^{(m)}(1-u^{(m)})}\alpha (1-u)\right)+(1-p_n^{(m)})log\left(\frac{n{\alpha }^{(m)}(1-u^{(m)})+(N-n)v^{(m)}}{v^{(m)}}v\right) \\ \propto p_n^{(m)}\left(log(\alpha )+log(1-u)\right)+(1-p_n^{(m)})log(v), \end{gathered}$$ (61)

where

$$p_n^{(m)}=\frac{n{\alpha }^{(m)}(1-u^{(m)})}{n{\alpha }^{(m)}(1-u^{(m)})+(N-n)v^{(m)}}.$$ (62)

Although (61) and (62) may appear complicated, this minorization has the effect of separating the parameters α and u in the surrogate function, allowing closed-form updates. In a similar way, we minorize the log-death rate log(μ_n) as

$$q_n^{(m)}log(1-v)+(1-q_n^{(m)})\left(log(\alpha )+log(u)\right),$$ (63)

where

$$q_n^{(m)}=\frac{(N-n)(1-v^{(m)})}{(N-n)(1-v^{(m)})+n{\alpha }^{(m)}{u}^{(m)}}.$$ (64)

We form the complete minorizing function H as

$$\begin{gathered} H(\theta )=\sum _{k=0}^N{B}_k\left[p_k^{(m)}\left(log(\alpha )+log(1-u)\right)+(1-p_k^{(m)})log(v)\right] \\ +D_k\left[q_k^{(m)}log(1-v)+(1-q_k^{(m)})\left(log(\alpha )+log(u)\right)\right] \\ -\frac{T_k}{N^2}\left[(N-k)k\alpha (1-u)+{(N-k)}^{2}v+(N-k)k(1-v)+k^2\alpha u\right], \end{gathered}$$ (65)

and the surrogate function is Q(θ) = 𝔼(H(θ)|Y, θ^(m)). A simple way to proceed is to find updates for each of the unknowns individually, conditional on the previous (mth) estimate of the others, giving a cyclic coordinate ascent algorithm. The update for α is

$${\alpha }^{(m+1)}=\frac{{\displaystyle \sum _{k=0}^N}{p}_k^{(m)}{B}_k+(1-q_k^{(m)})D_k}{\frac{1}{N^2}{\displaystyle \sum _{k=0}^N}{T}_k\left[(N-k)k(1-u^{(m)})+k^2{u}^{(m)}\right]}.$$ (66)

The update for u is the positive solution of the quadratic equation

$$0=\sum _{k=0}^N-u{B}_k{p}_k^{(m)}+(1-u)D_k(1-q_k^{(m)})-u(1-u)\frac{T_k}{N^2}\left[k^2{\alpha }^{(m)}-(N-k)k{\alpha }^{(m)}\right],$$ (67)

when 0 < u < 1. The update for v is obtained by similar manipulations.

EXAMPLE: MAXIMUM A POSTERIORI ESTIMATION FOR THE KENDALL PROCESS

In a Bayesian setting, a prior distribution f(θ) on the unknown parameters θ is given, and we seek to maximize the log-posterior distribution of the parameters, given the data, Pr(θ | Y ) ∝ Pr(Y | θ)f(θ) to obtain the maximum a posteriori (MAP) estimate of θ. Here the surrogate function becomes Q(θ|θ^(m)) = 𝔼(ℓ(θ)|Y, θ^(m)) + log [f (θ)]. To illustrate, suppose that independent observations from a BDP follow the simple linear model, and we believe that λ and μ are a priori independent and are Gamma-distributed:

$$\lambda ~\text{Gamma}(k_{\lambda },{\beta }_{\lambda })\text{and}\mu ~\text{Gamma}(k_{\mu },{\beta }_{\mu }).$$ (68)

Then the unknowns are θ = (λ, μ) and the log-prior for θ is

$$logf(\theta )\propto (k_{\lambda }-1)log(\lambda )+(k_{\mu }-1)log(\mu )-{\beta }_{\lambda }\lambda -{\beta }_{\mu }\mu$$ (69)

Ignoring irrelevant terms, the surrogate function becomes

$$Q\left(\theta \mid {\theta }^{(m)}\right)=\mathbb{E}(U\mid Y)log(\lambda )+\mathbb{E}(D\mid Y)log(\mu )-\mathbb{E}(T_{\text{particle}}\mid Y)(\lambda +\mu )+(k_{\lambda }-1)log(\lambda )+(k_{\mu }-1)log(\mu )-{\beta }_{\lambda }\lambda -{\beta }_{\mu }\mu$$ (70)

The MAP updates are

$${\lambda }^{(m+1)}=\frac{\mathbb{E}(U\mid Y)+k_{\lambda }-1}{\mathbb{E}(T_{\text{particle}}\mid Y)+{\beta }_{\lambda }},\text{and}$$ (71a)

$${\mu }^{(m+1)}=\frac{\mathbb{E}(D\mid Y)+k_{\mu }-1}{\mathbb{E}(T_{\text{particle}}\mid Y)+{\beta }_{\mu }}.$$ (71b)

EXAMPLE: REGRESSION FOR COUNT DATA

Perhaps the most interesting use of EM algorithms for BDP inference is to provide a unified framework for regression estimation. To illustrate, consider a collection of n independent BDPs, Xⁱ(t) with ${\lambda }_k^i=exp[{\mathbf{Z}}_i\beta ]$ and ${\mu }_k^i=0$ for i = 1, … , n, where Z_i is a d × 1 vector of covariates and β is a covariate vector of corresponding dimension and μ_k = 0 for all k. Then letting Xⁱ(0) = 0 and Xⁱ(1) = x_i for each i, the log-likelihood becomes

$$\ell (\mathit{\beta })=\sum _{i=1}^n{x}_i{\mathbf{Z}}_i\mathit{\beta }-exp[{\mathbf{Z}}_i\mathit{\beta }].$$ (72)

This is the log-likelihood for classical Poisson regression, and updates are found using a Newton-Raphson step (Dobson, 2001).

It is possible to formulate an analogous model for the Kendall process. Let ${\lambda }_k^i=kexp[{\mathbf{Z}}_i\mathit{\beta }]$ and ${\mu }_k^i=kexp[{\mathbf{W}}_i\mathit{\gamma }]$ be the birth and death rates of a BDP Xⁱ(t). The log-likelihood is

$$\ell (\mathit{\beta },\mathit{\gamma })\propto \sum _{i=1}^n\sum _{k=0}^{\infty }{U}_k^i{\mathbf{Z}}_i\mathit{\beta }+D_k^i{\mathbf{W}}_i\mathit{\gamma }-k{T}_k^i(exp[{\mathbf{Z}}_i\mathit{\beta }]+exp[{\mathbf{W}}_i\mathit{\gamma }]),$$ (73)

where the statistics $U_k^i,D_k^i$, and $T_k^i$ correspond to observation i. When the process is discretely-observed, we form the surrogate as before, and find the gradient vector

$${\nabla }_{\beta }Q=\sum _{i=1}^n\mathbb{E}[U^i\mid Y_i]{\mathbf{Z}}_i-\mathbb{E}[T_{\text{particle}}^i\mid Y_i]{\mathbf{Z}}_{i}exp[{\mathbf{Z}}_i\mathit{\beta }]$$ (74)

for β. The Hessian matrix is

$$d_{\beta }^{2}Q=-\sum _{i=1}^n\mathbb{E}[T_{\text{particle}}^i\mid Y_i]{\mathbf{Z}}_i{\mathbf{Z}}_i^{\prime }exp[{\mathbf{Z}}_i\mathit{\beta }].$$ (75)

Then, the Newton-Raphson update for β becomes

$${\mathit{\beta }}^{(m)}={\mathit{\beta }}^{(m-1)}-{\left(d_{\beta }^{2}Q\right)}^{-1}{\nabla }_{\beta }Q.$$ (76)

A similar update is available for γ. We contrast the simplicity of the update expressions (76) with the formula for the Kendall process transition probability (9).

INTEGRAL FUNCTIONALS OF BDPS

Many important real-life applications of BDPs can be characterized as questions about the distribution of summary statistics. A common feature of stochastic processes in decision-making contexts is that the parameters estimated by the statistical inference procedure are not always the ones of interest in the application. Often the quantity of interest is a summary statistic related to the time-integral of the process. To illustrate, let g : ℕ → [0,∞) be a function and let S be a set of “taboo” or prohibited states. Suppose the initial state of the BDP is X(0) = i ∈ ℕ\S. Define the functional

$$C_i={\int }_0^{{\tau }_i}g\left(X(t)\right)\mathrm{d}t,$$ (77)

where the upper limit of integration is the first passage time

$${\tau }_i=inf\{t:X(t)\in S\mid X(0)=i\}.$$ (78)

Here, C_i is a functional because it maps a realization of the stochastic process g(X(t)( to its integral. Figure 2 shows an example realization of a BDP and its integral C_i with S = {0}. The left-hand side shows a BDP beginning at X(0) = 1, and ending at X(τ₁) = 0. The right-hand plot shows g(X(t)( over the same time interval, and the area under the trajectory is C_i.

Figure 2.

Illustration of the integral of a functional of a general birth-death process (BDP). On the left, a BDP begins at X(0) = 1 and ends when the process reaches the absorbing state 0 just before time t = 2. On the right, $C_1={\int }_0^{{\tau }_1}g\left(X(t)\right)\mathrm{d}t$ is the area under the trajectory of g(X(t)), where g : ℕ → [0,∞) is an arbitrary positive “reward” or “cost” function. The upper limit of integration τ₁ is the first passage time to zero, beginning at X(0) = 1.

Expressions like (77) arise often in applied work. For example, epidemiologists usually estimate the parameters (contact/infection rate and recovery rate) of an epidemic process from data, but their objective is to make inference of the predictive distribution of the cost of the epidemic in the future. Operations researchers may estimate the arrival rate λ and service rate μ in a queuing process, but the object of inference is the distribution of customer-hours waited. Traffic engineers may be interested in the number of vehicle-hours waited in models for highway accident delays (Gaver, 1969).

To illustrate the role of integral summaries of BDPs in statistical prediction, let p(c|θ) be the density of C_i given θ. The posterior predictive uncertainty about the statistic is the marginal distribution

$$\begin{gathered} p(c\mid Y)={\int }_{\mathrm{\Theta }}p(c\mid \theta )p(\theta \mid Y)\mathrm{d}\theta \\ \propto {\int }_{\mathrm{\Theta }}p(c\mid \theta )p(Y\mid \theta )p(\theta )\mathrm{d}\theta , \end{gathered}$$ (79)

where p(θ|Y ) is the sampling distribution of θ given the realized data Y. In a Bayesian context, p(θ|Y ) is a posterior distribution, and we might estimate p(c|Y ) by a Monte Carlo approximation involving N draws θ_j ~ p(θ|Y ) via

$$p(c\mid Y)\approx \frac{1}{N}\sum _{j=1}^{N}p(c\mid {\theta }_j).$$ (80)

BACKGROUND ON INTEGRALS OF BDPS

Karlin and McGregor (1957a, 1957b) provided the first theoretical tools for working with integral functionals of general BDPs. P. Puri (1966); P. S. Puri (1968) derives the characteristic function for the joint distribution of simple linear BDP and its integral and gives expressions for moments and limiting distributions (P. Puri, 1972; P. S. Puri, 1971, 1972). McNeil (1970) gives the first results for general BDPs, Gani and McNeil (1971) derive expressions for the joint distribution of a general BDP and its integral, and Kaplan (1974) provides limit theorems for integrals of simple BDPs with immigration. Straightforward methods for moments of integrals of general BDPs using Laplace transforms are also available (Gani & Swift, 2008; Hernández-Suárez & Castillo-Chavez, 1999; Pollett, 2003; Pollett & Stefanov, 2003). However, most analyses of integral functionals of general BDPs are limited to simple analytically tractable models or focused on moments.

Now we consider the problem of computing the distribution of (77). Our emphasis on first-passage times as the upper limit of integration in (77) has two benefits. First, our analyses need not be conditional on an arbitrary time in the future. Second, first passage times allow us to exploit powerful analytic tools that establish a correspondence between transition probabilities and first-passage times, enabling analytic progress on integrals for arbitrary well-behaved processes. Our presentation follows the outline given by McNeil (1970). Let c_i(s) = 𝔼 (e^−sC_i] be the Laplace transform of C_i. Note that if X(0) = i ∈ S then τ_i = 0, C_i = 0, and so c_i(s) = 1. Now by an analogous conditioning argument for X(0) = i ∉ S, we re-write the Laplace transform as

$$\begin{gathered} c_i(s)={\int }_0^{\infty }\mathbb{E}\left[e^{-s(C_{i+1}+ug(i))}\right]Pr(\text{birth},U=u\mid X(0)=i)\mathrm{d}u+{\int }_0^{\infty }\mathbb{E}\left[e^{-s(C_{i-1}+ug(i))}\right]Pr(\text{death},U=u\mid X(0)=i)\mathrm{d}u \\ =\mathbb{E}\left[e^{-s{C}_{i+1}}\right]{\int }_0^{\infty }{e}^{-\mathit{sug}(i)}{\lambda }_i{e}^{-({\lambda }_i+{\mu }_i)u}\mathrm{d}u+\mathbb{E}\left[e^{-s{C}_{i-1}}\right]{\int }_0^{\infty }{e}^{-\mathit{sug}(i)}{\mu }_i{e}^{-({\lambda }_i+{\mu }_i)u}\mathrm{d}u \\ =c_{i+1}(s){\lambda }_i{\int }_0^{\infty }{e}^{-u(sg(i)+{\lambda }_i+{\mu }_i)}\mathrm{d}u+c_{i-1}(s){\mu }_i{\int }_0^{\infty }{e}^{-u(sg(i)+{\lambda }_i+{\mu }_i)}\mathrm{d}u, \end{gathered}$$ (81)

that gives

$$\left(sg(i)+{\lambda }_i+{\mu }_i\right)c_i(s)={\lambda }_i{c}_{i+1}(s)+{\mu }_i{c}_{i-1}(s).$$ (82)

Now dividing both sides of the above by g(i), we find that

$$\left(s+{\lambda }_i^{\ast }+{\mu }_i^{\ast }\right)c_i(s)={\lambda }_i^{\ast }{c}_{i+1}(s)+{\mu }_i^{\ast }{c}_{i-1}(s),$$ (83)

where ${\lambda }_i^{\ast }={\lambda }_i/g(i)$ and ${\mu }_i^{\ast }={\mu }_i/g(i)$. Therefore, we see that (83) is simply the backward equation for a modified process with birth and death rates ${\lambda }_i^{\ast }$ and ${\mu }_i^{\ast }$ for i ∈ ℕ. The forward equation for the cumulative distribution function of c_i is therefore equivalent to (5) with the modified birth and death rates.

Pollett (2003) gives the conditions, analogous to those for (16), under which this modified process explodes. We note that differentiation of solutions of (83) yields the moments of C_i, as noted by McNeil (1970) and subsequently refined by Hernández-Suárez and Castillo-Chavez (1999), Stefanov and Wang (2000), and Pollett (2003). We refer interested readers to those papers and focus here on results for the distribution of C_i that are more useful in statistical and decision applications.

To take advantage of (83), we modify (29) as follows. Fix S ⊂ ℕ and suppose X(t) is a general BDP with rates {λ_n} and {μ_n} with starting state X(0) = i ∈ ℕ\S. Suppose g(n) is a positive function defined for all n ∈ ℕ. Let Y (t) be a general BDP with rates ${\lambda }_n^{\ast }={\lambda }_n/g(n)$ and ${\mu }_n^{\ast }={\mu }_n/g(n)$ for all n ∈ ℕ\S, and ${\lambda }_n^{\ast }={\mu }_n^{\ast }=0$ for every n ∈ S. Then let $P_{ij}^{\ast }(t)=Pr(Y(t)=j\mid Y(0)=i)$. We then have

$$H(c)=Pr(C_i<c)=\sum _{j\in S}{P}_{ij}^{\ast }(c).$$ (84)

If instead of the cumulative distribution function H(c) of C_i, we wish to have the probability density, we could numerically differentiate (84). However, using the properties of the Laplace transform,

$$\begin{gathered} h(c)=\frac{\mathrm{d}}{\mathrm{d}c}Pr(C_i<c) \\ =\sum _{j\in S}\frac{\mathrm{d}}{\mathrm{d}c}{P}_{ij}^{\ast }(c) \\ =\sum _{j\in S}{\mathcal{L}}^{-1}\left[s{f}_{ij}^{\ast }(s)-P_{ij}^{\ast }(0)\right](c) \\ =\sum _{j\in S}{\mathcal{L}}^{-1}\left[s{f}_{ij}^{\ast }(s)\right](c) \end{gathered}$$ (85)

where $f_{ij}^{\ast }(s)$ is the Laplace transform of $P_{ij}^{\ast }(t)$, ℒ⁻¹[·] denotes Laplace inversion, and $P_{ij}^{\ast }(0)=0$ for all j ∈ S since we have assumed i ∉ S.

EXAMPLE: PROBABILISTIC CONTROL OF AN EPIDEMIC

In infectious disease epidemiology, stochastic modeling can give valuable insight into both disease dynamics and optimal intervention strategies (Ball, 1986; Wickwire, 1977). The total cost of an infectious disease epidemic is proportional to the area under the time trajectory of the number of infected people (Gani & Jerwood, 1972; Jerwood, 1970). To illustrate, we model the number of infected persons in a homogeneously mixing population as a type of general BDP. This simple model, called the susceptible-infected-susceptible (SIS) model, keeps track of the number of infected in a finite population of size N (N. T. Bailey, 1975). If there are currently n < N infected persons in the population, the rate of new infections is proportional to the product of the number infected n and susceptible N − n. The contact/transmission rate between infected and susceptible persons is λ. Infected persons recover and revert to susceptible status with constant per-person rate μ. For a SIS process X(t), the addition and removal rates are

$${\lambda }_n=\lambda n(N-n)\text{and}{\mu }_n=n(\mu +\epsilon ),$$ (86)

where ε is a positive control parameter related to vaccination or some other public health intervention strategy. Suppose the initial number of infected is X(0) = i ≤ N and we are interested in the total cost of the epidemic until its eventual extinction, so S = {0}. Let the cost of managing the epidemic per unit time be aε. Additionally, let the cost per infected person per unit time be b > 0, so the cost function becomes g(n) = aε + bn. Then the total cost is

$$C_i={\int }_0^{{\tau }_i}\left[a\epsilon +bX(t)\right]\mathrm{d}t=a\epsilon {\tau }_i+b{\int }_0^{{\tau }_i}X(t)\mathrm{d}t,$$ (87)

where τ_i is the time to extinction of the epidemic.

Most optimal control models seek a policy that minimizes the expected total cost, corresponding to the expectation of (77) under certain conditions on the intervention and cost functions (Cai & Luo, 1994; Clancy, 1999; Guo & Hernández-Lerma, 2009; Lefévre, 1981). The availability of probability distributions for the total cost allows us to seek the minimal intervention policy that guarantees that the total cost of the epidemic is small with high probability. Let X(t) be the process with rates given by (86) for a certain control setting ε. Then we wish to find the smallest ε such that

$$Pr\left(C_i<c\right)<1-\alpha ,$$ (88)

where c is a desired bound on the total cost, and 0 < α < 1 is a small probability. Assuming this probability is continuous and increases monotonically with ε near 1 − α, it is straightforward to find the smallest ε that satisfies (88).

Figure 3 shows how to find the minimal ε for a SIS process with N = 100 individuals, X(0) = 50, infectivity λ = 0.1, recovery rate μ = 8, control cost a = 0.1, and per-infected cost b = 0.3 per unit time. The top traces show the cumulative distribution function of the total cost for ε = 0, 0.5, 1, 1.5, 2. The vertical gray line shows C_i = 7, and we wish to keep the total cost less than 7 with probability 1− α = 0.95. The bottom trace shows Pr(C_i < 7) as a function of ε. The horizontal gray dashed line shows 0.95 probability, and the vertical gray dashed line shows the smallest value of ε (ε ≈ 3.4) that achieves this bound.

Figure 3.

Probabilistic control of a stochastic SIS epidemic. At top, the distribution of total epidemic cost C_i for different values of a control parameter ε. The dashed gray vertical line is at w = 7, and we wish to keep C_i < 7 with high probability. At bottom, the probability that C_i < 7 as a function of the control parameter ε. The horizontal gray dashed line denotes 0.95, and the vertical dashed line is the smallest epsilon that achieves Pr(C_i < 7) > 0.95; this yields ε ≈ 3.4. In this way, we can easily find the smallest value of a control parameter that bounds the probability that the epidemic will exceed a certain threshold.

DISCUSSION: COMPUTATION AND LIKELIHOOD-BASED INFERENCE FOR BDPS

BDPs are vital tools for modeling stochastic counting processes in epidemiology, evolution, ecology, chemistry, physics, and other fields. Modeling with BDPs is often straightforward; by considering rates of addition of new particles and removal of existing particles, conditional on the number already present, researchers can specify the birth and death rates ${\{{\lambda }_k,{\mu }_k\}}_{k=0}^{\infty }$. The ease of modeling with BDPs stands in stark contrast to the computational difficulty of inference using stochastic realizations of BDPs. Routine use of BDPs in statistical settings has been thwarted by intractable likelihoods and burdensome computations. A unified perspective on BDPs with arbitrary birth and death rates has remained elusive, until recently.

Laplace transforms of transition probabilities provide the essential analytic tools for bridging this gap in practice. Our approach for computing transition probabilities (likelihoods) in (25) and conditional expectations in the E-step (46) is general, robust, and computationally efficient. Laplace transforms of first-passage times also play an important role in finding the distribution of integral functionals of BDPs in applications. As a theoretic tool, this Laplace-perspective is not new; Karlin and McGregor (1957a, 1958a, 1957b) discuss the fundamental importance of Laplace transforms for analysis of BDPs. More recent results related to combinatorial properties of BDPs also rely on Laplace transforms (Flajolet & Guillemin, 2000; Guillemin & Pinchon, 1998, 1999).

Extensions to the general approach presented here have been developed for multivariate BDPs, where progress has been slower and analytic approaches to reducing computation are less readily available. Analytic formulae of transition probabilities for the simplest multivariate Kendall-like processes – called monomolecular reaction systems – have only recently been derived (Jahnke & Huisinga, 2007). Notably, Xu, Guttorp, Kato-Maeda, and Minin (2015) propose a fast algorithm to compute the transition probabilities of multi-type branching processes using a generating function approach. However, this method is only applicable for processes whose transition rates are linear. The first result for non-linear multivariate BDPs has recently been established by Ho, Xu, Crawford, Minin, and Suchard (2017), who consider a subclass of bivariate BDPs called birth/birth-death processes and develop an efficient computation method for the transition probabilities by deriving recursion formulae for their Laplace transforms. Nonetheless, the problem of computing the transition probabilities of general multivariate BDPs remains open and will be an exciting research direction for the future.

In this chapter, we have outlined new tools for practical likelihood-based analysis inference of BDP parameters under discrete and continuous observation of the process. In particular, BDP generalizations of Poisson regression yield more flexible and easy-to-fit models for count data. We have intentionally limited our discussion to basic computation of likelihoods, algorithms for maximum likelihood estimation, and finding the distribution of integral summary statistics for general BDPs. But these are only the first steps toward a comprehensive theory of estimation for BDPs. Ideally, we would like to see an analysis of identifiability, consistency and other statistical properties, like the groundbreaking work of Guttorp (1991) for Galton-Watson branching processes. We hope this review will stimulate statistical research related to BDPs with a view to bringing this rich class of stochastic models into wider use by applied scientists.