Evaluating Semi-Markov Processes and Other Epidemiological Time-to-Event Models by Computing Disease Sojourn Density as Partial Differential Equations

Abstract

Introduction

Epidemiological models benefit from incorporating detailed time-to-event data to understand how disease risk evolves. For example, decompensation risk in liver cirrhosis depends on sojourn time spent with cirrhosis. Semi-Markov and related models capture these details by modeling time-to-event distributions based on published survival data. However, implementations of semi-Markov processes rely on Monte Carlo sampling methods, which increase computational requirements and introduce stochastic variability. Explicitly calculating the evolving transition likelihood can avoid these issues and provide fast, reliable estimates.

Methods

We present the sojourn time density framework for computing semi-Markov and related models by calculating the evolving sojourn time probability density as a system of partial differential equations. The framework is parametrized by commonly used hazard and models the distribution of current disease state and sojourn time. We describe the mathematical background, a numerical method for computation, and an example model of liver disease.

Results

Models developed with the sojourn time density framework can directly incorporate time-to-event data and serial events in a deterministic system. This increases the level of potential model detail over Markov-type models, improves parameter identifiability, and reduces computational burden and stochastic uncertainty compared with Monte Carlo methods. The example model of liver disease was able to accurately reproduce targets without extensive calibration or fitting and required minimal computational burden.

Conclusions

Explicitly modeling sojourn time distribution allows us to represent semi-Markov systems using detailed survival data from epidemiological studies without requiring sampling, avoiding the need for calibration, reducing computational time, and allowing for more robust probabilistic sensitivity analyses.

Highlights

• Time-inhomogeneous semi-Markov models and other time-to-event–based modeling approaches can capture risks that evolve over time spent with a disease.

• We describe an approach to computing these models that represents them as partial differential equations representing the evolution of the sojourn time probability density.

• This sojourn time density framework incorporates complex data sources on competing risks and serial events while minimizing computational complexity.

Graphical Abstract.

This is a visual representation of the abstract.

Health economic modeling evaluates the impact and cost-effectiveness of interventions by synthesizing and extrapolating data on health and cost outcomes. Almost all health economic models are state-transition models, ¹ capturing a set of discrete health states an individual can be in and the likelihood of transitioning between these states. These models can simulate health care scenarios to predict outcomes, aiding informed decision making in resource allocation. There are various mathematical and computational approaches to state-transition modeling, each with their own strengths and limitations. ²

Many health economic analyses use Markov models, which are easy to comprehend, implement, and compute, and they often have high parameter identifiability (i.e., transition rates can be supported by the available data) due to the clear delineation of health states. However, Markov models have notable disadvantages. The Markov property restricts the likelihood of a state transition to be either constant (time homogeneous) or dependent on only the current model time (time inhomogenous); they cannot depend on the time spent in the current state (the sojourn time). Consequently, shorter sojourn times for any particular state transition at a given time are more likely than any longer times. ³ While this may be addressed with the addition of hidden states, this can lead to serious problems such as overfitting ⁴ due to lack of identifiability, ⁵ leading to poorly fitted models and spurious findings. A poor choice of time discretization/cycle length ⁶ can also dramatically bias a Markov model’s outcomes, and altering the cycle length can lead to numerical errors. ⁷

Semi-Markov models (or equivalently, Markov renewal processes)^8–10 can capture significantly more detail than Markov models can, allowing the time between state transitions to vary according to any distribution ¹¹ rather than being constrained to a geometric distribution. This can dramatically increase model fidelity. This means the risk of developing a disease can depend directly on the length of time spent with a precursor condition,^12,13 the sojourn time. Detailed survival analyses from prior studies can be used directly to inform disease risk, which has been identified as a priority in health technology assessments.^14,15 This approach conserves identifiability by explicitly including states for which survival data are available. By further generalizing to time-inhomogeneous semi-Markov processes, ¹¹ transitions can change over time, allowing the model to also capture risks that vary by age and secular trends. Semi-Markov models are also used in fields such as actuarial science^16–18 and engineering. ¹⁹ For example, semi-Markov models of cancer control can be used to increase the fidelity of estimates of disease progression and the impact of interventions.^20,21

Unlike Markov models, the dynamics of semi-Markov models cannot usually be analytically computed. Instead, Monte Carlo sampling methods are typically used to estimate patient-level dynamics.^22–24 In this formulation, the semi-Markov model will describe a patient-level simulation approach ²⁵ equivalent to a microsimulation or discrete event simulation,^26–29 which allows for a significant level of depth and detail.^2,30 However, Monte Carlo sampling introduces stochastic uncertainty to the model (first-order uncertainty).^25,31 In general, there is no consistent method of estimating the number of samples required to estimate a particular outcome. As a first-order approximation, if outcomes are assumed to be binomially distributed by the model, the length of confidence interval for the outcome is approximately $\mathcal{O}(1/\sqrt{N})$ , meaning an increasing number of simulations is required to accurately estimate an outcome, particularly for rare events. This can dramatically increase the computation time required for modeling, which both slows model development and limits the ability to conduct extensive sensitivity analyses, a priority for health economic estimates. ³² The tradeoff between discrete-time Markov-type models and continuous-time semi-Markov simulation models can lead to many potential sources of bias and inaccuracy. ³³

We developed the sojourn density model framework for explicit evaluation of semi-Markov and related models. This approach models the dynamics of the probability density of the sojourn time as a system of partial differential equations (PDEs), tracking both the likelihood of being in a given state and the time spent in that state, avoiding the need for stochastic sampling methods. The modeling framework is parametrized by the cause-specific hazard rates^34,35 (analogous to transition intensities ³⁶ or the force of increment ³⁷ ), allowing us to exploit survival data and methods ³⁸ such as Kaplan–Meier estimators^34,39,40 and risk ratios. ⁴¹ This captures the same detail as survival models, with the addition of serial state transitions. This approach is adapted from existing methods used in actuarial science,^11,16–18 which use the Kolmogorov forward equation. ⁴² We have modified this for an epidemiology and health economic context, including adapting the notation and simplifying the construction where possible.

In this study, we describe the model structure for sojourn density models, compare their strengths and weaknesses versus other modeling approaches, and describe a numerical scheme for calculating these models. To demonstrate the advantages of this approach, we include an illustrative model of liver disease. In Australia, liver cancer is the seventh most common cause of cancer-related death, ⁴³ with increasing incidence and mortality trends. ⁴⁴ Routine surveillance of patients with liver cirrhosis (late-stage liver scarring) can increase detection at early stages and improve survival ⁴⁵ but requires an understanding of the competing risk of liver cancer versus other-cause mortality. Our modeling approach allows us to incorporate competing risks of decompensation, cancer, and other-cause mortality and complex survival data,^46,47 which have been noted as key considerations in liver disease modeling. ⁴⁸

Methods

We now describe the sojourn density model framework and demonstrate its useful mathematical properties. We also develop a numerical scheme for the calculation of these models.

Sojourn Density Model

Consider a state transition model with $N$ possible discrete states labeled $1,2,...,N$ , such that at any time $t\in T\subset \mathbb{R}$ (typically $T=[0,\infty )$ ), an individual is in exactly 1 of these states. We not only describe the likelihood of an individual being in a given state but also track the distribution of the length of time spent in that state, the sojourn time $\tau \in T$ .

Doing so defines a continuous-time random process $\{X_t{\}}_{t\in T}$ with discrete state space $\{1,2,...,N\}$ such that $P(X_t=i)=P\left(\mathrm{individual}\mathrm{in}\mathrm{state}i\mathrm{at}\mathrm{time}t\right)$ . We can equivalently define the jump process $\{Y_n,T_n{\}}_{n=0,1,2,\dots }$ such that $Y_n=X_t\forall t\in [T_n,T_{n+1})$ , $Y_n\ne Y_{n+1}$ . ⁴⁹

The cause-specific hazard rates

$${\lambda }_{i,j}(t,\tau ),$$ (1)

depending on both the model time $t$ and sojourn time $\tau$ , are the key parameters for the model. These are the instantaneous transition rates for an individual in state $i$ to transition to state $j$ at time $t$ , given that the individual has so far spent time $\tau$ in $i$ without any transitions (i.e, entered state $i$ at time $t-\tau$ ). Hazard rates are widely used parameters in public health modeling. ²² In some cases, $h$ is used for hazard rates. ⁵⁰ They are equivalent to the force of increment discussed in equation 4.6 of Hoem ¹¹ in the case $s=t$ (i.e., the reference time when the model is in state $i$ is the same as the start of the period over which the transition may occur).

As the transition likelihoods depend on only the current state and the sojourn time, the random process $X_t$ is a semi-Markov proccess, ³⁶ as the sequence of transitions is Markov, but the time between transitions is not neccesarily exponentially distributed. By allowing the hazard rate to depend on $t$ , this generalizes to time-inhomogeneous semi-Markov models. This $t$ dependency can capture temporal trends as well as evolving risk with age $a=a_0+t$ , where $a_0$ is the age at $t=0$ . While $X_t$ is a semi-Markov process, the jump process $(Y_n,T_n)$ is a Markov renewal process. ⁵¹

For the hazard rate ${\lambda }_{i,j}(t,\tau )$ to represent the transition rate from state $i$ to state $j$ at time $t$ given sojourn time $\tau$ , by equation (3) in Król et al., ²² the following should hold:

$$\begin{array}{c}\multicolumn{1}{c}{{\lambda }_{i,j}(t,\tau )=\underset{\Delta t\to 0}{lim}\frac{1}{\mathrm{\Delta }t}P(Y_{n+1}=j,}\\ \multicolumn{1}{c}{T_{n+1}\in [t,t+\mathrm{\Delta }t)|Y_n=i,T_n=t-\tau )}\\ \multicolumn{1}{c}{=\underset{\Delta t\to 0}{lim}\frac{1}{\mathrm{\Delta }t}P(X_{t+\mathrm{\Delta }t}=j|X_s=i\forall s\in [t-\tau ,t),X_{t-\tau }\ne i).}\end{array}$$ (2)

For convenience, assume ${\lambda }_{i,i}\left(t,\tau \right)=0$ (i.e., there are no self-loops). If there is no possible transition between $i$ and $j$ , then ${\lambda }_{i,j}=0$ .

To track the model state and the sojourn time, we introduce the dynamic density functions for the sojourn time:

$$f_i(t,\tau ):T\times T\to [0,\infty )$$ (3)

for $i=1,2,\dots ,N$ . For a fixed time $t\in T$ , these are probability density functions on the space of sojourn times and states $(\tau ,i)\in T\times \{1,2,...N\}$ , describing the likelihood of being in a particular state with a particular sojourn at the present time. This notation is based on the notation introduced by Asanjarani et al., ³⁶ who described an equivalent density function with a single time/sojourn time variable.

Then we can define the process $X_t$ by the total probability mass in that state:

$$\begin{array}{c}\begin{array}{c}P(X_t=i)={\int }_0^{\infty }{f}_i(t,s)\mathrm{ds}=:g_i\left(t\right).\hfill \end{array}\hfill \end{array}$$ (4)

We should expect the total probability mass to be 1, that is,

$$\sum _i{g}_i\left(t\right)=1.$$ (5)

Our goal now is to describe the dynamics of $f_i\left(t,\tau \right)$ and then show this definition satisfies (2) and (5). Let $f_i(t,\tau )$ satisfy the partial differential equation

$$\frac{\partial }{\partial t}{f}_i(t,\tau )+\frac{\partial }{\partial \tau }{f}_i(t,\tau )=-\left(\sum _j{\lambda }_{i,j}(t,\tau )\right)f_i(t,\tau )$$ (6)

with boundary condition

$$f_i(t,0)=\sum _j\left({\int }_0^{\infty }{\lambda }_{j,i}(t,\tau )f_j(t,\tau )d\tau \right).$$ (7)

This equation is equivalent to the Kolmogorov forward equation.^17,42 The left-hand side of (6) represents the transport equation along lines of constant $t-\tau$ , representing the likelihood of remaining in state $i$ while both $t$ and $\tau$ increase, whereas the right-hand side captures the likelihood of making a transition to a subsequent state. Along the directional derivative ${\nabla }_{(1,1)}{f}_i$ , the likelihood decays proportionally to the hazard rate (see the “Method of Characteristics and Survival Analysis” section). More rigorous derivations of this PDE are included elsewhere.^16,17 The findings here have been simplified for a health economic context and assume the hazard rates ${\lambda }_{i,j}(t,\tau )$ are smooth.

The boundary condition (7) represents the accumulation of probability mass in state $i$ post-transition. By (5), the initial conditions $f_i(0,\tau )$ should satisfy

$$\sum _i{\int }_0^{\infty }{f}_i(0,\tau )d\tau =1.$$ (8)

In addition,

$$\underset{\tau \to \infty }{lim}{f}_i(t,\tau )=0$$ (9)

(i.e., supported on finite time).

Compare (6) to equation (2.37.1) in Helwich. ¹⁷ Helwich considered the likelihood $p_{ij}\left(s,t,u,v\right)$ of transition from state $y$ to state $z$ such that $X_s=i$ with sojourn time $u$ and $X_t=j$ with sojourn time less than $v$ . In our notation, this is equivalent to fixing the model at state $i$ with sojourn time $u$ at time $s$ (i.e., $f_k(s,\tau )=\delta (\tau -u)$ if $k=i$ else $0$ ), then tracking the evolution of $f_j(t,\tau )$ for $t$ in a small neighbourhood of $s$ .

We now have a complete description of the random process $X_t$ through (4), (6), (7) and appropriate initial conditions. This constitutes what we will call a sojourn density model. With this current definition, sojourn density models are a type of semi-Markov model as a specific parametrization using hazard rates and computed through the sojourn density functions. In the sections “Sojourn-Preserving Transitions” and “Inhomogeneous Hazard Rates and Infectious Disease Modeling,” we introduce modifications that expand the sojourn density framework to include some non-semi-Markov models.

We now show that the process defined by (4) is well-defined and driven by the hazards ${\lambda }_{i,j}(t,\tau )$ as expected.

• Theorem 2.1. The random process defined by (4) with dynamics defined by (6) and (7) satisfies the conditions (2) and (5).

Proof. The boundary condition (7) can be justified by calculating the derivative of $g_i\left(t\right)$ , that is, the rate of change in likelihood of being in state $i$ at time $t$ :

$$\begin{array}{c}\multicolumn{1}{c}{\frac{d}{\mathrm{dt}}{g}_i\left(t\right)=\frac{d}{\mathrm{dt}}{\int }_0^{\infty }{f}_i(t,\tau )d\tau }\\ \multicolumn{1}{c}{={\int }_0^{\infty }\frac{\partial }{\partial t}{f}_i(t,\tau )d\tau }\\ \multicolumn{1}{c}{=-{\int }_0^{\infty }\frac{\partial }{\partial \tau }{f}_i(t,\tau )d\tau -{\int }_0^{\infty }\left(\sum _j{\lambda }_{i,j}(t,\tau )\right)f_i(t,\tau )d\tau }\\ \multicolumn{1}{c}{=f_i(t,0)-{\int }_0^{\infty }\left(\sum _j{\lambda }_{i,j}(t,\tau )\right)f_i(t,\tau )d\tau }\\ \multicolumn{1}{c}{=\sum _j\left(\underset{\mathrm{entering}\mathrm{state}i}{\underbrace{{\int }_0^{\infty }{\lambda }_{j,i}(t,\tau )f_j(t,\tau )d\tau }}\right)-{\int }_0^{\infty }\left(\underset{\mathrm{leaving}\mathrm{state}i}{\underbrace{\sum _j{\lambda }_{i,j}(t,\tau )}}\right)f_i(t,\tau )d\tau .}\end{array}$$ (10)

This satisfies the definition of the hazard rate: the likelihood of being in state $i$ decreases proportionally to the hazard of every transition leaving state $i\sum _j{\lambda }_{i,j}(t,\tau )$ and increases for every transition entering state $i\sum _j{\lambda }_{j,i}(t,\tau )$ proportional to the weight of those states $f_j(t,\tau )$ . In addition,

$$\frac{d}{\mathrm{dt}}\left(\sum _i{g}_i\left(t\right)\right)=0,$$ (11)

that is, that the total probability mass is conserved. Therefore, with initial conditions satisfying equation (8) and boundary conditions (7), (5) is satisfied.

The propagation of hazards (2) through (6) can be observed through analogy with corollary 2.37 in Helwich ¹⁷ as noted above; a proof in the current notation is included in Appendix A.    ■

Method of Characteristics and Survival Analysis

We now compute the survival curves using method of characteristics. We refer to survival in the sense of the time until an event of interest, which may or may not be mortality, as is often used in epidemiological analyses.^34,52 The use of characteristics is equivalent to the approach in section 2.5 of Buchardt et al. ¹⁶ or the explicit solution in equation 9 from Asanjarani et al. ³⁶ As noted, equation (6) is best interpreted along characteristics of constant $t-\tau$ corresponding to the state entry time (Figure 1). This allows explicit solutions through method of characteristics as well as analogies to other survival modeling approaches.

Figure 1.

Characteristics of constant $t^{*}=t-\tau$ for $f_i(t,\tau )$ (left) and the corresponding elements $F_i^{m,n}$ with constant $m-n$ for the numerical scheme (right).

Consider the characteristic $t=t^{*}+\tau$ for fixed $t^{*}\in \mathbb{R}$ (Figure 1). Then, along this characteristic,

$$\frac{d}{d\tau }{f}_i(t^{*}+\tau ,\tau )=\frac{\partial }{\partial t}{f}_i(t^{*}+\tau ,\tau )\frac{d}{d\tau }(t^{*}+\tau )+\frac{\partial }{\partial \tau }{f}_i(t^{*}+\tau ,\tau )$$ (12)

$$=-\left(\sum _j{\lambda }_{i,j}(t^{*}+\tau ,\tau )\right)f_i(t^{*}+\tau ,\tau ).$$ (13)

If $t^{\ast }>0$ , then $t>\tau$ , which corresponds to a characteristic starting from the boundary $\tau =0$ ; if $t^{*}\le 0$ , then $t\le \tau$ , which corresponds to a characteristic from the initial condition $t=0$ .

Defining $t_0=\max (t^{*},0)$ , ${\tau }_0=\max (-t^{*},0)$ (so that $t^{*}=t_0-{\tau }_0$ , $t_0,{\tau }_0\ge 0$ ), the initial condition (at the observable start of the characteristic; not necessarily at $t=0$ ) is at $(t_0,{\tau }_0)$ and so (12) has solution

$$f_i(t^{*}+\tau ,\tau )=f_i(t_0,{\tau }_0)e^{-{\lambda }_i(t^{*}+\tau ,\tau )}$$ (14)

where

$${\lambda }_i(t,\tau )=\sum _j\left({\int }_{{\tau }_0}^{\tau }{\lambda }_{i,j}(t-\tau +s,s)\mathrm{ds}\right).$$ (15)

The function ${\lambda }_i\left(t,\tau \right)$ is the overall cumulative hazard ⁴¹ along the characteristic $t-\tau =t^{*}$ . Then,

$$S_i(t,\tau )=e^{-{\lambda }_i(t,\tau )},$$ (16)

is the overall survival (i.e., the likelihood no transitions occur) along this characteristic, and $f_i\left(t^{\ast }+\tau ,\tau \right)=f_i\left(t_0,{\tau }_0\right)S_i\left(t^{\ast }+\tau ,\tau \right)$ .

We can correspondingly calculate the time-to-event distribution after entering state $i$ at time $t-\tau$ as the distribution

$${\lambda }_{i,j}(t,\tau )S_i(t,\tau )$$ (17)

across $j=1,2,\dots ,N$ . This allows translation to time-to-event epidemiological models and access to time-to-event methodology.^35,53,54

It follows that the probability of the next transition being from $i\to j$ before time $t=t^{*}+\tau$ given entry to state $i$ at time $t_0$ , is

$$\begin{array}{c}\begin{array}{c}P(\mathrm{next}\mathrm{transition}\mathrm{time}<t\cap \mathrm{next}\mathrm{transition}i\to j|\hfill \\ \multicolumn{1}{c}{\mathrm{entered}\mathrm{state}i\mathrm{at}\mathrm{time}{t}_0)}\\ \multicolumn{1}{c}{=P(Y_{n+1}=j,T_{n+1}<t|Y_n=i,T_n=t_0)}\\ \multicolumn{1}{c}{={\int }_{t_0}^t{\lambda }_{i,j}(s,s-t^{*})S_i(s,s-t^{*})\mathrm{ds}.}\end{array}\hfill \end{array}$$ (18)

The limit of this is the likelihood of the next transition being to a particular state:

$$\begin{array}{c}\multicolumn{1}{c}{P(\mathrm{next}\mathrm{transition}i\to j|\mathrm{entered}\mathrm{state}i\mathrm{at}\mathrm{time}{t}_0)}\\ \multicolumn{1}{c}{={\int }_{t_0}^{\infty }{\lambda }_{i,j}(s,s-t^{*})S_i(s,s-t^{*})\mathrm{ds}.}\end{array}$$ (19)

We can also calculate the cause-specific survival, assuming other transitions (or events) are censored:

$$S_{i,j}(t,\tau )=e^{-{\int }_{{\tau }_0}^{\tau }{\lambda }_{i,j}(t-\tau +s,s)\mathrm{ds}}.$$ (20)

Where cause-specific survival data (equivalently, cause-specific cumulative incidence) is available with competing events censored, this may provide a calibration target for the hazard rates, which can be directly identified from the survival model (see “Hazard and Cumulative “Hazard in Clark et al. ³⁴ ); parametrization of ${\lambda }_{i,j}\left(t,\tau \right)$ across $t^{*}$ can then be identified.

Compare this with the common approach in analytical epidemiological models in which the likelihood of transition is computed explicitly as the convolution of the $\tau$ -dependent hazard rate and the time of entry.^55–57 In our formulation, this is reflected in the boundary conditions

$$\begin{array}{c}\begin{array}{c}{f}_i(t,0)=\sum _j\left({\int }_0^{\infty }{\lambda }_{j,i}(t,\tau )f_j(t,\tau )d\tau \right)\hfill \\ \multicolumn{1}{c}{=\sum _j\left({\int }_0^{\infty }{f}_j(t-\tau ,0){\lambda }_{j,i}(t,\tau )S_j(t,\tau )d\tau \right).}\end{array}\hfill \end{array}$$ (21)

If we define entry time distribution $F_j\left(x\right)=f_j\left(x,0\right)$ and survival-adjusted cause-specific hazard rate $H_{(j,i)}\left(x\right)={\lambda }_{j,i}(t,x)S_j(t,x)$ , then $F_i\left(t\right)=\sum _j(F_j*H_{j,i})\left(t\right)$ , the convolution of these. Our approach generalizes this to incorporate serial transitions.

Initial Conditions

The appropriate initial conditions are highly dependent on the system being modeled. The Dirac delta function around $\tau =0$ is a potential initial condition, corresponding to starting in state $i$ with no existing sojourn time; explicitly,

$$f_j(0,\tau )=\{\begin{array}{cc}\delta \left(\tau \right)\hfill & \mathrm{if}i=j\hfill \\ \multicolumn{1}{c}{0}& \mathrm{if}i\ne j.\hfill \end{array}$$ (22)

In practice, modeling may start in situ, that is, after some sojourn time, informing a distribution for the initial condition. For example, this could reflect the time an individual would have spent in a health state by the start of a trial (“Incorporating Lead-Time Biases” section).

Numerical Scheme

The characteristic solutions developed in the “Method of Characteristics and Survival Analysis” section can be used to develop a numerical scheme. For a discretization of $T$ by fixed time steps $\Delta t\in {\mathbb{R}}^{+}$ , define

$$\begin{array}{c}\begin{array}{c}\multicolumn{1}{c}{F_i^{m,n}:=f_i(m\Delta t,n\Delta t)}\\ \multicolumn{1}{c}{=\{\begin{array}{c}{f}_i((m-n)\Delta t,0)e^{-{\lambda }_i(m\mathrm{\Delta }t,n\mathrm{\Delta }t)}\mathrm{for}m\ge n\hfill \\ \multicolumn{1}{c}{f_i(0,(n-m)\Delta t)e^{-{\lambda }_i(m\mathrm{\Delta }t,n\mathrm{\Delta }t)}\mathrm{for}m<n}\end{array}}\\ \multicolumn{1}{c}{=\{\begin{array}{c}{F}_i^{m-n,0}{e}^{-{\lambda }_i(m\mathrm{\Delta }t,n\mathrm{\Delta }t)}\mathrm{for}m\ge n\hfill \\ \multicolumn{1}{c}{F_i^{0,n-m}{e}^{-{\lambda }_i(m\Delta t,n\Delta t)}\mathrm{for}m<n}\end{array}}\end{array}\hfill \end{array}$$ (23)

for $m,n\in {\mathbb{Z}}^{+}$ . Each value of $m-n\in \mathbb{Z}$ corresponds to a characteristic with $t^{*}=(m-n)\mathrm{\Delta }t$ . For $m<n$ , the initial condition is $F_i^{0,n-m}=f_i(0,(n-m)\mathrm{\Delta }t)$ . For $m\ge n$ , the initial condition $F_i^{m-n,0}=f_i((m-n)\mathrm{\Delta }t,0)$ is the boundary condition of the full PDE.

The cumulative hazard functions ${\lambda }_i(t,\tau )$ may be computed by (15) in simple cases or otherwise approximated. Equivalently, each $F_i^{m,n}$ element can be calculated iteratively by

$$F_i^{m,n}=F^{m-1,n-1}{e}^{-\sum _j{\int }_0^{\Delta t}{\lambda }_{i,j}((m-1)\Delta t+s,(n-1)\mathrm{\Delta }t+s)\mathrm{ds}}.$$ (24)

(see Figure 1 for an illustration of this; compare Buchardt et al. ¹⁶ ). In many cases, the integral in (24) can be explicitly computed; otherwise, it can be approximated by the midpoint ${\int }_0^{\delta t}{\lambda }_{i,j}\left(\left(m-1\right)\Delta t+s,\left(n-1\right)\Delta t+s\right)ds\approx \Delta t{\lambda }_{i,j}\left(\left(m-0.5\right)\Delta t,\left(n-0.5\right)\Delta t\right)$ . The exponential function could similarly be approximated using the appropriate Taylor polynomial for improved performance.

A first-order scheme for the boundary conditions can be computed as

$$\begin{array}{c}\begin{array}{c}\multicolumn{1}{c}{F_i^{m,0}=f_i(m\mathrm{\Delta }t,0)}\\ \multicolumn{1}{c}{=\sum _j\left({\int }_0^{\infty }{\lambda }_{j,i}(m\mathrm{\Delta }t,\tau )f_j(m\mathrm{\Delta }t,\tau )d\tau \right)}\\ \multicolumn{1}{c}{\approx \sum _j\left(\sum _{k>0}\mathrm{\Delta }t{\lambda }_{j,i}(m\mathrm{\Delta }t,k\mathrm{\Delta }t)f_j(m\mathrm{\Delta }t,k\mathrm{\Delta }t)\right)}\\ \multicolumn{1}{c}{=\sum _{k>0,j}\left(\mathrm{\Delta }t{\lambda }_{j,i}(m\mathrm{\Delta }t,k\mathrm{\Delta }t)F_j^{m,k}\right)}\end{array}\hfill \end{array}$$ (25)

for $m\ge 0$ .

One can subsequently estimate the evolution of the random process $X_t$ as

$$\begin{array}{c}\begin{array}{c}\multicolumn{1}{c}{P(X_t=i)=g_i\left(m\mathrm{\Delta }t\right)}\\ \multicolumn{1}{c}{\approx \sum _{k=0}^{\infty }\mathrm{\Delta }t{f}_i(m\mathrm{\Delta }t,k\mathrm{\Delta }t)}\\ \multicolumn{1}{c}{\approx \sum _{k=0}^{\infty }\mathrm{\Delta }{\mathrm{tF}}_i^{m,k}.}\end{array}\hfill \end{array}$$ (26)

An improvement can be made to this numerical scheme by adjusting the boundary conditions to exactly preserve the total mass at all timesteps:

$$F_i^{m,0}=\sum _{j,k}{\tilde{h}}_{j,i}^{m,k}(F_j^{m-1,k-1}-F_j^{m,k})$$ (27)

where

$${\tilde{h}}_{i,j}^{m,k}=\frac{{\lambda }_{i,j}((m-\frac{1}{2})\Delta t,(k-\frac{1}{2})\Delta t)}{\sum _l{\lambda }_{i,l}((m-\frac{1}{2})\Delta t,(k-\frac{1}{2})\Delta t)}.$$ (28)

We can then verify that $\sum _{i,k}{F}_i^{m,k}$ is constant; that is, the total probability mass $\sum _i{g}_i\left(t\right)$ is conserved. We have here used the midpoint estimates for ${\lambda }_{i,j}$ . This scheme can improve accuracy when there is a significant scale difference between hazards for competing events.

In epidemiological contexts, hazards are often modeled as continuous and smooth functions, so this scheme will have high accuracy and be convergent due to the inclusion of attractor death states.

A straightforward way to approach the numerical scheme is to calculate $F_i^{m,n}$ across all $i$ and all $m<0$ , tracking the accumulated boundary conditions $F^{m,0}$ for $m>0$ , and then computing $F_i^{m,n}$ for each $m\ge 0$ across all $i,n$ . This has been implemented in Supplementary Material: Example Code.

Hazard Functions and Common Parametric Distributions

The sojourn time density modeling scheme is determined by the hazard rates ${\lambda }_{i,j}(t,\tau )$ . It is common to describe these rates parameterically^25,58 to define a manageable set of model parameters. For example, constant hazard rates ${\lambda }_{i,j}(t,\tau )=c$ correspond to an exponential time-to-event distribution, while ${\lambda }_{i,j}(t,\tau )=\mathrm{bk}{\tau }^{k-1}$ and ${\lambda }_{i,j}(t,\tau )=p{e}^{r\tau }$ correspond to Weibull and Gompertz distributions, respectively. ⁵⁹ There are many other potential choices for ${\lambda }_{i,j}(t,\tau )$ , which may vary with $\tau$ , $t$ , or both.

The choice of hazard function is highly dependent on the nature of the system being modeled and the available data sources. ⁶⁰ van Wijk et al. ⁵⁸ provided a tutorial and general overview of determining appropriate parametrized hazard functions, including periodic functions that can represent seasonal variations or other cyclical patterns. Hazard rates can be estimated from processed survival data, such as smoothed ⁶¹ Kaplan–Meier or Fine–Grey estimates or time-to-event distributions.^36,62 Covariates can be directly included in hazard rates; for instance, Cox proportional hazard ratios can be used to define ${\tilde{\lambda }}_{i,j}(t,\tau |\mathbf{x})={\lambda }_{i,j}(t,\tau )e^{\beta ·\mathbf{x}}$ for some vector of covariates $\mathbf{x}\in {\mathbb{R}}^n$ and a baseline hazard function ${\lambda }_{i,j}(t,\tau )$ .^41,63 Constant multipliers for the hazards can be used as calibration targets, to maintain the shape of the distribution. ⁶⁴

Asanjarani et al. ³⁶ showed how time-to-event/discrete event simulation distributions^38,65 can be translated into hazard rates. Hazard rate can often be defined piecewise, ⁶⁴ for example, from life tables.^66,67 There are also more advanced methods for calculating and representing hazard rates, including nonparametric smoothing ⁶¹ and rational polynomial fraction parametrization. ⁶⁸ Arbitrarily detailed and complex hazard rates can be used in sojourn density models; an appropriate level of detail supported by the data must be chosen to avoid overfitting.

Sojourn-Preserving Transitions

In an epidemiological model, there are often transitions that are independent of the underlying growth of disease. For instance, in a model of cancer development and screening, entry into an “surveillance” cohort would not affect the progress of the disease. This can be reflected in our model framework by introducing transitions that preserve the sojourn time.

To achieve this, we can introduce a new set of hazard rates ${\mu }_{i,j}(t,\tau )$ that are the transition rates for an individual in state $i$ to transition to state $j$ , at time $t$ , given that the individual has already spent time $\tau$ in the state without any transitions occurring, while conserving the sojourn time. These rates ${\mu }_{i,j}$ will depend on the type of transition being modeled but would typically have form similar to the ${\lambda }_{i,j}$ hazard rates (“Hazard Functions and Common Parametric Distributions” section). Then, equation (6) can be modified as follows:

$$\begin{array}{c}\begin{array}{c}\frac{\partial }{\partial t}{f}_i(t,\tau )+\frac{\partial }{\partial \tau }{f}_i(t,\tau )=-\left(\sum _j{\lambda }_{i,j}(t,\tau )\right)f_i(t,\tau )\hfill \\ \multicolumn{1}{c}{-\left(\sum _j{\mu }_{i,j}(t,\tau )\right)f_i(t,\tau )+\left(\sum _k{\mu }_{k,i}(t,\tau )f_k(t,\tau )\right)}\end{array}\hfill \end{array}$$ (29)

with boundary condition (7). With this formulation, theorem 2.1 holds with minor alterations.

This approach may be useful for models in which there are multiple categories of states; for example, a health economic model may have a “health outcome” state space $\{1,2,3,\dots \}$ (e.g., no lesions, precancerous lesions, cancer) as well as an “intervention” state space $\{a,b,c...\}$ (e.g., screened, unscreened). The state space corresponding to the Cartesian product of these $(1,a),(1,b),(2,a)$ and so forth can be used for a model that tracks both, with transitions between intervention states governed by the $\lambda$ parameters and transitions between the intervention states governend by $\mu$ .

This can be further extended by continuing to track the sojourn time across multiple independent types of transitions. For instance, one could classify $k$ independent “types” of transitions ${\lambda }_{i,j}^1$ , ${\lambda }_{i,j}^2$ , … ${\lambda }_{i,j}^k$ corresponding to $k$ independent sojourn times ${\tau }^1,{\tau }^2,...,{\tau }^k$ which are each tracked by the model. Then (29) can be generalized to keep track of multiple sojourn times as

$$\begin{array}{c}\begin{array}{c}\frac{\partial }{\partial t}{f}_i(t,\mathrm{\tau })+\sum _k\frac{\partial }{\partial {\tau }^k}{f}_{i,j}(t,\mathrm{\tau })=-\left(\sum _{k,j}{\lambda }_{i,j}^k(t,\mathrm{\tau })\right)f_i(t,\mathrm{\tau })\hfill \end{array}\hfill \end{array}$$ (30)

where $\tau =\left({\tau }^1,{\tau }^2,\dots ,{\tau }^k\right)$ , with new boundary conditions for each sojourn time ${\tau }^i$

$$f_i(t,{\tau }^1(1-{\delta }_{1,n}),{\tau }^2(1-{\delta }_{2,n}),...,{\tau }^k(1-{\delta }_{k,n}))=\sum _{k,j}\left({\int }_0^{\infty }{\lambda }_{i,j}^k(t,\mathrm{\tau })f_j(t,\mathrm{\tau })d{\tau }^k\right)$$ (31)

for $n=1,2,\dots k$ where ${\delta }_{x,y}$ is the Kronecker delta function. Then, each transition type ${\lambda }^k$ “sends” the density to the corresponding boundary condition at ${\tau }^k=0$ . This expansion allows for a more general and class of models.

Computing Health Economic Outcomes

In addition to the likelihood of being in each state, there are other outcomes relevant for health economic analyses that can be computed from these models. These can usually be assessed via either the expected time spent in each state or the likelihood of transition into a state.

For example, the quality-adjusted life-years (QALYs ⁶⁹ ) can be calculated based on the expected time spent in each state. If a person in state $i$ has a QALY value of $x_i$ , the expected QALY value over the period $t_0$ to $t_1$ is the weighted sum

$$\sum _i{x}_i{\int }_{t_0}^{t_1}{g}_i\left(t\right)\mathrm{dt}.$$ (32)

Outcomes such as costs associated with entering a particular state can be calculated via the boundary condition. For instance, the likelihood of entering state $i$ over the period $t_0$ to $t_1$ is

$${\int }_{t_0}^{t_1}{f}_i(t,0)\mathrm{dt}.$$ (33)

If there is a cost ${\alpha }_i$ associated with entering state $i$ , the total expected cost across this period is

$$\sum _i{\alpha }_i{\int }_{t_0}^{t_1}{f}_i(t,0)\mathrm{dt}.$$ (34)

If there are ongoing costs associated with the time spent in a particular state, these can be calculated in the same way as QALYs above.

Inhomogeneous Hazard Rates and Infectious Disease Modeling

By relaxing the assumptions on the form of the hazard functions, the sojourn time density models can be used to simulate a more general class of models than semi-Markov models. For example, we could allow the hazard to depend on the current model state, that is, have the form

$${\lambda }_{i,j}(t,\tau ,\mathbf{g}\left(t\right))$$ (35)

where $g\left(t\right)=\left\{g_i\left(t\right)\right\}$ is a vector of the values (4). One can then simulate models where the effect size is dependent on the current state of the system.

For example, the standard SIR model ⁷⁰ has states $S$ (susceptible), $I$ (infectious), $R$ (recovered). The risk of infection is proportional to the currently infected population, leading to the hazard

$$\begin{array}{c}\begin{array}{c}\multicolumn{1}{c}{{\lambda }_{S,I}(t,\tau ,\mathbf{g}\left(t\right))=\beta P(X_t=I)}\\ \multicolumn{1}{c}{=\beta {\int }_0^{\infty }{f}_I(t,\tau )d\tau }\\ \multicolumn{1}{c}{=\beta g_I\left(t\right).}\end{array}\hfill \end{array}$$ (36)

The recovery rate for infected individuals is constant, that is,

$${\lambda }_{I,R}(t,\tau ,\mathbf{g}\left(t\right))=\gamma .$$ (37)

These hazards can be used to define a sojourn density model; we can confirm that

$$\frac{d{g}_S}{\mathrm{dt}}=-\beta g_S\left(t\right)g_I\left(t\right),\frac{d{g}_I}{\mathrm{dt}}=\beta g_S\left(t\right)g_I\left(t\right)-\gamma g_I\left(t\right),\frac{d{g}_R}{\mathrm{dt}}=\gamma g_I\left(t\right),$$ (38)

that is, the defining equations for the standard SIR model. Using this starting point, we could modify this to introduce more complex risks of infection and recovery. For instance, a distribution for time to disease recovery could be simulated by modifying ${\lambda }_{I,R}$ to be a function of $\tau$ , the time in infectious state, to reflect an inflection that can clear after only a few days minimum. In the sojourn density model framework, new dynamics can be included by modifying the hazard functions rather than by adjusting the design of the model.

This addition of heterogeneous generalizes the sojourn density model framework and could allow for the synthesis of different model types into one framework. For example, a model of both a viral infection and a resulting disease (such as hepatitis and liver cancer) could be directly modeled. The addition of proportional terms to the hazard does not affect theorem 2.1 but does mean the method of characteristics described in the Method of Characteristics and survival analysis section may no longer be applied.

Incorporating Lead-Time Biases

As the sojourn time is modeled explicitly in this framework, this allows us to incorporate lead-time bias. Lead-time biases occur in many contexts; for instance, screen detection of cancer artificially increases survival times due to the increased observation window.^71,72 A related bias occurs in modeling that has been informed by survival data measured from the start of a trial rather than onset of disease.

These biases can be addressed by modifying the hazard rate. A highly simplistic approach may be to define the adjusted hazard rate

$${\tilde{h}}_{i,j}(t,\tau )=\{\begin{array}{cc}0\hfill & \mathrm{for}\tau <{\tau }_0\hfill \\ \multicolumn{1}{c}{{\lambda }_{i,j}(t,\tau -{\tau }_0)}& \tau \ge {\tau }_0\hfill \end{array}$$ (39)

where $h_{i,j}\left(t,\tau \right)$ is the observed cause-specific hazard rate from a trial that may include a lead-time bias and ${\tau }_0$ is a cutoff time. This assumes that there is no risk of transition until some given time ${\tau }_0$ . Alternatively, the hazard could be assumed to increase to the observed rates at time ${\tau }_0$ , linearly or otherwise. In practice, a useful estimate for ${\tau }_0$ may be difficult to determine, requiring a sensitivity analysis over a range; there are many useful methods for estimating the lead time.^73–76

Example Model: Liver Disease

To demonstrate our new approach to modeling, we now describe a model of liver disease and development of cancer. A key intervention in the prevention of liver cancer death is early detection via routine surveillance. Recommendations like 6-monthly ultrasound for people with cirrhosis (late-stage liver scarring) can increase the likelihood of detection at early stages when curative treatment is possible. ⁷⁷ However, people with cirrhosis are also at high risk of both liver decompensation (irreversably worsened liver damage) and noncancer death. People who develop liver decompensation or die before the onset of liver cancer cannot benefit from early detection. It is therefore critical to understand the competing risks of liver cancer, liver decompensation, and other-cause death when assessing whether routine surveillance is advisable or whether the costs and patient discomfort caused by surveillance outweigh the health benefits. This model shows how the risk of liver cancer, and competing risk of decompensation, can be captured in a way that evolves with the length of time spent with liver cirrhosis.

The structure of the simplified model is shown in Figure 2 and captures individuals with liver cirrhosis, decompensation events, onset of liver cancer, and death. As the model is designed to illustrate the potential utility of surveillance for liver cancer in the presence of competing risks of decompensation and death, liver cancer and death were used as terminal states. Hazard rates for relevant transitions (decompensation, onset of liver cancer, death) were calibrated by identifying appropriate parametric forms and calibrating to relevant data sources. With this approach, the design of the model states and flow is kept simple, while the complexity is shifted to computing the hazard rates determining the transitions between states. Each cause-specific hazard was fitted in turn, and the overall fit assessed. For this iteration of the model, a cohort aged 55 y was simulated, based on the mean age of cirrhosis diagnosis.

Figure 2.

Schematic of the example liver disease model and the sojourn- and time-dependent hazards for competing risks for people with compensated cirrhosis.

The modeling described in this article is part of a larger model developed to inform the “Clinical Practice Guidelines for Liver Cancer Surveillance for People at High Risk in Australia.” ⁷⁸ The development of these guidelines, including the modeling, was led by clinicians in the area of liver disease and liver cancer alongside a multidisciplinary working party, which included health care and clinical representatives, representatives with lived experience, and other community representatives. The full model includes detailed liver cancer stage progression and survival as well as economic details. For further details, see Worthington et al. ⁷⁹

For this analysis, published survival curves were used as target data for the modeling. Where patient-level data are available, this could be used as a calibration target instead, using methods in the existing literature on semi-Markov models. ²²

Results

Performance of Sojourn Density Models

The described model structure can capture the same level of detail as semi-Markov models and many other patient-level simulations, increasing the level of detail while allowing for deterministic evaluation. The computational complexity of the numerical algorithm described in the “Numerical Scheme” section is $\mathcal{O}\left(T^2{S}^2\right)$ , where $T$ is the number of timesteps and $S$ is the number of model states. Vectorization across characteristics can improve performance considerably, for example, via the Python NumPy library or other scientific computing libraries. For models in which not every state has a transition to every other state, the complexity is $\mathcal{O}\left(T^{2}U\right)$ where $U<S^2$ is the number of possible transitions. Compare this with a patient-level simulation, including most implementations of semi-Markov models, in which the computational complexity is driven by the number of patients simulated $\mathcal{O}\left(\mathrm{TN}\right)$ , where $N$ is the number of simulated patients. As noted previously, the precision of such models is approximately $\mathcal{O}(1/\sqrt{N})$ , and so in some applications, $N$ may need to be very large for precise estimates. This is avoided in the sojourn time density model. For comparison, a standard numerical algorithm for a Markov model would have computational complexity $\mathcal{O}\left(T{S}^2\right)$ .

In practice, sojourn density models can run in less than 1 s on a standard computer. This allows for runs with a variety of parameter values to assess sensitivity analyses, particular with parallel computing. In most cases, the optimal timestep $\Delta t$ can be identified by assessing a particular parameter set, selecting a small enough value for the results of interest to converge. This choice will be highly dependent on the problem being assessed, roughly scaling with the expected transition time. Future work is planned to assess the effect of the timestep.

With an appropriate implementation of the numerical algorithm described in the “Numerical Scheme” section, computing a sojourn density model requires specifying only the model states and the form of the hazard rates between them. Listing 1 shows the syntax of the code for this model; the full code, including the implementation of the numerical scheme, has been included in Supplementary Material: Example Code.

Listing 1.

Excerpt Showing Code Syntax for the Liver Disease Model ^a

Liver_Disease_Model=Sojourn_Model.Model() state_names=[“Compensated cirrhosis”,“Decompensated cirrhosis”,“Liver ↪ cancer”,“Death”] Liver_Disease_Model.add_state(state_names) #Liver cancer onset transition Liver_Cancer_Hazard=Hazard_Functions.Hazard( Hazard_Type = ↪ “gompertz_sojourn”, Parameters={“p”:1.197e-02,“r”:6.654e-02}) Liver_Disease_Model.add_transition(“Compensated cirrhosis”,“Liver ↪ cancer”,Liver_Cancer_Hazard) ///Other transitions omitted for brevity res_df=Liver_Disease_Model.run_model(dt=0.01, max_t=10, ↪ init_conds=[“Compensated cirrhosis”])

^aFull code is included in Supplementary Material: Example Code.

Liver Disease Model Calibration

The model was calibrated to reproduce key targets, with model parameters shown in Table 1. For each target, an appropriate parameterized hazard rate was selected, and optimal parameters were calculated using the BFGS algorithm in the SciPy package. We also calibrated distributions for hazard ratios for each of the hazard rate functions. These were calibrated to reproduce the 95% confidence intervals in each data source across the range of hazard ratio values and act as multipliers for each distribution for the probabilistic sensitivity analysis. The model was well-fitted to the target data, reproducing terminal values for the survival curves and remaining within the 95% confidence intervals for the duration. Key targets are shown in Figure 3. For all-cause death rates, a piecewise constant hazard was used based on age-specific rates published by the Australian Bureau of Statistics. ⁸²

Table 1.

Calibrated Parameters for Example Model of Liver Disease

Parameter	Model Value (95% PI)	Target (95% CI)	Note
Cirrhosis decompensation hazard	$\lambda =0.0129t+0.00668\mathrm{for}t<6,0.0359\mathrm{for}t\ge 6$		Calibrated to Vilar-Gomez et al. 80
Hazard ratio distribution for PSA	$\mathrm{Lognormal}(\mu =-0.0213,{\sigma }^2={0.117}^2)$		Calibrated to Vilar-Gomez et al. 80
Annual decompensation risk (target)	1.03% (0.9%–1.2%)	1.59% (0.17%–3.01%)	Calibration target 80
Ten-year decompensation risk	34.5% (29.5%–40.2%)	33.7% (28.4%–39.1%)
Liver cancer onset hazard	$\lambda =0.0102{\tau }^{1.377}$		Calibrated to Vilar-Gomez et al. 80
Hazard ratio distribution for PSA	$\mathrm{Lognormal}(\mu =-0.00171,{\sigma }^2={0.158}^2)$		Calibrated to Vilar-Gomez et al. 80
Annual liver cancer risk	0.99% (0.8%–1.3%)	1.53% (0.14%–2.93%)	Calibration target 80
Ten-year liver cancer risk	15.2% (11.8%–19.2%)	15.6% (11.5%–19.7%)
Death hazard (compensated cirrhosis)	$\lambda =0.0336{\tau }^{-0.238}$		Calibrated to D’Amico et al. 81
Hazard ratio distribution for PSA	$\mathrm{Lognormal}(\mu =-0.0254,{\sigma }^2={0.147}^2)$		Calibrated to D’Amico et al. 81
Annual death risk (compensated)	4.72% (3.8%–5.8%)	3.97% (2.62%–5.31%)	Calibration target 81
Ten-year death risk (compensated)	39.4% (35.2%–43.6%)*	39.4% (36.1%–42.8%)
Death hazard (decompensated cirrhosis)	$\lambda =0.447-0.150\tau +0.0237{\tau }^2$		Calibrated to D’Amico et al. 81
Hazard ratio distribution for PSA	$\mathrm{Lognormal}(\mu =0.0478,{\sigma }^2={0.0895}^2)$		Calibrated to D’Amico et al. 81
Annual death risk (decompensated)	37.8% (33.1%–42.4%)*	38.3% (35.0%–41.6%)	Calibration target 81
Ten-year death risk (decompensated)	90.6% (89.0%–91.9%)*	91.1% (89.2%–93.0%)
Death hazard (other causes)	Age-specific annual rates		Australian life tables 82

CI, confidence interval; PSA, probabilistic sensitivity analysis.

^*95% PI: 95% prediction interval of model outcomes, based on samples from the specified hazard ratio distributions.

Figure 3.

Selected calibration outcomes. Top row: Liver cancer incidence rates (left; target Vilar Gomez et al. ⁸⁰ ) and decompensation rates (right). Bottom row: all-cause death rates in patients with compensated (left; includes any decompensation events) and decompensated (right) cirrhosis.

A code snippet showing the syntax to define a model using the included code is shown in Listing 1. Examples of the kinds of outputs that can be generated are shown in Figure 4. This includes the likelihood of $X_t$ being in a given state by time (through $g\left(t\right)$ ). This was used to calculate the 5-y risk of liver cancer or non–liver cancer death in people who have not yet developed liver cancer or died, a key outcome for assessing the benefits of liver surveillance.

Figure 4.

Outcomes from the example model of liver disease. Left: evolving likelihood of liver decompensation, liver cancer, and non–liver cancer death. Right: 5-y risk of liver cancer versus 5-y risk of non–liver cancer death for patients who have not yet developed liver cancer/died, including 95% model range based on probabilistic sensitivity analysis.

A probabilistic sensitivity analysis was run by sampling hazard ratios from the distributions given in Table 1. The sensitivity analysis ran 100,000 parameter sets, which was completed in under 2 h. In the 5-y risk outcomes in Figure 4, this was used to develop a prediction interval; 95% of the simulated parameter sets led to outcomes in the indicated range.

Discussion

We described a novel approach to epidemiological modeling by calculating the evolution of the probability density of the sojourn time. This allows detailed survival data sources to be used to model competing risks. We demonstrated this sojourn density model approach by presenting a model of liver disease. The model was calibrated and reproduced the targets accurately with low computational burden.

The novel sojourn density model structure allows for flexible analyses that rely on detailed survival data and flexible time scales. This gives advantage over typical epidemiological modeling approaches, improving detail without sacrificing interprability while limiting computational burden. Designing models that reduce computational burden to facilitate probabilistic sensitivity analyses has been identified as a priority for health technology assessments, ³² often requiring surrogate methods such as meta-modeling/emulators to reduce the reliance on computationally expensive models.^83–86

By using the hazard rates as parameters, the models can be interpreted directly in the context of source epidemiology studies, and parameter values can be directly inferred from survival data. In the context of liver disease modeling, this means the model can reflect the true health benefits of early-stage diagnosis by accurately capturing disease progression. The ability to numerically evaluate our model at any time scale also allows us to accurately model phenomena such as surveillance intervals, which were not been included in real-world trials with a high degree of precision. Avoiding a fixed time discretization reduces the likelihood of numerical errors if chosen poorly ⁸⁷ and improves the ability to capture short time scale events.

There is great potential to refine survival analysis estimates designed to guide policy recommendations, ¹⁵ and it is hoped that this modeling approach allows for the development of straightforward models that incorporate existing data sources in a clear and flexible way, reducing the need for model calibration and minimizing data postprocessing and model design effort. The benefits of this approach are demonstrated in the closeness of fit to the calibration targets (Figure 3) as well as the flexibility to evaluate surveillance at a wide range of intervals. ⁷⁹ By providing a numerical implementation, we have also reduced the implementation burden on health economic practitioners, requiring only the specification of the model states and the hazard rates associated with the transitions between them.

The limitations of this model structure include the semi-Markov property; each state transition depends only on the current state, the sojourn time, and the model time. This approach may therefore not be appropriate in contexts where multiple prior events are required to inform future likelihoods.

The liver disease model described here is a part of a larger model of liver disease, ⁷⁹ which was used to evaluate surveillance recommendations for the “Clinical Practice Guidelines for Hepatocellular Carcinoma Surveillance for People at High Risk in Australia.” ⁷⁸ The flexible sojourn density model allowed us to analyze combinations of surveillance technologies and intervals to analyze more complex surveillance algorithms and optimize these algorithms through iterative design enabled by the fast computation time and high precision.

There are many potential future developments to build on the work presented here. Higher-order approximations for the integrals in (25) and (24) could be used to derive more accurate numerical schemes, which may help when dealing with pathological hazards such as those that are nonsmooth or have discontinuities. It may also be possible to develop spectral methods to approximate solutions without time discretization. ⁸⁸ Further work could also use these models to assess the potential impact of lead-time biases, and the loss in precision when these are not considered in modeling. It is hoped that the method described here will be of general use to health economic practitioners.

Conclusions

To inform health policy and make an impact, epidemiological models must be able to provide reliable, timely, and interpretable results. Sojourn density models can incorporate published survival data and generate competing risk estimates quickly, allowing for simple design and fast calibration. We hope this approach will be of use to modelers in public health.