Microsimulation Estimates of Decision Uncertainty and Value of Information Are Biased but Consistent

Abstract

Purpose.

Individual-level state-transition microsimulations (iSTMs) have proliferated for economic evaluations in place of cohort state transition models (cSTMs). Probabilistic economic evaluations quantify decision uncertainty and value of information (VOI). Previous studies show that iSTMs provide unbiased estimates of expected incremental net monetary benefits (EINMB), but statistical properties of iSTM-produced estimates of decision uncertainty and VOI remain uncharacterized.

Methods.

We compare iSTM-produced estimates of decision uncertainty and VOI to corresponding cSTMs. For a 2-alternative decision and normally distributed incremental costs and benefits, we derive analytical expressions for the probability of being cost-effective and the expected value of perfect information (EVPI) for cSTMs and iSTMs, accounting for correlations in incremental outcomes at the population and individual levels. We use numerical simulations to illustrate our findings and explore the impact of relaxing normality assumptions or having >2 decision alternatives.

Results.

iSTM estimates of decision uncertainty and VOI are biased but asymptotically consistent (i.e., bias approaches 0 as number of microsimulated individuals approaches infinity). Decision uncertainty depends on 1 tail of the INMB distribution (e.g., P[INMB <0]), which depends on estimated variance (larger with iSTMs given first-order noise). While iSTMs overestimate EVPI, their direction of bias for the probability of being cost-effective is ambiguous. Bias is larger when uncertainties in incremental costs and effects are negatively correlated since this increases INMB variance.

Conclusions.

iSTMs are useful for probabilistic economic evaluations. While more samples at the population uncertainty level are interchangeable with more microsimulations for estimating EINMB, minimizing iSTM bias in estimating decision uncertainty and VOI depends on sufficient microsimulations. Analysts should account for this when allocating their computational budgets and, at minimum, characterize such bias in their reported results.

Model-based economic evaluations are increasingly used to inform decisions about which interventions to implement and whether there is value in conducting potential future research. The use of probabilistic analyses within economic evaluations has gained increasing traction and is now the recommended approach for base-case analyses.^1–3 Probabilistic analyses permit unbiased estimates of expected outcomes for nonlinear models and quantification of decision uncertainty stemming from uncertainty in population-level model inputs (i.e., secondorder parameter uncertainty, the uncertainty in quantities such as probabilities estimated using finite, empirical data).⁴ Because uncertainty in model parameters can be correlated, prior work has developed approaches for estimating, sampling from, and propagating the joint uncertainty distribution of model parameters.^4–11

While cohort state-transition models (cSTMs) (i.e., mean-process models) were the standard modeling technique in the past, individual-level state-transition microsimulations (iSTMs) are increasingly used for economic evaluations.^12,13 For any cSTM, a corresponding and equivalent iSTM that models the same underlying processes can be formulated.¹² The expected outcomes produced by simulations using an iSTM are estimates of the corresponding mean-process outcomes obtained from simulations with the equivalent cSTM.¹⁴

Several trends in medicine and public health have driven the increased use of iSTMs. These include 1) the proliferation of high-performance computing,^10,12 2) the focus on sophisticated and personalized interventions coupled with the rise of sophisticated risk prediction models that depend on a complex set of dynamic individual characteristics,^15,16 and 3) the move to proactively address health disparities, with health equity defined across multiple dimensions (e.g., race/ethnicity, socioeconomic status, and social determinants of health).^17–19

When a sufficiently large number of individuals are simulated using an iSTM, the outcomes produced converge with those that would have been produced by a corresponding cSTM.^12,20–23 However, when smaller numbers of individuals are simulated, modeled outcomes produced by the iSTM can differ appreciably from the cSTM’s outcomes due to Monte Carlo noise (i.e., firstorder uncertainty). Prior work conceptualizes the rate of convergence and hence the necessary number of microsimulated individuals needed to estimate expected outcomes and expected incremental outcomes within a given error bound.^20–23

Prior work also considers probabilistic analyses using iSTMs,^21–23 which involves both first-order and secondorder uncertainty. Specifically, this prior work developed methods to produce estimates with as few samples as possible from the joint model parameter uncertainty distribution because each sample then involves computationally costly microsimulations of many individuals. A key finding is that for a given number of samples from the joint model parameter uncertainty distribution, estimates of outcomes made with an iSTM will be unbiased but that their uncertainty will be systematically overestimated because they also contain Monte Carlo noise.

While the prior work cited here has focused on making estimates of expected outcomes and expected incremental outcomes providing guidance on the number of samples to take from the joint uncertainty distribution and number of individuals to microsimulate, it does not provide guidance for other quantities of interest related to decision uncertainty and the value of information. The present study provides analytic results and numerical simulations to characterize the precision and bias of estimating such quantities using microsimulation models.

Theoretical Model

We explore the precision and bias of estimates of decision uncertainty and the value of information using a simple decision model with 2 alternatives (treatment v. no treatment). We make these estimates using an iSTM with various numbers of sampled individuals $\left(n_{\mathit{\text{micro}}}\right)$ and compare them to estimates produced with a cSTM (i.e., an iSTM where $n_{\mathit{\text{micro}}}=\infty$).

Cohort State-Transition Markov Model

Let $\mu =\left\{{\mu }_{\Delta Q},{\mu }_{\Delta C}\right\}$ be the hyperparameters determining the incremental quality-adjusted life-years (QALYs) and costs of the decision model with $\left\{{\sigma }_{\Delta Q},{\sigma }_{\Delta C}\right\}$ reflecting the current level of second-order uncertainty regarding these values. A realization of our parameters would then be a sample from our second-order uncertainty distribution

$$\begin{gathered} {\theta }_{\Delta Q}~N\left({\mu }_{\Delta Q},{\sigma }_{\Delta Q}\right) \\ {\theta }_{\Delta C}~N\left({\mu }_{\Delta C},{\sigma }_{\Delta C}\right) \\ \Theta =\left\{{\theta }_{\Delta Q},{\theta }_{\Delta C}\right\} \end{gathered}$$ (1)

We first specify the cSTM, which is a function of our parameters $\left(f\right(\Theta \left)\right)$.

At the population level, we assume that the uncertainty in the incremental discounted lifetime QALYs, $\Delta Q_{\mathit{\text{psa}}}$, and incremental discounted lifetime costs, $\Delta C_{\mathit{\text{psa}}}$, with treatment compared with with no treatment are characterized by

$$⟨\Delta Q_{\mathit{\text{psa}}},\Delta C_{\mathit{\text{psa}}}⟩~f\left(N\left({\mu }_{\Delta Q},{\sigma }_{\Delta Q}\right),N\left({\mu }_{\Delta C},{\sigma }_{\Delta C}\right)\right)$$ (2)

While the random variables $\Delta Q_{\mathit{\text{psa}}}$ and $\Delta C_{\mathit{\text{psa}}}$ that we will consider below need not be normally distributed, we make this assumption because doing so will permit the analytic expressions we derive later to be more concrete. Furthermore, for simplicity, we assume that $f\left(\Theta \right)$ is the identity map, which implies that the second-order uncertainty in our model inputs is equal to the second-order uncertainty of our model output (i.e., $\left\{{\sigma }_{\Delta Q}^2,{\sigma }_{\Delta C}^2\right\}$, describing the variance of both $⟨{\theta }_{\Delta Q},{\theta }_{\Delta C}⟩$ and $\left.⟨\Delta Q_{\mathit{\text{psa}}},\Delta C_{\mathit{\text{psa}}}⟩\right)$.

More generally, as the uncertainty of incremental QALYs and incremental costs may be correlated, we can rewrite their joint uncertainty as

$$⟨\Delta Q_{\mathit{\text{psa}}},\Delta C_{\mathit{\text{psa}}}⟩~f\left(\mathit{\text{MVN}}\left(⟨{\mu }_{\Delta Q},{\mu }_{\Delta C}⟩,{\Sigma }_{\mathit{\text{psa}}}\right)\right),$$ (3)

where ${\Sigma }_{\mathit{\text{psa}}}$ is the variance-covariance matrix, where the diagonal consists of ${\sigma }_{\Delta Q}^2$ and ${\sigma }_{\Delta C}^2$, and the off-diagonal elements are defined as ${\rho }_{\mathit{\text{psa}}}{\sigma }_{\Delta Q}{\sigma }_{\Delta C}$, with correlation in the population uncertainty of incremental QALYs and incremental costs equal to ${\rho }_{\mathit{\text{psa}}}$ (i.e., the second-order correlation).

For the cSTM, a given draw from the population distribution of incremental QALYs and incremental costs (i.e., Probabilistic Sensitivity Analysis [PSA] sample) can be expressed in terms of incremental net monetary benefit:

$$\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}=\lambda \Delta Q_{\mathit{\text{psa}}}-\Delta C_{\mathit{\text{psa}}}$$ (4)

where $\lambda$ is the willingness to pay for QALYs gained. The variance of the incremental net monetary benefit across PSA samples is then

$$\begin{gathered} \mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)=\mathit{\text{Var}}\left(\lambda \Delta Q_{\mathit{\text{psa}}}-\Delta C_{\mathit{\text{psa}}}\right) \\ {\lambda }^2\mathit{\text{Var}}\left(\Delta Q_{\mathit{\text{psa}}}\right)+\mathit{\text{Var}}\left(\Delta C_{\mathit{\text{psa}}}\right) \\ -2\lambda \mathit{\text{Cov}}\left(\Delta Q_{\mathit{\text{psa}}},\Delta C_{\mathit{\text{psa}}}\right) \\ ={\left(\lambda {\sigma }_{\Delta Q}\right)}^2+{\left({\sigma }_{\Delta C}\right)}^2-2\lambda {\rho }_{\mathit{\text{psa}}}{\sigma }_{\Delta Q}{\sigma }_{\Delta C} \end{gathered}$$ (5)

Note that equation (5), the variance of $\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}$, does not rely on the normality assumptions above.

For the cSTM, the expected incremental net monetary benefit of treatment for a given finite number of PSA samples $\left(n_{\mathit{\text{psa}}}\right)$ can be estimated as

$$\begin{gathered} \mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}=E_{\Theta }\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right) \\ \overline{\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}}=\frac{{\sum }_{i=1}^{n_{\mathit{\text{psa}}}}\mathit{\text{INM}}{B}_{\mathit{\text{psa}},i}}{n_{\mathit{\text{psa}}}} \end{gathered}$$ (6)

As the number of PSA samples is finite $\left(n_{\mathit{\text{psa}}}\right)$, there is uncertainty in this unbiased estimate of $\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}$, given by

$$\begin{gathered} \mathit{\text{Var}}\left(\overline{{\mathit{\text{EINMB}}}_{\mathit{\text{psa}}}}\right)=\frac{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)}{n_{\mathit{\text{psa}}}-1} \\ =\frac{{\left(\lambda {\sigma }_{\Delta Q}\right)}^2+{\left({\sigma }_{\Delta C}\right)}^2-2\lambda {\rho }_{\mathit{\text{psa}}}{\sigma }_{\Delta Q}{\sigma }_{\Delta C}}{n_{\mathit{\text{psa}}}-1} \\ ={\lambda }^2{\left(s{e}_{\Delta Q}\right)}^2+{\left(s{e}_{\Delta C}\right)}^2-2\lambda {\rho }_{\mathit{\text{psa}}}s{e}_{\Delta Q}s{e}_{\Delta C}, \end{gathered}$$ (7)

where $s{e}_{\Delta Q}$ and $s{e}_{\Delta C}$ are standard errors for the estimated incremental QALYs and costs, respectively. The variance of $\overline{\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}}$ in equation (7) does not rely on the normality assumptions above.

In terms of decision uncertainty, that is, the probability that a given strategy is the most cost-effective (often represented as a cost-effectiveness acceptability curve [CEAC]), the probability that treatment is the optimal strategy for the cSTM is

$$\begin{gathered} \mathit{\text{CEA}}{C}_{\mathit{\text{psa}}}=P\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}>0\right) \\ =1-\Phi \left(\frac{0-\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}}{\sqrt{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)}}\right) \\ =1-\Phi \left(\frac{0-\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}}{\sqrt{{\left(\lambda {\sigma }_{\Delta Q}\right)}^2+{\left({\sigma }_{\Delta C}\right)}^2-2\lambda {\rho }_{\mathit{\text{psa}}}{\sigma }_{\Delta Q}{\sigma }_{\Delta C}}}\right) \end{gathered}$$ (8)

where $\Phi (.)$ is the cumulative density function of the standard normal distribution.

The expected value of perfect information (EVPI) for the cSTM is defined as

$$\mathit{\text{EVP}}{I}_{\mathit{\text{psa}}}=\left\{\begin{array}{cc}E\left(\mathit{\text{max}}\left(0,-\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)\right)& \mathit{\text{if}}\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}>0\\ E\left(\mathit{\text{max}}\left(0,\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)\right)& \mathit{\text{if}}\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}<0\end{array}\right.$$ (9)

We require both cases because, at some values of $\lambda$, the expected incremental net monetary benefit of treatment $\left(\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}\right)$ could be negative, implying that the preferred strategy on expectation is no treatment. This, in turn, implies that information will be valuable for PSA samples in which the incremental benefit of treatment $\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)$ is positive.

In the case of $\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}>0$, we can express the value of information as

$$\begin{gathered} \mathit{\text{EVP}}{I}_{\mathit{\text{psa}}}= \\ \underset{-\mathrm{\infty }}{\overset{0}{\int }}\left(-\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\frac{1}{\sqrt{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)}}\phi \left(\frac{\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}-\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}}{\sqrt{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)}}\right)\right)\mathit{\text{dINM}}{B}_{\mathit{\text{psa}}} \end{gathered}$$ (10)

where $\phi (.)$ is the probability density function of the standard normal distribution. This expression is a form of the unit normal loss integral function, which was first discussed by Raiffa and Schlaifer in the 1960s.^24–26 Likewise, in the case of $\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}<0$, this expression is

$$\begin{gathered} \mathit{\text{EVP}}{I}_{\mathit{\text{psa}}}= \\ \underset{0}{\overset{\infty }{\int }}\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\frac{1}{\sqrt{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)}}\phi \left(\frac{\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}-\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}}{\sqrt{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)}}\right)\right)\mathit{\text{dINM}}{B}_{\mathit{\text{psa}}} \end{gathered}$$ (11)

Individual-Level Microsimulation Model

If an iSTM equivalent to the above-described cSTM is used to estimate the quantities described for the cSTM using a given number of microsimulated individuals $\left(n_{\mathit{\text{micro}}}\right)$ for each PSA sample, we arrive at the corresponding set of equations.

For any PSA sample from the population distribution, the distribution of incremental QALYs with treatment compared to with no treatment that is estimated from an iSTM defined in terms of $\Delta Q_{\mathit{\text{psa}}}$, the population incremental QALYs with treatment sampled from the second-order uncertainty distribution, and ${\sigma }_{\Delta Q,\mathit{\text{micro}}}$, the square root of the variance of incremental QALYs with treatment across individuals in the microsimulation, can be written as

$$\begin{gathered} \Delta Q_{\mathit{\text{micro}}}~N\left(\Delta Q_{\mathit{\text{psa}}},s{e}_{\Delta Q,\mathit{\text{micro}}}\right)\mathit{\text{where}} \\ s{e}_{\Delta Q,\mathit{\text{micro}}}=\frac{{\sigma }_{\Delta Q,\mathit{\text{micro}}}}{\sqrt{\left(n_{\mathit{\text{micro}}}-1\right)}} \end{gathered}$$ (12)

For our simple example, we assume that the first-order noise from the microsimulations is homoscedastic with respect to the level of incremental QALYs with treatment, that is, that the variance across microsimulated individuals in terms of incremental QALYs from treatment $\left({\sigma }_{\Delta Q,\mathit{\text{micro}}}\right)$ is independent of $\Delta Q_{\mathit{\text{psa}}}$. Hence, equation (12) can then be expressed equivalently as

$$\begin{gathered} \Delta Q_{\mathit{\text{micro}}}=\Delta Q_{\mathit{\text{psa}}}+{\epsilon }_{\Delta Q,\mathit{\text{micro}}} \\ \text{where}{\epsilon }_{\Delta Q,\mathit{\text{micro}}}~N\left(0,{\mathit{\text{se}}}_{\Delta Q,\mathit{\text{micro}}}\right) \end{gathered}$$ (13)

In other words, an estimate of the population incremental QALYs generated from an iSTM will have the same expectation as the value produced by the cSTM with the addition of zero-centered noise due to the finite number of simulated individuals in the iSTM.

Likewise, for any PSA sample from the population distribution, the distribution of incremental costs with treatment compared to with no treatment estimated from an iSTM is

$$\begin{gathered} \Delta C_{\mathit{\text{micro}}}~N\left(\Delta C_{\mathit{\text{psa}}},s{e}_{\Delta C,\mathit{\text{micro}}}\right) \\ \mathit{\text{where}} \\ s{e}_{\Delta C,\mathit{\text{micro}}}=\frac{{\sigma }_{\Delta C,\mathit{\text{micro}}}}{\sqrt{\left(n_{\mathit{\text{micro}}}-1\right)}} \end{gathered}$$ (14)

For equation (14), the assumptions are parallel to those for equation (12) regarding incremental QALYs, which we then express as

$$\begin{gathered} \Delta C_{\mathit{\text{micro}}}=\Delta C_{\mathit{\text{psa}}}+{\epsilon }_{\Delta C,\mathit{\text{micro}}} \\ \text{where}{\epsilon }_{\Delta C,\mathit{\text{micro}}}~N\left(0,s{e}_{\Delta C,\mathit{\text{micro}}}\right) \end{gathered}$$ (15)

Since the uncertainty of incremental QALYs and incremental costs may be correlated within an individual in the microsimulation (i.e., some individuals could gain more than average number of QALYs with treatment and also have higher than average incremental costs with treatment in the case of positive within-individual correlation), we can rewrite the joint distribution as

$$⟨\Delta Q_{\mathit{\text{micro}}},\Delta C_{\mathit{\text{micro}}}⟩~\mathit{\text{MVN}}\left(⟨\Delta Q_{\mathit{\text{psa}}},\Delta C_{\mathit{\text{psa}}}⟩,{\Sigma }_{\mathit{\text{micro}}}\right)$$ (16)

where ${\Sigma }_{\mathit{\text{micro}}}$ is the variance-covariance matrix with diagonal elements defined by ${\left(s{e}_{\Delta Q,\mathit{\text{micro}}}\right)}^2$ and ${\left(s{e}_{\Delta C,\mathit{\text{micro}}}\right)}^2$ and off-diagonal elements defined as ${\rho }_{\mathit{\text{micro}}}s{e}_{\Delta Q,\mathit{\text{micro}}}s{e}_{\Delta C,\mathit{\text{micro}}}$, with correlation in the incremental QALYs and incremental costs experienced by individuals equal to ${\rho }_{\mathit{\text{micro}}}$.

For the iSTM, given a draw from the population distribution of incremental QALYs and incremental costs (i.e., PSA sample), the microsimulated estimate of the incremental net monetary benefit can be expressed as

$$\begin{gathered} \mathit{\text{INM}}{B}_{\mathit{\text{micro}}}=\lambda \Delta Q_{\mathit{\text{micro}}}-\Delta C_{\mathit{\text{micro}}} \\ \mathit{\text{INM}}{B}_{\mathit{\text{micro}}}=\lambda \left(\Delta Q_{\mathit{\text{psa}}}+{\epsilon }_{\Delta Q,\mathit{\text{micro}}}\right)-\left(\Delta C_{\mathit{\text{psa}}}+{\epsilon }_{\Delta C,\mathit{\text{micro}}}\right) \end{gathered}$$ (17)

where $\lambda$ is the willingness to pay for QALYs gained. The variance of the microsimulated incremental net monetary benefit across PSA samples is

$$\begin{gathered} \mathit{\text{Var}}(\left.\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right) \\ =\mathit{\text{Var}}\left(\lambda \Delta Q_{\mathit{\text{micro}}}-\Delta C_{\mathit{\text{micro}}}\right) \\ =\mathit{\text{Var}}\left(\lambda \left(\Delta Q_{\mathit{\text{psa}}}+{\epsilon }_{\Delta Q,\mathit{\text{micro}}}\right)-\left(\Delta C_{\mathit{\text{psa}}}+{\epsilon }_{\Delta C,\mathit{\text{micro}}}\right)\right) \\ ={\lambda }^2\mathit{\text{Var}}\left(\Delta Q_{\mathit{\text{psa}}}\right)+{\lambda }^2\mathit{\text{Var}}\left({\epsilon }_{\Delta Q,\mathit{\text{micro}}}\right)+\mathit{\text{Var}}\left(\Delta C_{\mathit{\text{psa}}}\right) \\ +\mathit{\text{Var}}\left({\epsilon }_{\Delta C,\mathit{\text{micro}}}\right)-2\lambda \mathit{\text{Cov}}\left(\Delta Q_{\mathit{\text{psa}}},\Delta C_{\mathit{\text{psa}}}\right) \\ -2\lambda \mathit{\text{Cov}}\left({\epsilon }_{\Delta Q,\mathit{\text{micro}}},{\epsilon }_{\Delta C,\mathit{\text{micro}}}\right) \\ ={\left(\lambda {\sigma }_{\Delta Q}\right)}^2+{\left({\sigma }_{\Delta C}\right)}^2+{\left(\lambda s{e}_{\Delta Q,\mathit{\text{micro}}}\right)}^2+{\left(s{e}_{\Delta C,\mathit{\text{micro}}}\right)}^2 \\ -2\lambda {\rho }_{\mathit{\text{psa}}}{\sigma }_{\Delta Q}{\sigma }_{\Delta C}-2\lambda {\rho }_{\mathit{\text{micro}}}s{e}_{\Delta Q,\mathit{\text{micro}}}s{e}_{\Delta C,\mathit{\text{micro}}} \\ =\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)+\left({\left(\lambda s{e}_{\Delta Q,\mathit{\text{micro}}}\right)}^2+{\left(s{e}_{\Delta C,\mathit{\text{micro}}}\right)}^2\right. \\ \left.-2\lambda {\rho }_{\mathit{\text{micro}}}s{e}_{\Delta Q,\mathit{\text{micro}}}s{e}_{\Delta C,\mathit{\text{micro}}}\right) \end{gathered}$$ (18)

This expression for the variance of $\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}$ does not rely on the normality assumptions above. Note that for simplicity, in the example we are considering here, we assume that the covariances between the microsimulation noise terms (${\epsilon }_{\Delta Q,\mathit{\text{micro}}}$ and ${\epsilon }_{\Delta C,\mathit{\text{micro}}}$) and the PSA sampled quantities ($\Delta Q_{\mathit{\text{psa}}}$ and $\Delta C_{\mathit{\text{psa}}}$) are 0. In principle, if this were not the case, then we would have additional terms in the variance equation of the form $-2{\rho }_{X,Y}\mathit{\text{XY}}$, which when the correlation was less than 0 would lead to even larger variances. From equation (18), as the number of microsimulated individuals increases, that is, $n_{\mathit{\text{micro}}}\to \infty ,\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right)\to \mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right)$.

The expected incremental net monetary benefit (EINMB) estimated with the iSTM, where expectations are taken over the parameter uncertainty $\left(\Theta \right)$, is then

$$\begin{gathered} \mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}=E_{\Theta }\left(\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right) \\ =E\left(\lambda \left(\Delta Q_{\mathit{\text{psa}}}+{\epsilon }_{\Delta Q,\mathit{\text{micro}}}\right)-\left(\Delta C_{\mathit{\text{psa}}}+{\epsilon }_{\Delta C,\mathit{\text{micro}}}\right)\right) \\ =\lambda E\left(\Delta Q_{\mathit{\text{psa}}}+{\epsilon }_{\Delta Q,\mathit{\text{micro}}}\right)-E\left(\Delta C_{\mathit{\text{psa}}}+{\epsilon }_{\Delta C,\mathit{\text{micro}}}\right) \\ =\lambda \left(E\left(\Delta Q_{\mathit{\text{psa}}}\right)+E\left(0\right)\right)-E\left(\Delta C_{\mathit{\text{psa}}}\right)-E\left(0\right) \\ =E\left(\lambda \Delta Q_{\mathit{\text{psa}}}-\Delta C_{\mathit{\text{psa}}}\right) \\ =E\left(\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}\right) \\ =\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}} \end{gathered}$$ (19)

Note that the final equation above implies that the expected incremental net monetary benefit estimated with the iSTM is equal to the expected incremental net monetary benefit estimated with the cSTM.

The uncertainty of the expected incremental net monetary benefit estimated with the microsimulation over the PSA samples is

$$\begin{gathered} \mathit{\text{Var}}\left(\mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}\right)=\frac{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right)}{n_{\mathit{\text{psa}}}-1} \\ =\frac{{\left(\lambda {\sigma }_{\Delta Q}\right)}^2+{\left({\sigma }_{\Delta C}\right)}^2+{\left(\lambda s{e}_{\Delta Q,\mathit{\text{micro}}}\right)}^2+{\left(s{e}_{\Delta C,\mathit{\text{micro}}}\right)}^2-2\lambda {\rho }_{\mathit{\text{psa}}}{\sigma }_{\Delta Q}{\sigma }_{\Delta C}-2\lambda {\rho }_{\mathit{\text{micro}}}s{e}_{\Delta Q,\mathit{\text{micro}}}s{e}_{\Delta C,\mathit{\text{micro}}}}{n_{\mathit{\text{psa}}}-1} \\ ={\lambda }^2{\left(s{e}_{\Delta Q}\right)}^2+{\left(s{e}_{\Delta C}\right)}^2-2\lambda {\rho }_{\mathit{\text{psa}}}s{e}_{\Delta Q}s{e}_{\Delta C}+\left(\frac{{\lambda }^2{\left(s{e}_{\Delta Q,\mathit{\text{micro}}}\right)}^2+{\left(s{e}_{\Delta C,\mathit{\text{micro}}}\right)}^2-2\lambda {\rho }_{\mathit{\text{micro}}}s{e}_{\Delta Q,\mathit{\text{micro}}}s{e}_{\Delta C,\mathit{\text{micro}}}}{n_{\mathit{\text{psa}}}-1}\right) \\ =\mathit{\text{Var}}\left(\overline{\mathit{\text{EINM}}{B}_{\mathit{\text{psa}}}}\right)+\left(\frac{{\lambda }^2{\left(s{e}_{\Delta Q,\mathit{\text{micro}}}\right)}^2+{\left(s{e}_{\Delta C,\mathit{\text{micro}}}\right)}^2-2\lambda {\rho }_{\mathit{\text{micro}}}s{e}_{\Delta Q,\mathit{\text{micro}}}s{e}_{\Delta C,\mathit{\text{micro}}}}{n_{\mathit{\text{psa}}}-1}\right) \end{gathered}$$ (20)

This expression for the variance of $\mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}$ in equation (20) does not rely on the normality assumptions above.

In terms of decision uncertainty and the value of information, the probability that treatment is the optimal strategy for the iSTM is

$$\begin{gathered} \mathit{\text{CEA}}{C}_{\mathit{\text{micro}}}=P\left(\lambda \left(\Delta Q_{\mathit{\text{psa}}}+{\epsilon }_{\Delta Q,\mathit{\text{micro}}}\right)-\left(C_{\mathit{\text{psa}}}+{\epsilon }_{\Delta C,\mathit{\text{micro}}}\right)>0\right) \\ =1-\Phi \left(\frac{0-\mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}}{\sqrt{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right)}}\right) \\ =1-\Phi \left(\frac{0-\mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}}{\sqrt{{\left(\lambda {\sigma }_{\Delta Q}\right)}^2+{\left({\sigma }_{\Delta C}\right)}^2+{\left(\lambda s{e}_{\Delta Q,\mathit{\text{micro}}}\right)}^2+{\left(s{e}_{\Delta C,\mathit{\text{micro}}}\right)}^2-2\lambda {\rho }_{\mathit{\text{psa}}}{\sigma }_{\Delta Q}{\sigma }_{\Delta C}-2\lambda {\rho }_{\mathit{\text{micro}}}s{e}_{\Delta Q,\mathit{\text{micro}}}s{e}_{\Delta C,\mathit{\text{micro}}}}}\right) \end{gathered}$$ (21)

The EVPI for the iSTM is defined as

$$\mathit{\text{EVP}}{I}_{\mathit{\text{micro}}}=\left\{\begin{array}{cc}E\left(\mathit{\text{max}}\left(0,-\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right)\right)& \mathit{\text{if}}\mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}>0\\ E\left(\mathit{\text{max}}\left(0,\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right)\right)& \mathit{\text{if}}\mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}<0\end{array}\right.$$ (22)

In the case of $\mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}>0$, we can express the value of information as

$$\mathit{\text{EVP}}{I}_{\mathit{\text{micro}}}=\underset{-\infty }{\overset{0}{\int }}\left(-\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\frac{1}{\sqrt{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right)}}\phi \left(\frac{\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}-\mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}}{\sqrt{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right)}}\right)\right)\mathit{\text{dINM}}{B}_{\mathit{\text{micro}}}$$ (23)

Likewise, in the case of $\mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}<0$, this expression is then

$$\mathit{\text{EVP}}{I}_{\mathit{\text{micro}}}=\underset{0}{\overset{\infty }{\int }}\left(\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\frac{1}{\sqrt{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right)}}\phi \left(\frac{\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}-\mathit{\text{EINM}}{B}_{\mathit{\text{micro}}}}{\sqrt{\mathit{\text{Var}}\left(\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}\right)}}\right)\right)\mathit{\text{dINM}}{B}_{\mathit{\text{micro}}}$$ (24)

Definition of Bias

As noted in the introduction, the expected outcomes from simulations with the iSTM provide estimates of corresponding outcomes from the mean-process represented by cSTM. We define bias as

$$\mathit{\text{Bia}}{s}_{\mathit{\text{micro}}}(\mathit{\text{Outcom}}{e}_{\mathit{\text{micro}}})=E(\mathit{\text{Outcom}}{e}_{\mathit{\text{micro}}}-\mathit{\text{Outcom}}{e}_{\mathit{\text{cSTM}}})$$ (25)

In other words, the iSTM estimate of an expected outcome for the mean-process it represents is biased if it differs systematically from that produced by the meanprocess model (cSTM). In (25), since it is typical to compute outcomes of both the iSTM and cSTM with finite numbers of PSA samples, what is produced is an estimate of the bias. However, if one replicated the process many times (repeatedly generating a finite number of samples from the PSA distribution and then computing outcomes with an iSTM and cSTM), one could compute the expected bias by averaging across these replicates. It is possible, as we have noted above, that for some outcomes, the iSTM estimate may be unbiased (i.e., the expected ${\mathit{\text{Bias}}}_{\mathit{\text{micro}}}=0$), while for others, it may be biased. Furthermore, if for some outcome where the iSTM estimate is biased on expectation and the expected ${\mathit{\text{Bias}}}_{\mathit{\text{micro}}}$ approaches 0 as $n_{\mathit{\text{micro}}}$ approaches $\infty$, then the iSTM will be biased but consistent.

Numerical Simulations

To illustrate the results implied by the equations above, we perform numerical simulations. The goal is to show that there exist cases in which the decision uncertainty and VOI outcomes generated by the iSTM are biased estimates of the outcomes from the corresponding cSTM and to show that the direction of bias for the decision uncertainty outcome can be either upwards or downwards.

First, we replicate outcomes from a cSTM accounting for parameter uncertainty by sampling from various distributions directly defined for the $\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}$ (normal distribution or mixture of normal distributions [that might occur due to bimodal priors or heterogeneity]) for decision problems involving a given number of strategies (2 strategies or 3 strategies), which then implies either 1 or 2 INMB distributions relative to the comparator strategy. For each of these, to replicate outcomes from a corresponding iSTM, we then simulate samples from the corresponding distributions of $\mathit{\text{INM}}{B}_{\mathit{\text{micro}}}$ by adding normally distributed zero-mean noise to the samples of $\mathit{\text{INM}}{B}_{\mathit{\text{psa}}}$ where the variance of this zero-mean noise decreases with more microsimulated individuals (see the online appendix for further details). With these samples, we compute the EINMB, the probability of being costeffective, and the EVPI for the cSTM and iSTM per the expressions above for these outcomes.

Next, we illustrate these results at various numbers of PSA samples and microsimulated individuals and with various correlation structures for the incremental costs and incremental QALYs at the population and/or individual levels. We perform additional numerical simulations using the parameter values defined in Table 1. As Table 1 shows, there are, in fact, 9 sets of parameter values as we consider combinations of second-order correlation of incremental QALYs and incremental costs with treatment (i.e., nonzero covariance in the joint parameter uncertainty distribution, where higher average incremental QALYs implies higher [or lower] average incremental costs) as well as how incremental QALYs and incremental costs may be correlated within microsimulated individuals (i.e., people who have higher than average incremental QALYs from treatment are also more likely to have higher [or lower] than average incremental costs with treatment). For each set of parameter values, we compute cSTM results with two sizes of PSA samples $\left(n_{\mathit{\text{psa}}}\in \left\{\mathrm{1000,4000}\right\}\right)$. For each PSA sample size, we then compute iSTM results with different numbers of microsimulated individuals $(n_{\mathit{\text{micro}}}\in \{10,40,160,640,2560\left\}\right)$. For each PSA sample size and number of microsimulated individuals, we bootstrap uncertainty intervals for the iSTM results by repeating the microsimulations 400 times and then taking the 2.5th and 97.5th percentile of the results.

Table 1.

Parameter Values for Numerical Simulations

Parameter	Description	Value for Example
$\Delta Q_{\mathit{\text{psa}}}$	Mean incremental QALYs with treatment for second-order uncertainty distribution	0.02
$\Delta C_{\mathit{\text{psa}}}$	Mean incremental costs with treatment for second-order uncertainty distribution	250
${\sigma }_{\Delta Q}$	Standard deviation of incremental QALYs with treatment for second-order uncertainty distribution	0.01
${\sigma }_{\Delta C}$	Standard deviation of incremental costs with treatment for second-order uncertainty distribution	20
${\rho }_{\mathit{\text{psa}}}$	Correlation in the second-order uncertainty of incremental QALYs and incremental costsa	{−0.9, 0, 0.9}
${\sigma }_{\Delta Q,\mathit{\text{micro}}}$	Standard deviation of incremental QALYs with treatment across individuals in the microsimulation	0.20
${\sigma }_{\Delta C,\mathit{\text{micro}}}$	Standard deviation of incremental costs with treatment across individuals in the microsimulation	400
${\rho }_{\mathit{\text{micro}}}$	Correlation within individuals of incremental QALYs and incremental costsa	{−0.9, 0, 0.9}

^aCorrelations show multiple values as we conduct numerical simulations with all combinations of positive, negative, and no correlations in the second-order uncertainty distribution and the within-person outcomes.

To enable reproducibility, all code for the numerical simulations is included in an online repository accessible via persistent URL (https://purl.stanford.edu/sk494zp7768).

Results

Overall, in probabilistic analyses, while individual-level microsimulations produced unbiased estimates of EINMB when compared with equivalent cohort state transition Markov models, their estimates of decision uncertainty (e.g., the probability of being cost-effective at a given willingness-to-pay threshold) and EVPIs are systematically biased (but asymptotically consistent).

Figure 1A provides the underlying intuition. In the case of a normally distributed INMB in a 2-strategy decision problem, 1) the EINMB is the same for both the cSTM and iSTM, 2) there is greater variance in the INMB (and hence greater uncertainty in the estimate of the EINMB) with the iSTM due to the added Monte Carlo noise of microsimulating a finite number of individuals, 3) the iSTM estimate of the probability of not being cost-effective differs from the cSTM’s estimate, and 4) the iSTMs overestimates EVPI compared with the cSTM. The misestimates and overestimates occur because while the distributions for the cSTM and iSTM are symmetric and have equal centers, statistics concerning decision uncertainty depend on one tail of the distribution, and hence, the differentially large variance of the iSTM’s INMB distribution generates bias in its estimates of these statistics.

Figure 1.

Distributions of samples from the incremental net monetary benefit (INMB) distribution of treatment compared with no treatment using a cohort state transition model (cSTM; red lines) versus using an individual-level state-transition microsimulation model (iSTM; blue lines). (A) Results when the population uncertainty distribution of the INMB is normal. (B) Results when the distribution is bimodal (nonnormal). In both panels, estimates of expected INMB (EINMB) are the same for the cSTM and iSTM, but the distribution generated with the iSTM has a higher variance, which leads to bias in estimates of the probability that treatment is costeffective compared with the cSTM estimates (1 minus the shaded areas to the left of the INMB = 0 under the red and blue curves) as well as to biased estimates of expected value of perfect information (EVPI; see table inset). The direction of the bias for the probability of being cost-effective with normally distributed INMB differs from the direction of bias in the nonnormally distributed case.

Figure 1B extends the intuition for a case in which the INMB distribution is a mixture of normal distributions (i.e., not normally distributed). As with the normally distributed INMB, the EINMB for the CSTM and ISTM is equal, and the EVPI from the iSTM is overestimated. However, unlike the normally distributed case, the probability of being cost-effective is now underestimated with the iSTM. Hence, while this statistic is biased in both cases, the direction of bias is not obvious a priori.

Biases in the Case of 2 Decision Alternatives

Our numerical simulations for the iSTM estimates of the EINMB show them to be unbiased estimates of those produced by a corresponding cSTM (Figure 2A and B). This is the case regardless of whether incremental costs and incremental QALYs are correlated at the individual level or at the population level (Appendix Figure 1). One can increase the precision of an iSTM’s estimate of the EINMB (i.e., reduce its variance) by increasing both the number of PSA samples and the number of individuals microsimulated. Notably, the estimates of the EINMB produced by an iSTM are less precise (i.e., have higher variance) with greater individual-level anticorrelation between incremental QALY gains with treatment and incremental costs with treatment (Figure 2C).

Figure 2.

Distributions of estimates of the expected incremental net monetary benefit (EINMB) derived from a cohort state transition model (cSTM; black vertical line) versus an individual-level state-transition microsimulation model (iSTM) using different numbers of PSA samples and different numbers of microsimulated individuals per PSA sample (the colors denote the number of microsimulated individuals per sample). (A) Results at a higher willingness-to-pay threshold. (B) Results at a lower willingness-to-pay threshold. Variance in the estimate of the EINMB made with an iSTM is reduced by having larger numbers of PSA samples and by having larger numbers of microsimulated individuals per PSA sample. (C) Because the INMB involves subtracting costs from the monetary equivalent of quality-adjusted life-years (QALYs), a higher negative individual-level correlation between incremental QALYs and incremental costs leads to larger variance in the iSTM estimate of EINMB. For all panels, the distributions of EINMB are bootstrapped by replicating the entire procedure (PSA sampling and microsimulation for each PSA sample) 400 times. PSA, Probabilistic Sensitivity Analysis.

While iSTM estimates of EINMB are unbiased, their estimates of the probability of being cost-effective are biased (Figure 3A). In the case of a 2-alternative decision problem and normally distributed incremental costs and QALYs, iSTM estimates are downward biased when the cSTM estimate of the probability of being cost-effective is greater than 50% and upward biased when the cSTM estimate is less than 50%. This is due to additional first-order Monte Carlo noise leading to increased variance, increasing the proportion of PSA samples for which the INMB is on the opposite below (above) 0 when treatment is the optimal (nonoptimal) action. More generally, for decisions with larger numbers of alternatives and different distributions, patterns of bias can differ and are more complex (see below).

Figure 3.

Cost-effectiveness acceptability curves estimated for the probability that treatment is cost-effective at different willingness-to-pay thresholds using a cohort state transition model (cSTM; black line) versus an individual-level state-transition microsimulation model (iSTM) using different numbers of PSA samples and different numbers of microsimulated individuals per PSA sample (colored lines denote the number of microsimulated individuals per sample; panel A). Increasing the number of PSA samples does not decrease the bias of the iSTM estimates; only increasing the number of microsimulations per PSA sample does. (B) Because the incremental net monetary benefit (INMB) involves subtracting costs from the monetary equivalent of quality-adjusted life-years (QALYs), a higher negative individual-level correlation between incremental QALYs and incremental costs leads to larger expected bias in the iSTM estimates. For each PSA sample size and number of microsimulated individuals, we bootstrap uncertainty intervals (shaded colored regions around each line in A) for the iSTM results by repeating the microsimulations 400 times and then taking the 2.5th and 97.5th percentile of the results. PSA, Probabilistic Sensitivity Analysis.

While the iSTM estimates of the probability of being cost-effective are biased, they are asymptotically consistent, in that as the number of microsimulated individuals is increased, the size of the bias is expected to decrease to 0. Importantly, the bias is not decreased by having more PSA samples because those represent second-order uncertainty. In addition, bias is larger when the within-person incremental QALYs with treatment and within-person incremental costs with treatment are more negatively correlated (Figure 3B). Appendix Figure 2 shows that there are some differences in the bias in the probability of being cost-effective due to correlation at the individual and population level in incremental QALYs and incremental costs, but across all cases, the number of microsimulated individuals exerts an important influence on the size of this bias.

The iSTM estimates of EVPI are also biased (Figure 4). In our example, the direction of the bias is upward. The estimates, while biased, are asymptotically consistent, in that as more individuals are microsimulated, the bias tends to 0. Since information has value when it allows one to shift from a suboptimal decision to an optimal decision and since with iSTMs there is a larger fraction of PSA samples for which INMB estimates fall on the opposite side of 0 from the EINMB, a greater proportion of PSA samples will appear to have a positive value of information. Furthermore, since the variance of the INMB estimates with the iSTM is greater, there will be a higher proportion of samples with larger magnitude values of information. As can be seen in Appendix Figure 3, especially at lower WTP thresholds (e.g., below ~ $12,500/QALY gained in the example), the bias in the EVPI is larger when the within-person incremental QALYs with treatment and within-person incremental costs with treatment are more negatively correlated. Appendix Figure 3 shows that there are some differences in the EVPI bias due to correlation at the individual and population level in incremental QALYs and incremental costs, but across all cases, the number of microsimulated individuals exerts an important influence on the size of this bias.

Figure 4.

The figure shows the expected value of perfect information at different willingness-to-pay thresholds using a cohort state transition model (cSTM; black line) versus an individual-level state-transition microsimulation model (iSTM) using different numbers of PSA samples and different numbers of microsimulated individuals per PSA sample (colored lines denote number of microsimulated individual per sample). Increasing the number of PSA samples does not decrease the bias of the iSTM estimates; only increasing the number of microsimulations per PSA sample does. For each PSA sample size and number of microsimulated individuals, we bootstrap uncertainty intervals (shaded colored regions around each line) for the iSTM results by repeating the microsimulations 400 times and then taking the 2.5th and 97.5th percentile of the results. PSA, Probabilistic Sensitivity Analysis.

Bias in the Case of 3+ Decision Alternatives

We examine the case of a decision problem with 3 alternatives where the INMBs for 2 of the alternatives (strategies 1 and 2) are defined with respect to the status quo (strategy 0). To illustrate some of the potentially complex patterns of bias in the probability of cost-effectiveness and EVPI, we simulate many cases of this decision problem that differ in terms of the means and variances of the INMB distributions (normal) defined for the alternatives and the amount of 0-mean Monte Carlo noise that exists for each alternative when using an iSTM to simulate it. In all cases, strategy 1 has the highest EINMB and is therefore the preferred option with current information. Strategy 2 has a lower EINMB, which can either be positive (better than the status quo strategy 0) or negative (worse than the status quo). In total, we simulate 5,760 different combinations of EINMBs and amount of Monte Carlo noise in the INMB distributions due to microsimulation (Appendix Table 1).

Figure 5 shows patterns of bias for 3 illustrative cases. Given the normality assumption for the INMB distributions, the probability that strategy 1 (the preferred strategy on expectation) is optimal computed with the iSTM is downward biased in these cases. Because of this, the probability of another alternative being optimal is overestimated for at least 1 of the 2 other alternatives. However, the cases show that sometimes the probabilities for both strategies 0 and 2 can be overestimated, but at other times, for one alternative, the probability is overestimated, and for the other, the probability is underestimated. For these cases, just as with the 2-alternative decision problems considered above, the bias in EVPI estimated with the iSTM is ≥0 (upward biased; see Appendix Figures 4–6 for full details of the patterns of bias for the probabilities of being cost-effective and the EVPI for the 5,760 combinations we simulated; the appendix figures show that the patterns in the illustrative examples are present in many of the combinations we simulated).

Figure 5.

Complex patterns of bias that can emerge when using an individual-level state-transition microsimulation model (iSTM) to assess decision uncertainty for decision problems involving 3 alternatives. The figure shows summary results for 3 example problems. All 3 examples have expected incremental net monetary benefit (EINMB) for strategy 1 (the strategy preferred on expectation) of 0.2 with the EINMB for strategy 0 (status quo) and strategy 1 being 0 (a toss-up for second best). The 3 examples differ in terms of how much stochastic (Monte Carlo) noise there is for the iSTM’s INMB estimates for strategy 1 (from least to most noise going down the panels of the figure) while holding the noise for the strategies fixed and equal to or less than for strategy 1. The probability of being cost-effective is underestimated for strategy 1 in all 3 examples; however, strategy 2’s probability of being cost-effective can be either overestimated or underestimated such that, in some examples, the probabilities of being cost-effective for both strategy 0 and strategy 2 are overestimated whereas in other cases only 1 of them is overestimated (see the appendix for results for the 5,760 variants of the 3-alterantive decision problem that we explored numerically, which further illustrate the complexity of patterns of bias that can arise).

Discussion

Economic evaluation in health and medicine is witnessing 3 converging trends. The first is a shift from cohort state transition models (i.e., Markov cohort models, cSTMs) to individual-level state transition models (i.e., microsimulation or iSTMs) to enable modelers to reflect more complex and heterogeneous diseases and intervention strategies. The second is the use of probabilistic analyses to appropriately estimate the expectations of outcomes simulated using these decision models and to propagate population-level parameter uncertainty in these expectations. The third is a growing interest in gauging current levels of decision uncertainty and the value of collecting additional information to resolve that uncertainty.

Given these trends, our study considers the potential for bias introduced into estimates of decision uncertainty and value of information through iSTMs whose outputs also necessarily contain first-order Monte Carlo noise. We confirm both analytically and through a range of numerical simulations for iSTMs that while their estimates of the expected incremental value of decision alternatives are unbiased with respect to corresponding cSTMs, their estimates of decision uncertainty and the value of information are biased but asymptotically consistent.

A key implication of the study findings involves how analysts conducting economic evaluations with iSTMs allocate their computational budgets.^21–23 For computing EINMB, the number of parameter sets sampled from the parameter uncertainty distributions (i.e., outer loops) and the number of microsimulated individuals simulated for each parameter set sampled (i.e., inner loops) are interchangeable. However, for decision uncertainty and value of information, taking more samples from the parameter uncertainty distribution does not reduce the bias in the estimates produced by the iSTM; only simulating larger numbers of microsimulated individuals will reduce this bias. Hence, the appropriate balance between inner and outer loop samples depends on the quantities that the analyst is interested in estimating and the precision required for each of them requiring tradeoffs in their accuracy given the computation budget.

Given the study findings, one area of future research is developing efficient diagnostics to gauge the magnitude of the bias for given combinations of inner and outer loop samples, which can be used to suggest efficient computational budget allocations. For example, while more samples are better if one ignores computational budgets, how many samples would be sufficient to make the iSTM bias suitably small depends on the model used and the decision being considered. A model focused on comparing 2 highly effective strategies for preventing a relatively rare disease outcome (e.g., 2 aggressive screening strategies for a relatively rare cancer) may have to simulate many more individuals to form stable estimates of incremental net benefits of one strategy versus the other than if the outcome were common or the strategies being compared were substantially different in their costs and the health benefits they produced. Approaches might use a fraction of the overall computational budget to perform a probabilistic analysis with a given number of outer (e.g., 1,000) and inner loops (e.g., 400) and then act as if all of these samples were not available to consider how the model estimates change if there were 1,000 outer and 100 inner samples or 1,000 outer and 200 inner samples. One could then use a form of convergence analysis to examine how and at what rate the estimates changed as a function of the number of inner loop samples. In principle, one could adapt value-of-information approaches concerning optimal sample size and study design to this situation as well.²⁷

While our study has established that there exist cases in which iSTMs generate biased but asymptotically consistent estimates of decision uncertainty and value-of-information quantities with respect to equivalent cSTMs, the characterization of the situations in which these biases are most pronounced and most consequential will require additional work. One important extension is greater consideration of various distributions of incremental net monetary benefit as well as the distribution of the incremental cost and incremental health benefits and their correlations within and between the population and individual levels. While the present analysis explored a wide range of correlations, empirical characterization of these correlations from data and from outputs of simulation models used in practice would be informative. Another important extension is considering these biases as the number of interventions increases. While the analyses visualized in Figure 5 demonstrate that biases exist for decision problems involving 3 alternatives like they do in the case of 2-decision alternatives, the complexity in the potential patterns of bias appears to grow with the number of alternatives. A third extension would be to consider the biases with iSTMs for estimating other measures of value of information such as partial perfect information and the expected net benefit of sampling and optimal sample size. A fourth area would be to gauge the potential size of biases in the existing applied literature by examining the estimates of these quantities from published model-based analyses using iSTMs and reimplementing the equivalent cSTMs to calculate the biases and perhaps re-performing the iSTM analyses with different inner and outer loop sample sizes. In summary, there are numerous opportunities for promising future work in this area.

Our study is careful to define bias as a systematic deviation in estimating decision-analytic outcomes of the mean process (cSTM) using a corresponding iSTM. The mean-process outcomes are those relevant to a risk-neutral decision maker and in situations in which the actual population for whom the decision is relevant is large. In this case, the additional noise from microsimulation is essentially a nuisance that should be reduced and removed to the extent possible.^28–30 However, in the case of risk-averse decision makers in situations where the population for whom the decision is relevant is small (i.e., so that the number of microsimulated individuals might be equal to the number of actual people in that population itself), the noise from the iSTM may be relevant. For example, if one were deciding between 2 alternative actions for an institution housing 100 people, the fraction of time that 1 alternative action will produce the highest INMB should include Monte Carlo noise, although such an estimate systematically differs from what one would have estimated for a very large institution using a cSTM. In other words, in these situations, the iSTM is not introducing bias but rather telling us something very important that we may well want to know in addition to what meanprocess estimates might tell us.

In conclusion, despite being unbiased in identifying the best strategy on expectation, iSTMs introduce bias in estimates of decision uncertainty and value of information. Fortunately, as the bias is asymptotically consistent, the size of the bias can be reduced by using sufficiently large numbers of microsimulated individuals per probabilistic sample. Analysts using iSTMs for economic evaluations that include probabilistic analyses should explicitly demonstrate that the number of microsimulated individuals per probabilistic sample is sufficiently large to make the bias in these estimates small.

Supplementary Material

Supplementary material for this article is available online at https://doi.org/10.1177/0272989X241305414.

Highlights.

• Individual-level state-transition microsimulation models (iSTMs) produce biased but consistent estimates of the probability that interventions are cost-effective.

• iSTMs also produce biased but consistent estimates of the expected value of perfect information.

• The biases in these decision uncertainty and value-of-information measures are not reduced by more parameter sets being sampled from their population-level uncertainty distribution but rather by more individuals being microsimulated for each parameter set sampled.

• Analysts using iSTMs to quantify decision uncertainty and value of information should account for these biases when allocating their computational budgets and, at minimum, characterize such bias in their reported results.

Data Availability

All code and data used to carry out the analysis have been deposited in a public archive to facilitate transparency and replicability (https://purl.stanford.edu/sk494zp7768).