Individualized Treatment Rules Based on Cost-Effectiveness Criteria in Microsimulations

Abstract

Background and Objective

In cost-effectiveness analysis, treatment decisions are analysed at the population level. Combinations of treatment strategies that account for the heterogeneity of costs and effects across patients can be more cost-effective than a “one size fits all” approach. Individualized treatment rules (ITRs) assign a specific treatment to every patient based on their relevant characteristics, such that overall cost-effectiveness is optimized, but do not include feasibility or ethical considerations. We propose an approach for the design of ITRs based on simulated patient data from microsimulation models using statistical learning techniques.

Methods

We mathematically define the optimal ITR and how to measure the value of an ITR in a cost-effectiveness context. We explore least absolute shrinkage and selection operator (LASSO) regression, classification trees, and policy trees to illustrate how standard statistical learning techniques can be used to derive ITRs. We compare the strengths and limitations of these three approaches in terms of three criteria: the incremental value of the ITRs compared to optimal treatment assignment in terms of net monetary benefit (NMB), computational speed, and the interpretability of the ITRs. We propose methods to describe the impact of parameter uncertainty on the ITRs. We also explore how stochastic uncertainty can impact the ITR incremental value. We illustrate the methods by applying them to a microsimulation model for haemophilia B comparing four treatment strategies as a case study. The relevant patient characteristics in this model are the annualized bleeding rate, age, and sex.

Results

In our case study, a simple two-layer-deep classification tree is best suited based on the three criteria. This classification tree allocates treatments depending on whether the annualized bleeding rate of a patient is above or below 30 and whether their age is above or below 51. The optimal threshold values are uncertain based on the 95% credible ranges from the probabilistic analysis: 21–46 for annualized bleeding rate and 42–56 for age. Scenarios show that stochastic uncertainty has an impact on the incremental value of the ITR.

Discussion

Based on methodological considerations and the empirical findings in our case study, we expect the superiority of classification trees for the derivation of ITRs to be generalizable to other microsimulation models. This finding needs to be confirmed in future applications. Stochastic uncertainty has significant impacts on the ITRs, such that accurate representations of individual patient pathways are particularly crucial when designing ITRs. Future research could explore further empirical models and analytical approaches for ITRs or consider the translation of ITRs into the real-world decision-making context.

Key Points for Decision Makers

Microsimulation cost-effectiveness models can be used to design individualized treatment rules.

Individualized treatment strategies can be more cost-effective than uniform treatment strategies but are also subject to uncertainty.

Individualized treatment rules can be expressed graphically as decision trees or as a list of IF statements.

Introduction

In cost-effectiveness analysis, treatments are typically evaluated at the cohort level, overlooking heterogeneity in prognostic factors and outcomes. However, the most cost-effective treatment for a population may not be optimal for every individual. Considering variations in treatment costs and effects across patients in the treatment assignment can lead to more cost-effective approaches than uniform treatment strategies [1–4]. Beyond understanding the value of individualized care, it is important to present individual-level cost-effectiveness findings in an easily interpretable manner. Individualized treatment rules (ITRs) can be used to determine how treatments can be assigned on the basis of patient characteristics [5, 6] that influence cost-effectiveness outcomes. They provide useful insight by identifying subgroups or individuals for whom a given treatment is cost-effective, without pre-specification of subgroups by the modellers. ITRs may not always be suitable for real-world implementation due to the costs of implementation [3] or due to ethical concerns, e.g. about discrimination or exacerbation of inequity [7–9]. Although ethical and feasibility considerations, as well as clinical relevance, are crucial for real-world applications, they fall outside the scope of this study, which focuses solely on methods for designing ITRs.

Recent studies have presented novel approaches for designing ITRs on the basis of individual patient data for cost-effectiveness analysis via machine learning [10–12]. Machine learning methods can help to individualize care in different ways [13, 14]. First, they can be used to identify heterogeneity in treatment effects [15], and thereby make counterfactual predictions of the cost-effectiveness of different treatments for individual patients. Second, machine learning can be used to identify optimal ITRs that maximize mean cost-effectiveness outcomes, based on the observed individual patient data as well as the predicted counterfactuals [16].

However, cost-effectiveness analysis based on individual patient data is not appropriate for decision-making if the observation period is too short to capture all relevant consequences of treatment decisions over the patients’ lifetime or if the data do not include the entire patient population affected by a decision. In such cases, decision-analytic models become necessary [17]. Microsimulations are patient-level decision-analytic models in which the heterogeneity of patient characteristics, health events, and treatment sequences can be emulated. Microsimulations, like other modelling approaches, can be used to extrapolate costs and effects beyond the observation period of clinical data [18–20].

We propose a conceptual framework for ITRs based on cost-effectiveness microsimulations. We compare multiple approaches for designing ITRs in microsimulation models via machine learning. These leverage the availability of counterfactual outcomes under alternative treatments for every simulated patient. The availability of these simulated counterfactuals removes the need to predict counterfactuals via statistical machine learning techniques and instead allows for the direct identification of ITRs in the simulated patient data. This approach can be pursued with any cost-effectiveness microsimulation model and does not require any modifications to the underlying model.

As with other cost-effectiveness analyses, uncertainty should be considered when determining an optimal ITR. Uncertainty can be categorized as parameter uncertainty, stochastic uncertainty, or structural uncertainty [21]. Parameter uncertainty refers to the statistical uncertainty of the microsimulation model parameter values. We use probabilistic analysis to assess the impact of parameter uncertainty on the uncertainty of the ITR parameters and present it as credible intervals and graphically. We study how stochastic uncertainty can influence the ability of ITRs to optimally individualize care by varying the degree of stochastic uncertainty in the model. Structural uncertainty about the underlying cost-effectiveness model can affect ITRs but is not the focus of this article. Patient heterogeneity is distinct from uncertainty but modelled using uncertain model parameters. Hence, parameter uncertainty also creates uncertainty about heterogeneity, which we aimed to capture in the probabilistic analysis.

The remainder of the article is structured as follows. First, we provide a mathematical definition of the optimal ITR and outline how to measure the value of an ITR in a cost-effectiveness context. Second, we explore regressions with the least absolute shrinkage and selection operator (LASSO), classification trees, and policy trees to demonstrate how standard statistical learning techniques can be used to derive ITRs based on microsimulation models. Third, we compare the strengths and limitations of these three approaches in terms of three criteria: the incremental value of ITRs, computational speed, and the ability to express ITRs in an easily interpretable manner. Fourth, we propose methods to describe the impact of parameter uncertainty on ITRs. Fifth, we explore how random events, which, due to stochastic uncertainty, cannot be predicted with certainty based on patient characteristics and have heterogeneous impacts on patient outcomes, can impact the incremental value of an ITR. Sixth, we introduce an extension of a previously published microsimulation for haemophilia B [22], in which the gene therapy etranacogene dezaparvovec (ED) is compared with prophylactic and on-demand administration of extended half-life factor IX (EHL-FIX), as a case study. The relevant and observable characteristics in the model are the annualized bleeding rate (ABR), baseline age, and sex of patients. Seventh, we present the described methods and their results by applying them to the haemophilia B model.

Methods

Decision Rules

A decision rule, denoted as $\pi (X)$, is a function of a matrix of patient characteristics $X$ that returns a vector of treatment assignments for all patients. $\pi (X)$ determines which patient $i\in \{\text{1,2},\cdots ,N\}$ should receive which treatment $j\in \{1,2,\cdots ,M\}$. The expression $\pi \left(X\right)=j$ describes a situation in which the decision rule assigns a specific but unspecified treatment scheme $j$. In cost-effectiveness analysis, the value of treatment $j$ for a specific patient $i$ can be defined as the patient’s net monetary benefit, ${\text{NMB}}_{\mathit{ij}}$, which is calculated by multiplying the effectiveness $E_{\mathit{ij}}$ with the willingness to pay, $\text{WTP}$, and subtracting the costs, $C_{\mathit{ij}}$ [23, 24].

$${\text{NMB}}_{\mathit{ij}}=E_{\mathit{ij}}\ast \text{WTP}-C_{\mathit{ij}}$$

Traditional cost-effectiveness analysis applies a uniform treatment rule $\overline{ȷ}$ to all patients [1, 2]. $\pi =\overline{ȷ}$ indicates that each patient is assigned the same treatment, independently of their individual characteristics. The value $V(\overline{ȷ})$ of a uniform treatment rule $\pi =\overline{ȷ}$ is the expected value of individual net monetary benefit, ${\text{NMB}}_{i\overline{ȷ}},$ under that treatment, given the matrix of patient characteristics $X$ in the population. The expected value of ${\text{NMB}}_{i\overline{ȷ}}$ is determined by the conditional expectation $E\left[\text{NMB}|X=x,,\overline{ȷ}\right]$ and the joint density of observable characteristics $p(x)$ in the population. In a simulation setting, the expected value of a uniform treatment $V\left(\overline{ȷ}\right)$ can be calculated as the mean ${\text{NMB}}_{i\overline{ȷ}}$ across all patients.1

$$\begin{array}{cc}\hfill V\left(\overline{ȷ}\right)& =E\left[{\text{NMB}}_{i\overline{ȷ}}\right]\hfill \\ \hfill & =\int E\left[\text{NMB|}X=x,\pi =\overline{ȷ}\right]p\left(x\right)\text{d}x\hfill \\ \hfill & \simeq \frac{1}{N}\sum _{i=1}^{N}E\left[\text{NMB|}X=x_i,\pi =\overline{ȷ}\right]\hfill \\ \hfill \end{array}$$

The optimal treatment in a uniform treatment strategy $\overline{j}_{}^{\ast }$ (* denotes an optimal treatment strategy) is chosen to maximize the expected NMB when all patients receive the same treatment. In regular (non-individualized) cost-effectiveness analysis, $\overline{ȷ}_{}^{\ast }$ is simply the cost-effective treatment strategy.

$$\overline{ȷ}_{}^{\ast }=\underset{\overline{ȷ}}{\text{argmax}}V\left(\overline{ȷ}\right)$$

An ITR $\pi \left(X\right)$ that allows different treatments to be assigned to individual patients can be described by a set of IF statements based on threshold values $\rho$, which depend on patient characteristics $X$. We denote specific patient characteristics $k\in \{\text{1,2},\cdots ,K\}$ as $x^{(k)}$. In some but not all instances, $\pi \left(X\right)$ can be described by a combination of multiple IF statements based on multiple characteristics with different threshold values $\rho$ for patient characteristics $X$. For example, an ITR $\pi \left(X\right)$ for two relevant patient characteristics $x^{(1)},x^{(2)}$ that sort patients into treatments $j\in \{1,2,3\}$ could follow the structure below:

1. If $x_i^{(1)}\ge {\rho }^{(1)}$ and $x_i^{(2)}<{\rho }^{(2)}$, patient $i$ is treated with $j=1$

2. If $x_i^{(1)}\ge {\rho }^{(1)}$ and $x_i^{(2)}\ge {\rho }^{(2)}$, patient $i$ is treated with $j=2$

3. If $x_i^{(1)}<{\rho }^{(1)}$, patient $i$ is treated with $j=3$

The value of an ITR $V\left(\pi ,\left(X\right)\right)$ is the expected NMB for the individual treatments $\pi \left(X\right)=j$ assigned to each patient based on all patients’ observable characteristics $X$.

$$\begin{array}{cc}\hfill V\left(\pi \left(X\right)\right)& =\int E\left[\text{NMB|}X=x,\pi \left(X\right)=j\right]p\left(x\right)\text{d}x\hfill \\ \hfill & \simeq \frac{1}{N}\sum _{i=1}^{N}E\left[\text{NMB|}X=x_i,\pi \left(X\right)=j\right]\hfill \\ \hfill \end{array}$$

The incremental value $\mathrm{\Delta }\left(\pi ,\left(X\right)\right)$ of an ITR compared with that of the most cost-effective uniform treatment $\overline{ȷ}_{}^{\ast }$ is the difference between the values of the two treatment strategies.

$$\mathrm{\Delta }\left(\pi ,\left(X\right)\right)=V\left(\pi ,\left(X\right)\right)-V\left(\overline{ȷ}_{}^{\ast }\right)$$

The optimal ITR $\pi _{}^{\ast }\left(X\right)$ maximizes the average NMB across the treated population by assigning each patient the most cost-effective treatment.

$$\pi _{}^{\ast }\left(X\right)=\underset{\pi (X)}{\text{argmax}}V\left(\pi ,\left(X\right)\right)$$

The Expected Value of Individualized Care (EVIC) has been defined in the literature as the incremental value of the optimal ITR $\pi _{}^{\ast }\left(X\right)$ compared with that of the optimal uniform treatment strategy $\overline{ȷ}_{}^{\ast }$ [1, 2].

$$\text{EVIC}(\pi _{}^{\ast }\left(X\right))=\mathrm{\Delta }\left(\pi _{}^{\ast },\left(X\right)\right)=V\left(\pi _{}^{\ast },\left(X\right)\right)-V\left(\overline{ȷ}_{}^{\ast }\right)$$

The optimal ITR $\pi _{}^{\ast }\left(X\right)$ can also serve as a benchmark for ITRs $\pi \left(X\right)$. The regret $R\left(\pi ,\left(X\right)\right)$ of an ITR $\pi \left(X\right)$ measures how much the ITR $\pi \left(X\right)$ deviates from the optimal $\pi _{}^{\ast }\left(X\right)$ in terms of NMB and is calculated as the difference between the value of the optimal ITR $V\left(\pi _{}^{\ast },\left(X\right)\right)$ and the value of the assessed ITR $V(\pi \left(X\right))$. As $V\left(\pi _{}^{\ast },\left(X\right)\right)\ge V\left(\pi ,\left(X\right)\right)$, regret $R\left(\pi ,\left(X\right)\right)$ is never negative. While regret is not relevant for decision-making, it is an important technical metric for understanding how close to a theoretical optimum a given ITR is, and is therefore often used to tune machine learning models [16, 25].

$$R\left(\pi ,\left(X\right)\right)=V\left(\pi _{}^{\ast },\left(X\right)\right)-V\left(\pi ,\left(X\right)\right)\ge 0$$

Designing Data-Driven Individualized Treatment Rules in Microsimulation Data

In this study, we use machine learning techniques to design ITRs by re-analysing simulated patient-level data from a microsimulation model of haemophilia B. This analysis is based on efficacy parameters derived from a published randomized clinical trial and secondary sources.

The main strength of microsimulations for designing ITRs is that they compute counterfactual outcomes under alternative treatments in the same population and thus provide conditional average treatment effects without the need to employ causal machine learning techniques as in observational data settings. Thanks to this property of microsimulation models, standard predictive statistical learning methods can be used to identify predictors of NMB under alternative treatments and to derive ITRs based on these predictors.

Let $f(X,j)$ be a statistical model’s conditional expectation function of the NMB. For regression models, this conditional expectation function is a parametric function of patient characteristics and treatment. For tree-based models, the conditional expectation function is a set of average NMBs under alternative treatments for subgroups with different combinations of patient characteristics $X$. The conditional expectation function can be used to calculate the fitted ${\widehat{\text{NMB}}}_{\mathit{ij}}$ for each simulated patient $i$ under all alternative treatments $j$.

$${\widehat{\text{NMB}}}_{\mathit{ij}}=f(x_i,j)$$

The ITR $\pi (x_i)$ assigns to each patient the treatment $\widehat{\pi }(x_i)$ with the highest expected individual net monetary benefit ${\widehat{\text{NMB}}}_{\mathit{ij}}$ based on the conditional expectation function $f(x_i,j)$.

$$\widehat{\pi }(x_i)=\underset{j}{\text{argmax}}{\widehat{\text{NMB}}}_{\mathit{ij}}$$

In microsimulation data, the modelled “true” ${\text{NMB}}_{\mathit{ij}}$ under all counterfactual therapies $j$ are recorded, and the “truly” optimal individual treatment $\widehat{\pi }_{}^{\ast }(x_i)$ can be determined. This benchmark does not necessarily represent the perfectly individualized treatment for observed heterogeneity in real patients but only the perfectly individualized treatment for the modelled heterogeneity.

$$\widehat{\pi }_{}^{\ast }(x_i)=\underset{j}{\text{argmax}}{\text{NMB}}_{\mathit{ij}}$$

This information enables the estimation of the value of the empirical ITR $V\left(\widehat{\pi },\left(X\right)\right)$ and the value of the “truly” optimal ITR $V\left(\pi _{}^{\ast },\left(X\right)\right)$ in the microsimulation data. Note that the value of the ITR is the expected value of the “true” $\text{NMB}$ under the treatments that were favoured by the conditional expectation function, not the expected value of the conditional expectation function.

$$\begin{array}{cc}\hfill V(\widehat{\pi }(X))& =\frac{1}{N}\sum _{i=1}^{N}E[\text{NMB}\mid X=x_i,\pi (X)=\widehat{\pi }(x_i)]\hfill \\ \hfill V(\pi _{}^{\ast }(X))& =\frac{1}{N}\sum _{i=1}^{N}E[\text{NMB}\mid X=x_i,\pi (X)=\pi _{}^{\ast }(x_i)]\hfill \end{array}$$

The incremental value of an ITR, $\mathrm{\Delta }\left(\widehat{\pi },\left(X\right)\right),$ can be measured by the distance of the value of the ITR, $V\left(\widehat{\pi },\left(X\right)\right)$, from the value of the optimal uniform treatment strategy, $V\left(\overline{ȷ}_{}^{\ast }\right)$.

$$\begin{array}{cc}\hfill \mathrm{\Delta }\left(\widehat{\pi }\left(X\right)\right)& =V\left(\widehat{\pi }\left(X\right)\right)-V\left(\overline{ȷ}_{}^{\ast }\right)\hfill \\ \hfill & =\frac{1}{N}\sum _{i=1}^{N}E\left[\text{NMB|}X=x_i,\pi \left(X\right)=\widehat{\pi }\left(x_i\right)\right]-E\left[\text{NMB|}X=x_i,\pi =\overline{ȷ}_{}^{\ast }\right]\hfill \\ \hfill \end{array}$$

Empirical Models

There are many potentially viable approaches to derive ITRs via outcome modelling in microsimulation patient data. In this study, we explore LASSO regression, classification trees, and policy trees to demonstrate how standard statistical learning techniques can be used to derive ITRs from microsimulation patient data and examine key considerations for selecting an appropriate approach. We chose these three empirical models, all capable of assigning treatments, because they represent widely used off-the-shelf machine learning approaches, yet vary sufficiently in their properties, offering differing advantages and disadvantages.

LASSO regression: Regression models are flexible tools for analysing patient characteristics and predicting the treatment with the highest NMB for individual patients. LASSO regression [26] is a machine learning regularization technique that reduces the coefficients of covariates with low predictive power, and allows the selection of a subset of covariates with non-zero coefficients, simplifying the decision rule.

We first select covariates with non-zero coefficients via LASSO and then use these covariates in a nonregularized regression model. In the candidate set of covariates, we include treatment dummies $\mathit{T}$, binary patient characteristics ${\mathit{X}}_b$, continuous patient characteristics ${\mathit{X}}_{\mathit{c}}$, polynomials of continuous characteristics up to the 5th degree ${\mathit{Z}}^d=poly({\mathit{X}}_{\mathit{c}},d)$, interactions between the treatment dummies and binary patient characteristics $\mathit{T}·{\mathit{X}}_b$, interactions between the treatment and continuous patient characteristics $\mathit{T}·{\mathit{Z}}_c^d$, and interactions between all patient characteristics $\mathit{X}·\mathit{X}$, excluding polynomial terms. This regression model is described by the following equation:

$$\text{NMB}={\beta }_0+\mathit{T}\mathit{\beta }+{\mathit{X}}_b{\mathit{\gamma }}_b+\sum _{d=1}^5{\mathit{Z}}_c^d{\mathit{\gamma }}_c^d+\left(\mathit{T}·{\mathit{X}}_b\right){\mathit{\delta }}_b+\sum _{d=1}^5\left(\mathit{T}·{\mathit{Z}}_c^d\right){\mathit{\delta }}_c^d+\left(\mathit{X}·\mathit{X}\right)\mathit{\theta }+\epsilon$$

We tuned the LASSO model via tenfold cross-validation2 with mean squared error as the loss function and selected the covariates with non-zero coefficients at the penalty parameter $\lambda$ that minimizes the cross-validation error.3 Once the regression coefficients ${\beta }_0$, $\mathit{\beta }$, ${\mathbf{\gamma }}_b$, ${\mathbf{\gamma }}_c^d$, ${\mathbf{\delta }}_b$, ${\mathbf{\delta }}_c^d$, and $\mathit{\theta }$ are estimated, ${\widehat{\text{NMB}}}_{\mathit{ij}}$ can be predicted for all patients in the simulated data. The treatment with the highest ${\widehat{\text{NMB}}}_{\mathit{ij}}$ for patient $i$ is the treatment assigned by the ITR based on the linear regression function. This regression function is an ITR $\pi \left(X\right)$ in the sense that for any given combination of patient characteristics, a clear prediction of the optimal treatment is made.

Classification tree: Recursive partitioning can be used to create classification trees that attempt to correctly classify observations by repeatedly splitting the patient population into a treatment at patient-characteristic threshold values, creating subgroups in the process [29]. Different combinations of patient characteristics and threshold values $\rho$ are algorithmically tested to construct a tree that minimizes the classification error for each patient in the dataset. In this context, a class refers to the optimal treatment for any given patient, meaning that if a patient receives optimal treatment $j$, this is also their class. The threshold values for splitting are determined by the Gini impurity $G$ in the data for combinations of patient characteristics and threshold values [30]. In this application, the Gini impurity coefficient $G$ is calculated as 1 minus the sum of the squared proportions of each class $P[j_{}^{\ast }]_{}^2$ in the subgroup.

$$G=1-\sum _j^{M}P[j_{}^{\ast }]_{}^2$$

If every patient in a subgroup belongs to the same class, the Gini impurity reaches a minimum of 0. As each split creates two subgroups, the Gini impurity coefficient is calculated for each of the two subgroups following the split, $G_1$ and $G_2$. Then, a weighted Gini impurity $G_{\text{w}}$ with weights given by sample sizes in the two subgroups, $N_1$ and $N_2$, is calculated.

$$G_{\text{w}}=\frac{N_1}{N}{G}_1+\frac{N_2}{N}{G}_2$$

The split threshold value $\rho$ that minimizes the weighted Gini impurity $G_{\text{w}}$ is chosen for each split. Within each node, the optimal treatment for most patients (the majority class) is assigned to all patients in that node. By repeating this process multiple times, a tree of nodes, each corresponding to a treatment assignment, is created. This classification tree is the graphical representation of an ITR $\pi \left(X\right)$. For this application, we generated a classification tree with a depth of two layers to keep the complexity of the corresponding ITR low.4 This choice was not based on the empirical properties of the model but rather serves as an example of a case study with the goal of creating a decision rule with limited user-defined complexity.

Policy tree: Bonander and Svensson [10], Glynn et al. [12], and Hattab et al. [11] have previously applied the policy tree algorithm [16] to develop ITRs with the aim of assigning each patient the treatment with the highest value for money for this individual. As with classification trees, policy trees are generated via recursive partitioning, but the algorithm generates nodes and sets split thresholds $\rho$ that minimize regret and thereby maximize the value of a given policy tree, rather than minimizing the Gini impurity as the classification tree algorithm does. Minimizing regret is equivalent to maximizing the incremental value of the ITR [25]. For the present application, we generated a policy tree with a depth of two layers, in order to have a similar level of complexity when comparing it to the classification tree.5

Assessing the Impact of Parameter Uncertainty on Individualized Treatment Rules

The influence of parameter uncertainty on cost-effectiveness outcomes also affects ITRs. Therefore, variations in the optimal ITRs due to parameter uncertainty should be examined via probabilistic analysis [21]. To ensure that the assessments of the parameter uncertainty of cost-effectiveness outcomes and ITRs are congruent, they should be performed using the same random parameter draws. The assessment of the parameter uncertainty of ITRs can be implemented as a direct extension of the probabilistic analysis functionality in the microsimulation model.

We assume that $P$ populations of $N$ patients are simulated, with random population-level parameters resampled for each simulated population, so that we draw a unique set of parameter values for each population. The resulting simulated dataset with $P\times N$ simulated patients can be used to design the ITR with the highest expected value under consideration of uncertainty of the model inputs and outputs. We can also design a new ITR in each simulated population $P$ and describe the variation in ITRs to assess the impact of parameter uncertainty on the ITRs themselves.

Variation in ITRs can occur in two forms: structural variation or ITR parameter value variation. It occurs based on the counterfactual patient outcomes that are used as inputs into the empirical models for deriving ITRs. Structural variation refers to changes in the included patient characteristics and, for decision trees, also to changes in the order in which the patient characteristics are considered. ITR parameter value variation refers to differences in estimated coefficients for regression models or differences in estimated threshold values for decision trees. In our usage of the terms, structural variation and ITR parameter value variation both refer specifically to variation in the ITRs themselves, not the underlying microsimulation model.

Allowing for structural variation makes summarizing parameter uncertainty complex and difficult to interpret. We thus focus purely on the impact of parameter uncertainty on ITR parameter value variation. We design the ITR structure using data from all the simulated populations and then estimate the optimal ITR parameter values while keeping the structure fixed. The resulting ITR parameter value variation can be quantified, for example, using a credible interval. If 95% of the ITR parameter values fall within a specific range, this provides a useful measure of uncertainty stemming from parameter variability.

Criteria for Model Assessment

We formulated three criteria for assessing the three methods for designing ITRs.

Incremental value: The ITR’s incremental value $\mathrm{\Delta }\left(\widehat{\pi },\left(X\right)\right)$ should be as large as possible. When the incremental value of an ITR reaches the possible maximum, it becomes equivalent to the EVIC.

Computational speed: From a practical perspective, tools that require less time to run on widely available hardware grant an advantage when performing cost-effectiveness analyses. We measure the time needed to apply each of the three empirical models on a standard laptop6 for our case study. The runtime of the empirical models will differ depending on the exact hardware, but the recorded times serve as a reasonable benchmark.

Interpretability: To be useful in practice, an ITR should not only assign treatments to each individual on the basis of individual characteristics. It should also be possible to express it as a set of easily interpretable rules, determining which patients would receive which treatments on the basis of characteristic value thresholds. The corresponding rule thresholds should be determined by the empirical model itself and optimized for any number of characteristics across the simulated patient data. It is a further advantage if the complexity of the rules can be determined by the user who is technically implementing the ITRs.

Stochastic Uncertainty

In the microsimulation model, due to stochastic uncertainty, certain random events occur only for some patients or at random times based on event probabilities [21]. Due to randomness, even patients with identical characteristics and/or treatments can experience different outcomes. If these different outcomes cannot be predicted with certainty from included patient characteristics, then no ITR is able to assign the optimal treatment to all patients, despite being familiar with all counterfactuals. Random events with impacts on outcomes in the model therefore influence the incremental value and regret of ITRs.

Case Study: Haemophilia B Microsimulation

We adapt a previously published microsimulation cost-effectiveness model of multiple treatments for haemophilia B in Germany [22] as a case study to illustrate the described methods. In this model, the gene therapy ED is compared with prophylactic administration of EHL-FIX. When the effectiveness of ED wanes, patients switch treatment to prophylactic EHL-FIX. We do not alter the underlying model for the purposes of this case study but add two additional treatment options: on-demand treatment with FIX and ED followed by on-demand treatment if effectiveness wanes (“ED into on-demand”). These two treatments are not standard practice for the treatment of severe haemophilia B in Germany and hence were not included in the original analysis. However, we included them to compare more than two different treatment options as a demonstration of the methods used in this study. A WTP of EUR 50,000 per quality-adjusted life year (QALY) was assumed in this study.

The microsimulation contains three relevant and observable patient characteristics that impact the outcomes of treatments: the ABR, baseline age, and sex (which influences weight and thus FIX dosage). An ITR can therefore assign treatments based on these three patient characteristics.

In the haemophilia B case study [22], 2000 simulations are run with random draws of the probabilistic parameters. One hundred patients are generated per simulation, leading to 200,000 patients in total. The same 200,000 patients are used to compare each treatment strategy. The initial ITR is designed to optimize individualized treatment across all 200,000 patients and provides the structure that is used to subsequently assess the uncertainty of the ITR parameter values. We use grid searches [32] to determine which combination of ITR parameter values is optimal within the defined structure for each simulation, producing 2000 combinations of locally optimal ITR parameter values. We describe the 95% credible range of the individual ITR parameter values and depict the density of the ITR parameter value combinations graphically to understand the impact of parameter uncertainty on the ITRs.

In the original case study microsimulation model [22], there are two potentially relevant sources of stochastic uncertainty. First, for each patient treated with ED, there is a small probability of initial treatment failure. Second, in each cycle, there is a probability of death for every patient, based on ABR, age, and sex. However, in the base case of the case study, death is not modelled stochastically as a binary, random event in each cycle, as it was expected that stochastic modelling of death would increase the variability in patient outcomes, requiring a larger simulated population, without having a relevant impact on mean patient outcomes. Instead, survival status in each cycle is implemented as a survival probability based on the probabilities of death in all previous cycles. To show how stochastic uncertainty influences the ability of an ITR to assign optimal treatment to patients, we present two additional scenarios. In the first scenario, we implement death as a binary, random event to fully capture the stochastic uncertainty around the event of death. In the second scenario, we remove the randomly modelled treatment failure with ED from the microsimulation model, decreasing stochastic uncertainty. We compare the incremental value and regret of ITRs emerging from the model’s base case and the scenarios to show how increasing or decreasing the stochastic uncertainty covered by the microsimulation model influences the ability of the ITRs to optimally assign treatments.

Results

Comparison of Empirical Models

All methods for deriving ITRs yield an expected NMB that is much closer to the optimal individualized treatment than uniform treatment strategies are (Table 1). The model produces a negative expected NMB for all treatment strategies, and the most cost-effective strategy exhibits the least negative expected NMB. The first four rows show that under uniform treatment, when all patients receive the same treatment, on-demand treatment is the most cost-effective treatment strategy, with an expected NMB of EUR − 1,657,944. However, making treatment decisions at the population level is not cost-effective when compared to individualized treatment. The highest expected NMB is obtained by assigning the optimal treatment to each patient individually, leading to an expected NMB of EUR − 1,392,709. The incremental value of optimal individualized treatment, EUR 265,235, is equivalent to the EVIC. The ITRs based on the three empirical models all produce very similar expected NMBs ranging from EUR − 1,450,973 (LASSO regression) to EUR − 1,447,264 (policy tree).

Table 1.

Comparison of the discounted mean outcomes depending on the treatment strategies

Treatment strategy	QALYs	Costs (EUR)	NMB (EUR)	Regret (EUR)	Inc. value (EUR)
Uniform treatment
On-demand	11.62	2,238,940.48	− 1,657,944.49	265,235.39	–
ED into on-demand	14.26	2,499,796.58	− 1,786,915.54	394,206.44	–
Prophylaxis	16.51	6,427,659.72	− 5,601,952.15	4,209,243.05	–
ED into prophylaxis	17.13	5,308,144.04	− 4,451,697.16	3,058,988.06	–
Optimal individualized treatment	12.97	2,034,709.87	− 1,392,709.10	0.00	265,235.39
Individualized treatment rules
LASSO regression	12.83	2,092,625.56	− 1,450,973.32	58,264.22	206,971.17
Classification tree	12.83	2,090,196.74	− 1,448,847.51	56,138.41	209,096.98
Policy tree	12.70	2,082,204.32	− 1,447,263.85	54,554.75	210,680.64

Rows 1–4 show the mean outcomes per patient when all patients in the population receive the same treatment. Rows 5–8 show the mean outcomes when patients are assigned treatments at the individual level

ED etranacogene dezaparvovec, Inc. incremental, LASSO least absolute shrinkage and selection operator, NMB net monetary benefit, QALY quality-adjusted life year

Table 2 presents a comparison of the three empirical models based on the three criteria we established. All models achieve approximately the same incremental value compared to uniform treatment, ranging from EUR 206,971 (LASSO regression) to EUR 210,681 (policy tree), with a higher value indicating a better individualization of care. These values are similar, and we consider all three models roughly equivalent in terms of incremental value.

Table 2.

Comparison of three methods for assigning treatments to individual patients on the basis of defined criteria

Criterion	LASSO regression	Classification tree	Policy tree
Criterion #1: incremental value	EUR 206,971	EUR 209,097	EUR 210,681
Criterion #2: computational speed	26 s	3 s	292 s
Criterion #3: interpretability	Complex	Simple	Simple

LASSO least absolute shrinkage and selection operator

The relative differences in computational speed are considerable. The classification tree requires approximately 3 s, the LASSO regression requires 26 s, and the policy tree requires 292 s. These relative differences can translate into large absolute differences in the presence of more features, complex decision trees, or larger data.

In terms of the ability to generate decision rules that determine treatment assignment, we find regression approaches to be unsuitable. Regressions can be used to make individual predictions by multiplying patient characteristics with the estimated coefficients and thus determine the optimal treatment for any combination of patient characteristics. In theory, it would be possible to compute on the basis of the estimated coefficients at what threshold values the optimal treatment changes. These computations could be used to define the decision rules that determine which patient subgroups receive which treatments. However, when there are many coefficients, this becomes increasingly complex. Classification and policy trees, by design, split populations into subsamples, and represent decision rules as decision trees, which are easily interpretable.

In this comparison, we find that classification trees show comparable predictive performance, the shortest runtime, and are easy to interpret. Therefore, we only apply classification trees for the remaining illustration of the case study.

Individualized Treatment Rules via Classification Tree

Figure 1 shows the classification tree that was generated via recursive partitioning. The iterative logic can be traced by following the branches. If only one treatment were assigned, then on-demand treatment would be cost-effective on average. However, this approach would only be the cost-effective treatment for 123,320 of 200,000 patients. As a first step, patients are split into two subsamples: patients with an ABR ≥ 30 receive ED followed by on-demand treatment, and patients with an ABR < 30 receive on-demand treatment. In the second step, patients who were assigned ED followed by on-demand treatment in the first step are again split into two subsamples: patients with an age < 51 years receive ED followed by on-demand treatment, and patients with an age ≥ 51 years receive on-demand treatment. Both of these steps are intuitively plausible. The expensive gene therapy ED is cost-effective for patients with a higher ABR and thus greater disease severity. However, among patients with high disease severity, high upfront costs are only worthwhile for patients below a certain age, who thus have a greater life expectancy. The recursive partitioning algorithm generating the classification tree did not select sex as a characteristic, as sex was less important than ABR or age for optimal treatment assignment. This tree can also be described in terms of three IF statements:

1. If ABR ≥ 30 and age < 51, patient is treated with ED into on-demand.

2. If ABR ≥ 30 and age ≥ 51, patient is treated on-demand.

3. If ABR < 30, patient is treated on-demand.

Fig. 1.

Individualized treatment rules generated as a classification tree via recursive partitioning. ABR annualized bleeding rate, ED etranacogene dezaparvovec

Parameter Uncertainty

The impact of the parameter uncertainty on the uncertainty of the threshold values is considerable. Across the 2000 simulations, we observe a 95% credible range of 21–46 for the optimal ABR threshold value and a range of 42–56 for the optimal age threshold value, holding the decision tree in Fig. 1 constant. The density graph of optimal threshold values shows that the greatest density of optimal threshold values is around an ABR of 30 and a patient age of 51 (Fig. 2). However, these two values are not independent: we find a slight negative correlation between the optimal ABR and the age threshold, indicating that the thresholds are not independent of each other. This means that changing one threshold value has an impact on the optimal value of the other threshold.

Fig. 2.

A density graph showing the distribution of optimal threshold value combinations across the 2000 simulations. The dark red colour indicates a high density of observations, whereas a bright yellow colour represents a low density. The outliers not contained within these densities are depicted by black dots. ABR annualized bleeding rate

Stochastic Uncertainty

The random event scenarios are shown in Table 3. In the microsimulation model’s base case, the incremental value of the classification tree ITR is EUR 209,097.

Table 3.

Comparison between the base case and two scenarios in terms of the net monetary benefit (EUR)

	Base case	Scenario 1: random death	Scenario 2: no treatment failure
Optimal individualized treatment strategy (a)	− 1,392,709	− 1,222,067	− 1,365,318
Optimal uniform treatment strategy (b)	− 1,657,944	− 1,643,699	− 1,643,699
ITR via classification tree (c)	− 1,448,848	− 1,462,515	− 1,387,446
Regret (a–c)	56,138	240,448	22,128
Incremental value (b–c)	209,097	181,184	256,253

ITR individualized treatment rule

Scenario 1: If death is modelled randomly, the incremental value decreases to EUR 181,184 and regret increases to EUR 240,448. With optimal individualized treatment assignment, patients with random early deaths are not assigned ED, as it carries high upfront costs and is only cost-effective when patients gain sufficient life expectancy from the treatment. With knowledge of all counterfactuals, we could, in theory, avoid assigning ED to those patients who die early, but these early deaths cannot be predicted via the observable patient characteristics. Therefore, an ITR that assigns treatments on the basis of patient characteristics is less successful in individualizing treatment and worse at minimizing regret. However, the expected NMB of the classification tree ITR itself decreases by only EUR 13,667, and it remains cost-effective versus uniform treatment assignments.

Scenario 2: If treatment failure is removed from the model, i.e. there is no stochastic uncertainty in the model, the incremental value increases to EUR 256,253. The regret is reduced to EUR 22,128, as the ITR is able to almost perfectly assign individualized treatments. However, as treatment failure is a realistic modelling assumption and carries a random element with it, this implies an overestimation of the incremental value and an underestimation of the regret of this ITR.

Discussion

This study examines the use of ITRs in cost-effectiveness models, and three empirical approaches are compared using haemophilia B microsimulation as a case study [22]. We find classification trees to be a suitable method for three reasons. First, it produces ITRs comparable to those of the other tested empirical models in terms of incremental value and regret. Second, it has low computational demands. Third, its decision tree structure allows ITRs to be expressed in an easily interpretable manner [29]. We believe that the viability of classification trees is generalizable to other microsimulation models: its computational efficiency holds even for large datasets, and the interpretation of a classification tree always follows the same principle, no matter what microsimulation model it is applied to. The incremental value of the derived ITR depends on the complexity of the classification tree, but this can be controlled by the modeller: a deeper classification tree allows for better ITRs at the cost of greater complexity. These findings can and should be confirmed via other microsimulation models.

We consider the ability to express an ITR in an easily interpretable manner important for the translation of the model results into a format that can be considered for real-world decision-making. Our case study demonstrates this facet: on the basis of a decision tree consisting of only two threshold values and three IF statements, treatment can be assigned to all simulated patients, leading to a mean NMB higher than that of any uniform treatment strategy. While a regression model may allow for the calculation of an expected NMB based on patient characteristics, it would be difficult to formulate simple cost-effectiveness treatment recommendations on this basis. We further consider it advantageous for the empirical model to derive an optimal ITR structure in a data-driven manner, as the manual specification of an ITR structure including many patient characteristics may be prone to bias [33].

When real patient data are used, classification trees can be unsuitable because they adhere too closely to the data, which can lead to overfitting, particularly with deeper trees, and may exhibit biased variable selection [34]. However, with simulated patient data, this is not a drawback. The correlations between patient characteristics and outcomes stem from simplifying yet realistic modelling assumptions about disease and treatment effects. It is therefore desirable for any ITR to fit itself to the data closely and capture the impact of these modelling assumptions as completely as possible. When microsimulation model results lack real-world generalizability, the ITR will also lack real-world generalizability.

In our case study, the classification tree-based ITR assigns either on-demand or ED followed by on-demand treatment if effectiveness wanes, based on the ABR and age threshold values. These ITR parameter values are subject to considerable parameter uncertainty. We also find it important to consider the role of stochastic uncertainty in model events, as highlighted by two scenarios that consider random death events and random treatment failures. These scenarios show that if stochastic uncertainty is not sufficiently considered in the underlying microsimulation model, the incremental value of ITRs may be overestimated. Certain modelling simplifications that reduce stochastic uncertainty, which may be acceptable when only considering mean outcomes [35], become an issue when a microsimulation is used to derive ITRs. With such a purpose, making microsimulation models representational of individual patient pathways and avoiding abstract expected values of outcomes becomes even more crucial. While our case study is limited to three patient characteristics, our approaches allow us to derive ITRs with more complex patient data when available.

The approach described in this paper has various limitations. While we compare ITRs based on three empirical models, there may be other suitable approaches. However, such approaches can be compared based on our suggested criteria, and a gold standard may eventually be established. As determining the uncertainty of the ITRs is based on probabilistic analysis, we inherit the limitations of this approach: only parameter uncertainty is captured by this [21]. For the parameter uncertainty, we assume that the underlying effect estimates are unbiased, which may not be the case. In the case study, as the input parameters are derived from various secondary sources, we do not capture the correlations between these parameters [36].

Stochastic uncertainty in our microsimulation case study was represented by random events that varied in their occurrence or timing between patients. Alternatively, it can be approached in a different manner: Glynn et al. [12] simulated each individual patient 10,000 times for 8746 real-world patient profiles to optimize the ITR, leading to 87.46 million simulations, and then took the average of each patient’s simulations. The equivalent simulated population of 200,000 in this case study, multiplied by the four evaluated treatments to generate all counterfactuals, would require 8 billion simulations. To reduce the simulated population size, probabilistic uncertainty can also be considered at the individual level, as Glynn et al. did. Considering both stochastic and probabilistic uncertainty at the individual level in microsimulations could be an interesting avenue for further research.

Our ITRs focus purely on expected NMB, which has some drawbacks. Calculating the expected NMB requires an explicit WTP threshold, which does not exist in many countries. However, the described methods also allow for the optimization of other outcome parameters, e.g. QALYs or costs, when they are relevant for decision-making. In future studies, the methods could be extended, e.g. to consider aspects of equity. For example, an ITR could be designed to maximize NMB within constraints to avoid sacrificing too many QALYs at the individual patient level. Alternatively, the ITR could consider the Gini index [37] and keep inequality below a target value. ITRs based on such extensions would allow for more balanced optimization approaches and might be more suitable for informing real-world decision-making than the examples given in this article can. We believe that there are many additional avenues to explore, ranging from different empirical models to types of analysis, that would allow ITRs to become a common method for interpreting the results of microsimulation models.

Author contributions

All authors conceived and designed the study. NM implemented the described methods in R and drafted the manuscript. MP made important contributions to the underlying methods. ACQG, MP, and MS performed detailed reviews of the initial manuscript drafts and approved the final version.

Funding

Open access funding provided by University of Basel. No funding was received for conducting this study.

Declarations

Conflict of interest

Mark Pletscher is an employee of Novartis Pharma AG and holds stock of Novartis Pharma AG. The other authors report no conflicts of interest.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and material

All data generated or analysed during this study are included in this published article or are available from the corresponding author (NM) on reasonable request for non-commercial purposes.

Code availability

The microsimulation model and the methods described in this article are implemented in R, and the full code is available at https://github.com/NiklausMeier/HB_CE_microsimulation.