Meta-Modeling as a Variance-Reduction Technique for Stochastic Model–Based Cost-Effectiveness Analyses

Abstract

Purpose.

When using stochastic models for cost-effectiveness analysis (CEA), run-to-run outcome variability arising from model stochasticity can sometimes exceed the change in outcomes resulting from an intervention, especially when individual-level efficacy is small, leading to counterintuitive results. This issue is compounded for probabilistic sensitivity analyses (PSAs), in which stochastic noise can obscure the influence of parameter uncertainty. This study evaluates meta-modeling as a variance-reduction technique to mitigate stochastic noise while preserving parameter uncertainty in PSAs.

Methods.

We applied meta-modeling to 2 simulation models: 1) a 4-state Sick-Sicker model and 2) an agent-based HIV transmission model among men who have sex with men (MSM). We conducted a PSA and applied 3 meta-modeling techniques—linear regression, generalized additive models, and artificial neural networks—to reduce stochastic noise. Model performance was assessed using R² and root mean squared error (RMSE) values on a validation dataset. We compared PSA results by examining scatter plots of incremental costs and quality-adjusted life-years (QALYs), cost-effectiveness acceptability curves (CEACs), and the occurrence of unintuitive results, such as interventions appearing to reduce QALYs due to stochastic noise.

Results.

In the Sick-Sicker model, stochastic noise increased variance in incremental costs and QALYs. Applying meta-modeling techniques substantially reduced this variance and nearly eliminated unintuitive results, with R² and RMSE values indicating good model fit. In the HIV agent-based model, all 3 meta-models effectively reduced outcome variability while retaining parameter uncertainty, yielding more informative CEACs with higher probabilities of being cost-effective for the optimal strategy.

Conclusions.

Meta-modeling effectively reduces stochastic noise in simulation models while maintaining parameter uncertainty in PSA, enhancing the reliability of CEA results without requiring an impractical number of simulations.

Mathematical models have an important role in the cost-effectiveness analysis (CEA) of infectious disease interventions. Among model types, stochastic agent–based models have the advantage of accommodating complex dynamic processes, including realistic representation of transmission-relevant exposures between individuals and greater individual heterogeneity.^1-7 However, stochastic models may require a large number of simulations or large simulated populations to achieve stable estimates of the average cost and quality-adjusted life-year (QALYs) outcomes needed for CEAs. This is especially true when evaluating interventions with low individual-level efficacy (e.g., a vaccine that reduces infection risk by 5% or 10%). Averaging over an insufficient number of simulations can lead to counterintuitive results, such as a harm-free intervention worsening population health simply due to random variation. The most straightforward solution would be to increase the number of model simulations, but this may be computationally burdensome for complex models with long run times and for models with a large number of potential intervention scenarios. Probabilistic sensitivity analysis (PSA) to quantify impacts of parameter uncertainty in a CEA further exacerbates this computational burden: the desired number of sampled PSA parameter sets combined with the ideal number of simulations per PSA sample may require millions of simulations.⁸ In these situations, variance reduction techniques (VRTs)^9,10 may prove useful to reduce the stochastic noise inherent in the model while still reflecting parameter uncertainty variation.

The most well-known VRT is the use of common random numbers, in which an identical stream of random numbers is used across simulations of different interventions and/or PSA samples to reduce variability due to stochastic noise.¹¹ However, in large-scale simulations of interacting agents, implementing common random numbers can be challenging due to the presence of many different and interacting stochastic processes. In these cases, meta-modeling may be a more viable approach to reduce unwanted stochastic noise. A meta-model is a mathematical representation that approximates the input–output relationship of a more complex model.¹² There are different meta-modeling techniques, including linear regression, generalized additive models (GAMs), and artificial neural networks (ANNs).¹² The simplest meta-model is a linear function, which is often sufficient for predicting the typical CEA outcomes of costs and QALYs.¹³ By fitting a linear meta-model to PSA data, the impact of uncertain parameters on model outputs is captured in the variance of the deterministic components, while the impacts of stochastic noise can be discarded as the variance in the error term. While meta-models have been proposed to reduce computational burden in a variety of applications, including conducting PSA,¹³ model calibration,^14,15 and value-of-information analysis,^16-19 there has been little discussion regarding the use of meta-modeling as a VRT in stochastic models.

In this article, we evaluated the effectiveness of meta-modeling as a VRT in conducting a PSA with a stochastic model. We first fit 3 different meta-models (linear regression, GAM, and ANN) to a PSA dataset from a 4-state, stochastic microsimulation model. We evaluated the variance reduction performance of the different meta-models by comparing meta-model–based results to PSA results from a deterministic implementation of the same 4-state model, which served as a “noise-free” comparator. We then fit the different meta-model structures to PSA results from a more complex agent-based sexual network model of HIV transmission, prevention, and treatment used in a previous CEA¹ to demonstrate the impact of a meta-modeling VRT in a more realistic application.

Methods

Formalizing Meta-Modeling as a VRT

In a decision-analytic model, uncertainty arises from 2 main sources: stochastic (first-order) uncertainty and parameter (second-order) uncertainty. Parameter uncertainty is typically characterized through PSA. However, in complex stochastic models, it can be difficult to disentangle parameter uncertainty from stochastic uncertainty, which can obscure the relationship between parameter uncertainty and model outcomes.

When fitting a meta-model, the outcome being modeled ($Y$) is split into 2 distinct parts: a deterministic component ($f(X)$) that reflects the predicted values based on the input parameters ($X$) and a stochastic component representing the residual error ($ϵ$), which captures statistical noise. The relationship can be expressed as

$$Y=f(X)+ϵ.$$

The rationale behind using meta-modeling as a VRT is that the deterministic component captures the effects of parameter uncertainty on model outcomes, while the stochastic component isolates the stochastic uncertainty. For example, when fitting a linear meta-model to PSA data, the variance explained by the deterministic component reflects the influence of uncertain parameters on model outputs, whereas the variance of the error term accounts for stochastic noise, which can effectively be discarded. However, as meta-models are approximations of the original model, if they are misspecified, the deterministic component may fail to accurately capture parameter uncertainty, leading to biases in PSAs. This underscores the importance of meta-model validation.

Evaluating Meta-Modeling as VRT in a Simple 4-State Model

We first applied meta-modeling VRT to a stochastic microsimulation implementation of the simple, 4-state Sick-Sicker model that is often used as a toy example for illustrating methods in cost-effectiveness application.^20,21 The 4 states of the Sick-Sicker model are Healthy (H), Sick (S1), Sicker (S2), and Dead (D). Each time step, individuals face a risk of progressing from Healthy to Sick, from which they can recover back to Healthy or progress to Sicker, where there is no possibility of recovery. Death is possible from all states, reflecting a baseline background mortality (from Healthy) and an increased mortality from disease-specific causes (from Sick and Sicker). The model’s stochasticity arises from the probabilistic transitions between health states for each individual at every time step. In our example, the Sick-Sicker model evaluates a hypothetical treatment of modest benefit (with no harm) to those in the Sick state by reducing the rate of progression from Sick to Sicker. This example was designed to illustrate a situation in which the intervention’s benefit may be overwhelmed by model stochasticity, resulting in some simulations that may estimate negative incremental QALYs with the intervention relative to the status quo simply due to random chance.

We conducted a PSA by sampling 1,000 parameter sets from distributions reflecting parameter uncertainty (Table 1) and running the Sick-Sicker model as a stochastic microsimulation with 80 individuals (to ensure a significant amount of stochastic noise) and calculating the average total lifetime cost and QALYs for the status quo and intervention strategies. For each PSA parameter set, we also ran the Sick-Sicker model as a deterministic cohort model to generate PSA results without stochastic noise.

Table 1.

Parameters and Distributions for Probabilistic Sensitivity Analysis for 2 Example Models

Model Parameter Description	Distribution Parameters	Sample Mean (95% Quantiles)	Source
Sick-Sicker state-transition model
State transitions (annual)
Probability of disease onset (H to S1)	Beta (alpha = 30, beta = 170)	0.15 (0.11, 0.20)	References 20 and 21
Probability of recovery (S1 to H)	Beta (alpha = 60, beta = 60)	0.50 (0.41, 0.59)	—
Probability of disease progression (S1 to S2)	Beta (alpha = 5, beta = 45)	0.11 (0.06, 0.17)	—
Hazard ratio of death in S1 v. H	Lognormal (x = log(3), s = 0.01)	3.00 (2.94, 3.06)	—
Hazard ratio of death in S2 v. H	Lognormal (x = log(10), s = 0.02)	10.00 (9.63, 10.41)	—
Annual costs
Healthy individuals	Gamma (shape = 100, scale = 20)	2,003 (1,634, 2,417)	—
Sick individuals in S1	Gamma (shape = 177.8, scale = 22.5)	4,028 (3,474, 4,656)	—
Sick individuals in S2 Utilities	Gamma (shape = 225, scale = 66.7)	24,967 (21,934, 28,166)	—
Utilities			—
Healthy individuals	Beta (alpha = 200, beta = 3)	0.98 (0.96, 0.99)	—
Sick individuals in S1	Beta (alpha = 37.5, beta = 12.5)	0.75 (0.61, 0.87)	—
Sick individuals in S2	Beta (alpha = 230, beta = 230)	0.50 (0.45, 0.54)	—
Intervention parameters
Hazard ratio of disease progression with intervention	Lognormal (log[0.90], s = 0.005)	0.899 (0.892, 0.908)	—
Annual cost of intervention	Gamma (shape = 100, scale = 50)	5,003 (4,079, 6,006)	—
HIV agent-based network model
Status quo model parameters Probability of starting PrEP	Beta (alpha = 16.9, beta = 6.8)	0.715 (0.527, 0.874)	Reference 1
Probability of discontinuing PrEP	Beta (alpha = 179.5, beta = 85.7)	0.0215 (0.018, 0.025)	Reference 22
PrEP adherence levels (proportion with low, medium, and high adherence)	Dirichlet (low = 91.74, medium = 130.66, high = 333.6)	Low = 0.165, medium = 0.235, high = 0.60 (0.56, 0.64)	References 1, 23, and 24
Adherence intervention
Proportion with high PrEP adherence after intervention	Beta (alpha = 13.4, beta = 6.3)	0.68 (0.47, 0.85)	Reference 25
Per-person intervention cost (one time)	Gamma (shape = 62, scale = 15.7)	975.68 (754.03, 1231.60)	Reference 1

H, Healthy; S1, Sick; S2, Sicker; PrEP, preexposure prophylaxis.

We fit 3 different meta-model types to the PSA results from the stochastic model: linear regression, GAM, and ANN. These models were commonly used in health economics research and selected to represent a spectrum of complexity¹²: from linear regression, the simplest model, which was easy to fit and interpret; to ANN, which had a highly flexible structure capable of capturing complex nonlinear relationships and interactions between parameters. The linear regression and GAM meta-models were fit separately to each outcome (costs, QALYs) for each strategy (status quo, intervention), for a total of 4 meta-models of each type. In the linear regression models, each outcome was regressed on all parameters included in the PSA (Table 1) without incorporating quadratic or interaction terms. For GAM, we applied smooth functions using thin plate regression splines for each term.²⁶ The ANN meta-model was fit to both outcomes for both strategies together to capture correlation in outcomes, resulting in a single ANN meta-model. Full specifications of each meta-model are provided in the appendix.

Following meta-modeling best practices,¹² we divided the PSA into an 80% training set and a 20% validation set. To assess the fit of each meta-model, we calculated the R² and root mean squared error (RMSE) on the validation set. We also assessed whether each meta-model was able to recover the relationship between parameter uncertainty and model outcomes while discarding stochastic noise by comparing the meta-model’s deterministic and stochastic variance components to the variance decomposition of the original Sick-Sicker model. For the original model, we calculated the percentage of variance that was deterministic as the variance in the deterministic model implementation (i.e., accounting only for parameter uncertainty) divided by the total variance of the stochastic model implementation, which included both parameter uncertainty and stochastic noise. For each meta-model, the percentage deterministic variance was calculated as the variance in the deterministic component (predictions) divided by the total variance, while the percentage stochastic variance was calculated as the variance in the error term divided by the total variance. To demonstrate the impact of meta-modeling VRT on CEA results, we compared scatterplots of PSA results as well as cost-effectiveness acceptability curves (CEACs) derived from the stochastic and deterministic implementations of the Sick-Sicker model alongside PSA results predicted by the meta-models. Again, the extent to which meta-model results match those of the deterministic model is an indication of their efficacy in removing stochastic noise while retaining parameter uncertainty.

Applying Meta-Modeling VRT to a Stochastic Agent–Based HIV Transmission Model

We also applied the meta-modeling VRT to a more complex stochastic model example, for which a deterministic implementation was not possible. Previously, we used an agent-based, sexual network HIV transmission model, built on the EpiModel software platform,²⁷ to evaluate the cost-effectiveness of interventions to improve the use of preexposure prophylaxis (PrEP) for HIV prevention in men who have sex with men (MSM).¹ In this example, we expanded on the CEA by conducting a PSA on the uncertainty in the cost and effectiveness parameters of an intervention developed to improve adherence to daily oral PrEP medication among current users (Table 1). Because the adherence intervention applied only to those already on PrEP and only modestly increased adherence among its participants (increased the proportion of high adherence population from 60% to 68% in average), this was again a situation in which the model would find a reduction in QALYs with the intervention relative to the status quo simply due to random chance. A total of 1,980 sets of parameter values were sampled from distributions estimated from the published literature (see appendix). For each PSA sample, total discounted health care costs and QALYs were averaged over 256 simulations of each intervention (status quo and adherence intervention), requiring 2 × 256 × 1,980 = 1,013,760 model runs; for each model run, we simulated a population of 12,000 and extrapolated results to reflect the total population of approximately 103,000 MSM in Atlanta; we assumed this was a reasonable upper limit for a typical high-performance computing system.

We applied the same 3 meta-model structures (linear regression, GAM, ANN) to the agent-based model PSA results. Like in the Sick-Sicker model, the linear regression and GAM meta-models were fit separately to each outcome (costs, QALYs) for each strategy (status quo, intervention), for a total of 4 meta-models of each type, while a single ANN was fit to both outcomes for both strategies together. For the linear regression meta-model of the status quo scenario, we included main effects and interactions for all parameters varied in the PSA. In the adherence intervention scenario, we incorporated additional terms related to intervention efficacy. For GAM, we further applied smooth functions to all terms. Full meta-model specifications are provided in the appendix.

Meta-models were again fit using 80% of PSA samples, and performance was assessed in terms of R² and RMSE on the remaining 20% of PSA samples that constituted the validation set. In this more complex model, there was no deterministic version of the model that can be used to decompose variance into deterministic and stochastic components; instead, we present the reduction in variance achieved by each meta-model and visual changes in the PSA scatterplot and CEAC.

Results

Sick-Sicker State Transition Model Example

In the simple 4-state model with stochasticity, the variance of incremental costs and QALYs increased significantly compared with the deterministic model that included only variability from PSA parameter sampling (Figure 1). Although the benefit of the intervention was small for certain PSA samples, the deterministic model always indicated a QALY gain from the intervention compared with the status quo; this was not the case when using the stochastic model, which, due to random chance, estimated QALY reductions under the intervention compared with the status quo in 37% of PSA parameter sets. This was also visible in the resulting CEAC constructed from stochastic model results, which showed the probability that the intervention is cost-effective not exceeding 63% no matter how high the willingness to pay (WTP), with the status quo always considered costeffective for the 37% of samples for which the intervention was estimated to reduce QALYs.

Figure 1.

Cost-effectiveness scatter plots with model outputs from a deterministic and stochastic simple 4-state model and the predicted values from 3 meta-models. Incremental costs and QALYs are shown for the intervention versus the status quo across PSA parameter sets. Percentages in parentheses indicate the proportion of PSA parameter sets yielding unintuitive results (i.e., lower total QALYs for the intervention than for status quo). ANN, artificial neural network; GAM, generalized additive model; PSA, probabilistic sensitivity analysis; QALY, quality-adjusted life-year.

The 3 meta-models demonstrated comparable performance in fitting the average costs and QALYs across PSA samples, with similar R² values and RMSEs in the validation set (Table 2). The variance of predicted values from the 3 meta-models was greatly reduced compared with the stochastic model, and the occurrence of unintuitive results (incremental QALYs < 0) was nearly eliminated in PSA samples (Figure 1). In addition, the percentages of total variance captured in the deterministic component for each of the 3 meta-models was consistent with that of the original model (Table 3), demonstrating the effectiveness of meta-modeling techniques in reducing stochastic variance.

Table 2.

R² Values and Root Mean Squared Errors (RMSEs) of the Validation Set of the 3 Meta-Models for the Sick-Sicker State Transition Model

Metamodel	Intervention	R2 Values for Validation Set	RMSEs for Validation Set
Linear	Cost, status quo	0.72	14,956
Regression	Cost, intervention	0.67	17,799
	QALYs, status quo	0.73	0.65
	QALYs, intervention	0.67	0.70
GAM	Cost, status quo	0.71	15,124
	Cost, intervention	0.67	17,856
	QALYs, status quo	0.73	0.65
	QALYs, intervention	0.70	0.67
ANN	Cost, status quo	0.67	17,048
	Cost, intervention	0.68	17,643
	QALYs, status quo	0.69	0.71
	QALYs, intervention	0.67	0.69

ANN, artificial neural network; GAM, generalized additive model; QALY, quality-adjusted life-year.

Table 3.

Variance Decomposition of Sick-Sicker State Transition Model and 3 Meta-Models

Model	Outcome	Variance of Deterministic Component,a % of Total Variance	Variance of Stochastic Component, % of Total Variance
Sick-Sicker	Cost, status quo	71.9	28.8
State transition	Cost, intervention	70.3	29.2
Model	QALY, status quo	77.7	25.7
	QALY, intervention	75.3	26.3
Meta-Model	Outcome	Variance of Predicted Values,b % of Total Variance	Residual Variance,b % of Total Variance
Linear	Cost, status quo	69.2	29.8
Regression	Cost, intervention	71.2	30.2
	QALY, status quo	74.3	26.1
	QALY, intervention	74.6	27.3
GAM	Cost, status quo	70.4	28.3
	Cost, intervention	72.4	28.3
	QALY, status quo	75.1	24.3
	QALY, intervention	75.3	25.4
ANN	Cost, status quo	66.3	32.1
	Cost, intervention	69.4	30.4
	QALY, status quo	74.3	27.0
	QALY, intervention	75.4	27.7

ANN, artificial neural network; GAM, generalized additive model; QALY, quality-adjusted life-year.

^aThe variance of the deterministic component in the Sick-Sicker state transition model was calculated using the deterministic state transition model implementation.

^bThe sum of variances may not equal 100% due to small covariances between the deterministic and stochastic components.

The construction of the CEACs from the meta-model predictions was more informative and showed a clearer cost-effectiveness relationship (Figure 2), with the probability of the intervention being cost-effective increasing with WTP, than the CEAC of the stochastic model. The meta-modeling–derived CEACs better approximated the CEAC constructed from the deterministic model results, although with slower increases in the probability of the intervention being cost-effective with increasing WTP.

Figure 2.

Cost-effectiveness acceptability curves (CEACs) with model outputs from deterministic and stochastic simple 4-state model and the predicted values from 3 meta-models. ANN, artificial neural network; GAM, generalized additive model.

HIV Agent-Based Network Transmission Model Example

All 3 meta-models showed comparable goodness of fit in fitting the average costs and QALYs across PSA samples, with similar R² values and RMSEs in the validation set (Table 4). When meta-models were applied, outcome variances were greatly reduced, and the proportion of samples with unintuitive results dropped to nearly 0% for all 3 meta-models (Figure 3). The CEAC of the stochastic agent–based model suggested that the adherence intervention had a 50% probability of being cost-effective, regardless of WTP, reflecting the 40% of simulations in which the intervention appeared harmful due to random chance (Figure 4). When meta-models were used, the CEAC became more informative and demonstrated a clear cost-effectiveness relationship, with the probability of being cost-effective increasing with WTP.

Table 4.

R² Values and Root Mean Squared Errors (RMSEs) of the Validation Set of the 3 Meta-Models for HIV Agent-Based Network Model Fit to Total Cost and QALY Outcomes for Both the Status Quo and Adherence Intervention

Meta-Model	Intervention	R2 Values for Validation Set	RMSEs for Validation Set
Linear regression	Cost, status quo	0.98	31,858,348
	Cost, intervention	0.98	34,085,398
	QALYs, status quo	0.85	661
	QALYs, intervention	0.84	717
GAM	Cost, status quo	0.98	30,127,082
	Cost, intervention	0.98	33,385,587
	QALYs, status quo	0.85	649
	QALYs, intervention	0.84	707
ANN	Cost, status quo	0.97	38,673,171
	Cost, intervention	0.98	37,268,930
	QALYs, status quo	0.83	760
	QALYs, intervention	0.85	704

ANN, artificial neural network; GAM, generalized additive model; QALY, quality-adjusted life-year.

Figure 3.

Cost-effectiveness scatter plots comparing model outputs from the original HIV agent-based model and predicted values from 3 meta-models. Incremental costs and QALYs are shown for adherence intervention versus the status quo across PSA parameter sets. Percentages in parentheses indicate the proportion of PSA parameter sets yielding unintuitive results (i.e., lower total QALYs for the adherence intervention than for the status quo). ANN, artificial neural network; GAM, generalized additive model; PSA, probabilistic sensitivity analysis; QALY, quality-adjusted life-year.

Figure 4.

Cost-effectiveness acceptability curves (CEACs) with model outputs from the original HIV agent-based model and the predicted values from 3 meta-models. ANN, artificial neural network; GAM, generalized additive model.

Discussion

Meta-models have been used in decision-analytic models as emulators to alleviate computational burden, facilitate value-of-information analysis, and overcome challenges in model calibration for complex models.^12,15,28 However, their application as a VRT for stochastic models has been relatively limited. Our study demonstrates that meta-models can effectively reduce stochastic noise while preserving parameter uncertainty. In both examples—the Sick-Sicker state-transition model and HIV agent-based network model—the small, individual-level benefits of the intervention made simulated CEA outcomes susceptible to the stochastic noise inherent in the complex individual-level model being used to simulate outcomes. Although increasing the number of simulations per PSA parameter set may mitigate stochastic noise, this approach may require substantial computational resources that may be unavailable or cost prohibitive. Instead, fitting meta-models to the PSA dataset and reporting the CEA outcomes with predicted values from these meta-models significantly reduced the stochastic uncertainty while retaining parameter uncertainty. This finding holds across both a hypothetical simple toy model and a more complex real-world application. Our study underscored the potential of meta-models to enhance the interpretability of PSA outcomes generated with complex, stochastic models.

The meta-modeling approach involved some assumptions related to the type of meta-model used and its specification. In our study, we explored 3 different meta-models: linear regression, which assumes a linear relationship between model parameters and cost and QALY outcomes; GAM, which relaxes the linearity assumption and instead estimates a smooth relationship; and ANN, which can capture more complex nonlinear relationships. Despite varying levels of complexity, all 3 meta-models demonstrated strong predictive performance, as indicated by R² values and RMSEs from validation sets. In addition, each meta-model substantially reduced stochastic noise, suggesting that different meta-modeling approaches can be effective in variance reduction.

In the example of the Sick-Sicker state-transition model, we assessed the reliability of the meta-modeling approach for variance reduction. Specifically, we ran the model deterministically as a cohort-based state transition model and stochastically as an individual-level microsimulation. This allowed us to decompose the variance into parameter uncertainty and stochastic uncertainty. The variance of costs and effectiveness in the deterministic model was entirely due to parameter uncertainty, free from stochastic noise. When comparing this variance to that of the meta-model–predicted CEA outcomes, we found that they were closely aligned, with only minor differences. This suggests that meta-models successfully filter out stochastic noise while retaining parameter uncertainty. The slight variance discrepancy may stem from prediction errors or the structural misspecifications of the meta-models.

Validation is a critical step in meta-modeling applications. When training meta-models, it is essential to use a validation set and minimize prediction errors to prevent overfitting. Overfitting would cause the meta-model to recover both parameter and stochastic uncertainty when predicting the CEA outcomes, negating its intended role in variance reduction. In real-world applications, analysts should follow best practices outlined in Degling et al.¹²: 1) identifying a suitable type of meta-model; 2) fitting the meta-model to the PSA dataset; 3) assessing the performance of the meta-model, particularly on the validation set; 4) tuning the meta-model; and 5) reporting PSA results based on meta-model–predicted values.

There are limitations to our analysis. First, we did not compare the impact of combinations of different varying numbers of PSA samples (outer loop) or simulations (inner loop) to the meta-model VRT. The selection of inner and outer loop sample sizes will affect the variance of the original model and also the performance of the meta-model.²⁹ In both examples, the ANN slightly underperformed compared with the linear regression model and GAM, likely due to its higher data requirements (i.e., larger sample size) for effective fitting. Second, we focused only on meta-modeling without comparing it to other common VRTs, such as common random numbers. Implementing CRNs in the HIV agent-based network model presents challenges due to the numerous interacting stochastic processes. However, the combination of both techniques may yield further improvements in variance reduction. Future research should explore the combination of different VRTs and the tradeoffs between VRTs and inner and outer loop sample sizes.

In conclusion, meta-modeling proved to be an effective VRT in decision-analytic modeling. With a reduction in stochastic noise, the PSA of the low-cost, low-efficacy intervention became more informative and useful for public health decision making.

Supplementary Material

Supplementary material for this article is available online at https://doi.org/10.1177/0272989X251352210.

Highlights.

• When using complex stochastic models for cost-effectiveness analysis (CEA), stochastic noise can overwhelm intervention effects and obscure the impact of parameter uncertainty on CEA outcomes in probabilistic sensitivity analysis (PSA).

• Meta-modeling offers a solution by effectively reducing stochastic noise in complex stochastic simulation models without increasing computational burden, thereby improving the interpretability of PSA results.

Data Availability

The dataset and code required to reproduce this study can be found at https://github.com/expn123/metamodel_VRT_MDM.