Table 1.

Considerations for Approximate Bayesian Computation Algorithms

	Description	Recommendation	Notation
Step 1: Establish Calibration Problem
Define Calibration Problem and Identify Calibration Target	Identify calibration targets, model predicted outcome(s) that should be similar to outcome(s) from an external source. Generally, these consists of summary measures such as means or proportions.	Target outcomes should be meaningful for decision makers or model end users.	D: target data S(D): summary of target data $\widehat{D}$: model prediction of target data
Identify Model Parameters to Calibrate	Identify model parameters to calibrate.	Parameters should be selected based on whether or not they affect the calibration target outcome, and whether or not a good estimate for the parameter in question exists.	θ: Set of parameters being calibrated.
Determine Prior Distribution of Parameters to be Calibrated	Describes what is known about the distribution of the parameters to be calibrated. The calibration procedure samples from this distribution.	If possible, incorporate known information about the distribution of a target parameter. Otherwise, a uniform prior or other uninformative prior can be chosen. Posterior predictive checks can eliminate unlikely parameter values from consideration.	π(θ): prior distribution for model parameters to be calibrated
Determine Number of Simulations	Determine how many patients are simulated and how many times at each step of the calibration algorithm. The simulation process can be computationally taxing, and there is a trade-off between computation time and variance of the distribution of predicted outcome P(D|θ).	Use a sufficiently large number of simulations to ensure reasonable computation time and small enough variance. Or, conduct simulations with varying levels of n and k to determine optimal combination.	n: number of patients k: number of times the n patients are simulated
Determine Sample Size of the Posterior Distribution	The size of the sample drawn from the posterior distribution P(θ|D) in the calibration procedure.	Limited guidance in the literature in the application of approximate Bayesian calibration, but in practice the sample size should be sufficiently large that the researcher can obtain summary measures from the posterior distribution with minimal error. Size of the sample may also be limited by the computational budget of the researcher.	N: size of the sample drawn from the posterior distribution P(θ|D): posterior distribution of new model parameter conditioned on the calibration target data.
Distance Measure	Calculates how far the model predicted outcomes S($S(\widehat{D})$ are from the target outcomes S(D) given a sampled parameter set ($\widehat{\theta }$).	A measure appropriate for the data type and how the predictions are expected to fit the observed data. There many possible distance measures, including Euclidean distance, (root) mean squared error, Mahalanobis distance.	||·||
Step 2: Implement ABC Approach
ABC Rejection Sampler Considerations
Threshold	Determines if a parameter value is likely to be sampled from the posterior distribution of target outcome P(θ|D)	When calibration is only used for prediction than ε ≥ 0 should be as close to 0 as feasible. When calibration is used for prediction and uncertainty analysis, then an ε ≥ 0 should be chosen that replicates the uncertainty observed in the target outcome.	ε
MCMC Rejection Sampler Considerations
Kernel function	Is a weighted function of the difference between model predicted $\widehat{D}$ and target data D. The function gives greater importance to predicted $\widehat{D}$ values that are closer to D. Kernel smoothing functions are often applied in statistics.	A Gaussian kernel has been recommended over kernel functions with compact support, since they have been shown in an empirical study to reduce the convergence time of the algorithm.28	K
Scale parameter	Scales the curve of the kernel function and helps determine the acceptance probability. Larger scale parameter values cause a higher proportion of tested values being accepted, while small scale parameter values cause a smaller proportion of tested values to be accepted.	Empirically test different values until a reasonable proportion of tested parameter values are accepted.	h
Proposal density	Determines the next proposed parameter value in the MCMC chain conditional on the current sampled parameter value.	Random normal variable (mean = 0 and SD = scaled to the parameters magnitude) added to the previous value.	q(θ′|θ)
Step 3: Final Reporting Considerations
Compare Posterior Predictive Distribution With Calibration Target	Visually compare the posterior predictive distribution with the calibration targets. Targets lying outside of the distribution indicate poor fit.	Use boxplot to visualize the posterior predictive distribution and include calibration targets in the plot.
Other Considerations	Comprehensively report all elements of the calibration procedure.	Graphically present approximate posterior and posterior predictive distributions. Include choices of distance function, number of simulations, sample size of posterior distribution, and other necessary algorithm inputs.