Bayesian Calibration of Microsimulation Models

Abstract

Microsimulation models that describe disease processes synthesize information from multiple sources and can be used to estimate the effects of screening and treatment on cancer incidence and mortality at a population level. These models are characterized by simulation of individual event histories for an idealized population of interest. Microsimulation models are complex and invariably include parameters that are not well informed by existing data. Therefore, a key component of model development is the choice of parameter values. Microsimulation model parameter values are selected to reproduce expected or known results though the process of model calibration. Calibration may be done by perturbing model parameters one at a time or by using a search algorithm. As an alternative, we propose a Bayesian method to calibrate microsimulation models that uses Markov chain Monte Carlo. We show that this approach converges to the target distribution and use a simulation study to demonstrate its finite-sample performance. Although computationally intensive, this approach has several advantages over previously proposed methods, including the use of statistical criteria to select parameter values, simultaneous calibration of multiple parameters to multiple data sources, incorporation of information via prior distributions, description of parameter identifiability, and the ability to obtain interval estimates of model parameters. We develop a microsimulation model for colorectal cancer and use our proposed method to calibrate model parameters. The microsimulation model provides a good fit to the calibration data. We find evidence that some parameters are identified primarily through prior distributions. Our results underscore the need to incorporate multiple sources of variability (i.e., due to calibration data, unknown parameters, and estimated parameters and predicted values) when calibrating and applying microsimulation models.

1. Introduction

Microsimulation models (MSMs) provide a way to estimate population-level effects of medical interventions on health outcomes by integrating results from randomized controlled trials, observational studies, and expert opinion. MSMs have been developed for a wide range of diseases, including prostate cancer (Etzioni et al. 1999; Feuer et al. 2004), breast cancer (Berry et al. 2005), and colorectal cancer (Loeve et al. 1999). MSMs, which have the potential to influence policy making decisions, have been used to estimate the impact of overdiagnosis due to screening on breast cancer incidence (de Koning et al. 2006), to compare the effect of risk factors and screening behaviors on colorectal cancer (CRC) mortality rates (Vogelaar et al. 2006), to evaluate the cost-effectiveness of cervical cancer (van den Akker-van Marle et al. 2002) and breast cancer screening (Shen and Parmigiani 2005), and to examine mortality and reoperation rates after aortic valve replacement (Puvimanasinghe et al. 2004). The National Cancer Institute recognized the value of microsimulation models when it formed the Cancer Intervention and Surveillance Modeling Network (CISNET; cisnet.cancer.gov) (Croyle 2006).

MSMs are characterized by simulation of individual event histories for an idealized population of interest. These individual event histories catalog landmarks in the disease process, such as the development of an incident cancer or onset of myocardial infarction. Simulation of event histories requires mathematical models for key components of the disease process. In general, these processes are not directly observable, although their outcomes may be. For example, the process of developing CRC cannot be observed, but the prevalence of both precancerous lesions (adenomas) and preclinical CRC can be estimated from screening trials.

MSM model calibration involves selecting parameter values that are consistent with observed data and expected findings. Once parameters are selected, MSMs can be used to make predictions about hypothetical interventions and/or future trends in population disease outcomes. The simplest calibration method involves perturbing parameters one at a time and subjectively judging the goodness of fit to calibration data (Ramsey et al. 2000). More recently, chi-squared (Ness et al. 2000) and deviance (Loeve et al. 2004) statistics have been used to evaluate how closely MSM estimates match calibration data at different points in the parameter space. Statistical criteria for MSM goodness of fit also have been combined with search algorithms to determine the best parameters. Salomon et al. (2002) used simulation-based estimates of likelihoods in combination with a sequential search algorithm to determine the best-fitting model parameters. Chia et al. (2004) demonstrated the use of three maximum likelihood methods for estimating parameters associated with a complex two-parameter model describing breast cancer tumor growth, including grid search using the Nelder–Mead algorithm (Nelder and Mead 1965); the Kiefer–Wolfowitz algorithm (Kiefer and Wolfowitz 1952), based on likelihood differences estimated by simulating the likelihood; and the Robbins–Monro algorithm (Robbins and Monro 1951), based on directly simulating the likelihood gradient.

Here we propose a Bayesian method for calibrating MSM parameters that parallels Bayesian sampling-based estimation approaches. We place prior distributions on all parameters and use Markov chain Monte Carlo (MCMC) to estimate parameters using data from multiple sources. The Bayesian approach allows us to describe an appropriately complex model for disease processes while incorporating information from both prior distributions and observed data. By using informative prior distributions, we can include model parameters that otherwise might not be identifiable based on available data. This approach acknowledges uncertainty in parameters as opposed to fixing selected parameters to obtain the identifiability of a potentially inferior model (Gustafson 2005). We suggest using model diagnostics to examine parameter identifiability. Finally, our approach allows point and interval estimation of both model parameters and functions of these parameters, including posterior predictive estimates that can be used to assess model fit.

In the next section we briefly describe our MSM for the natural history of CRC. In Section 3 we describe our calibration approach, examine its relationship to Bayesian analysis, and present its asymptotic characteristics. In Section 4 we describe a simulation study used to examine the behavior of our MCMC calibration with finite sample sizes. We provide calibration results for our CRC MSM in Section 5 and concluding remarks in Section 6.

2. Microsimulation Model for the Natural History of Colorectal Cancer

CRC is the second leading cause of cancer death in the United States (American Cancer Society 2008; Jemal et al. 2008). CRC mortality rates have declined since 1980 (Wingo et al. 2003; Stewart et al. 2004; American Cancer Society 2008); and CRC incidence declined between 1985 and 1995 and then stabilized (Howe et al. 2001, 2006). The reasons behind these trends are unclear. Clinical trials have shown that screening for CRC by fecal occult blood tests can reduce CRC mortality (Hardcastle et al. 1996; Kronborg et al. 1996; Towler et al. 1998); however, a 1999 study found low rates of CRC screening, with fewer than half of Americans over age 50 reporting ever being screened (CDC 2001). The possibility that lifestyle changes have reduced the population's risk for CRC holds promise for primary prevention. The use of MSMs allows estimation of the effects of screening and population risk on changes in CRC incidence and mortality, offering new insights into factors associated with observed trends. In this article, we develop and calibrate an MSM for CRC.

Our MSM for the natural history of CRC is based on the adenoma–carcinoma sequence (Muto, Bussey, and Morson 1975; Leslie et al. 2002) and assumes that all CRCs arise from an adenoma. Four model components describe the natural history of CRC: adenoma risk, adenoma growth, transition from adenoma to preclinical cancer, and transition from preclinical to clinical cancer. Table 1 summarizes each component's parameters and prior distributions.

Table 1.

Summary of CRC MSM components

		Posterior estimates
Component	Prior distribution	Mean	95% CI	Estimated overlap
Adenoma risk [eqn. (2.1)]
Baseline log-risk	α0 ∼ N(−6.7, 0.27)	−6.6	(−7.5, −5.9)	0.79
Main sex effect	α1 ∼ N(−0.3, 0.04)	−0.24	(−0.35, −0.14)	0.53
Standard deviation, baseline log-risk	σα ∼ U(0.10, 3.0)	1.1	(0.35, 1.6)	0.46
Age effect, age ∈ [20, 50)	α20 ∼ TN0(0.03, 0.003)	0.037	(0.024, 0.049)	0.42
Age effect, age ∈ [50, 60)	α21 ∼ TN0(0.03, 0.003)	0.031	(0.011, 0.047)	0.51
Age effect, age ∈ [60, 70)	α22 ∼ TN0(0.03, 0.003)	0.029	(0.011, 0.047)	0.50
Age effect, age ≥ 70	α23 ∼ TN0(0.03, 0.003)	0.030	(0.011, 0.049)	0.51
Time to 10 mm [eqn. (2.2)]
Colon: Location	β1c ∼ U(1, 100)	28.6	(24.3, 34.2)	0.14
Scale	β2c ∼ U(1, 4)	2.7	(1.1, 3.9)	0.89
Rectum: Location	β1r ∼ U(1, 100)	10.3	(5.9, 13.8)	0.11
Scale	β2r ∼ U(1, 4)	2.7	(1.4, 3.9)	0.83
Transition to cancer [eqn. (2.3)]
Men
Colon, size	γ1cm ∼ U(0.02, 0.05)	0.045	(0.040, 0.049)	0.37
Colon, age at initiation	γ2cm ∼ U(0.0, 0.02)	0.008	(0.002, 0.016)	0.64
Rectum, size	γ1rm ∼ U(0.02, 0.05)	0.035	(0.021, 0.049)	0.90
Rectum, age at initiation	γ2rm ∼ U(0.0, 0.02)	0.010	(0.001, 0.019)	0.91
Women
Colon, size	γ1cf ∼ U(0.02, 0.05)	0.048	(0.044, 0.050)	0.23
Colon, age at initiation	γ2cf ∼ U(0.0, 0.02)	0.005	(0.000, 0.013)	0.58
Rectum, size	γ1rf ∼ U(0.02, 0.05)	0.043	(0.030, 0.050)	0.57
Rectum, age at initiation	γ2rf ∼ U(0.0, 0.02)	0.015	(0.008, 0.019)	0.56
Mean sojourn time [Lognormal(μ, τμ)]
Colon	μc ∼ U(0.5, 5.0)	1.9	(1.0, 3.9)	0.55
	τc ∼ U(0.1, 1.5)	0.80	(0.15, 1.4)	0.89
Rectum	μr ∼ U(0.5, 5.0)	2.7	(1.1, 4.7)	0.80
	τr ∼ U(0.1, 1.5)	0.84	(0.15, 1.4)	0.90

NOTE: Shown are calibrated parameters associated with the 4 components of the natural history model: including parameter notation, associated equations, prior distributions and posterior estimates (mean and 95% credible interval). N(μ, σ) denotes a Normal distribution with mean μ and standard deviation σ. TN₀(μ, sigma) denotes a truncated Normal distribution with mean μ and standard deviation sigma, based on a Normal distribution restricted to range over (0, ∞). U(a, b) denotes a Uniform distribution over (a, b). The estimated overlap statistic is based on ∫ min(g(θ), h(θ|y)) dθ (Garret and Zeger 2000).

2.1 Adenoma Risk Model

We modeled the occurrence of adenomas using a nonhomogeneous Poisson process with a piecewise age effect. The ith individual's instantaneous risk of an adenoma at time t is given by

$$\begin{array}{c}{\psi }_i(t)=\text{exp}({\alpha }_{0i}+{\alpha }_1{\mathit{\text{sex}}}_i+\sum _{k=1}^4\delta (A_k<{\mathit{\text{age}}}_i(t)\le A_{k+1})\hfill \\ \times \left\{{\mathit{\text{age}}}_i(t){\alpha }_{2k}+\sum _{j=2}^k{A}_j({\alpha }_{2j-1}-{\alpha }_{2j})\right\}),\hfill \end{array}$$ (2.1)

where α_0i ~ N(α₀, σ_α) describes an individual's baseline risk, α₁ describes the difference in risk for men (sex_i = −1) versus women (sex_i = +1), α_2k describes changes in risk with age (in years) in the kth interval, and δ(·) is an indicator function with δ(x) = 1 when x is true and δ(x) = 0 otherwise. We assumed that before age 20, individuals are not at risk of developing adenomas and allowed for the following age-risk intervals: A₁ = 20, A₂ = 50, A₃ = 60, A₄ = 70, and A₅ = 120.

The prior distributions for α₀, α₁, α_2k, and α_3k, k ∈ {1, 2, 3, 4}, were based on results from a Bayesian meta-analysis of autopsy data that modeled a log-linear age effect that remained constant across age and sex groups (Rutter, Miglioretti, and Yu 2007). The meta-analysis did not provide information about σ_α, so we used a minimally informative uniform prior distribution for this parameter.

Once adenomas were initiated, we assigned their location (colon vs. rectum). We specified P(rectum) = 0.09 based on analysis of data from 9 autopsy studies (Blatt 1961; Chapman 1963; Stemmermann and Yatani 1973; Eide and Stalsberg 1978; Rickert et al. 1979; Williams, Balasooriya, and Day 1982; Bombi 1988; Johannsen, Momsen, and Jacobsen 1989; Szczepanski, Urban, and Wierzchowski 1992a) and one colonoscopy study (Szczepanski, Urban, and Wierzchowski 1992b).

2.2 Adenoma Growth Model

We modeled adenoma growth using an extension to the Janoschek growth curve model (Janoschek 1957; Gille and Salomon 2000), d_ij(t) = d_∞ − (d_∞ − d₀)e^−λijt, where d_ij(t) is the diameter of the jth adenoma in the ith individual at time t after initiation, d_∞ is the maximum possible adenoma diameter, d₀ is the minimum possible adenoma diameter, and λ_ij is the growth rate for the jth adenoma within the ith individual. This model is asymmetric, with early exponential growth that slows to allow an asymptote at d_∞.

To improve our ability to obtain prior information from clinicians, we reparameterized the growth model in terms of the time needed for the adenoma to grow to 10 mm in diameter, $t_{10\text{mm}}=-\frac{1}{\lambda }\text{ln}(\frac{d_{\infty }-10}{d_{\infty }-d_0})$. We assumed that t_{10 mm} has a type 2 extreme value distribution with cumulative distribution function given by

$$F(t)=\text{exp}\left[-{\left(\frac{t}{{\beta }_1}\right)}^{-{\beta }_2}\right]$$ (2.2)

for t ≥ 0. Because the type 2 extreme value distribution has a long right tail, with skewness that can persist as the mean moves away from 0, increasing the proportion of slow-growing adenomas does not also require increasing the proportion of fast-growing adenomas. The parameter β₁ roughly determines the location of the distribution, and β₂ corresponds to the scale. We modeled separate growth distributions for colon and rectal adenomas, with parameters (β_1c, β_2c) and (β_1r, β_2r), respectively. We chose relatively uninformative prior distributions for the β's. Under these priors, the median time to 10 mm ranged from 1.1 to 144.3 years.

2.3 Model for Transition From Adenoma to Preclinical Invasive Cancer

Our adenoma transition model and priors for associated parameters are informed by an analysis of colorectal polyp registry data reported by Nusko et al. (1997) and a recent study of follow-up colonoscopy that provides evidence suggesting that the probability of transition depends on an individual's age at the time of adenoma onset (Yamaji et al. 2006). We use a lognormal cumulative distribution function to describe the cumulative transition probability as a function of sex, size, and age at adenoma onset. For an adenoma initiated at age a in the colon of a man, the probability of transition to preclinical cancer at or before size s is given by the lognormal cumulative distribution,

$${\xi }_c(s,a)=\Phi \left(\{\text{ln}({\gamma }_{1\mathit{\text{cm}}}s)+{\gamma }_{2\mathit{\text{cm}}}(a-50)\}/{\gamma }_3\right),$$ (2.3)

where Φ(·) is the standard Normal cumulative distribution function.

Cumulative transition probabilities for adenomas in the male rectum have the same form and are parameterized by γ_1rm, γ_2rm, and γ₃. Similarly, the probability of adenoma transition in women is based on eq. (2.3), parameterized by γ_1cf, γ_2cf, and γ₃ for adenomas in the colon and γ_1r, γ_2rf, and γ₃ for adenomas in the rectum. The γ₁ parameters rescale adenoma size and are associated with the magnitude of the survival function. The γ₂ parameters allow the probability of transition to increase with age at adenoma onset. The γ₃ parameter is associated with the standard deviation of the lognormal distribution and determines the shape of the cumulative probability curve (i.e., how rapidly the probability of transition increases with size) and is closely associated with the probability of transition in small (≤ 10 mm) adenomas.

Because we lacked sufficient calibration data to estimate γ₃, we set γ₃ = 0.5 to restrict the transition probability for small (≤10 mm) adenomas, based on expert opinion that such small adenomas are unlikely to undergo malignant transition. Priors on the remaining transition parameters impose clinically reasonable restrictions on the range of cumulative transition probabilities; for example, cumulative transition probabilities of a 10-mm adenoma can range from 0.0006 to 0.08 for a 50-year-old and up to 0.28 for a 70-year-old with the maximum age effect. Cumulative transition probabilities for a 20-mm adenoma can range from 0.03 to 0.50 at age 50 up to 0.79 at age 70.

2.4 Model for Time From Preclinical to Clinical Cancer (Sojourn Time)

We define sojourn time as the time from the onset of preclincal cancer to clinical detection. For individuals with multiple cancers, each sojourn time is assumed to be independent, and cancer survival is dependent on the earliest clinically detected cancer. Sojourn time is modeled with a lognormal distribution that depends on whether an adenoma is located in the colon or in the rectum. Let t represent sojourn time. For adenomas in the colon, log(t) ∼ Normal(ξ_c,ν_c), where ξ_c and ν_c denote the mean and standard deviation of log(t), and t ∼ lognormal with mean μ_c and standard deviation τ_cμ_c, where ${\mu }_c=\text{exp}({\xi }_c+\frac{1}{2}{\nu }_c^2)$ and ${\tau }_c=\sqrt{\text{exp}({\nu }_c^2)-1}$. Sojourn time for adenomas in the rectum is modeled by a lognormal distribution with mean μ_r and standard deviation τ_rμ_r. Prior distributions for mean sojourn time are based on data from the TAMACS study Chen et al. (1999), which reported an estimated mean sojourn time of 2.85 years with a 95% confidence interval (CI) of 2.15–4.30. Under our prior distributions for sojourn time parameters (μ_c, τ_c, μ_r, τ_r), sojourn time has a median ranging from 0.3 to 5.0 years, a 5th percentile ranging from 0 to 4.2 years, and a 95th percentile ranging from 0.6 to 16.5 years.

Once a cancer becomes clinically detectable, we simulate size and stage at clinical detection and survival. We specify an overall (unconditional) distribution for tumor size at clinical detection and a conditional distribution of stage given size, using observed SEER data describing size at detection in 1975–1979 (Surveillance, Epidemiology, and End Results (SEER) Program (www.seer.cancer.gov) 2004). Little or no CRC screening was done during this time period. We simulate survival time based on a Cox proportional hazards model for relative survival. Using SEER data on cancer survival for cases diagnosed in 1975–1979, we estimate multiple proportional hazard models using the CANSURV program (http://srab.cancer.gov/cansurv/). These proportional hazards models are stratified by location (colon or rectum) and AJCC stage, with age and sex included as covariates. Finally, we model other-cause mortality using survival probabilities based on product-limit estimates for age and birth year cohorts from the National Center for Health Statistics Databases (US Life Tables 2000).

2.5 Calibration Data

Table 2 presents calibration data from 4 studies based on the evaluation of minimally screened asymptomatic individuals. These data include the prevalence of adenomas and preclinical cancers in individuals undergoing screening colonoscopy (Strul et al. 2006), the size of the largest adenoma found in veterans participating in a study of screening colonoscopy (Lieberman et al. 2000), the number and size of adenomas found in individuals participating in a study comparing virtual and optical colonoscopy (Pickhardt et al. 2003), and the prevalence of adenomas found in individuals participating in a workplace study of screening colonoscopy (Imperiale et al. 2000).

Table 2.

Calibration data from studies reporting individual-level outcomes

Prevalence of adenomas and preclinical cancers (Strul et al. 2006) (m = 50,000)
Age range	Mean age (SD)	% male	Sample size		Prevalence
40–49	49.2 (3.0)	49.2%	183		0.10
50–75	61.2 (7.6)	47.5%	917		0.22
76–80	77.7 (1.4)	37.7%	77		0.29
Size of largest adenoma given at least 1 adenoma (Lieberman et al. 2000) (m = 10,000)
Age range	Mean age (SD)	% male	Sample size		Proportion of 1,411 adenomas ≥ 10 mm
50–75	61.2 (7.3)	96.8%	3121		0.23
Overall number and size distribution of adenomas (Pickhardt et al. 2003) (m = 10,000)
Age range	Mean age (SD)	% male	Sample size		Size distribution of 554 adenomas
40–79	57.8	59%	1233	< 6 mm:	0.620
				[6, 10) mm:	0.287
				≥ 10 mm:	0.092
Number of preclinical cancers (Imperiale et al. 2000) (m = 50,000)
Age range	Mean age (SD)	% male	Sample size		Prevalence
50+	59.8 (8.3)	59%	1994		0.0035

NOTE: Age is given in years. m indicates the number of embedded simulations used to estimate the data parameters for each Metropolis–Hastings step.

Data from 2 clinical series inform our adenoma transition model (Table 3). These studies focused on adenomas and did not include individual characteristics, such as age and sex. The first series, reported by Church (2004), describes the pathology of 5,722 adenomas removed between January 1995 and September 2002 in a single endoscopist's practice. The second series, reported by Odom et al. (2005), describes the pathology of 3,225 adenomas removed between January 1999 and December 2003, excluding those obtained from bowel resection and CRC not associated with a polyp. We calibrate to preclinical cancer rates in adenomas ≥ 5 mm, because the rates of preclincal cancer are near 0 in smaller adenomas (0.05% in the Church data and 0.03% in the Odom data), which causes problems with embedded simulations (as described in the next section).

Table 3.

Calibration studies reporting adenoma transition to preclinical disease

Source	Sample size	Preclinical cancer rate by adenoma size
Church (2004)	1,341	6–10 mm 1/666 (0.15%)	>10 mm 21/675 (3.1%)
Odom et al. (2005)	374	6–10 mm 1/152 (0.65%)	11–20 mm 3/191 (1.6%)	>20 mm 6/31 (19.4%)

NOTE: Data parameters associated with multinomial distributions are based on m = 200,000 embedded simulations.

SEER colon and rectal cancer incidence rates in 1975–1979 are a key calibration component (Table 7, Section 5.2). The incidence rates reported are per 100,000 individuals. These rates are based on the first observed invasive colon or rectal cancer during 1975–1979, the last period before the dissemination of CRC screening tests.

Table 7.

Model fit to calibration data

	Observed	Estimated	95% predictive interval
Adenoma prevalence
Among individuals 40–49 years old1	0.10	0.14	(0.09, 0.21)
Among individuals 50–75 years old1	0.22	0.27	(0.21, 0.33)
Among individuals 76–80 years old1	0.29	0.38	(0.25, 0.52)
Number of adenomas found in 1,233 individuals2	554	575	(450, 747)
Adenoma size
Percent of individuals with an adenoma ≥10 mm, given ≥1 adenoma3	0.23	0.28	(0.22, 0.35)
Percent of adenomas <6 mm2	0.62	0.61	(0.53, 0.68)
Percent of adenomas [6, 10) mm2	0.29	0.22	(0.16, 0.28)
Percent of adenomas ≥10 mm2	0.09	0.17	(0.12, 0.22)
Preclinical cancers
Among employed individuals ≥504	0.004	0.003	(0.001, 0.006)
Among adenomas 6–10 mm5	0.002	0.008	(0.000, 0.028)
Among adenomas >10 mm5	0.030	0.048	(0.000, 0.109)
Among adenomas 6–10 mm6	0.0065	0.008	(0.000, 0.049)
Among adenomas 11–20 mm6	0.0160	0.036	(0.000, 0.133)
Among adenomas >20 mm6	0.19	0.14	(0.00, 0.667)
Clinical cancers in 1975–1979, per 100,0007
Colon cancer in women
20–49 y.o.	4.8	4.4	(3.1, 5.8)
50–59 y.o.	43.3	45.5	(37.9, 53.1)
60–69 y.o.	100.7	99.3	(88.1, 111.9)
70–84 y.o.	216.7	207.3	(181.5, 240.7)
Rectal cancer in women
20–49 y.o.	1.87	2.0	(1.4, 2.8)
50–59 y.o.	20.4	18.6	(14.6, 22.6)
60–69 y.o.	42.5	39.7	(32.8, 46.7)
70–84 y.o.	73.9	82.1	(66.8, 97.2)
Colon cancer in men
20–49 y.o.	4.51	4.2	(2.9, 5.6)
50–59 y.o.	45.9	50.6	(41.4, 60.1)
60–69 y.o.	121.4	120.0	(106.2, 136.8)
70–84 y.o.	268.4	263.4	(224.9, 306.4)
Rectal cancer in men
20–49 y.o.	2.3	3.2	(2.3, 4.4)
50–59 y.o.	30.0	29.8	(24.1, 35.5)
60–69 y.o.	71.4	63.2	(53.5, 72.4)
70–84 y.o.	128.0	123.8	(101.5, 145.9)

¹Strul et al. (2006);

²Pickhardt et al. (2003);

³Lieberman et al. (2000);

⁴Imperiale et al. (2000);

⁵Church (2004);

⁶Odom et al. (2005);

⁷Surveillance, Epidemiology, and End Results (SEER) Program (www.seer.cancer.gov) (2004).

3. Bayesian Microsimulation Model Calibration

We propose a MCMC calibration (MCMCC) approach that is similar to MCMC estimation methods for Bayesian analysis (Gelman et al. 1995). Parametric data analysis has two basic components: data, y, and a data distribution that is conditional on some unknown parameter vector, f(Y|θ). Bayesian models add a third component, the prior distribution of the unknown parameters, θ ∼ π(θ).

In the context of MSM calibration, θ represents MSM parameters, and prior distributions are specified for these θ. But the distribution of the calibration data, y ∼ f(Y|g(θ)), is parameterized by an unknown function of MSM parameters, g(θ).

3.1 Markov Chain Monte Carlo Calibration

Let M(θ) denote a MSM parameterized by the vector θ. We assume that the MSM is composed of K separate components and that the corresponding parameter vectors are distinct and a priori mutually independent. Let M_i(θ_i) denote the ith model component and let π_i(θ_i) denote the prior distribution associated with θ_i, where ${\theta }^{\prime }=({\theta }_1^{\prime },{\theta }_2^{\prime },\dots ,{\theta }_k^{\prime })$.

Let Y = {y₁, …, y_N} denote the full set of calibration data from N independent sources. We assume that each y_j has a known distributional form, y_j|θ ∼ f_j(g_j(θ)). In general, calibration data are statistics from published studies that can be described using distributions for count and categorical data, such as the binomial, multinomial, and Poisson distributions.

We calibrate θ by simulating draws from the posterior distribution of θ given Y, h(θ|Y), using MCMC with Metropolis–Hastings (MH) steps within Gibbs iterations. Gibbs iterations are based on the distinct parameter sets associated with each of the K model components. One complete iteration of the algorithm involves sequentially drawing candidate values $({\theta }_i^{\ast })$ and accepting or rejecting these candidates for each of the K components (Smith and Roberts 1993). We use a random-permutation sweep approach to improve the mixing of the sampler (Roberts and Sahu 1997). Within Gibbs steps, we use a random-walk MH algorithm to approximately sample from the full conditional distributions, h(θ_i|θ_(−i), Y) (Tierney 1994), where θ_(−i) denotes the parameter vector excluding the ith component.

For a symmetric jumping distribution, such as the Normal centered at the current value, the MH algorithm accepts ${\theta }_i^{\ast }$ based on the transition probability $\alpha ({\theta }_i,{\theta }_i^{\ast })$, where

$$\alpha ({\theta }_i,{\theta }_i^{\ast })=\{\begin{array}{ccc}\text{min}(r_i({\theta }_i,{\theta }_i^{\ast }),1)\hfill & \text{if}\hfill & {\pi }_i({\theta }_i)\prod _{j=1}^N{f}_j(y_j|\theta )>0\hfill \\ 1\hfill & \text{if}\hfill & {\pi }_i({\theta }_i)\prod _{j=1}^N{f}_j(y_j|\theta )=0\hfill \end{array}$$

and

$$r_i({\theta }_i,{\theta }_i^{\ast })=\frac{h({\theta }_i^{\ast },{\theta }_{(-i)}|Y)}{h(\theta |Y)}=\frac{{\pi }_i({\theta }_i^{\ast }){\prod }_{j=1}^N{f}_j(y_j|{\theta }_i^{\ast },{\theta }_{(-i)})}{{\pi }_i({\theta }_i){\prod }_{j=1}^N{f}_j(y_j|{\theta }_i)}.$$

In most data analyses, calculating $r_i({\theta }_i,{\theta }_i^{\ast })$ is straightforward, because both the priors, π(θ), and the data distributions, f(y|θ), are known and have closed form. In the case of MSM calibration, $r_i({\theta }_i,{\theta }_i^{\ast })$ cannot be calculated directly, because g_j(θ) are unknown functions of θ. For example, when calibrating to observed cancer incidence, the associated probability of incident clinical cancer contains information about the full disease process, described by multiple MSM parameters, from the risk of an adenoma through the transition to clinically detected cancer, and the exact functional form of this relationship is unknown.

We propose an approximate MH algorithm that includes an embedded simulation to estimate g(θ). We assume that the MSM, M(θ), and data distributions, f(y|g(θ)), are specified correctly and use M(θ) to simulate m draws from f_j(y_j|g_j(θ)): {ỹ_j1, …, ỹ_jm}. Using the simulated sample, we calculate the maximum likelihood estimate of g_j(θ), denoted by ĝ_j(θ); for example, for Poisson and binomial distributions, we estimate ${\widehat{g}}_j(\theta )=\frac{1}{m}\sum {\stackrel{\sim }{y}}_{ji}$. To simulate draws from h(θ|Y), we use the Metropolis-within-Gibbs steps described earlier, substituting f_j(y_j|ĝ_j(θ)) for f_j(y_j|g_j(θ)). The resulting transition probability function, α̂(θ, θ*), is based on $\widehat{r}({\theta }_i,{\theta }_i^{\ast })$, where

$${\widehat{r}}_i({\theta }_i,{\theta }_i^{\ast })=\frac{{\pi }_i({\theta }_i^{\ast }){\prod }_{j=1}^N{f}_j(y_j|{\widehat{g}}_j({\theta }_i^{\ast },{\theta }_{(-i)}))}{{\pi }_i({\theta }_i){\prod }_{j=1}^N{f}_j(y_j|{\widehat{g}}_j({\theta }_i))}.$$

To maintain ergodicity of the chain, we obtain independent estimates of ĝ(θ), with a new embedded simulation for both candidate and current parameter values at each iteration. Heuristically, this repeated simulation is required because f(y|ĝ(θ)) may be large by chance. The MH algorithm tends to accept candidate values associated with large estimated likelihoods, including both those with large true likelihoods and those with small true likelihoods that by chance are estimated to be large. Without repeated simulation of g(θ) at both candidate and current values of θ, the sampler can get stuck where the true likelihood is small but an estimated likelihood is large. In the Appendix we show that this approach, based on an approximate jumping rule, converges to the target posterior distribution as m increases.

4. Simulation Study

We performed a simulation study to explore the characteristics of MSM parameter estimates based on MCMCC with finite m and fixed chain length, focusing on the effects of the number of embedded simulations (m) and the number of estimated MSM parameters on bias and credible interval length and coverage. Our simulation study was based on a relatively simple hypothetical MSM describing transitions among 4 disease states with 6 potentially unknown parameters.

4.1 Hypothetical Microstimulation Model

Under the hypothetical MSM, individuals begin in a disease-free state (S₀) and may progressively transition through 3 disease-related states (S₁, S₂, and S₃), with transition to a death state (S_d) possible from any of these 4 states. We select true parameter values so that the hypothetical model produces prevalence patterns that are similar to those seen for cancers, with a precursor state (S₁) that is relatively common and preclinical and clinical cancer outcomes (S₂ and S₃) that are rare. Let t_k indicate the transition time from S_k−1 to S_k, and let t_d indicate the transition time from S₀ to S_d. We assume that t₁ ∼ exponential(λ₁), where log(λ₁) ∼ N(Λ₁, σ₁). Given a transition to S₁, individuals have the potential to transition to S₂ with probability p₂. For individuals with this potential, we assume t₂ ∼ exponential(λ₂) and t₃ ∼ lognormal with mean μ₃ and standard deviation τ₃μ₃. Finally, t_d ∼ Gumbel(λ_d, σ_d). Although the time scale is arbitrary, we refer to units of time as years and to a simulated individual's time since initiation as his or her age. True values for model parameters are set to Λ₁ = 4, σ₁ = 0.5, p₂ = 0.05, λ₂ = 10, μ₃ = 3, σ₃ = 0.5, λ_d = 65, and σ_d = 15. Associated population characteristics based on this hypothetical MSM, calculated for 10 million simulated individuals, are shown in Table 4. Under the hypothetical true MSM, membership in S₁ is relatively common and prevalence increases with age, but membership in S₂ and transition into S₃ are rare. In this sense, the hypothetical MSM is similar to CRC, in which adenomas are relatively common and prevalence increases with age, but preclinical cancer and transition to clinically detectable cancer are rare.

Table 4.

Estimated population characteristics for the hypothetical 3-state MSM, based on 10,000,000 simulated individuals

Time, T	P(∈ S1)	P(∈ S2)	$P({\sum }_{i=1}^3{t}_i\in [T,T+10))$	P(tD > T)
10	0.18	0.0014	0.059	1.00
20	0.31	0.0018	0.064	1.00
30	0.42	0.0016	0.057	0.99
40	0.51	0.0014	0.049	0.99
50	0.57	0.0011	0.043	0.93
60	0.63	0.0010	0.038	0.75
70	0.68	0.0008	0.033	0.51
80	0.71	0.0006	0.029	0.31

NOTE: For i ∈ (1, 2, 3), t_i is time from S_i−1 to S_i, and t_D is the time to S_d, the death state.

4.2 Simulated Calibration Data

For each run of our simulation study, we used the hypothetical MSM model with the true MSM parameter values to generate 3 calibration data sets. Calibration set 1 describes the number of individuals in S₁ at age 10, 20, 30, 40, and 50 years, with 250 individuals at each age. Calibration set 2 describes the number of individuals in S₂ at age 10, 20, and 50 years, based on 5,000 individuals at each age. Calibration set 3 describes the proportion of individuals (per life year) who transition to S₃ within 20-year age intervals, [0, 20), [20, 40), [40, 60), and [60, 80), based on a cohort of 10,000 individuals.

A subset of simulations examines the effect of additional data on parameter estimates by adding a fourth calibration data set. Calibration set 4 provides the categorical distributions of t₂ and t₃ across 5 intervals (t₂: [0, 2), [2, 5), [5, 10), [10, 15), and ≥15; t₃: [0, 1), [1, 2), [2, 3), [3, 4), and ≥4) for 500 individuals transitioning from S₁ to S₂ and from S₂ to S₃, respectively.

4.3 Simulation Study Design

We examined 4 estimation scenarios that vary in terms of the number of MSM parameters estimated and the calibration sets used for estimation. Scenarios 1, 2, and 3 all use calibration sets 1, 2, and 3, differing only in terms of the number of MSM parameters estimated. Scenario 1 examines estimation of Λ₁ and σ₁. Scenario 2 examines estimation of Λ₁, σ₁, p₂, and σ₂. Scenario 3 examines estimation of all 6 model parameters. Scenario 4 is identical to scenario 3, but with the addition of calibration data set 4.

We used different embedded simulation sizes to estimate the data parameters associated with each calibration data set. For calibration set 1, we simulated m₁ individuals who are distributed equally across the 5 corresponding calibration ages. For calibration set 2, we simulated m₂ individuals who are distributed equally across the 3 corresponding calibration ages. For calibration set 3, we simulated a cohort of m₃ individuals. For calibration set 4, we fixed the embedded simulation size, simulating 5,000 individuals undergoing each transition (S₁ to S₂ and S₂ to S₃). We examined 3 combinations of embedded simulation sizes: small (m₁ = 6,250, m₂ = 75,000, m₃ = 50,000), medium (m₁ = 12,500, m₂ = 150,000, m₃ = 100,000), and large (m₁ = 25,000, m₂ = 300,000, m₃ = 200,000).

We simulated MCMCC estimation for scenario 1 (estimation of S₁ parameters) using all 3 embedded simulation sizes, for scenarios 2 and 3 using medium and large embedded simulation sizes, and for scenario 4 using medium embedded simulation sizes.

4.4 Implementation of Markov Chain Monte Carlo Calibration

In this simulation study, the MSM was correctly specified, and we set unestimated MSM parameters to their true values. We specified uniform priors for all estimated parameters. We chose the prior mean to be twice the true value, so that (prior mean − true mean)/true mean equals 1 for all parameters and 100% bias corresponds to an average estimate of posterior mean that is equal to the prior mean. We selected starting values to be 3 times the true value, so that true value < prior mean < starting value for all parameters. Table 5 gives the true values of hypothetical MSM parameters, their prior ranges and means, and starting values used for MCMC calibration.

Table 5.

Hypothetical MSM parameters: true values, starting values used for MCMC calibration, and the mean and range of associated prior distributions

Parameter	True value	Prior mean	Starting value	Prior range
Λ1	4	8	12	[0.1, 16.1]
σ1	0.5	1	1.5	[0.0001, 2.0001]
p2	0.05	0.1	0.15	[0.0001, 0.2001]
λ2	10	20	30	[0.1, 40.1]
μ3	3	6	9	[1, 13]
τ3	0.5	1	1.5	[0.001, 2.001]

The parameter estimates were based on a single chain with 32,500 iterations: 5,000 iterations for burn-in, the next 2,500 iterations to estimate the between-parameter covariance (based on every fifth iteration), and every fifth iteration from the last 25,000 to estimate parameters. We took the same approach when calibrating our MSM for CRC, using initial Gibbs steps to estimate the correlation matrix of estimated parameters and subsequent steps based on sampling the full parameter vector using this correlation matrix to direct the random-walk steps. The parameters were estimated by the mean across simulated posterior draws, and the 95% CIs were estimated based on the upper and lower 2.5th percentiles across draws.

4.5 Simulation Results

For each combination of scenario and embedded sample size, we simulated 1,000 runs. For each run, we simulated calibration data and used MCMCC to estimate the parameters. Using these 1,000 estimates, we calculated the percent bias (i.e., bias as a percentage of the true parameter value), the length of the 95% CI, and the coverage of the 95% CI. We assessed the identifiability of model parameters by comparing prior and posterior distributions using an estimated overlap measure proposed by Garrett and Zeger (2000), ∫ min(π(θ), h(θ|y)) dθ. This measure of overlap ranges from 0 to 1, with values closer to 1 corresponding to greater similarity between the specified prior and estimated posterior distributions.

Table 6 gives results from the simulation study. Increasing embedded simulation sizes (m) reduced the percent bias and decreased the length of the 95% CIs, although the changes tended to be small. Under scenario 1, decreases in percent bias were greatest for the initial increase from small to medium m; another doubling of m (from medium to large) resulted in only minor changes in percent bias. An estimated increase in absolute percent bias of σ̂₁ was small (0.04 times the standard deviation of the difference, SD_diff), reflecting the variability of σ̂₁, which did not change with increasing m. Under scenarios 2 and 3, increasing m from medium to large resulted in slight decreases in percent bias, ranging from 0.02 times SD_diff for μ̂₃ estimated under scenario 3 to 0.12 times SD_diff for p̂₂ estimated under either scenario 2 or scenario 3.

Table 6.

Percent bias, length, and coverage of 95% CIs and Garrett and Zeger (2000) “overlap” statistics associated with MSM parameter estimates derived using MCMCC

			95% CI
	Embedded simulation size	Percent bias	Length	Coverage	Overlap statistic
Scenario 1: 2 parameters estimated
Λ1	Small	2.2 (17.7)	0.41 (0.97)	0.99	0.04 (0.08)
	Medium	0.1 (1.2)	0.26 (0.02)	0.99	0.03 (0.003)
	Large	0.1 (1.1)	0.23 (0.02)	0.99	0.03 (0.003)
σ1	Small	3.7 (29.8)	0.93 (0.16)	0.99	0.51 (0.07)
	Medium	−1.1 (28.2)	0.84 (0.10)	0.99	0.47 (0.05)
	Large	−2.8 (28.1)	0.80 (0.09)	0.99	0.45 (0.05)
Scenario 2: 4 parameters estimated
Λ1	Medium	1.2 (1.8)	0.37 (0.06)	0.97	0.04 (0.01)
	Large	1.1 (1.6)	0.33 (0.05)	0.98	0.04 (0.01)
σ1	Medium	45.2 (47.9)	1.3 (0.23)	0.97	0.65 (0.09)
	Large	38.2 (48.3)	1.2 (0.22)	0.96	0.62 (0.09)
p2	Medium	7.9 (7.6)	0.02 (0.005)	0.98	0.16 (0.03)
	Large	6.7 (7.3)	0.02 (0.004)	0.98	0.14 (0.02)
λ2	Medium	33.2 (33.5)	16.67 (4.62)	0.94	0.45 (0.09)
	Large	28.6 (33.9)	14.85 (4.17)	0.93	0.42 (0.09)
Scenario 3: 6 parameters estimated
Λ1	Medium	1.1 (1.8)	0.36 (0.06)	0.97	0.04 (0.01)
	Large	1.0 (1.7)	0.32 (0.06)	0.96	0.04 (0.01)
σ1	Medium	44.9 (49.1)	1.30 (0.24)	0.97	0.65 (0.10)
	Large	36.8 (51.0)	1.19 (0.23)	0.95	0.61 (0.10)
p2	Medium	7.5 (7.7)	0.02 (0.005)	0.97	0.16 (0.03)
	Large	6.2 (7.4)	0.02 (0.004)	0.97	0.14 (0.02)
λ2	Medium	27.8 (34.1)	16.0 (4.44)	0.95	0.45 (0.09)
	Large	23.7 (34.0)	14.2 (4.09)	0.94	0.41 (0.09)
μ3	Medium	65.9 (30.4)	7.2 (0.97)	0.94	0.62 (0.07)
	Large	65.1 (32.8)	6.8 (1.02)	0.91	0.60 (0.08)
τ3	Medium	134.9 (24.1)	1.8 (0.09)	0.99	0.84 (0.06)
	Large	133.7 (26.9)	1.8 (0.09)	0.99	0.83 (0.06)
Scenario 4: 6 parameters estimated with additional data
Λ1	Medium	0.8 (5.7)	0.33 (0.19)	0.98	0.04 (0.01)
σ1	Medium	10.7 (32.2)	0.95 (0.14)	0.99	0.51 (0.06)
p2	Medium	2.6 (8.6)	0.02 (0.004)	0.99	0.11 (0.02)
λ2	Medium	3.2 (10.2)	3.7 (0.94)	0.99	0.13 (0.03)
μ3	Medium	1.6 (6.5)	0.61 (0.27)	0.99	0.08 (0.03)
τ3	Medium	1.9 (5.8)	0.15 (0.03)	0.99	0.11 (0.01)

NOTE: Results are based on 1,000 runs per combination of scenario and embedded simulation sizes. Means are shown with standard deviations in parentheses. The true values used to simulate data are Λ₁ = 12, σ₁ =1.5, p₂ = 0.15, λ₂ = 30, μ₃ = 9, and τ₃ = 1.5. Further details are provided in Section 4.

Across scenarios 1, 2, and 3, increasing m was associated with modest reductions in 95% CI length. Under scenario 1, increasing m from small to medium also decreased the variability of the 95% CI length. In most cases (30/32), the 95% CIs included the true values for at least 94% of the runs.

The overlap statistics of Garret and Zeger (GZ) were generally largest for parameters with largest percent bias and smallest for those with small bias. For example, under scenario 3, τ̂₃ had a percent bias > 130% and a GZ statistic > 0.80, indicating little difference between the prior and posterior distributions. When additional data were used for estimation (scenario 4), the percent bias of τ̂₃ dropped to 1.9%, and the mean GZ statistic was 0.11, indicating identifiability of τ₃ under this scenario. Similar patterns were seen for μ₃, λ₂, and, to a lesser extent, p₂. In all cases, Λ₁ appeared to be well informed by calibration data, whereas σ₁ appeared to be less well informed by the data and hence more reliant on the prior distribution. Simulated calibration data describes summaries across individuals and thus contain little information about the between-individual variability that σ₁ models. GZ statistics were most sensitive to the data used for calibration and were insensitive to the embedded simulation size, m. In this simulation study, the GZ statistics were not affected by the number of parameters estimated.

These simulations indicate that our proposed MCMCC approach can provide unbiased estimates of informed parameters with a finite number of draws and embedded simulations. They also demonstrate that GZ statistics are a useful measure of parameter identifiability.

5. Application of Markov Chain Monte Carlo Calibration to Our Colorectal Cancer Microsimulation Model

Parameters associated with our MSM for CRC are grouped into 4 categories corresponding to our 4 model components: adenoma risk, adenoma growth, cancer transition, and cancer detection (Table 1). Initially, each Gibbs iteration updated the 4 parameter vectors in random order, updating the adenoma risk and transition parameter vectors twice in each pass. This unequal sampling approach improved the mixing of the sampler, because the relatively high dimension of the adenoma risk parameter vector resulted in lower acceptance rates of the associated MH steps. After several thousand iterations, we switched to a single draw of all parameters using a covariance matrix for the random walk based on earlier Gibbs steps. This approach saved a significant number of computational steps required to estimate the data parameters, g(θ).

We ran 2 MCMC chains, beginning at different locations in the parameter space. We evaluated convergence using trace plots and the corrected potential scale-reduction factor based on the ratio of within- and between-chain variability (Brooks and Gelman 1998). Once estimation was complete, we assessed the identifiability of model parameters by comparing prior and posterior distributions visually and using the Garrett and Zeger (2000) overlap statistic, described in Section 4.5.

5.1 Data Likelihoods

We assumed that data from different sources and different individuals within sources are independent. We modeled prevalence data using a binomial distribution (Imperiale et al. 2000, 2002; Lieberman et al. 2000; Surveillance, Epidemiology, and End Results (SEER) Program (www.seer.cancer.gov) 2004; Strul et al. 2006). For the SEER data, we assumed that the number of clinical CRC cases in each age–sex–location group follow independent binomial distributions. For the Pickhardt et al. (2003) data, we modeled the total number of adenomas across all screened individuals using a Poisson distribution, and modeled the distribution of adenoma size given the total number of adenomas using a multinomial distribution. We modeled the number of preclinical cancers among adenomas grouped by size (Church 2004; Odom et al. 2005) using a multinomial distribution. Tables 2 and 3 note the number of embedded simulations, m, used to estimate data-likelihood parameters. We simulated 2,000,000 individuals to estimate SEER rates. Each embedded simulation corresponds to 1 simulated life history (i.e., 1 pseudoindividual). We chose the number of embedded simulations based on 3 criteria: observed rates, with larger m for rare events; our desire to obtain both point and interval estimates; and limitations on computation time.

When calibrating to colonoscopy data (Imperiale et al. 2000, 2002; Lieberman et al. 2000; Strul et al. 2006), we incorporated the accuracy of colonoscopy into the embedded simulations. We assumed a simple quadratic model for miss rates: P(miss|size = s < 20 mm) = 0.34 − 0.035s + 0.0009s² and P(miss|size = s ≥ 20 mm) = 0, producing miss rates consistent with observed findings (Hixson et al. 1990; Rex et al. 1997). For the study comparing virtual and optical colonoscopy (Pickhardt et al. 2003), we assumed that all adenomas were found, because individuals underwent both procedures and calibration data include adenomas found by either modality.

The parameters associated with data likelihoods, π(θ), depend on both MSM parameters and the age- and sex-distribution of the data modeled. For SEER data [Surveillance, Epidemiology, and End Results (SEER) Program (www.seer.cancer.gov) 2004], embedded simulations used the U.S. age and sex distribution in 1978. For other published data, we made assumptions about the age and sex distribution of study subjects. In general, studies described their samples by reporting the percentage of men and women, their average age, and the standard deviation of age. Unless information by sex was provided, we assumed the same age distribution for men and women. We modeled age distributions (used for embedded simulations) using a truncated Normal distribution based on the reported mean, standard deviation, and age range. We used a grid search to select a truncated Normal distribution with a mean and standard deviation closest to the observed values, based on a simple distance measure. For the adenoma case series data (Church 2004; Odom et al. 2005), no information on either age or sex was provided; for these 2 studies, we assumed an equal probability of being male and female, restricted individuals to be under age 90 years, and specified a truncated Normal distribution on age with a mean of 65 and standard deviation of 5.

5.2 Calibration Results

We began our MCMCC algorithm with 2 chains initiated at different locations in the parameter space. We ran these for 14,500 iterations with block sampling of parameters, followed by 75,000 additional iterations based on draws of the full parameter vector. Drawing the full parameter vector reduces computation due to the embedded simulation, but it requires a reasonable estimate of the posterior covariance matrix for MSM parameters, because this guides the direction of multidimensional random-walk steps. We evaluated convergence using visual assessment of trace plots and Gelman and Rubin statistics for multiple chains (Gelman and Rubin 1992). The chain was slowly mixing but appeared to converge, with Gelman and Rubin statistics ranging from 1.00 to 1.05 for all parameters. We based the estimation on 6,000 iterations resulting from systematically selecting every 25th iteration from the last 75,000 iterations from each chain. Because calculation of prediction intervals is computationally intensive, we based the predictive values on 3,000 iterations resulting from systematically selecting every 50th iteration.

Table 1 gives estimated posterior means, 95% CIs, and overlap statistics for all MSM parameters. The estimated posterior distribution of baseline adenoma risk was similar to our prior distribution. Posterior distributions for all other adenoma risk parameters were shifted from prior distributions. Adenoma growth means differed from the wide priors that we specifiedl however, posterior distributions for scale parameters were similar to the uniform distributions specified, indicating that the data provide little information about variation in adenoma growth. Posterior distributions for adenoma transition parameters were shifted from specified priors, except for posterior distributions associated with the transition of rectal adenomas in men. This highly parameterized transition component, with different parameters by sex and location, was required to obtain adequate fit to SEER cancer incidence data. Overlap statistics indicate that information about sojourn time came largely from the prior distributions, with the exception of mean sojourn time associated with colon cancers.

Table 7 gives estimated posterior means together with 95% predictive intervals (PIs) for the MSM model parameters. In general, the MSM demonstrated good prediction of calibration data, particularly for SEER cancer incidence rates. For the SEER cancer rates, estimates are similar to observed data, and PIs cover the observed values, except for rectal cancer rates in men age 20–49 (observed, 2.28 per 100,000; estimated, 3.23 per 100,000; 95% PI = 2.33–4.37). The predicted adenoma prevalence was somewhat high compared with that found by Strul et al. (2006), although predicted adenoma counts across individuals were similar to those reported by Pickhardt et al. (2003), suggesting that model fit might be improved by incorporating additional data describing the number of adenomas within individuals. Predicted adenoma sizes were close to observed data, but with more large adenomas than found by either Lieberman et al. (2000) or Pickhardt et al. (2003). Wide PIs demonstrate uncertainty in predicted transition probabilities. The model accurately predicted preclinical cancer rates in individuals, but predicted too many cancers in small adenomas and too few in large adenomas.

6. Discussion

We have proposed an MCMCC method based on a random-walk MH algorithm that uses an estimated transition rule. Our proposed approach is similar to an approximate MCMC method proposed by Christen and Fox (2005), who based their transition rule on an approximation to the data likelihood. In the scenario that those authors examined, computation of the likelihood was possible but expensive. When calibrating a complex MSM, the likelihood could not be calculated exactly, because the associated data parameters were unknown functions of MSM parameters. Our MCMCC method uses simulation to approximate these data parameters. Although MCMCC is computationally intensive, the price of computation is offset the advantages of MCMCC over previously proposed MSM calibration methods.

MCMCC provides 2 distinct avenues for incorporating information into the MSM: via prior distributions for MSM parameters and via calibration data. Specification of prior distributions formalizes the process of incorporating expert information into MSMs. Uniform priors can be used when little prior knowledge is available. Priors are updated using calibration data to obtain posterior estimates. Care must be taken when choosing priors for parameters that are not informed by calibration data, because there is little or no updating of the prior information. Overlap statistics, used to compare prior and posterior distributions, provide information on how resulting posterior distributions differ from prior beliefs. These overlap statistics also provide insight into parameter identifiability. This information is invaluable, because it can serve to temper findings from MSM studies, indicating areas in which variability in model parameters is especially important to consider. Results describing parameter identifiability also point out areas requiring further research to inform the disease processes.

Simultaneous calibration to multiple data sources via MCMCC is straightforward, and the algorithm implicitly weights calibration data based on sampling variability. MCMCC provides both point and interval estimates of model parameters, as well as functions of parameters, offering flexibility in the presentation of results. Calibration methods based on a grid search to obtain the best fit to data do not provide interval estimates. When applying frequentist maximum likelihood approaches, lack of model identifiability can hamper the ability to obtain interval estimates.

We have not discussed estimation based on direct Monte Carlo simulation of the likelihood, which could be implemented using embedded simulation to obtain estimates of data-likelihood parameters used in calculating likelihoods. Such a Monte Carlo approach could provide maximum likelihood estimates of MSM parameters. But because the functional form of the likelihood is not readily available, the information matrix also is not readily available, and thus Monte Carlo estimates cannot directly provide measures of precision. In any case, asymptotic precision likely is not relevant for MSM estimation when only limited calibration data are available.

The accuracy of MSM models depends on appropriate modeling of disease processes. Between-model comparisons (Berry et al. 2005) offer one approach to model assessment. Model checking based on both calibration and validation data also provides information on model adequacy. How closely the model should calibrate to observed data is unclear, especially when calibration data are variable and may provide conflicting information. Prediction intervals from our MSM cover 15 of the 16 SEER cancer rates considered to be key calibration points, as well as 12 of 14 other calibration data points. Is this enough? It depends on how modelers trade-off concerns about possibly overparameterizing and overfitting calibration data relative to the importance of exactly replicating observed or expected results. We lean toward model parsimony. For complex disease processes, even a relatively parsimonious model may require the inclusion of parameters that are not well informed by available data.

The data available to inform MSMs often span decades, in this case ranging from SEER cancer incidence data from 1975–1979 to colonoscopy data collected in 2002–2003. Much of the information needed to calibrate the model parameters became available only after the advent of colonoscopy. We used all available data and assumed no secular trends in MSM parameters. Other assumptions are possible; for example, we could make stronger prior assumptions about model parameters and restrict our calibration data to specific date ranges, or could make assumptions about secular trends in adenoma parameters and use all available data.

Once MSM parameters are calibrated to reproduce known or expected results, MSMs may be used to examine the impact of potential interventions, including the cost-effectiveness of interventions (Ness et al. 2000; Will et al. 2001; Vogelaar et al. 2006). These results may in turn influence public policy decisions. Because of their potential policy impact, MSM assumptions and calibration methods should be clearly described to allow for their critical evaluation. Ideally, point estimates of effects should be accompanied by measures of precision so that policy makers can assess the strength of evidence models provide.

Appendix: Convergence of the Approximate Markov Chain Monte Carlo

Here we show that our proposed MCMCC approach converges to the target distribution, h(θ|y), by showing that the detailed balance condition holds asymptotically, as m → ∞, when ĝ(θ) is a maximum likelihood estimator. The detailed balance condition is given by h(θ|y)α(θ, θ*) = h(θ*|y)α(θ*, θ). Detailed balance of a Markov chain is a sufficient, but not necessary, condition for reversibility and convergence to h(θ|y) (Casella and Robert 1999). Detailed balance for the MH algorithm is easy to confirm when g(θ) is known.

When r is estimated, the probability of transition from θ to θ*, unconditional on r̂, equals P(r̂ > 1) + E(r̂|r̂ ≤ 1)P(r̂ ≤ 1), where the expectation results from integration over the distribution of r̂. Note that E(α̂|r̂ > 1) = 1 and E(α̂|r̂ ≤ 1) = E(r̂|r̂ ≤ 1). When r is estimated, the asymptotic behavior of the MCMC depends on the distribution of the estimators, r̂ and 1/r̂. Next, we show that r̂ and 1/r̂ are asymptotically Normally distributed.

Let G = (g(θ), g(θ*)) and Ĝ_m = (ĝ_m(θ), ĝ_m(θ*)). Assume that θ and θ* are fixed but arbitrary, and given θ and θ*, ĝ_m(θ) and ĝ_m(θ*) are independent. As m → ∞, $\sqrt{\mathit{\text{m}}}$, Ĝ_m converges in law to Normal(G, V), where V is a diagonal 2 × 2 matrix with diagonal terms v₁₁ = var(y|g(θ)) and v₂₂ = var(y|g(θ*)). If ṙ(G) = (∂r/∂g_i(θ), ∂r/∂g_i(θ*)) is continuous in a neighborhood of G then, given ĝ_m(θ) and ĝ_m(θ*), the sequence $\sqrt{m}r({\widehat{G}}_m)$ converges in law to a Normal distribution centered at r(G), with variance

$$\begin{array}{c}{r}^2[{\left(\frac{\partial f(y|g(\theta ))/\partial g(\theta )}{f(y|g(\theta ))}\right)}^2\text{var}(y|g(\theta ))\hfill \\ +{\left(\frac{\partial f(y|g({\theta }^{\ast }))/\partial g({\theta }^{\ast })}{f(y|g({\theta }^{\ast }))}\right)}^2\text{var}(y|g({\theta }^{\ast }))]\hfill \end{array}$$

(Ferguson 1996). Similarly, for large m, 1/r(Ĝ_m) is approximately Normally distributed with mean 1/r(G).

Given the asymptotic Normality of r̂ and 1/r̂, we can show that detailed balance holds for MCMCC. Suppose that r ≤ 1; then

$$\begin{array}{c}h(\theta |y)E(\widehat{\alpha }(\theta ,{\theta }^{\ast })|r\le 1)\hfill \\ =h(\theta |y)[P(\widehat{r}>1|r\le 1)+P(\widehat{r}\le 1|r\le 1)E(\widehat{r}|r\le 1\&\widehat{r}\le 1)].\hfill \end{array}$$

As m → ∞, E(r̂|r ≤ 1 & r̂ ≤ 1) → r. Using $h(\theta |y)=\frac{1}{r}h({\theta }^{\ast }|y)$, it is easy to show that

$$h(\theta |y)E(\widehat{\alpha }(\theta ,{\theta }^{\ast })|r\le 1)=h({\theta }^{\ast }|y)\left[1+\left(\frac{1}{r}-1\right)P(\widehat{r}>1|r\le 1)\right].$$

For fixed r such that 0 < r + ε < 1 for some ε > 0,

$$\underset{m\to \infty }{\text{lim}}\left(\frac{1}{r}-1\right)P(\widehat{r}>1|r\le 1)=0.$$

At the upper boundary,

$$\underset{r\to 1}{\text{lim}}\left(\frac{1}{r}-1\right)P(\widehat{r}>1|r\le 1)=0,$$

because ${\text{lim}}_{r\to 1}(\frac{1}{r}-1)\to 0$ and lim_r→1 P(r̂ > 1|r ≤ 1) → 0.5. At the lower boundary,

$$\underset{r\to 0}{\text{lim}}\left(\frac{1}{r}-1\right)P(\widehat{r}>1|r\le 1)=0,$$

by l'Hôpital's rule. Thus, for m → ∞, h(θ|y)E(α̂(θ, θ*)|r ≤ 1) → h(θ*|y) for 0 ≤ r ≤ 1.

Similarly,

$$\begin{array}{c}h({\theta }^{\ast }|y)E(\widehat{\alpha }({\theta }^{\ast },\theta )|r\le 1)\hfill \\ =h({\theta }^{\ast }|y)[P\left(\frac{1}{\widehat{r}}\ge 1|\frac{1}{r}\ge 1\right)\hfill \\ +P\left(\frac{1}{\widehat{r}}<1|\frac{1}{r}\ge 1\right)E\left(\frac{1}{\widehat{r}}|\frac{1}{\widehat{r}}<1\&\frac{1}{r}\ge 1\right)].\hfill \end{array}$$

As m → ∞, $E(\frac{1}{\widehat{r}}|\frac{1}{\widehat{r}}<1\&\frac{1}{r}\ge 1)\to 1$, so that

$$\begin{array}{c}\underset{m\to \infty }{\text{lim}}E(\widehat{\alpha }({\theta }^{\ast },\theta )|r\le 1)\hfill \\ =\underset{m\to \infty }{\text{lim}}[P({\widehat{r}}^{-1}\ge 1|r^{-1}\ge 1)+P({\widehat{r}}^{-1}<1|r^{-1}\ge 1)]=1\hfill \end{array}$$

and lim_{m → ∞} E(α̂(θ*, θ)|r ≤ 1) = h(θ*|y). Thus, when r ≤ 1, as m → ∞, h(θ|y)E(α̂(θ, θ*)|r ≤ 1) = h(θ*|y)E(α̂(θ*, θ)|r ≤ 1) and detailed balance holds. Similar arguments show that detailed balance holds when r ≥ 1.