Accuracy of MSI testing in predicting germline mutations of MSH2 and MLH1: a case study in Bayesian meta-analysis of diagnostic tests without a gold standard

SUMMARY

Microsatellite instability (MSI) testing is a common screening procedure used to identify families that may harbor mutations of a mismatch repair (MMR) gene and therefore may be at high risk for hereditary colorectal cancer. A reliable estimate of sensitivity and specificity of MSI for detecting germline mutations of MMR genes is critical in genetic counseling and colorectal cancer prevention. Several studies published results of both MSI and mutation analysis on the same subjects. In this article we perform a meta-analysis of these studies and obtain estimates that can be directly used in counseling and screening. In particular, we estimate the sensitivity of MSI for detecting mutations of MSH2 and MLH1 to be 0.81 (0.73–0.89). Statistically, challenges arise from the following: (a) traditional mutation analysis methods used in these studies cannot be considered a gold standard for the identification of mutations; (b) studies are heterogeneous in both the design and the populations considered; and (c) studies may include different patterns of missing data resulting from partial testing of the populations sampled. We address these challenges in the context of a Bayesian meta-analytic implementation of the Hui–Walter design, tailored to account for various forms of incomplete data. Posterior inference is handled via a Gibbs sampler.

1. INTRODUCTION

Hereditary nonpolyposis colorectal cancer (HNPCC) (Lynch and de la Chapelle, 1999) is the most common hereditary cancer syndrome. It can be caused by a deleterious germline mutation of one of the mismatch repair (MMR) genes, mainly MSH2 and MLH1. These mutations confer substantially increased lifetime risk of cancer of the colorectum and other sites (Vasen et al., 2001; Lin et al., 1998). Microsatellite instability (MSI) is the amplification or deletion within DNA microsatellites (tandemly repeated sequences that appear throughout the genome). This phenotype is caused by a general failure of the DNA MMR system. All HNPCC cases, i.e. cases with inherited defects in the MMR genes, have a strong tendency to develop tumors with this phenotype. For a few years since MSI was discovered in association with HNPCC (Thibodeau et al., 1993), this phenomenon was not uniformly characterized. A variety of laboratories developed their own tools for measuring MSI, and related studies indicated that many markers could be used to distinguish MSI tumors from microsatellite stable (MSS) ones. In 1998, ‘The International Workshop on Microsatellite Instability and RER Phenotypes in Cancer Detection and Familial Predisposition’ was held to develop a concensus definition of MSI and to establish unifying criteria for its measurement (Boland et al., 1998). As a result, the following five markers are now used for the identification of MSI tumors: BAT26, BAT25, D5S346, D2S123, and D17S250. Instability in more than two of the five markers indicates an MSI tumor. To date, MSI is found in more than 70% of HNPCC-related cancers (Liu et al., 2000) and up to 10–20% of sporadic colorectal cancers (Salovaara et al., 2000; Lothe et al., 1993). MSI testing is aimed at detecting MMR deficiencies and has become widely used as a prescreening procedure for possible HNPCC germline mutations, thanks to the simpler and less expensive technique compared to most mutation analyses (Boland et al., 1998).

A reliable estimate of sensitivity and specificity of MSI for detecting germline mutations of MMR genes is critical in genetic counseling and colorectal cancer prevention. In particular, knowing one’s probability of carrying an inherited deleterious mutation on the MSH2 and MLH1 genes prior to genetic testing by mutation analysis is important for decision making about genetic testing, disease prophylaxis, family planning, and more. As many colorectal cancer patients undergo MSI testing before mutation analysis, pretest carrier probabilities need to incorporate information about MSI testing, as done in commonly used prediction software such as BayesMendel (Chen et al., 2004). This requires estimates of the sensitivity of MSI, defined as the probability of a subject’s tumor sample being microsatellite unstable (MSI = 1) given he/she is carrying a deleterious germline mutation of MSH2 or MLH1 (Mut = 1), and of the specificity of MSI, defined as the probability of the subject’s tumor sample being MSS (MSI = 0) given he/she is not carrying a mutation (Mut = 0). Also, accurate estimates of the test properties along with related costs are determinants in designing screening and surveillance programs for colorectal cancer (Ramsey et al., 2001; Reyes et al., 2002).

Several investigations have reported testing a set of tumors both for germline mutations and for MSI, and then comparing the results. Among these, studies using tumors from high-risk colorectal cancer cases have relatively small sample sizes. Yet accurate risk prediction and genetic counseling require a reliable estimate of MSI sensitivity and specificity. In this article we perform a meta-analysis of these studies and obtain estimates of sensitivity and specificity as defined above. Statistically, challenges arise from the following: (a) traditional mutation analysis methods used in published studies cannot be considered a gold standard for the identification of mutations (Yan et al., 2000); (b) studies are heterogeneous in both the design and the populations considered; and (c) studies may include different patterns of missing data resulting from partial testing of the populations sampled. We address these challenges using a Bayesian meta-analysis motivated by the Hui–Walter approach (Hui and Walter, 1980). This approach exploits the availability of multiple studies that sample populations with different mutation prevalences, to obtain estimates of sensitivity and specificity of two tests without requiring either one to be a gold standard. We develop specific adaptations designed to accommodate incomplete data and heterogeneity in the experimental designs of the available studies. Posterior inference is handled via a Gibbs sampler.

2. DATA AND METHODS

For our meta-analysis, we considered published studies that involved testing colorectal cancer tumor tissues for MSI and/or germline mutations. We initially identified a number of such studies by interviewing experts, searching on PUBMED, searching reference sets of previously identified papers, and searching citation indices. Not all studies with such results could be included in the analysis. The reason for exclusion and a more detailed description of each study are provided in the Appendix. We then selected studies with clearly reported test results. Examining the results of the literature search, the following issues arose.

The first issue is the absence of a gold standard. In the published studies, tumor samples were tested with a variety of mutation analysis strategies including direct sequencing alone, denaturing gradient gel electrophoresis (DGGE) alone, single-strand conformation analysis (SSCA) alone, polymerase chain reaction (PCR) + SSCA + heteroduplex analysis (HA), DGGE + reverse transcriptase-polymerase chain reaction (RT-PCR) + protein truncation test (PTT), and many others. We discuss the handling of these testing strategies in Section 4. For now, we refer to these tests collectively as ‘mutation analysis’. Studies have demonstrated that these conventional mutation testing strategies are not sensitive to certain complex mutations (Wang et al., 2001; Sophie et al., 2001), or large genomic deletions and rearrangements (Yan et al., 2000; Payne et al., 2000; Nakagawa et al., 2002; Wang et al., 2003; Renkonen et al., 2003). The mutations that are likely to be missed by conventional methods make up a significant fraction of all mutations related to colorectal cancers. Because of this limitation, in these studies, the true genotype is not always observed. Using germline testing results as the gold standard can lead to biased estimates of MSI sensitivity and specificity.

The second issue is population heterogeneity both between and within studies. Several studies we included tested families that can be categorized into different groups according to certain clinical criteria. It is likely that the prevalence of deleterious mutation differs from group to group, but the authors often chose not to report the results by groups but rather gave aggregate results. Meanwhile, other studies may focus on a single more homogeneous population, but the populations may be different across studies. Overall, this scenario is a challenge but also a strength because the existence of such strata allows us to account for the lack of a gold standard, as we will discuss later.

The third issue is incompleteness in the data. An ideal study is one where the same set of subjects are tested both for MSI and germline mutations. However, not all studies have complete data in this sense. In Terdiman et al. (2002), Wang et al. (2003), Loukola et al. (1999a), and Salovaara et al. (2000), it was expected that the yield of mutations would be low among MSI-negative tumors and only those samples that tested positive for MSI were genotyped. In Cederquist et al. (2001), only 23 out of the 35 MSI positives were genotyped. Ponz de Leon et al. (2004) only gave mutation analysis results, whereas Percesepe et al. (2001), Salahshor et al. (1999), and Dietmaier et al. (1997) only gave MSI results. While the data in these studies are incomplete, they may provide important information on MSI accuracy. Therefore, the inclusion of such data was judged to be worth the additional modeling effort required.

In our analysis, we stratified the study populations into a high-risk group and a low-risk group. The high-risk group includes families that fulfilled the Amsterdam criteria, or met any modified clinical guidelines for the purpose of identifying HNPCC families. In addition, we have included in the high-risk group all families and individual cases that can be classified as being at significantly elevated risk, based on descriptors, such as ‘with age of onset before 35’, ‘with two or more close relatives with colorectal cancer’, or ‘exhibiting familial clustering of colorectal cancer (strong and positive family history)’. The low-risk group includes cases from population-based registries. The grouping we use here attempts, within the limits imposed by the information available, to distinguish between data gathered in high-risk clinical settings, most of which is self-referred, and data gathered using population-based strategies. While our approach would have been applicable, and potentially more efficient, to a larger set of ordered categories of risk, we stopped at two because several studies tested a possibly heterogeneous sample and only reported aggregate results, and to assure that each group has a reasonable sample size. The stratified data are shown in Table 1.

Table 1.

A list of the studies included. Data are stratified into a high-risk group in the top portion and a low-risk group in the bottom portion. In each row, data are summarized in four columns by MSI test results (MSI = 0 or 1) and mutation analysis results (Mut = 0 or 1). Missing counts due to samples without one of the two tests are represented by x_. and y_.. For example, x3 stands for the potential number of mutation analysis positive subjects in Ponz de Leon et al. (2004) who would have been tested positive for MSI. Among the MSI-positive group of 35 in Cederquist et al. (2001), mutation analysis was performed on 23 samples among which eight were found to harbor germline mutations. No mutation analysis was performed on the other 12 samples. x₁ represents the potential number of mutation positives among these 12 untested subjects

Study	High risk	Low risk	MSI = 1, Mut = 1	MSI = 1, Mut = 0	MSI = 0, Mut = 1	MSI = 0, Mut = 0
Bapat et al. (1999)	•		16	1	2	20
Calistri et al. (2000)	•		8	8	0	9
Debniak et al. (2000)	•		5	4	1	15
Dieumegard et al. (2000)	•		7	7	0	2
Lamberti et al. (1999)	•		13	22	2	10
Liu et al. (2000)	•		16	6	1	36
Scartozzi et al. (2002)	•		0	1	4	22
Wahlberg et al. (2002)	•		14	14	0	20
Cederquist et al. (2001)	•		8 + x1	15 + (12 − x1)	x2	43 − x2
Ponz de Leon et al. (2004)	•		x3	x4	89 − x3	75 − x4
Terdiman et al. (2001)	•		21	11	x5	63 − x5
Wang et al. (2003)	•		92	88	x6	188 − x6
Debniak et al. (2000)		•	0	5	0	38
Dieumegard et al. (2000)		•	0	0	0	7
Loukola et al. (1999)		•	10	53	y1	446 − y1
Percesepe et al. (2001)		•	y2	28 − y2	y3	308 − y3
Salahshor et al. (1999)		•	y4	22 − y4	y5	159 − y5
Dietmaier et al. (1997)		•	y6	18 − y6	y7	130 − y7
Salovaara et al. (2000)		•	18	48	y8	469 − y8

If all missing counts in Table 1 were observed, then the stratified data would fit into the framework of a Hui–Walter design (Hui and Walter, 1980). In this design, the sensitivity and specificity of MSI testing can be estimated without ascertaining the true genetic status of the subjects, by exploiting the fact that the strata that are known to have different prevalences. Data consist of a three-way tabulation of the results of two tests and the stratum indicator. The Hui–Walter approach relies on two key modeling assumptions: independence of the testing modalities and different prevalences among the high- and low-risk groups (Parmigiani, 2002; Johnson et al., 2001). Further discussion of these assumptions is in Section 4.

To write the likelihood function, we define the multinomial probabilities of an individual falling into the four cells MSI = 1 & Mut = 1, MSI = 1 & Mut = 0, MSI = 0 & Mut = 1, and MSI = 0 & Mut = 0 as a, b, c, and d, respectively. Subscripts ‘h’ and ‘l’ will denote the high-risk group and low-risk group, respectively, while subscript + and - will denote true carrier status. For example, a_h+ are MSI positive, mutation analysis positive, and true carriers in the high-risk group. We denote the sensitivity and specificity of MSI testing by β_M and α_M, those of the mutation analysis by β_T and α_T, and the mutation prevalences in the high-risk and low-risk groups by π_h and π_l. Then, using the independence assumption for the two testing modalities, we can write for the high-risk group,

$$\begin{array}{c}{a}_{\mathrm{h}}\equiv a_{\mathrm{h}+}+a_{\mathrm{h}-}={\pi }_{\mathrm{h}}{\beta }_{\mathrm{M}}{\beta }_{\mathrm{T}}+(1-{\pi }_{\mathrm{h}})(1-{\alpha }_{\mathrm{M}})(1-{\alpha }_{\mathrm{T}}),\\ b_{\mathrm{h}}\equiv b_{\mathrm{h}+}+b_{\mathrm{h}-}={\pi }_{\mathrm{h}}{\beta }_{\mathrm{M}}(1-{\beta }_{\mathrm{T}})+(1-{\pi }_{\mathrm{h}})(1-{\alpha }_{\mathrm{M}}){\alpha }_{\mathrm{T}},\\ c_{\mathrm{h}}\equiv c_{\mathrm{h}+}+c_{\mathrm{h}-}={\pi }_{\mathrm{h}}(1-{\beta }_{\mathrm{M}}){\beta }_{\mathrm{T}}+(1-{\pi }_{\mathrm{h}}){\alpha }_{\mathrm{M}}(1-{\alpha }_{\mathrm{T}}),\\ d_{\mathrm{h}}\equiv d_{\mathrm{h}+}+d_{\mathrm{h}-}={\pi }_{\mathrm{h}}(1-{\beta }_{\mathrm{M}})+(1-{\beta }_{\mathrm{T}})+(1-{\pi }_{\mathrm{h}}){\alpha }_{\mathrm{M}}{\alpha }_{\mathrm{T}}.\end{array}$$

Similar relationships apply to the low-risk group. We also use θ to denote the parameter vector (α_T, α_M, β_T, β_M, π_h, π_l) and O to denote the totality of observed variables in Table 1.

Using this notation, we could write the contribution from each study to the likelihood function, depending on the stratum and missing data pattern. For example, the contribution from Bapat et al. (1999) was

$$a_{\mathrm{h}}^{16}{b}_{\mathrm{h}}^1{c}_{\mathrm{h}}^2{d}_{\mathrm{h}}^{20},$$

that from Terdiman et al. (2001) is

$$a_{\mathrm{h}}^{21}{b}_{\mathrm{h}}^{11}{(c_{\mathrm{h}}+d_{\mathrm{h}})}^{63},$$

and that from Percesepe et al. (2001) is

$${(a_{\mathrm{l}}+b_{\mathrm{l}})}^{28}{(c_{\mathrm{l}}+d_{\mathrm{l}})}^{308}.$$

Assuming independence of study results conditional on θ, the likelihood is the product of the contribution from each study, that is,

$$\begin{array}{ll}L(\theta |O)=\hfill & \{a_{\mathrm{h}}^{79}{b}_{\mathrm{h}}^{63}{c}_{\mathrm{h}}^{10}{d}_{\mathrm{h}}^{134}\}\{a_{\mathrm{h}}^8{b}_{\mathrm{h}}^{15}{(a_{\mathrm{h}}+b_{\mathrm{h}})}^{12}{(c_{\mathrm{h}}+d_{\mathrm{h}})}^{45}\}\{{(a_{\mathrm{h}}+c_{\mathrm{h}})}^{89}{(b_{\mathrm{h}}+d_{\mathrm{h}})}^{75}\}\hfill \\ \hfill & \times \{a_{\mathrm{h}}^{21}{b}_{\mathrm{h}}^{11}{(c_{\mathrm{h}}+d_{\mathrm{h}})}^{63}\}\{a_{\mathrm{h}}^{92}{b}_{\mathrm{h}}^{88}{(c_{\mathrm{h}}+d_{\mathrm{h}})}^{188}\}\{b_{\mathrm{l}}^5{d}_{\mathrm{l}}^{38}\}{d}_{\mathrm{l}}^7\{a_{\mathrm{l}}^{10}{b}_{\mathrm{l}}^{53}{(c_{\mathrm{l}}+d_{\mathrm{l}})}^{446}\}\hfill \\ \hfill & \times \{{(a_{\mathrm{l}}+b_{\mathrm{l}})}^{28}{(c_{\mathrm{l}}+d_{\mathrm{l}})}^{308}\}\{{(a_{\mathrm{l}}+d_{\mathrm{l}})}^{22}{(c_{\mathrm{l}}+d_{\mathrm{l}})}^{159}\}\hfill \\ \hfill & \times \{{(a_{\mathrm{l}}+b_{\mathrm{l}})}^{18}{(c_{\mathrm{l}}+d_{\mathrm{l}})}^{130}\}\{a_{\mathrm{l}}^{18}{b}_{\mathrm{l}}^{48}{(c_{\mathrm{l}}+d_{\mathrm{l}})}^{469}\}\hfill \\ \hfill =& a_{\mathrm{h}}^{200}{b}_{\mathrm{h}}^{177}{c}_{\mathrm{h}}^{10}{d}_{\mathrm{h}}^{134}{(a_{\mathrm{h}}+b_{\mathrm{h}})}^{12}{(c_{\mathrm{h}}+d_{\mathrm{h}})}^{294}{(a_{\mathrm{h}}+c_{\mathrm{h}})}^{89}{(b_{\mathrm{h}}+d_{\mathrm{h}})}^{75}\hfill \\ \hfill & \times a_{\mathrm{l}}^{28}{b}_{\mathrm{l}}^{106}{d}_{\mathrm{l}}^{45}{(a_{\mathrm{l}}+b_{\mathrm{l}})}^{68}{(c_{\mathrm{l}}+d_{\mathrm{l}})}^{1512},\hfill \end{array}$$ (2.1)

where each bracketed expression corresponds to a study. Under our modeling assumptions, all six parameters β_M, α_M, β_T, α_T, π_h, and π_l are identified, as long as the constraint π_h > π_l holds.

We drew inferences about θ via a Bayesian analysis. Advantages include the possibility to derive exact probability intervals without relying on asymptotic approximations, and ease of handling both the missing data patterns and the parametric constraint required for identifiability. Within this constraint, we adopted a uniform prior on all components of θ, with the exception of the specificity α_T of mutation analysis, which we assumed to be 100%, a value likely to be close to the true specificity. However, as a sensitivity analysis, we also reported results using a uniform prior on α_T, the opposite extreme in term of informativeness.

From Bayes theorem, the posterior is proportional to expression (2.1) wherever the prior is nonzero. Computation was simplified by introducing two sets of auxiliary variables: the counts of test results for subjects who have not been tested by both tests, and the counts of true genotypes for all subjects. Denoting auxiliary variables by A, we implemented a Markov chain Monte Carlo algorithm to draw from the distribution of (A, θ) given O (Tanner and Wong, 1987).

Our algorithm requires an expression for the so-called complete data likelihood, that is, p(A, O|θ). This is given by

$$\begin{array}{c}{a}_{\mathrm{h}+}^{n_{a,\mathrm{h}+}}·a_{\mathrm{h}-}^{n_{a,\mathrm{h}-}}·b_{\mathrm{h}+}^{n_{b,\mathrm{h}+}}·b_{\mathrm{h}-}^{n_{b,\mathrm{h}-}}·c_{\mathrm{h}+}^{n_{c,\mathrm{h}+}}·c_{\mathrm{h}-}^{n_{c,\mathrm{h}-}}·d_{\mathrm{h}+}^{n_{d,\mathrm{h}+}}·d_{\mathrm{h}-}^{n_{d,\mathrm{h}-}}\\ ·a_{\mathrm{l}+}^{n_{a,\mathrm{l}+}}·a_{\mathrm{l}-}^{n_{a,\mathrm{l}-}}·b_{\mathrm{l}+}^{n_{b,\mathrm{l}+}}·b_{\mathrm{l}-}^{n_{b,\mathrm{l}-}}·c_{\mathrm{l}+}^{n_{c,\mathrm{l}+}}·c_{\mathrm{l}-}^{n_{c,\mathrm{l}-}}·d_{\mathrm{l}+}^{n_{d,\mathrm{l}+}}·d_{\mathrm{l}-}^{n_{d,\mathrm{l}-}}.\end{array}$$ (2.2)

Each exponent in (2.2) arises as the sum of contributions from studies with different missing data structures, which needed to be simulated separately, requiring additional auxiliary variables defined in Table 2. To explain the significance of the auxiliary variables, for example, we observe in the high-risk group that 200 subjects are MSI positive and mutation analysis positive (MSI+ & Mut+) and 177 are MSI+ & Mut−. This corresponds to the first two factors in the expression (2.1), $a_{\mathrm{h}}^{200}{b}_{\mathrm{h}}^{177}$. After we assume the carrier status known, among the 200 MSI+ & Mut+ subjects, $n_{a,\mathrm{h}+}^{(1)}$ would be the true carriers and $n_{a,\mathrm{h}-}^{(1)}=200-n_{a,\mathrm{h}+}^{(1)}$ the true noncarriers. Similarly, among the 177 MSI+ & Mut− subjects, $n_{b,\mathrm{h}+}^{(1)}$ would be true carriers and $n_{b,\mathrm{h}-}^{(1)}$ noncarriers. Thus, the corresponding factors in the complete data likelihood were written as $a_{\mathrm{h}+}^{n_{a,\mathrm{h}+}^{(1)}}{a}_{\mathrm{h}-}^{n_{a,\mathrm{h}-}^{(1)}}{b}_{\mathrm{h}+}^{n_{b,\mathrm{h}+}^{(1)}}{b}_{\mathrm{h}-}^{n_{b,\mathrm{h}-}^{(1)}}$ with the constraints $n_{a,\mathrm{h}+}^{(1)}+n_{a,\mathrm{h}-}^{(1)}=200\text{and}{n}_{b,\mathrm{h}+}^{(1)}+n_{b,\mathrm{h}-}^{(1)}=177$. In other words, $a_{\mathrm{h}}^{200}{b}_{\mathrm{h}}^{177}$ is the result of integrating out the auxiliary variables $n_{a,\mathrm{h}+}^{(1)},n_{a,\mathrm{h}-}^{(1)},n_{b,\mathrm{h}+}^{(1)},n_{b,\mathrm{h}-}^{(1)}\text{from}{a}_{\mathrm{h}+}^{n_{a,\mathrm{h}+}^{(1)}}{a}_{\mathrm{h}-}^{n_{a,\mathrm{h}-}^{(1)}}{b}_{\mathrm{h}+}^{n_{b,\mathrm{h}+}^{(1)}}{b}_{\mathrm{h}-}^{n_{b,\mathrm{h}-}^{(1)}}$ within the constraints. Details of the Gibbs sampler and the various constraints placed on the components are available from the first author.

Table 2.

Notation: the number in each cell can be broken down into those with a true germline mutation (subscript ‘+’) and those without (subscript ‘−’). High risk and low risk are indicated by the letter ‘h’ and ‘l’, respectively, in the subscript. The superscripts in parentheses signify contributions from studies with different missing data structures

	Mut = 1	Mut = 0
High risk
MSI = 1	$n_{a,\mathrm{h}+}=n_{a,\mathrm{h}+}^{(1)}+n_{a,\mathrm{h}+}^{(2)}+n_{a,\mathrm{h}+}^{(3)}$	$n_{b,\mathrm{h}+}=n_{b,\mathrm{h}+}^{(1)}+n_{b,\mathrm{h}+}^{(2)}+n_{b,\mathrm{h}+}^{(3)}$
	$n_{a,\mathrm{h}-}=n_{a,\mathrm{h}-}^{(1)}+n_{a,\mathrm{h}-}^{(2)}+n_{a,\mathrm{h}-}^{(3)}$	$n_{b,\mathrm{h}-}=n_{b,\mathrm{h}-}^{(1)}+n_{d,\mathrm{h}-}^{(2)}+n_{b,\mathrm{h}-}^{(3)}$
MSI = 0	$n_{c,\mathrm{h}+}=n_{c,\mathrm{h}+}^{(1)}+n_{c,\mathrm{h}+}^{(2)}+n_{c,\mathrm{h}+}^{(3)}$	$n_{d,\mathrm{h}+}=n_{d,\mathrm{h}+}^{(1)}+n_{d,\mathrm{h}+}^{(2)}+n_{d,\mathrm{h}+}^{(3)}$
	$n_{c,\mathrm{h}-}=n_{c,\mathrm{h}-}^{(1)}+n_{c,\mathrm{h}-}^{(2)}+n_{d,\mathrm{h}-}^{(3)}$	$n_{d,\mathrm{h}-}=n_{d,\mathrm{h}-}^{(1)}+n_{d,\mathrm{h}-}^{(2)}+n_{d,\mathrm{h}-}^{(3)}$
Low risk
MSI = 1	$n_{a,\mathrm{l}+}=n_{a,\mathrm{l}+}^{(1)}+n_{a,\mathrm{l}+}^{(2)}$	$n_{b,\mathrm{l}+}=n_{b,\mathrm{l}+}^{(1)}+n_{b,\mathrm{l}+}^{(2)}$
	$n_{a,\mathrm{l}-}=n_{a,\mathrm{l}-}^{(1)}+n_{a,\mathrm{l}-}^{(2)}$	$n_{b,\mathrm{l}-}=n_{b,\mathrm{l}-}^{(1)}+n_{d,\mathrm{l}-}^{(2)}$
MSI = 0	nc,l+ = nc,l+	$n_{d,\mathrm{l}+}=n_{d,\mathrm{l}+}^{(1)}+n_{d,\mathrm{l}+}^{(2)}$
	nc,l− = nc,l−	$n_{d,\mathrm{l}-}=n_{d,\mathrm{l}+}^{(1)}+n_{d,\mathrm{l}-}^{(2)}$

3. RESULTS

Posterior means and 95% credible intervals (CIs) on the parameteres of interest are shown in Table 3. The estimated specificity of MSI testing was 92% (95% CI, 90–94%), indicating that MSI is a more specific test than previously suggested (Salovaara et al., 2000; Lothe et al., 1993). Combined with an estimated sensitivity of 81%, our estimates translate into a positive predictive value (PPV) of 0.93 (95% CI, 0.90–0.95) among high-risk families. However, the PPV of MSI is drastically lower among unselected colorectal cancer cases: we estimate it to be 0.34 (95% CI, 0.24–0.48). Both PPV estimates are consistent with practical clinical experiences in MSI screening. Our prevalence estimate of true carriers among unselected colorectal cancer, 3–7%, coincides with previous estimates (Aaltonen et al., 1994, 1998).

Table 3.

Summary of inferences on parameters. The left part of the table is obtained by setting α_T to 1. The right part is obtained by placing a uniform prior on α_T

		αT = 1		αT ~ u(0, 1)
		Mean	95% PI	Mean	95% PI
MSI specificity	αM	0.92	(0.90, 0.94)	0.92	(0.90, 0.93)
MSI sensitivity	βM	0.81	(0.73, 0.89)	0.85	(0.74, 0.96)
Mutation analysis specificity	αT	1		0.97	(0.91, 1.0)
Mutation analysis sensitivity	βT	0.62	(0.56, 0.67)	0.62	(0.57, 0.68)
Prevalence in high-risk group	πh	0.56	(0.50, 0.63)	0.53	(0.46, 0.61)
Prevalence in low-risk group	πl	0.049	(0.032, 0.071)	0.042	(0.022, 0.064)

We estimated the sensitivity of mutation analysis to be 62%. While this is a modest value, our estimates are comparable to those reported for similar mutation analysis techniques for other large cancer susceptibility genes. For example, Ford et al. (1998) reported an estimated sensitivity of 63% (51–77%) for direct sequencing in detecting mutations on BRCA1. They also find that the sensitivity for other mutation screening techniques is not significantly different from that of direct sequencing. While clinical use of sequencing-based testing is still justified in view of the much higher specificity of sequencing compared to MSI testing, our results further emphasize the need for cautious interpretation of negative test results in families with a large number of colorectal cancer cases (Yan et al., 2000).

At each iteration of the Gibbs sampler, we generated a draw of all entries in Table 2. These draws can be used to make inferences on the missing counts in Table 1. For example, in the Ponz de Leon et al. (2004) study, x₃ was imputed as 72 on average by the Gibbs sampler, which means that 72 out of the 89 mutation-positive subjects in the study would have been tested positive for MSI. In addition, x₁ was on average 7 and x₄ was on average 24. The other missing counts could be estimated up to a sum. The quantities x₂ + x₅ + x₆, y₂ + y₄ + y₆, and y₁ + y₃ + y₅ + y₇ + y₈ had posterior means around 38, 14, and 10, respectively.

With the Gibbs sampler, in addition to inferences on missing portions of the original studies, we also obtained inferences on the true genotypes for all subjects. Computing the total of each of the rightmost four columns of Table 1, and then breaking them down using imputed carrier status results, we produced Table 4. From Table 4 we can see that although MSI is quite specific, because of the low prevalence of mutations in the low-risk population, there will still be a significant portion of false positives (an estimated 133 out of 202).

Table 4.

The data table broken down by imputed genotype. The top table is imputated under the constraint that the specificity of mutation analysis is 1, the bottom without the constraint. In each cell, the first number is the expected number of true mutation carriers of all subjects in that corresponding column in Table 1; the second number is the number of noncarriers

	High risk		Low risk
	Mut = 1	Mut = 0	Mut = 1	Mut = 0
αT = 1
MSI = 1	279 + 0	171 + 35	42 + 0	27 + 133
MSI = 0	65 + 0	41 + 400	10 + 0	7 + 1541
αT < 1
MSI = 1	278 + 1	168 + 38	38 + 4	24+ 136
MSI = 0	51 + 13	32 + 410	7 + 45	4 + 1500

As a sensitivity analysis, we relaxed the assumption that the mutation analysis specificity is 100%, that is, that α_T ≡ 1, in favor of a uniform prior. Results are also shown in Table 3 and Table 4. The posterior mean of the mutation analysis specificity was 0.97 (95% CI, 0.91–1.0). Comparing the rest of the parameter estimates with those obtained under α_T ≡ 1, we can see that this departure from the assumption had a small impact on the posterior inference. The CIs were only slightly wider, as a result of increased uncertainty. This suggests that the likelihood is informative about the parameters of interest. Also, since some of the true positives (in terms of mutation analysis) under the specificity assumption were imputed as false positives, the posterior distributions of the proportions of carriers in the two groups shifted slightly to the left. The MSI sensitivity estimate increased slightly from 0.81 (0.73–0.89) to 0.85 (0.74–0.96). Comparing the upper portion of Table 4 to the lower portion, in all the mutation-positive cells, a few were imputed as false positives in the lower portion. However, such imputation occurred more often among MSI-negative subjects than among MSI-positive ones, causing a small increase in the overall MSI sensitivity. The only sensitive figure was the posterior mean of y₁ + y₃ + y₅ + y₇ + y₈, which increased from 12 to 47 when a uniform prior was assigned to α_T.

To further explore the sensitivity of our inferences to the choice of priors, we considered the study by Aaltonen et al. (1994), who found 25 out of 29 HNPCC patients to be MSI+. While their definition of an ‘HNPCC patient’ is different from the gold standard used in our paper, it can still provide a plausible reference for a sensitivity analysis. Thus, we explored using a Beta(25, 4) as the prior on β_M. The resulting posterior mean and 95% probability interval shifted from 0.81 (0.73, 0.89) to 0.85 (0.76, 0.93), suggesting robustness of our findings in terms of both location and spread. This suggests that the width of the probability intervals we report should adequately convey the level of uncertainty about the parameters based on currently available evidence.

4. DISCUSSION

In this paper we presented a meta-analysis of the accuracy of MSI testing in detecting MSH2 and MLH1 mutations. Our meta-analysis overcame the difficulty posed by the fact that no gold standard is available in the published literature about this issue, and provided estimates that can be directly used in genetic counseling and colorectal cancer screening. Our methodology was a Bayesian meta-analytic implementation of the Hui–Walter approach, adapted to account for various forms of incomplete data.

We found MSI to be a very sensitive and specific indicator of germline mutations. The estimates of sensitivity and specificity of MSI in detecting MSH2 and MLH1 mutations will be incorporated in the latest version of the carrier probability package BayesMendel (Chen et al., 2004) and the risk prediction model CRCAPRO. The false-positive fraction associated with MSI testing can be large among unselected colorectal cancers. We estimated the sensitivity of mutation analysis at a modest 62%. This result suggests using caution when interpreting negative genetic testing results for potential mutation carriers.

One of the limitations of our study is that we only considered the MSH2 and MLH1 genes. Other, less prevalent HNPCC genes have been found. For example, MSH6 has a mutation prevalence of about 1.7% among unselected colorectal or endometrial cancer cases (Goodfellow et al., 2003). At the present time, it would be difficult to consider other genes because data from published studies involving such genes are scarce.

A further concern is that the published studies have used a variety of mutation analysis techniques. We pooled all these as though they had the same sensitivity. The sensitivity of a particular mutation search method depends on the true spectrum of mutations, particularly the relative frequency of each type of mutation. In the context of HNPCC mutations, the pathogenic mutations discovered to date in MLH1 and MSH2 are varied in type, and the true spectrum of mutations is still unknown. Systematic studies aimed at comparing sensitivity among the mutation search methods are not presently available, and thus there is no evidence as to which methods are more sensitive over the true mutation spectrum than others. In addition, there are practical reasons why allowing for different sensitivities is unlikely to be successful in our meta-analysis: mutation screening strategies differed across studies, and in many cases were not reported in sufficient detail to allow stratified analysis. Also, if we were to assign each of the testing strategies a separate sensitivity and specificity, the sample sizes for each strategy would become too small.

To further explore this issue, we designed a simple simulation experiment where subjects are tested with genotyping approaches with different sensitivities. Our results indicated that, as long as the distributions of genotyping methods in the high-risk and low-risk populations are similar, estimates of MSI specificity and sensitivity remain unbiased. Specifically, we simulated a data set where we tested a highrisk population of 800 (prevalence, 0.6) and a low-risk population of 1700 (prevalence, 0.05) with a prescreen with sensitivity of 0.78 and specificity of 0.92. In each population, we randomly chose half of the subjects to be subsequently genotyped with sensitivity 0.60 and the other half with sensitivity 0.80 (both specificities are 1). To imitate the missing data situation in our meta-analysis, those who were prescreened negative did not receive a second test. With 100 simulations, all parameter estimates were unbiased and the mean relative errors were all of the order of magnitude 10⁻³ or less.

A key assumption in our analysis is that the results of MSI testing and mutation analysis are independent given the true genotype. In other words, we assume that if a specific mutation is difficult to detect by mutation analysis, the same mutation is not necessarily difficult to detect by MSI testing. As we write, there is no strong biological evidence that such dependence may arise. The errors associated with MSI with respect to the carrier status of MSH2 and MLH1 are likely to arise from links between MSI and changes in the MMR system other than a mutation on MSH2 or MLH1. On the other hand, limitations of sequencing and other forms of conventional mutation analysis arise from the complex spectrum of mutations on MSH2 and MLH1 themselves. Thus, the unknown mechanisms driving test errors are likely to be different, though further validation of this assumption would be helpful.

A second key assumption in our analysis is that the prevalence of mutations in the two groups should be different. This is not a restrictive assumption, given the way the groups were defined. However, groups were separated based on a sensible but subjective choice and it is conceivable that the results may be sensitive to this choice. To address this issue, we performed a simulation experiment. The results indicated that different groupings of the data according to risk do not change the unbiasedness of the estimates of MSI sensitivity and specificity, as long as the risk in one group is higher than the other. In this experiment, in addition to a homogeneous high-risk group of prevalence 0.6 and a low-risk group of prevalence 0.05, we simulated a group of ‘medium-high risk’, with prevalence of 0.3. The rest of the setup was similar to the simulation described above. In case 1 we pooled the medium-high-risk group with the high-risk group; in case 2 we pooled it with the low-risk group. In both cases, the posterior mean was unbiased for all parameters.

APPENDIX

Summary of studies

While we included as many studies as possible, we had to exclude studies where the subjects are a mixture of high-risk and low-risk cases with no breakdown given, such as Ruszkiewicz et al. (2002) and Banno et al. (2003). We disregarded all immunohistochemistry (IHC) results where reported because the literature suggests a very high correlation between MSI and IHC (Lindor et al., 2002; Debniak et al., 2000; Marcus et al., 1999), which may violate the conditional independence assumption.

In the searches for germline mutations, it is common for studies to sequence those samples that showed alterations that first underwent PTT or DGGE as a confirmation step. However, to our knowledge there was no gain of sensitivity or specificity in this extra step. A few studies used novel methods to search for germline mutations that are hard to detect by conventional methods (Renkonen et al., 2003; Nakagawa et al., 2003; Wang et al., 2003). Unfortunately, since these technologies are very different from conventional methods, pooling them with the conventional methods may introduce bias. We decided to exclude such studies or to exclude mutations found by using such technologies. However, such data can serve as independent validation of the estimated low sensitivity for conventional methods.

Table A1 lists the reasons for exclusion, while Table A2 gives the characteristics of the 19 studies included in the meta-analysis.

Table A1.

Relevant studies and their reasons for being excluded from the meta-analysis Paper Reason for exclusion

Paper	Reason for exclusion
Banno et al. (2003)	Endometrial tumors
Debniak et al. (2001)	Use of Alu-PCR (no conventional mutation analysis)
Loukola et al. (1999b)	CR adenomas
Nakagawa et al. (2003)	Use of multiplex ligation-dependent probe amplification method
Renkonen et al. (2003)	Use of the SNuPE technique
Terdiman et al. (2002)	Insufficient presentation of the data

Table A2.

Characteristics of studies that were included in the meta-analysis. Note that the Debniak et al. (2000) study and the Dieumegard et al. (2000) study were each subdivided into a high-risk set and a low-risk set, and thus appeared in Table 1 twice

Paper	Ethnic	Families	Mutation testing strategy	Notes
Bapat et al. (1999)	Mixed	14 ACa + 19 MCb	RT-PCR + PTT
Calistri et al. (2000)	Italian	13 AC + 7MGc + 4 fhd	Exon-by-exon SSCP
Cederquist et al. (2001)	Swedish	Double primary tumor at colorectum or endometrium	DHPLC confirmed by sequencing	MSH6 results excluded
Debniak et al. (2000)	Polish	11 def.hnpcce (or highly likely) + 15 sporadic late-onset CRC	Sequencing	IHC results discarded
Dietmaier et al. (1997)	German	148 sporadic CRC	None
Dieumegard et al. (2000)	French	15 ICGf + 1 fh + 7 sporadic < 50	SSCP	IHC results discarded
Lamberti et al. (1999)	German	47 AC + 5 looser AC	PCR + SSCA + HA, etc.	Some data excluded b/c of poor presentation
Liu et al. (2000)	Swedish	‘Highly suspicious for HNPCC’	DGGE + RT-PCR + PTT	Some AC, some fh, some early CRC, no break-down given
Loukola et al. (1999)	Finnish	CRC registry	DGGE or Sequencing on MSI+
Percesepe et al. (2001)	Italian	CRC registry	None
Ponz de Leon et al. (2004)	Italian	‘Consistent with HNPCC’	SSCP or Sequencing	MSI results n.a.
Salahshor et al. (1999)	Swedish	Sporadic CRC	None
Salovaara et al. (2000)	Finnish	Sporadic CRC	Search for 2 founder mutation with ASO and PCR-based, then sequencing the negatives
Scartozzi et al. (2002)	Italian	At least one of the BGg	Sequencing
Terdiman et al. (2001)	US	23% met AC, 70% BG	DGGE
Wahlberg et al. (2002)	US	38 AC or BG + 5 MG + 5 HNPCC-like	Sequencing
Wang et al. (2003)	German	BG	SSCP, HA, DHPLC followed by sequencing	Semiquantitative multiplex PCR result excluded

CRC = colorectal cancer; SSCP = single-strand conformation polymorphism; DHPLC = denaturing high-performance liquid chromatography; ASO = allele-specific oligonucleotide.

^aAmsterdam criteria.

^bMt. Sinai Hospital criteria.

^cModified guidelines.

^dPositive family history.

^eDefinitive HNPCC.

^fInternational Collaborative Group criteria.

^gBethesda guidelines.