Model-Integrated Estimation of Normal Tissue Contamination for Cancer SNP Allelic Copy Number Data

Abstract

SNP allelic copy number data provides intensity measurements for the two different alleles separately. We present a method that estimates the number of copies of each allele at each SNP position, using a continuous-index hidden Markov model. The method is especially suited for cancer data, since it includes the fraction of normal tissue contamination, often present when studying data from cancer tumors, into the model. The continuous-index structure takes into account the distances between the SNPs, and is thereby appropriate also when SNPs are unequally spaced. In a simulation study we show that the method performs favorably compared to previous methods even with as much as 70% normal contamination. We also provide results from applications to clinical data produced using the Affymetrix genome-wide SNP 6.0 platform.

1. Introduction

DNA in tumor cells can contain abnormalities in the form of copy number aberrations such as segments with losses or gains of one or several copies of either allele. The lengths of such aberrations can vary between short segments up to an entire chromosome, and their positions are essential both for detecting and for improving knowledge of various sorts of cancer. Therefore, methods that localize copy number aberrations are of great importance. In addition to changes in the total copy number of both alleles together, changes in the allelic copy numbers, ie, the number of copies of each allele, are also important. We denote the two different alleles at a given genomic location by A and B, so that for normal cells the possible genotypes are AA, AB and BB. One example of a genotype aberration is loss of heterozygosity (LOH), for which the only attainable genotypes are AA and BB.

Different techniques to measure DNA copy numbers have been developed, as have methods to evaluate the measurement data. One technique is array comparative genomic hybridization (aCGH), which provides ratios of the copy numbers of a sample DNA, compared to those of some reference DNA. Several different statistical methods have been applied to this kind of data, including different segmentation methods,^4,20,21 smoothing^6,12 and hidden Markov models.^{1,7,9,16,19,24,27,28,29} aCGH data provides information only about the total copy number and gives no information about the amount of each allele. Another drawback with such data is the limited number of probes on the arrays. For this reason there is an increased use of single nucleotide polymorphism (SNP) data, which offers denser measurements and provides intensities for the two alleles separately. Using SNP data it is possible not only to estimate copy number changes, but also to find allelic changes such as LOH. Indeed, a copy number amplification may be caused by different allelic changes. For example, a copy number of four could correspond either to {AAAA, AAAB, ABBB, BBBB}, to {AAAA, AABB, BBBB} or to {AAAA, BBBB}, depending on which allele that has gained extra copies.

SNP data has previously been analyzed using various sorts of methods, such as smoothing^11,15 and pattern recognition.²² The most frequently used methods are however based on hidden Markov models (HMMs).^{3,8,13,17,18,26,30,31} A brief introduction to Markov chains and HMMs is found in Appendix 1. HMMs suit SNP data well since genomic alterations often appear in longer or shorter segments, implying that copy numbers across probes in a small genomic region are correlated. For example, Wang et al³¹ and Colella et al³ model SNP data from the Illumina array, which provides log R ratio data (log₂-ratio of total observed intensities to total expected intensities) and BAF data (normalized measure of the relative intensities of the two alleles), using an HMM with six states, while Sun et al³⁰ apply a more comprehensive model with nine states. Korn et al¹³ combine an HMM to model copy number variants with a clustering algorithm to detect genotypes. Li et al¹⁸ also model the proportion of the major allele while Lamy et al¹⁷ use both allelic intensities provided by the Affymetrix array and model them using bivariate Normal distributions.

Several of the methods above assume that the ploidy, ie, mean copy number, of a chromosome is two. This holds for normal cells, but cancer cells are anueploid, ie, their ploidy may differ from two. The necessity for considering ploidy when modeling cancer data is well described by Greenman et al,⁸ but in brief one can say that the measured normalized intensity for a probe in a diploid chromosome is twice as large as for a probe with the same copy number in a quadroploid chromosome. Two methods that include ploidy are those of Attiyeh et al² and Greenman et al,⁸ which both contain a pre-processing step in which the ploidy is estimated. Greenman et al then continue by using an HMM while Attiyeh et al apply a window-based model.

Another feature common in tumor samples, arising from the difficulty to dissect tumor cells only from a tissue sample, is contamination of the tumor cell sample by normal cells. As a result the measured allelic intensities are mixtures of intensities from tumor and normal cells, thus yielding non-integer DNA copy numbers. One way to incorporate such contamination is to model total copy numbers of the mixed sample in a non-parametric way,^2,29 but this provides limited information about the copy numbers of the cancer cells. Sun et al³⁰ estimate the fraction of normal tissue contamination using an empirical method and Colella et al³ write that it is possible to extend their method to handle contamination, but without being more specific. Li et al¹⁸ show that their method can handle a fraction of normal tissue contamination up to 30%, while Lamy et al¹⁷ report a simulation study with slightly better results. Some tumors however form in a manner such that even with microdisection, a significant proportion of normal cells (say 50% or more) can arise in the sample, and none of the above methods provide results that are satisfactory for such high fractions of normal tissue contamination.

The purpose of the present paper is to devise a method to estimate allelic copy numbers, with ploidy and fraction of normal tissue contamination integrated in the model. Indeed, in all of the above papers, ploidy and/or normal fraction are estimated by adding more or less ad-hoc steps to a model that does not account for these parameters in itself. The model reported here is thus particularly suited for cancer data, for which both of these features are common. By including these parameters in the model they can be estimated alongside the other parameters using all data, rather than adding a pre-processing step or empirical methods using only a small subset of the data. In the simulation study presented below, samples with 30%, 50% and 70% normal contamination are simulated and even for the largest amount of contamination, 97% of the probes are reconstructed to the correct copy number state.

An additional feature of our model is that it is based on a continuous-index Markov chain, which accounts for the fact that the SNP probes are often unevenly spread over the genome. The relevance of a continuous-index model was highlighted by Gupta and Mitra¹⁰ (Section 5.3) for the different but related problem of classifying regions of DNA as nucleosome free regions (NFR) or non-NFR using a two-state HMM. Indeed, they showed that with irregularly spaced probes, a continuous-index model can provide substantially better results than a discrete-index model; 99% vs. 85% or 68% correct classifications in simulations for two different arrangements of the probes. Also the methods by Wang et al,³¹ Colella et al³ and Li et al,¹⁸ who apply discrete-index HMMs to SNP data, aim to take distances between probes into account by letting the Markov transition probabilities depend on these distances in different ways. Common to all of these methods is however that the stipulated transition probabilities violate the Chapman-Kolmogorov equation of Markov chains. That is, letting P(t) be the matrix of transition probabilities over a distance t between two probes, the equality P(t₁ + t₂) = P(t₁)P(t₂) does not hold. In essence this means that there is in fact no Markov chain with the given transition probabilities.

The paper is organized as follows. The model is described in in Section 3. Section 4 provides results from a simulation study as well as from an application to clinical data. Concluding remarks are given in Section 5.

2. Data

The data used in this study are the cancer samples in Greenman et al,⁸ produced using the Affymetrix genome-wide SNP 6.0 platform. We applied the algorithm to about 15 different cell line and primary tumor samples, representing various cancer forms including breast, lung and renal cancer. The primary tumor sample PD1753a for which results are reported in Section 4.2 are from a clear cell renal cell carcinoma sample.³²

For probes at SNPs the intensities of the two different alleles are provided, while at other positions only a single total copy number intensity is available. Following Greenman et al,⁸ the intensities are normalized by first dividing each measurement by the total intensity of the sample (ie, the sum of all probe intensities over the entire genome), to remove chip-to-chip variation. The mean signals for each allele (or probe at non-SNP positions) are then transformed into a copy number intensity and a genotype intensity that are indicators of total copy number and allelic ratio dosages. The model presented below incorporates intensities for SNP probes only, but is easily extendable to include also probes measuring total copy number only; we elaborate on this further in the Discussion. The cancer data is available from the Cancer Genome Project, subject to a manual transfer agreement, and our Matlab code is available on the WWW.³³

3. The Model

3.1. Basic structure

Let there be N_c probes on chromosome c, and denote these probes as probe (k, c), k = 1, 2, …, N_c. The genomic location of probe (k, c) is denoted by t_kc, measured in the unit base pairs (bp) starting from the beginning of the chromosome. We denote the two different alleles at any genomic location by A and B. We will write g = (g_A, g_B) for the allelic copy numbers, ie, g_A and g_B are the number of copies of the A and B allele respectively. For example, the genotype AAB corresponds to g = (2, 1). Obviously the genotype and the allelic copy numbers are in a one-to-one correspondence to each other, and at times we will make no real distinction between the two. The allelic intensities are modeled using an HMM for which each state i corresponds to one genotype set G_i as specified in Table 1. The Markov chain can be extended to include more states with copy numbers above six, but the model as stated here has proved to be enough for the studied samples. To explain the genotype sets in Table 1, we note that through cancer development any region in the genome starts with one parental copy of each region and ends up with m copies of one allele and n copies of the other. If the genotype was originally AA or BB then the genotype will be (m + n)A or (m + n)B, respectively. If the SNP was heterozygous then we must end up with either mA and nB, or mB and nA. These are the genotypes indicated in Table 1. We refer to state 4, with genotype set {AA, AB, BB} as the normal state, and by an abnormal state we mean any other state.

Table 1.

Genotype sets for the different states of the Markov chain, sorted in the order given by the total copy number and copy number of the minor allele.

Statei	(Total CN, minor CN)	Genotype set Gi
1	(0,0)	{ }
2	(1,0)	{A, B}
3	(2,0)	{AA, BB}
4	(2,1)	{AA, AB, BB}
5	(3,0)	{AAA, BBB}
6	(3,1)	{AAA, AAB, ABB, BBB}
7	(4,0)	{4A, 4B}
8	(4,1)	{4A, 3AB, A3B, 4B}
9	(4,2)	{4A, 2A2B, 4B}
10	(5,0)	{5A, 5B}
11	(5,1)	{5A, 4AB, A4B, 5B}
12	(5,2)	{5A, 3A2B, 2A3B, 5B}
13	(6,0)	{6A, 6B}
14	(6,1)	{6A, 5AB, A5B, 6B}
15	(6,2)	{6A, 4A2B, 2A4B, 6B}
16	(6,3)	{6A, 3A3B, 6B}

For each chromosome c the sequence of copy number states, according to Table 1, is modeled by a continuous-index Markov chain (X_c(t))_t1c≤t≤Tc, where t and T_c are respectively the genomic location (in bp) within the chromosome and the length (in bp) of the chromosome. The Markov chains for different chromosomes are assumed independent. The genomic location (in bp) is, strictly speaking, a discrete variable, but since the number of bp’s within a chromosome is much larger than the number of jumps of the Markov chain, the error caused by using a continuous approximation is negligible. With a discrete-index model the Markov transition probabilities would either be very close to unity (for staying in the same state from one bp to another) or close to zero (for changing state), and dealing with such probabilities is unstable numerically. For a continuous-index model, using transition rates rather than probabilities, this problem does not exist.

With 16 different states there are 240 different types of jumps and equally many transition rates (per chromosome). It is infeasible to estimate such many rates, and to make the model more parsimonious we assume a large number of them to agree. Specifically we assume, for chromosome c, a common rate λ_c for jumps from any state (normal or abnormal) to the group of abnormal states, with each such state, except for the current one in case the chain resides in an abnormal state, being equally likely, and another common rate η_c for jumps to the normal state from any abnormal state. The total rate out of any abnormal state, for chromosome c, is thus λ_c + η_c. This dynamic provides Markov chains whose stationary versions are time-reversible.²⁹ Finally we let δ_ic = P(X_c(t_1c) = i) denote the initial probability for Markov state i in chromosome c.

Write y_kc = (y_Akc, y_Bkc) for the measured allelic intensities at probe (k, c). Greenman et al⁸ studied the correlation between the allele A and B intensities, for each probe, using 460 wild-type samples. For probe (k, c), plotting the two allele intensities for all wild-type samples against each other reveals three clusters (see,⁸ Figure 1, for an example). These clusters correspond to the genotypes AA, AB and BB, with the coordinates of the cluster centers written as (A_0kc + 2A_1kc, B_0kc), (A_0kc + A_1kc, B_0kc + B_1kc) and (A_0kc, B_0kc + 2B_1kc) respectively for suitable parameters A_0kc, B_0kc, A_1kc and B_1kc. These parameters were all estimated by Greenman et al⁸ using the wild-type samples. Their interpretation is that A_0kc is the background intensity of the A allele (at diploid probes BB), and A_1kc is the increase in A allele intensity from BB to AB and from AB to AA; B_0kc and B_1kc have analogous interpretations.

Figure 1.

Proportions of probes at which the Markov state was incorrectly reconstructed by the Viterbi algorithm with MAP parameter estimates computed by the EM algorithm. Markov transition rates were λ_c = η_c = 10⁻⁷ (top left), λ_c = 10⁻⁷, η_c = 10⁻⁹ (top right), λ_c = 10⁻⁹, η_c = 10⁻⁷ (bottom left), λ_c = η_c = 10⁻⁹ (bottom right) (unit: bp⁻¹). Confidence intervals were obtained by exponentiating two-sided 95% student-t confidence limits based on the log-proportions for 10 genome replicates.

Further denote by (μ_Akcg, μ_Bkcg) the mean allele A and B intensities at probe (k, c) for allelic copy numbers g = (g_A, g_B). The cluster centers above then write

$$\begin{array}{lll}{\mu }_{kcg}\hfill & =\hfill & \hspace{0.17em}({\mu }_{\text{A}kcg},\hspace{0.17em}{\mu }_{\text{B}kcg})\hfill \\ \hfill & =\hfill & (A_{0kc}\hspace{0.17em}+\hspace{0.17em}{g}_{\text{A}}\hspace{0.17em}{A}_{1kc},\hspace{0.17em}{B}_{0kc}\hspace{0.17em}+\hspace{0.17em}{g}_{\text{B}}{B}_{1kc}),\hfill \end{array}$$ (1)

and this model applies for the normal Markov state i = 4, ie, for allelic copy numbers such that g_A + g_B = 2. Moreover, the clusters in Greenman et al⁸ (Fig. 1) are tilted ovals, indicating that the intensities for alleles A and B are correlated and have unequal variances. Greenman et al⁸ found that a suitable model for the covariance matrix is

$${\sum }_{kcg}\hspace{0.17em}=\hspace{0.17em}{v}_{kc}\left(\begin{array}{cc}\hfill {\mu }_{\text{A}kcg}^2\hfill & \hfill {\rho }_{kc}{\mu }_{\text{A}kcg}{\mu }_{\text{B}kcg}\hfill \\ \hfill {\rho }_{kc}{\mu }_{\text{A}kcg}{\mu }_{\text{B}kcg}\hfill & \hfill {\mu }_{\text{B}kcg}^2\hfill \end{array}\right);$$ (2)

note that the variances are taken proportional to the squared means. The probe-specific variance factors v_kc and correlations ρ_kc, as well as the means parameters A_0kc, B_0kc, A_1kc and B_1kc described above, were all estimated by Greenman et al⁸ using the wild-type samples and assuming a bivariate Normal distribution for each cluster.

We now carry this model further by assuming that for each probe, the allele intensities follow the mean-variance model given by Eqs. (1)–(2) also for genotypes (g_A, g_B) for which g_A and g_B do not sum to two, ie, for all pairs (g_A, g_B) corresponding to genotypes listed in Table 1. That is, we assume that the response from amount of each allele on the microarray to measured intensity is linear, with the variance also increasing linearly. In reality the allelic intensities have a linear response for lower copy numbers, while at higher copy numbers the intensities start to saturate and our method is approximate. This could be adjusted for by a non-linear transformation, cf. Section 5, but we have not attempted such an adjustment in the analyses presented in this paper.

The above model specifies the conditional density of Y_kc given a particular genotype. To specify the conditional density of Y_kc given a Markov state, we recall that each Markov state has a genotype set comprising between one and four different genotypes. Thus the conditional density of Y_kc, given the state, is a mixture of bivariate Normal distributions for which each mixture component represents a different genotype. The mixture weights were taken as the Hardy-Weinberg weights; for the copy number-aberrated genotypes, Hardy-Weinberg was used to compute the germline genotype proportions. Thus letting p_kc be the allele frequency for an A allele at probe (k, c), the probability for the different genotypes, denoted by w_kcig, are the binomial probabilities p_kc and 1 – p_kc for states with two genotypes, $p_{kc}^2$, 2p_kc(1 – p_kc) and (1 – p_kc)² for states with three genotypes, and $p_{kc}^2$, p_kc(1 – p_kc), p_kc (1 – p_kc) and (1 – p_kc)² respectively for states with four different genotypes. The frequencies p_kc were also estimated by Greenman et al,⁸ using the wild-type samples. The conditional density for a measurement Y_kc given the Markov state, often referred to as the emission density of the HMM, thus writes

$$f_{Y_{kc}|X_c\hspace{0.17em}(t_{kc})}\hspace{0.17em}(y|i)\hspace{0.17em}=\hspace{0.17em}\sum _{g\in G_i}{w}_{kcig}{f}_{Y_{kc}|G_{kc}}(y|g),$$ (3)

where G_kc is the allelic copy numbers for probe (k, c) and f_Ykc|Gkc (·|g) is the bivariate Normal density with mean and covariance matrix as in Eqs. (1)–(2).

As pointed out in the introduction we include the ploidy K, ie, average copy number over the entire genome, in the model to make it suitable for cancer data. The ploidy is defined genome-wide and not per chromosome, as the probe intensities are normalized per genome. The HMM described above models the normalized intensities, and its parameters were estimated for wild-type samples (ie, diploid samples; K = 2). For a sample with K > 2 the normalized intensities will thus be smaller by a factor 2/K (on average), so that the model for the normalized intensities becomes

$$Y_{kc}|G_{kc}\hspace{0.17em}=\hspace{0.17em}g\hspace{0.17em}\sim \hspace{0.17em}N\left(\frac{2}{K}{\mu }_{kcg},\hspace{0.17em}\frac{4}{K^2}\hspace{0.17em}{\Sigma }_{kcg}\right).$$ (4)

This completes the specification of the basic model. As described above, the parameters A_0kc, A_1kc, B_0kc, B_1kc, v_kc, ρ_kc and p_kc were all estimated from the wild type samples, and were thus considered as fixed when the model was applied to cancer cell data. The intensities λ_c and η_c, the initial probabilities δ_c and the ploidy K were on the other hand estimated from the actual cancer data.

3.2. Normal tissue contamination

As mentioned above it is often difficult to dissect cancer cells without including any surrounding normal tissue, ie, diploid tissue. Such contamination implies that the measured allelic intensities correspond to a mixture of cancer and normal cells. We denote the fraction of normal tissue in the sample by γ, and consequently the fraction of tumor tissue is 1 – γ. Then for a given probe with, as above, copy numbers g_A and g_B or alleles A and B in the tumor but also copy numbers $g_{\text{A}}^{\text{N}}$ and $g_{\text{B}}^{\text{N}}$ for the two alleles in the normal tissue, we assumed the same mean-covariance model as in Eqs. (1)–(2), but with (g_A, g_B) replaced by

$$\left(g_{\text{A}}^{\gamma },\hspace{0.17em}{g}_{\text{B}}^{\gamma }\right)\hspace{0.17em}=\hspace{0.17em}\left((1\hspace{0.17em}-\hspace{0.17em}\gamma )g_{\text{A}}\hspace{0.17em}+\hspace{0.17em}\gamma g_{\text{A}}^{\text{N}},\hspace{0.17em}(1\hspace{0.17em}-\hspace{0.17em}\gamma )g_{\text{B}}\hspace{0.17em}+\gamma g_{\text{B}}^{\text{N}}\right).\hspace{0.17em}$$ (5)

Similarly, the conditional distribution of Y_kc given Markov state i is a mixture of bivariate Normals, but now each four-tuple (g_A, g_B, $g_{\text{A}}^{\text{N}}$, $g_{\text{B}}^{\text{N}}$) contributes to a component of that mixture. Thus, the number of mixture components will for some Markov states be larger than without normal tissue contamination (see Table 2).

Table 2.

Combined genotype sets for the different states of the Markov chain, in a model with normal contamination γ. The weights for the respective combined genotypes are the Hardy-Weinberg weights as in the model without normal tissue contamination, and the total and minor copy numbers for the abberated components are as in Table 1.

Statei	Combined genotype setGi
1	{2γA, γAγB, 2γB}
2	{(1 + γ)A, AγB, γAB, (1 + γ )B}
3	{2A, (2 − γ)AγB, γA(2 − γ)B, 2B}
4	{AA, AB, BB}
5	{(3 − γ)A, (3 − 2γ)AγB, γA(3 − 2γ)B, (3 − γ)B}
6	{(3 − γ)A, (2 − γ)AB, A(2 − γ)B, (3 − γ)B}
7	{(4 − 2γ)A, (4 − 3γ)AγB, γA(4 − 3γ)B, (4 − 2γ)B}
8	{(4 − 2γ)A, (3 − 2γ)AB, A(2 − γ)B, (4 − 2γ)B}
9	{(4 − 2γ)A, (2 − γ)A(2 − γ)B, (4 − 2γ)B}
10	{(5 − 3γ)A, (5 − 4γ)AγB, γA(5 − 4γ)B, (5 − 3γ)B}
11	{(5 − 3γ)A, (4 − 3γ)AB, A(4 − 3γ)B, (5 − 3γ)B}
12	{(5 − 3γ)A, (3 − 2γ)A(2 − γ)B, (2 − γ)A(3 − 2γ)B, (5 − 3γ)B}
13	{(6 − 4γ)A, (6 − 5γ)AγB, γA(6 − 5γ)B, (6 − 4γ)B}
14	{(6 − 4γ)A, (5 − 4γ)AB, A(5 − 4γ)B, (6 − 4γ)B}
15	{(6 − 4γ)A, (4 − 3γ)A(2 − γ)B, (2 − γ)A(4 − 3γ)B, (6 − 4γ)B}
16	{(6 − 4γ)A, (3 − 2γ)A(3 − 2γ)B, (6 − 4γ)B}

The weights for the combined genotypes are Hardy-Weinberg weights as in the model without normal contamination. For example, for a state in Table 2 with three combined genotypes, the weights are $p_{kc}^2$, 2p_kc(1 – p_kc) and (1 – p_kc)² respectively.

3.3. Estimation of parameters and the Markov path

The parameters estimated from a tumor sample are the transition rates λ_c and η_c, the initial probabilities δ_c, the ploidy K and also the fraction γ of normal tissue contamination.

For a model like the present one, the maximum-likelihood estimator (MLE) typically overestimates the transition rates λ_c and η_c ²⁵ (Section 4.3), thereby letting an aposteriori reconstruction of the Markov chain trajectory capture also very short transients of the observed data. When using the EM algorithm to compute the MLE, this becomes visible as an overestimated number of jumps of the Markov chain. In order to control the jumps and make their number biologically plausible, we take a Bayesian approach and penalize overly large transition rates by placing Gamma distribution priors on each λ_c and η_c. Other parameters are assigned uniform (flat) priors. All parameters are apriori independent. We then compute the maximum aposteriori (MAP) parameter estimate using the EM algorithm, by incorporating the priors into the M-step⁵ (p. 6). Otherwise this algorithm is a variant of the EM algorithm described by Roberts and Ephraim,²³ designed to estimate parameters of a continuous-index HMM observed at discrete positions. The method is detailed in Appendix 2.1.

Finally, to construct an estimate of the trajectory of the hidden Markov chain we use a Viterbi algorithm adapted to continuous-index Markov chains (see Appendix 2.2).

4. Results

4.1. Application to simulated data

To evaluate our method’s ability of making correct reconstructions for different amounts of normal contamination, we simulated data from the assumed model, computed MAP parameter estimates using the EM algorithm, reconstructed the hidden Markov chain using the Viterbi algorithm, and finally computed the proportion of probes at which the Markov state was correctly reconstructed. For each simulated dataset we first simulated the Markov chain and the genotypes for each probe position, then computed μ_kcg and Σ_kcg using Eqs. (1)–(2), Eq. (5) and the fixed A₀, A₁, B₀, B₁, ρ and v (estimated from the wild-type samples), and finally simulated data from the bivariate Normal distributions of Eq. (4) with K = 2. Note that the actual value of K is irrelevant for these simulations, since the model given by Eqs. (1)–(2) describes the data after normalization.

The simulations were carried out for 30%, 50% and 70% normal contamination, and transition rates λ_c = η_c = 10⁻⁷, λ_c = 10⁻⁷ and η_c = 10⁻⁹, λ_c = 10⁻⁹ and η_c = 10⁻⁷, and λ_c = η_c = 10⁻⁹ (in units of bp⁻¹) respectively. For each combination of contamination and rates, 10 replicates were simulated. For the Gamma priors of λ_c and η_c we chose shape parameter 2 and means equal to the true transition rates. These choices yield priors that are not overly informative, but which are concentrated enough on small values to prevent the Markov chain from jumping too frequently in our samples.

To verify the convergence of the EM algorithm we present the EM iterations for three different simulated replicates in Figure 2. The proportions of incorrectly reconstructed probes are plotted in Figure 1.

Figure 2.

Estimates of normal contamination γ for iterations 1–10 of the EM algorithm and three simulated replicates with different values of γ: γ = 0.3 (top), γ = 0.5 (middle), and γ = 0.7 (bottom). The initial value for γ was 0.5 in all simulations.

These results can be compared to those from the simulation study by Lamy et al.¹⁷ For a normal contamination of 30% the results are similar, but for 45%, which is the largest fraction studied by Lamy et al, their method provides 8%–18% incorrectly estimated probes while at 50% contamination our model provides an error rate below 1%. In addition, the present model performs well even at such a high amount of normal contamination as 70%, when the Markov state is correctly reconstructed at more than 97% of the probes. Obviously the differences between our results and those of Lamy et al depend not only on the different estimation algorithms but also on differences between the number and location of the probes, and on the model for the observed allele intensities and its parameters. However, given the magnitude of the performance improvement, a significant part of it must be attributed to the estimation algorithm as such.

4.2. Application to clinical data

We applied our method to a number of samples from the data described in Section 2. An example is displayed in Figure 3, which shows the Viterbi reconstruction of the Markov chain as well as the corresponding copy numbers compared to the data, for chromosome 3 in primary sample PD1753. For the Gamma priors for λ_c and η_c we chose shape parameters 2 and means 10⁻¹⁵.

Figure 3.

Top: Viterbi reconstruction of the Markov path for chromosome 3 in PD1753. Bottom: sum of (standardized) allele intensities for probes within the same chromosome (grey dots), and the copy number of the corresponding state (black solid line).

The reconstruction divides the chromosome into two regions, reconstructed to state 2 ({A, B}) and state 4 ({AA, AB, BB}) respectively. As a simple check of this reconstruction we plotted the standardized allele intensities against each other for all probes in the respective region (Figs. 4–5). Figure 5, corresponding to the normal state, shows three clusters representing the three genotypes AA, AB and BB, while Figure 4 shows four clusters. In Table 1 state 2 is associated to two genotypes, A and B, but with normal contamination this state comprises four combined genotypes (1 + γ)A, AγB, γAB and (1 + γ)B (Table 2). Here γ is estimated at 0.53.

Figure 4.

Scatter plot of standardized measured allele intensities in the segment reconstructed to Markov state 2 in Figure 3. The fraction of normal contamination was estimated at 0.53.

Figure 5.

Scatter plot of standardized measured allele intensities in the segment reconstructed to Markov state 4 in Figure 3.

For some of the genomes the values of A_0kc, A_1kc, B_0kc and B_1kc needed small adjustments before applying our model; without it, the model did not produce a reasonable fit. A possible explanation for this adjustment being required is a drift in the measured intensities from when data from the wild-type samples, used to estimate most model parameters, was collected, to when the tumor samples were analyzed. A suitable construction of the adjustment was as a common, ie, genome-wide multiplier c₀ for all A_0kc and B_0kc, and another common multiplier c₁ for all A_1kc and B_1kc. The multipliers c₀ and c₁ were estimated using data from a chromosome segment known to belong to the normal state. The data within this segment was clustered into three parts using the k-means algorithm, and then c₀ and c₁ were estimated by a least squares fit.

5. Discussion

We have presented a method to estimate the number of copies of each of the two alleles in SNP data, taking three features common in cancer data into account; unequally spaced probes, aneuploidy, and normal contamination. Unequally spaced probes are modeled using a continuous-index Markov chain instead of a discrete-index one, which is the usual choice in the literature. The ploidy and fraction of normal contamination are both included as parameters in the model, which allows us to estimate them along with other variables and using all the data, rather than estimating them separately in a pre-processing step. This set-up also allows us to retain the integer structure of the allele copy numbers. The model’s ability to estimate the fraction of normal contamination has been demonstrated in a simulation study, with the results being far better than for previous methods and excellent even with as much as 70% normal contamination.

Above we denoted Markov state 4, ie, the state with genotypes {AA, AB, BB}, the normal state, irrespective of the ploidy of the chromosome. The reason for singling out this particular state is that it is often particularly interesting whether the Markov chain is in this state or not, at any given probe. One could argue that if the ploidy differs from two this is not ‘normal’, but it is straightforward to select a different state as ‘normal’ and then modify the transition rate structure and estimation algorithm accordingly.

The emission model, ie, Eqs. (1)–(4), assume that the means and variances of the measured intensities are both linear in the amount of each allele. In practice this assumption may fail, eg, because for large copy numbers the response is nonlinear. One could then include such a non-linearity in the model, and model the mean intensities as μ_kcg = h_kc(g;θ_kc) where h is some function and θ_kc parameters of this function. Ideally the functional form h as well as all its probe-speficic parameters θ_kc should be well estimated beforehand, so that they are essientially known when evaluating an unknown sample. Similar comments apply to the variance of the measured intensities.

In this paper we have only considered probes that provide allele-specific intensity measurements, but, as mentioned in Section 2, microarrays often also contain probes that measure the total copy number only, ie, the sum of the number of alleles. Such probes can easily be included in our model by speficying a corresponding suitable emission density, ie, a density corresponding to Eq. (3). For instance, this could be a univariate Normal density with mean μ_kcg = C_0kc + C_1kc(g_A + g_B) and variance ${\sigma }_{kcg}^2\hspace{0.17em}=\hspace{0.17em}{\nu }_{kc}\hspace{0.17em}{\mu }_{kcg}^2$ for parameters C_0kc, C_1kc and ν_kc that again need to be estimated prior to analyzing an unknown sample. Should the response function from total copy number to intensity not be linear for large copy numbers, this could be handled similarly to what can be done for SNP probes; cf. the previous paragraph.

Finally we mention some possible limitations of our method. Firstly, the accuracy of the method is likely to be reduced in regions of very high copy number where signal saturation occurs, such as in amplicons, and bespoke nonlinear adjustments may be required (as discused above). Secondly, we have ignored copy number polymorhisms. These will produce non-integer copy numbers in the cancer sample due to the skewed ratio between the cancer and the contaminating normal. If copy number data is available for the normal, it may be possible to generalise these methods to make such an adjustment, however, such regions are generally a lot smaller in scale than the somatic copy number changes seen in cancer and were not considered further. Lastly, we have assumed that the sample in question is derived from a homogeneous collection of cells. However, cell-to-cell variation is quite possibly going to produce a lot of different clones with differing copy numbers, and more general methods will be required to deal with such complexities.

To sum up this paper, copy number variations in cancer are common and their accurate determination is important for determining homozygous deletion, amplifications and breakpoints, all of which can be functionally implicated in cancer. This problem is compounded by normal contamination, making the accurate estimation of integer copy numbers in cancer samples with normal contamination difficult. Here we have introduced a method that addresses this problem.

Appendix

1. A Primer on Markov Chains and Hidden Markov Model

The purpose of this section is to provide a brief and rather elementary introduction to Markov chains with discrete and continuous index, and to hidden Markov models. A monograph entirely devoted to bioinformatics applications of hidden Markov models is the text by Koski.¹⁴

Consider a sequence t₁, t₂, …, t_N of locations (in our case these will be probe locations), and a set {1, 2, …, r} of states (which will in our case be as in Tables 1 or 2). At any location t_k there is an actual state x(t_k) (ie, a true copy number state), which we think of as the realization of a random variable X(t_k). These random variables are dependent, since copy number states at nearby probes are correlated. To model this dependence, we use Markov chains.

A discrete-index Markov chain (we use the term index rather than the more common ‘time’, since bp location is not a temporal variable) is specified by transition probabilities p_ij(t_k–1, t_k), giving the (conditional) probability that if the chain happens to be in state i at location t_k–1, it will move to state j at location t_k. For j = i, the probability concerns the event that the chain will stay in the same state, ie, not move at all. Implicit in this characterization is also the fact that if the states x(t₁), x(t₂), …, x(t_k–1) at all foregoing locations t₁, t₂, …, t_k–1 are known, this does not affect the conditional probability, which only depends on the state x(t_k–1) at the closest location t_k–1; this is the Markov property. To complete the specification of the Markov chain, we must also provide the initial probabilities, ie, the probabilities that at the first location t₁, the chain starts in state i for each respective state.

In our model, the probe locations t_k are separated by different distances t_k – t_k–1, ie, these distances are not equal. We wish to incorporate this feature into the Markov model, so that the transition probabilities p_ij(t_k–1, t_k) do not only depend on the states i and j that the chain moves from and to respectively, but also on the distance h_k = t_k – t_k–1 between the probes. One way to accomplish this is to think of the base pair location along a chromosome, which we denote by t, as a continuous variable rather than as a discrete one, and to model the state changes of the Markov chain using this continuous variable, or index. In contrast to a discrete-index Markov chain, a continuous-index Markov chain is specified in terms of transition rates. For any state i and any other state j, ie, different from i, there is a transition rate q_ij from state i state j. For any state i we also define the total rate out of i, q_i, as the sum of all transition rates out of this state, ie, q_i = Σ_j≠i q_ij. One way to interpret these rates is in terms of sojourn lengths and jump probabilities. Given that the chain has entered state i, it will stay there for a sojourn whose length is random and follows an exponential distribution with rate q_i (mean 1/q_i); the probability that this sojourn exceeds length s is thus the exponential exp(–q_i s). When then the chain eventually leaves state i, the probability that it jumps to state j is given by q_ij/q_i.

For a continuous-index Markov chain it is also possible to compute the probability that for two locations t_k–1 and t_k separated by distance h_k, if the chain is in state i at location t_k–1, it will be in state j at location t_k. Denoting these probabilities by p_ij(h_k) and collecting then into an r × r matrix P(h_k) (thus p_ij(h_k) is the row i column j element of this matrix), it holds that P(h_k) = exp(Qh_k), where Q is the r × r rate matrix (or intensity matrix, or generator) with off-diagonal elements given by the transition rates q_ij and diagonal elements q_ii = –q_i, ie, the negative of the total rates out of the respective states. Moreover, exp(Qh_k) is the matrix-exponential function, defined by the power series exp(A) = I + A + A²/2! + A³/3! + … for any square matrix A, where I is the identity matrix, ie, a matrix of the same size as A and with diagonal elements equal to one and all off-diagonal elements being zero, and k! is the factorial 1 × 2 × … × k. This definition is a direct generalization of the power series for the ordinary (real-valued) exponential function.

In a hidden Markov model (HMM), the Markov chain is not directly observable, but only as disturbed by noise. In the present setting the copy number state cannot be observed with certainty, but for any probe the intensity measurements, for each allele, provide partial information about the copy number state. In an HMM, the link between the state X(t_k) at some location t_k and the corresponding measurement Y_k (here, intensities) is specified through an emission density f_YkX(tk)(y|i), which is the conditional density of Y_k given that X(t_k) = i. In the present context the emission density is thus the density of the measured intensities given a certain copy number state. Since there are two intensities available, one for each allele, the density is a bivariate one. Furthermore, since each copy number state (Markov state) contains several genotypes, the emission density for a copy number state is a mixture (weighted average) of densities corresponding to each of these genotypes; this is Eq. (3).

Specifying the HMM thus amounts to specifiying the structure and parameters of the Markov chain, and those of the emission densities. When this has been done, typical tasks are to i) estimate parameters from data, and ii) find the most likely realization of the Markov chain, given data. The first task, parameter estimation, is commonly carried out using the so-called EM (expectation maximization) algorithm, which is an iterative procedure that in each iteration increases the likelihood of the model parameters. The purpose is thus to iterate until convergence, and then to report the resulting parameters as the MLE (maximum likelihood estimate); convergence to the MLE is not guaranteed, however. For our HMM, the algorithm is outlined in Appendix 2.1. The second problem above can be viewed as that of reconstructing the Markov trajectory, given data (and model parameters, usually estimated ones). This problem is solved using the so-called Viterbi algorithm, which is a dynamic programming algorithm that recursively finds the most likely path. This algorithm, for our HMM, is described in Appendix 2.2.

2. Methods

2.1. The EM algorithm

The parameters to estimate are the ploidy K, the fraction γ of normal tissue, and, for each chromosome c, the two transition rates λ_c and η_c and the initial distribution δ_c. Our starting point is the EM algorithm for continuous-index hidden Markov chains by Roberts and Ephraim.²³ As latent (unobserved) data we take the whole Markov trajectory (X_c(t))_t1c≤t≤Tc for each chromosome c, but the complete likelihood involves only the sufficient statistics consisting of the initial state X_c(t_1c), the total lengths T_nc and T_ac of sojourns in the normal state and in abnormal states respectively, and the numbers m_·ac and m_anc of jumps to abnormal states, and from abnormal states to the normal state respectively, for chromosome c.

With these sufficient statistics, and recalling that each λ_c has a Gamma prior with shape and intensity parameters say ${\alpha }_c^{\lambda }$ and ${\beta }_c^{\lambda }$, and analogously for each η_c, the complete log-posterior, ie, the sum of the complete log-likelihood and the log-prior, is, up to a constant not depending on the parameters,

$$\begin{array}{lll}{L}^c\hspace{0.17em}(\theta ;\mathit{\text{X}},\hspace{0.17em}\mathit{\text{y}})\hfill & =\hfill & \sum _c\{\text{log}\hspace{0.17em}{\delta }_{X(t_1),c}\hspace{0.17em}+\hspace{0.17em}{m}_{\cdot \text{a}c}\hspace{0.17em}\text{log}\hspace{0.17em}{\lambda }_c\hspace{0.17em}+\hspace{0.17em}{m}_{\text{an}c}\hspace{0.17em}\text{log}\hspace{0.17em}{\eta }_c\hspace{0.17em}\hfill \\ \hfill & \hfill & -{\lambda }_c\hspace{0.17em}{T}_{\text{n}c}\hspace{0.17em}-\hspace{0.17em}({\lambda }_c\hspace{0.17em}+\hspace{0.17em}{\eta }_c)T_{\text{a}c}\hfill \\ \hfill & \hfill & +\hspace{0.17em}\left({\alpha }_c^{\lambda }\hspace{0.17em}-\hspace{0.17em}1\right)\text{log}\hspace{0.17em}{\lambda }_c\hspace{0.17em}-\hspace{0.17em}{\beta }_c^{\lambda }\hspace{0.17em}{\lambda }_c\hfill \\ \hfill & \hfill & +\hspace{0.17em}\left({\alpha }_c^{\eta }\hspace{0.17em}-\hspace{0.17em}1\right)\text{log}\hspace{0.17em}{\eta }_c\hspace{0.17em}-\hspace{0.17em}{\beta }_c^{\eta }\hspace{0.17em}{\eta }_c\hfill \\ \hfill & \hfill & +\sum _{k=1}^{N_c}\text{log}\hspace{0.17em}{f}_{Y_{kc}|X_{kc}}\hspace{0.17em}(y_{kc}|X_{kc};\hspace{0.17em}\gamma ,\hspace{0.17em}K)\hspace{0.17em}\},\hfill \end{array}$$

where θ = (δ_c, λ_c, η_c, K, γ). Moreover, y = {y_kc} is the collection of all data and X = {(X_c(t))} is the collection of all (unobserved) Markov chain trajectories. The quantity to maximize in one iteration of the EM algorithm is

$$Q(\theta ;\theta \prime )\hspace{0.17em}=\hspace{0.17em}{E}_{\theta }\hspace{0.17em}[L^c(\theta \prime ;\hspace{0.17em}\mathit{\text{X}},\mathit{\text{y}})|\mathit{\text{y}}],$$

where maximization is with respect to θ′ and the notation E_θ indicates that the expectation is computed under the current parameter (estimate) θ. Note that L_c(θ′;X,y) and hence also Q(θ;θ′) split into two distinct parts, one of which depends on (δ_c, λ_c, η_c) only and one of which depends on K and γ only. Maximization with respect to (δ_c, λ_c, η_c) and with respect to (K, γ) can thus be carried out separately.

Also note that K and γ are common across the genome, and therefore estimated using the data for all chromosomes. For each iteration of the EM algorithm we compute the forward and backward variables for each chromosome, store them, and then re-estimate K and γ using the information from all chromosomes.

The M-steps for the transition rates read

$${\widehat{\lambda }}_c\hspace{0.17em}=\hspace{0.17em}\frac{{\alpha }_c^{\lambda }\hspace{0.17em}-\hspace{0.17em}1\hspace{0.17em}+\hspace{0.17em}{\widehat{m}}_{\cdot \text{a}c}}{{\beta }_c^{\lambda }\hspace{0.17em}+\hspace{0.17em}{\widehat{T}}_{\text{a}c}\hspace{0.17em}+\hspace{0.17em}{\widehat{T}}_{\text{n}c}},\hspace{0.17em}{\widehat{\eta }}_c\hspace{0.17em}=\hspace{0.17em}\frac{{\alpha }_c^{\eta }\hspace{0.17em}-\hspace{0.17em}1\hspace{0.17em}+\hspace{0.17em}{\widehat{m}}_{\text{a}nc}}{{\beta }_c^{\eta }\hspace{0.17em}+\hspace{0.17em}{\widehat{T}}_{\text{a}c}},$$

where m̂_·ac=E_θ[m_·ac|y_1c, ⋯, y_c,Nc] etc. Note that T_ac + T_nc equal the length of the Markov chain trajectory for chromosome c, ie, T_c – t_1c, so that also T̂_ac+ T̂_nc= T_c – t_1c. Moreover, the M-step for the initial distributions is

$${\widehat{\delta }}_{ic}\hspace{0.17em}=\hspace{0.17em}{P}_{\theta }(X(t_1)\hspace{0.17em}=\hspace{0.17em}i|y_{1c},\hspace{0.17em}\cdots ,\hspace{0.17em}{y}_{c,N_c}).$$

The M-step for the ploidy is

$$\widehat{K}\hspace{0.17em}=\hspace{0.17em}-\frac{V}{4U}\hspace{0.17em}+\hspace{0.17em}\sqrt{\frac{V^2}{16U^2}\hspace{0.17em}-\hspace{0.17em}\frac{{\Sigma }_c\hspace{0.17em}{N}_c}{U}},$$

where

$$\begin{array}{lll}U\hfill & =\hfill & -\sum _{c,k,i,g\in G_i}\frac{1}{8{\nu }_{kc}\hspace{0.17em}(1\hspace{0.17em}-\hspace{0.17em}{\rho }_{kc}^2)}\hspace{0.17em}\left(\frac{y_{\text{A}kc}^2}{{\mu }_{\text{A}kcg}^2}\hspace{0.17em}-\hspace{0.17em}\frac{2{\rho }_{kc}{y}_{\text{A}kc}{y}_{\text{B}kc}}{{\mu }_{\text{A}kcg}{\mu }_{\text{B}kcg}}\hspace{0.17em}+\hspace{0.17em}\frac{y_{\text{B}kc}^2}{{\mu }_{\text{B}kcg}^2}\right)\hfill \\ \hfill & \hfill & \times \hspace{0.17em}{P}_{\theta }(X_c\hspace{0.17em}(t_{kc})\hspace{0.17em}=\hspace{0.17em}i,\hspace{0.17em}{G}_{kc}\hspace{0.17em}=\hspace{0.17em}g|y_{1c},\hspace{0.17em}\cdots ,\hspace{0.17em}{y}_{c,N_c})\hfill \end{array}$$

and

$$\begin{array}{lll}V\hfill & =\hfill & -\sum _{c,k,i,g\in G_i}\frac{1}{2v_{kc}\hspace{0.17em}(1\hspace{0.17em}+\hspace{0.17em}{\rho }_{kc})}\hspace{0.17em}\left(\frac{y_{\text{A}kc}}{{\mu }_{\text{A}kcg}}\hspace{0.17em}+\hspace{0.17em}\frac{y_{\text{B}kc}}{{\mu }_{\text{B}kcg}}\right)\hfill \\ \hfill & \hfill & \times \hspace{0.17em}{P}_{\theta }(X_c\hspace{0.17em}(t_{kc})\hspace{0.17em}=\hspace{0.17em}i,\hspace{0.17em}{G}_{kc}\hspace{0.17em}=\hspace{0.17em}g|y_{1c},\hspace{0.17em}\cdots ,\hspace{0.17em}{y}_{c,N_c})\hfill \end{array}$$

For the fraction γ of normal contamination there is no closed form expression for the M-step, and to re-estimate γ we maximize Q(θ;·), as a function of γ, numerically. Note however that K̂ above depends on γ, which appears implicitly in the means μ_Akcg and μ_Bkcg used to compute U and V. Therefore, by maximizing w.r.t. K′ (using the current γ) and then w.r.t. γ′ (using the re-estimated K̂), as we do, and not w.r.t. K′ and γ′ jointly, we in fact obtain a generalized EM algorithm rather than an EM algorithm, in the terminology of Dempster et al⁵ (Eq. (3.5)).

The conditional expectations m̂_·ac etc. are computed in the E-step, which follows that of Roberts and Ephraim²³ with minor changes. Now write y_k:l,c for {y_kc, y_{k+ 1,c}, …, y_lc}, and let m_ijc be the number of jumps by the Markov chain from state i to state j, in chromosome c. Then

$$\begin{array}{lll}{\widehat{m}}_{ijc}\hfill & =\hfill & E_{\theta }[m_{ijc}\hspace{0.17em}|\hspace{0.17em}{y}_{1:N_c,c}]\hfill \\ \hfill & =\hfill & {\int }_{\theta }^{T_c}{P}_{\theta }(X_c(t-)\hspace{0.17em}=\hspace{0.17em}i,X_c(t)\hspace{0.17em}=\hspace{0.17em}j|y_{1:N_c,c})dt\hfill \\ \hfill & =\hfill & {\int }_0^{T_c}\frac{P_{\theta }(X_c(t-)=i,X_c(t)=j)}{p_{\theta }(y_{1:N_c,c})}\hfill \\ \hfill & \hfill & \times \hspace{0.17em}{p}_{\theta }(y_{1:N_c,c}|X_c(t-)\hspace{0.17em}=\hspace{0.17em}i,\hspace{0.17em}{X}_c(t)=\hspace{0.17em}j)dt\hfill \\ \hfill & =\hfill & \sum _{k=2}^{N_c}{\int }_{t_{k-1,c}}^{t_{kc}}\frac{P_{\theta }(X_c(t)=j|X_c(t-)=i)P_{\theta }(X_c(t-)=i)}{p_{\theta }(y_{1:N_c,c})}\hfill \\ \hfill & \hfill & \times p_{\theta }(y_{1:k-1,c},y_{k:N_c,c}|X_c(t-)=i,X_c(t)=j)dt;\hfill \end{array}$$

here the symbol P denotes probabilities as well as densities; note that P_θ(X_c(t) = j|X_c(t−) = i) = q_ijc, where q_ijc is the transition rate from state i to state j in chromosome c. Thus, with r_abnormal being the number of abnormal states, q_ijc is equal to λ_c/r_abnormal if i is the normal state and j is any abnormal state, equal to λ_c/(r_abnormal − 1) if i and j are both abnormal states (because the chain cannot jump from a state to itself), and equal to η_c if i is any abnormal state and j is the normal state. Given the HMM structure it follows that y_1:k–1,c and y_k:Nc,c are conditionally independent given X_c(t−) = i and X_c(t) = j, whence

$$\begin{array}{lll}{\widehat{m}}_{ijc}\hfill & =\hfill & \frac{q_{ijc}}{p_{\theta }(y_{1:N_c,c})}\sum _{\kappa =2}^{N_c}{\int }_{t_{k-1,c}}^{t_{kc}}{P}_{\theta }(X_c(t-)=i)\hspace{0.17em}{p}_{\theta }(y_{1:k-1,c}|X_c(t-)=i)\hfill \\ \hfill & \hfill & \times P_{\theta }(y_{k:N_c,c}|X_c(t)=j)dt\hfill \\ \hfill & =\hfill & \frac{q_{ijc}}{p_{\theta }(y_{1:N_c,c})}\sum _{\kappa =2}^{N_c}{\int }_0^{h_{kc}}{P}_{\theta }(y_{1:k-1,c},X_c(t_{k-1,c}+t-)=i)\hfill \\ \hfill & \hfill & \times P(y_{k:N_c,c}|X_c(t_{k-1,c}+t)=j)dt\hfill \end{array}$$

with h_kc = t_kc – t_k–1,c. Here, the two factors in the integrand on the right-hand side are the forward and backward densities respectively.

To compute these factors, and similar ones, we use a forward-backward type algorithm. Let r be the number of Markov states, and let B_kc be the r × r diagonal matrix whose (i, i)-th element is the probability density function of y_kc given Markov state i at position t_kc, ie, f_{YkcXc (tkc)} = i(y_kc) in Eq. (3). Further let F_kc be the r × r matrix whose (i, j)-th element is

$$\begin{array}{lll}{[F_{kc}]}_{ij}\hfill & =\hfill & P_{\theta }(y_{kc},\hspace{0.17em}{X}_c\hspace{0.17em}(t_{kc})\hspace{0.17em}=\hspace{0.17em}j|X_c(t_{k-1,c})\hspace{0.17em}=\hspace{0.17em}i)\hfill \\ \hfill & =\hfill & {[\text{exp}(Q_c{h}_{kc})]}_{ij}{[B_{kc}]}_{ij},\hfill \end{array}$$

where Q_c is the matrix with elements q_ijc, i, j = 1, 2, …, r for i ≠ j, and diagonal elements q_cii being the negative of the total rate out of state i for chromosome c (the row sums of Q_c then become zero). We note that the discrete-index process (X_c(t_kc))_1≤k≤Nc, ie, the continuous-index process (X_c(·)) sampled at the locations of the probes, is a non-homogeneous discrete-index Markov chain with transition probability matrices, from t_k–1,c to t_kc, given by exp(Q_ch_kc). With this matrix notation we have F_kc = exp(Q_ch_kc)B_kc, and the likelihood for chromosome c can be written

$$p_{\theta }(y_{1:N_c,c})\hspace{0.17em}=\hspace{0.17em}{\delta }_c{B}_{1c}\hspace{0.17em}\left(\prod _{k=2}^{N_c}{F}_{kc}\right)\mathbf{1}$$

where 1 is the r × 1 vector of all ones. The forward densities are

$$\begin{array}{l}{P}_{\theta }(y_{1:k-1,c,}{X}_c(t_{k-1,c}+t-)=i)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\sum _{s=1}^r{P}_{\theta }(y_{1:k-1,c},X_c(t_{k-1,c})=s)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\times P_{\theta }(X_c(t_{k-1,c}+t-)=i|X_c(t_{k-1,c})=s)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\sum _{s=1}^r\left({\delta }_c{B}_{1c}\prod _{\kappa =2}^{k-1}{F}_{\kappa c}\right){\mathbf{1}}_s{[\text{exp}(Q_{c}t)]}_{\mathit{\text{si}}},\end{array}$$

where 1_j is the r × 1 vector whose elements are zero except for element j which is one.

The backward densities are

$$\begin{array}{l}{p}_{\theta }(y_{k:N_c,c}|X_c(t_{k-1,c}+t)=j)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\sum _{s=1}^r{p}_{\theta }(y_{k+1:N_c,c}|X_c(t_{\mathit{\text{kc}}})=s)p_{\theta }(y_{\mathit{\text{kc}}}|X_c(t_{\mathit{\text{kc}}})=s)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\times P_{\theta }(X_c(t_{\mathit{\text{kc}}})=s|X_c(t_{k-1,c}+t)=j)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\sum _{s=1}^r{[B_{\mathit{\text{kc}}}]}_{\mathit{\text{ss}}}{[\text{exp}(Q_c(h_{\mathit{\text{kc}}}-t))]}_{\mathit{\text{js}}}{{\mathbf{1}}^{\prime }}_s\left(\prod _{\kappa =k+1}^{N_c}{F}_{\kappa c}\right)\mathbf{1}.\end{array}$$

The above matrix multiplications are numerically unstable, as the products will either tend to zero or infinity exponentially fast as the number of factors increases. Therefore scaled versions of these recursions are introduced. The scaled forward densities with normalizing constants d_kc at probe (k, c) are

$$L_c(k)=\frac{{\delta }_c{B}_{1c}}{d_{1c}}\prod _{\kappa =2}^k\frac{F_{\kappa c}}{d_{\kappa c}},$$

which we compute recursively as

$$L_c(k)=\frac{L_c(k-1)F_{\mathit{\text{kc}}}}{d_{\mathit{\text{kc}}}}$$

with d_1c = δ_cB_1c1, L_c(1) = δ_cB_1c/d_1c, d_kc = L_c(k – 1)F_kc1.

The scaled backward densities are

$$R_c(k)=\prod _{\kappa =k+1}^{N_c}\frac{F_{\kappa c}}{d_{\kappa c}}\mathbf{1},$$

which we compute as

$$R_c(k)=\frac{F_{k+1}{R}_c(k+1)}{d_{k+1,c}}$$

with R_c(N_c) = 1.

Using these scales quantities, the matrix m̂_c with entries m̂_ijc can be expressed as

$${\widehat{m}}_c=Q_c\odot {I^{\prime }}_{k,c},$$

where ⊙ denotes element-wise multiplication and

$$I_{\mathit{\text{kc}}}={\int }_0^{h_{\mathit{\text{kc}}}}\text{exp}(Q_c(h_{\mathit{\text{kc}}}-t))\hspace{0.17em}{V}_c\hspace{0.17em}\text{exp}(Q_{c}t)\mathit{\text{dt}}$$

with

$$V_c=\sum _{k=2}^{N_c}\frac{B_{\mathit{\text{kc}}}{R}_c(k)L_c(k-1)}{d_{k,c}}.$$

The integrals I_kc are evaluated using the matrix

$$D_c=\left(\begin{array}{ll}{Q}_c\hfill & V_c\hfill \\ 0\hfill & Q_c\hfill \end{array}\right);$$

I_kc is then upper right r × r block of exp(D_ch_kc).

Finally, recalling that the normal state is state 4,

$$\begin{array}{l}{\widehat{m}}_{.\text{a}c}=\sum _{1\le i\le r\hspace{0.17em}1\le j\le r,}\sum _{j\ne i,j\ne 4}{\widehat{m}}_{\mathit{\text{ijc}}},\\ {\widehat{m}}_{\text{an}c}=\sum _{1\le i\le r,i\ne 4}{\widehat{m}}_{i4c}.\end{array}$$

Using similar types of computations is follows that

$${\widehat{T}}_{\mathit{\text{ic}}}=E[T_{\mathit{\text{ic}}}|y_{1:N_c,c}]={\widehat{m}}_{\mathit{\text{iic}}}/q_{\mathit{\text{iic}}},$$

where T_ic is the total length of all sojourns of the Markov chain in state i within chromosome c. Moreover,

$$P_{\theta }(X(t_{1c})=i|y_{1:N_c,c})\propto L_c{(1)}_i{R}_c{(1)}_i,$$

and the conditional probabilities in the expressions for U and V are computed using

$$\begin{array}{l}P(X_c(t_{\mathit{\text{kc}}})=i,G_{\mathit{\text{kc}}}=g|y_{1:N_c})\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=P(G_{\mathit{\text{kc}}}=g|X_{\mathit{\text{kc}}}=i,y_{1:N_c,c})P(X_{\mathit{\text{kc}}}=i|y_{1:N_c,c})\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\propto w_{\mathit{\text{kcig}}}f(y_{\mathit{\text{kc}}}|G_{\mathit{\text{kc}}}=g)L_c{(k)}_i{R}_c{(k)}_i,\end{array}$$

where the weights and densities on the right-hand side are those in Eq. (3).

2.2. The Viterbi algorithm

We used a Viterbi algorithm, adapted to the continuous-index structure, to find the aposteriori most likely Markov chain trajectory. The algorithm is the usual Viterbi algorithm, but with transition probability matrices exp(Q_ch_kc) that vary with probe index (c, k). The algorithm thus finds the most likely sequence at the probe locations only. When the estimated reconstruction of each Markov state X(t_kc) is available, one may also estimate the corresponding genotype G_kc (see below).

For any chromosome c, the Viterbi algorithm is as follows. To ensure numeric stability, it operates on log-scale.

1. Put ξ_1c(i) = log(δ_ic[B_1c]_ii) for i = 1, …, r.

2. Iterate for k = 2, 3, …, N_c,

   $$\begin{array}{l}{\xi }_{\mathit{\text{kc}}}(j)=\underset{i}{\text{max}}\{{\xi }_{k-1}(i)+\text{log}{[\text{exp}(Q_c{h}_{\mathit{\text{kc}}})]}_{\mathit{\text{ij}}}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\text{log}{[B_{\mathit{\text{kc}}}]}_{\mathit{\text{jj}}}\}\end{array}$$

   for i = 1, 2, …, r.

3. Put x̂_c(N_c) = arg max_i ξ_Ncc(i).

4. Iterate for k = N_c – 1, N_c – 2, …, 1,

   $$\begin{array}{l}{\widehat{x}}_c(k)=\text{arg}\hspace{0.17em}\hspace{0.17em}\underset{i}{\text{max}}\{{\xi }_{\mathit{\text{kc}}}(i)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\text{log}{[\text{exp}(Q{h}_{k+1})]}_{i,{\widehat{x}}_c(k+1)}\}.\end{array}$$

Having reconstructed the states x_c(t_kc), it holds that the corresponding genotypes, given the Markov chain and intensity data, are conditionally independent with

$$\begin{array}{l}P(G_{\mathit{\text{kc}}}=g|X_c(t_{\mathit{\text{kc}}})=i,Y_{\mathit{\text{kc}}}=y)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\frac{w_{\mathit{\text{kcig}}}{f}_{Y_{\mathit{\text{kc}}}|G_{\mathit{\text{kc}}}}(y|g)}{{\Sigma }_{g^{\prime }\in G_i}{w}_{\mathit{\text{kci}}{g}^{\prime }}{f}_{Y_{\mathit{\text{kc}}}|G_{\mathit{\text{kc}}}}(y|g^{\prime })}\end{array}$$

for all g ∈ G_i; here f_Ykc|Gkc (y| g) is the bivariate Normal density as in Eq. (3). Selecting, for each probe (k, c), G_kc as the genotype g ∈ G_i maximizing this expression thus yields a maximum aposteriori (MAP) reconstruction of the genotypes at all probes.