Nonparametric Bayes Differential Analysis of Multigroup DNA Methylation Data

Abstract

DNA methylation datasets in cancer studies are comprised of measurements on a large number of genomic locations called cytosine-phosphate-guanine (CpG) sites with complex correlation structures. A fundamental goal of these studies is the development of statistical techniques that can identify disease genomic signatures across multiple patient groups defined by different experimental or biological conditions. We propose BayesDiff, a nonparametric Bayesian approach for differential analysis relying on a novel class of first order mixture models called the Sticky Pitman-Yor process or two-restaurant two-cuisine franchise (2R2CF). The BayesDiff methodology flexibly utilizes information from all CpG sites or biomarker probes, adaptively accommodates any serial dependence due to the widely varying inter-probe distances, and makes posterior inferences about the differential genomic signature of patient groups. Using simulation studies, we demonstrate the effectiveness of the BayesDiff procedure relative to existing statistical techniques for differential DNA methylation. The methodology is applied to analyze a gastrointestinal (GI) cancer dataset exhibiting serial correlation and complex interaction patterns. The results support and complement known aspects of DNA methylation and gene association in upper GI cancers.

1. Introduction

Recent advances in array-based and next-generation sequencing (NGS) technologies have revolutionized biomedical research, especially in cancer. The rapid decline in the cost of genome technologies has facilitated the availability of datasets involving intrinsically different sizes and scales of high-throughput data and provided genome-wide, high resolution information about the biology of cancer. A common analytical goal is the identification of differential genomic signatures between groups of samples corresponding to different treatments or biological conditions, e.g., treatment arms, response to adjuvant chemotherapy, tumor subtypes, or cancer stages. Challenges include the high dimensionality of genomic biomarkers or probes, usually in the hundreds of thousands, and the relatively small number of patient samples, usually no more than a few hundred. This “small $n$, large $p$” setting results in unstable inferences due to collinearity. Further, the data exhibit complex interaction patterns, such as signaling or functional pathway-based interactions, and location-based serial correlation for high-throughput sequencing data. These characteristics challenge statistical techniques for detecting differential signatures.

Differential DNA methylation in cancer studies

DNA methylation is an epigenetic mechanism that involves the addition of a methyl (CH₃) group to DNA, resulting in the modification of gene functions. It typically occurs at genomic locations called cytosine-phosphate-guanine (CpG) sites. Alterations in DNA methylation, e.g., hypomethylation of oncogenes and hypermethylation of tumor suppressor genes, are often associated with the development and progression of cancer (Feinberg and Tycko, 2004). It was previously believed that these alterations occur almost exclusively at promoter regions known as CpG islands, i.e., chromosomal regions with high concentrations of CpG sites. However, with the advent of high-throughput technologies, it has been shown that a significant proportion of cancer-related alterations do not occur in promoters or CpG islands (Irizarry et al., 2009), prompting higher resolution, epigenome-wide investigations.

Gastrointestinal (GI) cancer, the most common form of cancer in the U.S. (Siegel et al., 2017), refers to malignant conditions affecting the digestive system associated with epigenetic alterations (Vedeld et al., 2017). Molecular characterization of different cancer types, facilitated by the identification of differentially methylated CpG sites, is therefore key to better understanding GI cancer. In the motivating application, we analyze methylation profiles publicly available from The Cancer Genome Atlas (TCGA) project and comprising 1,224 tumor samples belonging to four GI cancers of the upper digestive tract: stomach adenocarcinoma (STAD), liver hepatocellular carcinoma (LIHC), esophageal carcinoma (ESCA) and pancreatic adenocarcinoma (PAAD). For 485,577 probes, where each probe is mapped to a CpG site, DNA methylation levels or Beta-values ranging from 0 (no methylation) to 1 (full methylation) were measured using the Illumina Human Methylation 450 platform.

Figure 1 displays the methylation levels for CpG sites near TP53, a tumor suppressor gene located on chromosome 17. A random subset of the tumor samples was chosen to facilitate an informal visual evaluation. Each plotted point represents the methylation level of a tumor sample at a CpG site. As indicated in the figure legend, the four sets of colors and shapes of the points represent the four upper GI cancers. The vertical dashed lines indicate the boundaries of the TP53 gene. Although differential methylation is clearly visible at some CpG sites, the differences are generally subtle, demonstrating the need for sophisticated statistical analyses. An obvious feature is the correlation of the apparent methylation statuses of nearby CpG sites (Eckhardt et al., 2006; Irizarry et al., 2008; Leek et al., 2010). The dependence of proximal CpG sites is also seen in Figure 6 of Supplementary Material with highly significant tests for serial correlations. Furthermore, the highly variable inter-probe spacings in Figure 1 suggests the need to model distance-based dependencies.

Figure 1:

Methylation levels of CpG sites near gene TP53 for randomly chosen tumor samples of the TCGA upper GI dataset. Each plotted point represents the methylation level at a CpG site, with the shapes and colors corresponding to the different GI cancers indicated in the legend. The vertical dashed lines demarcate the TP53 gene boundaries.

Existing statistical approaches for differential DNA methylation and limitations

Numerous frequentist and Bayesian methods have been developed for differential DNA methylation, and can be broadly classified into four categories: (i) Testing-based methods, such as Illumina Methylation Analyzer (IMA) (Wang et al., 2012), City of Hope CpG Island Analysis Pipeline (COHCAP) (Warden et al., 2013), and BSmooth (Hansen et al., 2012). These methods rely on two-sample or multiple-sample tests for the mean group differences at each CpG site. (ii) Regression based models, such as Methylkit (Akalin et al., 2012), bump hunting (Jaffe et al., 2012), Biseq (Hebestreit et al., 2013), and RADMeth (Dolzhenko and Smith, 2014). After applying smoothing or other adjustments, these methods fit individual regression models for each CpG site and test for significance. (iii) Beta-binomial model-based methods, such as MOABS (Sun et al., 2014), DSS (Feng et al., 2014), and methylSig (Park et al., 2014). These methods fit separate models to each CpG site. (iv) Hidden Markov models (HMMs), such as Meth-Pipe (Song et al., 2013), Bisulfighter (Saito et al., 2014), and HMM-DM (Yu and Sun, 2016). These methods detect differentially methylated sites using inferred hidden states.

The aforementioned methods have several deficiencies. Because they fit separate models to each probe, most methods ignore the strong correlations between neighboring probes, reducing detection power. Additionally, beta-binomial, HMM, and most testing-based methods are able to accommodate only two treatments and rely on inefficient adjustments for multiple treatments. The methods that account for serial dependence (e.g., HMMs) do not adjust for the widely varying inter-probe distances, and instead, assume uniform inter-probe dependencies. The few methods that account for inter-probe distances (e.g., Hansen et al., 2012; Jaffe et al., 2012; Hebestreit et al., 2013) rely on ad hoc parameter-tuning procedures that do not adjust for the data characteristics.

Motivated by these challenges, we propose general and flexible methodology for differential analysis in DNA methylation data, referred to as BayesDiff. Rather than fitting a separate model to each CpG site or probe, BayesDiff relies on a common analytical framework for simultaneous inferences that adapts to the unique data attributes. To diminish collinearity effects and achieve dimension reduction, the probes are allocated to a smaller, unknown number of latent clusters based on the similarities of probe-specific multivariate parameters. Posterior inferences are made on differential state variables to delineate the disease genomic signature of multiple treatments.

For realistically modeling the probe-cluster allocation mechanism of DNA methylation profiles, we devise an extension of Pitman-Yor processes (PYPs) (Perman et al., 1992) called the Sticky PYP (equivalently, the two-restaurant two-cuisine franchise). In addition to accounting for long-range biological interactions, this nonparametric process accommodates distance-based serial dependencies. Separately for the differential and non-differential probes, the data flexibly direct the choice between PYPs, and their special case, Dirichlet processes, in finding the best-fitting allocation schemes.

We implement an inferential procedure for Sticky PYPs using a Markov chain Monte Carlo (MCMC) algorithm specifically designed for posterior inferences in the typically large methylation datasets. Simulation results show that our approach significantly outperforms existing methods for multigroup comparisons in data with or without serial correlation. For the motivating TCGA dataset, in addition to confirming known features of DNA methylation and disease-gene associations, the analysis reveals interesting aspects of the biological mechanisms of upper GI cancers.

The rest of the paper is organized as follows. Section 2 describes the BayesDiff approach, with Section 2.1 introducing the Sticky PYP or two-restaurant two-cuisine franchise (2R2CF) for differential DNA methylation. Section 3 outlines an effective inference procedure for detecting differential probes. Section 4 uses artificial datasets with varying noise and correlation levels to assess the accuracy of BayesDiff in detecting disease genomic signatures and compares BayesDiff with established techniques for DNA methylation data. The motivating upper GI dataset is analyzed in Section 5. Finally, conclusions and future work are discussed in Section 6.

2. The BayesDiff Model

Sequencing technologies measure DNA methylation levels of $p$ biomarkers represented by CpG sites (“probes”) and $n$ matched patient or tissue samples (“individuals”). Usually, $p$ is much larger than $n$. The methylation levels, which belong to the interval [0, 1], are arranged in an $n\times p$ matrix of proportions, $\mathbf{X}=\left(\left(x_{ij}\right)\right)$, for individuals $i$ and probes $j$, with the probes sequentially indexed by their genomic locations. The distances between adjacent probes are denoted by $e_1,\dots ,e_{p-1}$, and typically exhibit high variability. For instance, in the upper GI TCGA dataset, the inter-probe distances range from 2 base pairs to a million base pairs; a base pair is a unit of DNA length consisting of two nucleobases bound to each other by hydrogen bonds (e.g., Baker et al., 2008).

Each individual $i$ is associated with a known experimental or biological condition (“treatment”) denoted by $t_i$ and taking values in $\{1,...,T\}$ with $T\ge 2$. In the motivating TCGA data, there are $T=4$ upper GI cancer types. We model the logit transformation of the methylation levels, $z_{ij}=\mathrm{l}\mathrm{o}\mathrm{g}\left(x_{ij}/\left(1-x_{ij}\right)\right)$, as follows:

$$z_{ij}\sim N\left({\xi }_i+{\chi }_j+{\theta }_{t_{i}j},{\sigma }^2\right)$$ (2.1)

where ${\xi }_i$ represents the ith subject’s random effect, ${\chi }_j$ represents the jth probe’s random effect, and ${\theta }_{tj}$ is the treatment $t$–probe $j$ interaction random effect; refer to the directed acyclic graph (DAG) in Figure 11 of Supplementary Material. Logit methylation levels differ from M-values (Du et al., 2010), commonly used in differential analyses, by $1-\mathrm{l}\mathrm{o}\mathrm{g}\left(2\right)=0.306$; however, the key results are identical for both transformations.

The main inferential goal is the detection of differential probes, i.e., probes $j$, for which ${\mathit{\theta }}_j={\left({\theta }_{1j},\dots ,{\theta }_{Tj}\right)}^{\prime }$ does not have all identical elements. Consequently, we define a binary differential state variable, $s_j$, with $s_j=1$ indicating that probe $j$ is not differential and $s_j=2$ indicating that it is differential:

$$s_j=\left\{\begin{array}{ll}1& \text{if}{\theta }_{1j}={\theta }_{2j}=\cdots ={\theta }_{Tj},\\ 2& \text{otherwise},\end{array}\right.$$ (2.2)

for $j=1,...,p.$ The parameters of interest are $s_1,\dots ,s_p$, with the differential genomic signature consisting of probes with $s_j=2$. Figure 11 of Supplementary Material positions the differential state variables in the BayesDiff model hierarchy. Motivated by the distance-dependent correlations of DNA methylation data and the deficiencies of existing statistical approaches, this paper fosters a Bayesian nonparametric framework for random effects ${\mathit{\theta }}_1,\dots ,{\mathit{\theta }}_p$ underlying the differential state variables.

Modeling probe clusters

In addition to high-dimensionality, the analytical challenges include pervasive collinearity caused by dependencies between physically proximal probes. Additionally, non-adjacent probes may have long-range dependencies due to biological interactions, e.g., signaling or functional pathways. To accommodate these complex dependence structures and extract information from the large number of probes, we allocate the $p$ probes to a much smaller number, $q$, of latent clusters based on the similarities of their random effects ${\mathit{\theta }}_j$. We favor clustering to dimension reduction methods such as principal components analysis (PCA); each PC being a linear combination of all $p$ biomarkers, PCA is less useful because it is unable to select sparse features, i.e., probes. By contrast, the proposed approach facilities biological interpretations by identifying CpG sites relevant to the differential genomic signatures between multiple treatments.

Suppose an allocation variable, $c_j$, assigns probe $j$ to one of $q$ latent clusters, where $q$ is unknown. The event $\left[c_j=k\right]$ indicates that the j^th probe is assigned to the k^th latent cluster, $k=1,...,q.$ We assume that the $q$ clusters are associated with latent vectors, ${\mathit{\lambda }}_1,\dots ,{\mathit{\lambda }}_q$, where the probe-specific random effects and cluster-specific latent vectors have the relation:

$${\mathit{\theta }}_j={\mathit{\lambda }}_k\text{if}{c}_j=k.$$ (2.3)

That is, all probes in a cluster are assumed to have identical random effects equal to that cluster’s latent vector. The differential state variables, defined in equation (2.2), then become a shared attribute of their parent cluster, and clusters as a whole are either differentially or non-differentially methylated. Further, if probe $j$ belongs to cluster $k$ (i.e., $c_j=k$), then the condition ${\theta }_{1j}={\theta }_{2j}=\cdots ={\theta }_{Tj}$ in equation (2.2) is equivalent to ${\lambda }_{1k}={\lambda }_{2k}=\cdots ={\lambda }_{Tk}$, and the differential cluster indexes comprise

$$𝒟=\left\{k:{\lambda }_{tk}\ne {\lambda }_{t^{\prime }k},\text{for}\text{some}t\ne t^{\prime },k=1,\dots ,q\right\}.$$ (2.4)

Mixture models for allocation

Bayesian infinite mixture models are a natural choice for allocating $p$ probes to a smaller, unknown number of latent clusters. Dirichlet processes (Ferguson, 1973) are arguably the most frequently used infinite mixture models; see Müller and Mitra (2013, chap. 4) for a comprehensive review. The use of Dirichlet processes to achieve dimension reduction has precedence in the literature, albeit in unrelated applications (see Medvedovic et al., 2004; Kim et al., 2006; Dunson et al., 2008; Dunson and Park, 2008; Guha and Baladandayuthapani, 2016). Lijoi, Mena, and Prünster (2007a) advocated the use of Gibbs-type priors (Gnedin and Pitman, 2006) for accommodating more flexible clustering mechanisms and demonstrated the utility of Pitman-Yor processes (PYPs) in genomic applications. An overview of Gibbs-type priors and characterization of the learning mechanism is provided by De Blasi et al. (2015). Formally, the PYP (Perman et al., 1992) relies on a discount parameter $d\in [0,1)$, positive mass parameter $\alpha$, and $T$-variate base distribution $W$, and is denoted by $𝒲(d,\alpha ,W)$. The value $d=0$ yields a Dirichlet process with mass parameter $\alpha$ and base distribution $W$. Suppose ${\mathit{\theta }}_1,\dots ,{\mathit{\theta }}_p$ are distributed as $𝒲(d,\alpha ,W)$. The stick-breaking representation of $𝒲(d,\alpha ,W)$ is ${\mathit{\theta }}_j\stackrel{\text{i.i.d.}}{\sim }𝒫$, where random distribution $𝒫$ is the discrete mixture ${\sum }_{v=1}^{\mathrm{\infty }} {\pi }_v{\delta }_{{\mathit{\varphi }}_v}$, with ${\delta }_{{\mathit{\varphi }}_v}$ denoting a point mass located at the atom ${\mathit{\varphi }}_v\stackrel{\text{i.i.d.}}{\sim }W$. The random stick-breaking probabilities have the form ${\pi }_1=V_1$, and ${\pi }_h=V_h{\prod }_{v=1}^{h-1} \left(1-V_v\right)$ for $h>1$, where $V_h\stackrel{\text{indep}}{\sim }\mathrm{b}\mathrm{e}\mathrm{t}\mathrm{a}(1-d,\alpha +hd)$. Guha and Baladandayuthapani (2016) introduced VariScan, a technique that utilizes PYPs and Dirichlet processes for clustering, variable selection, and prediction in high-dimensional regression problems in general, and in gene expression datasets in particular. They also demonstrated that PYPs are overwhelmingly favored over Dirichlet processes in gene expression datasets, which typically exhibit no serial correlation.

Limitations of existing mixture models

Although the aforementioned mixture models achieve dimension reduction and account for long-range biological interactions between non-adjacent probes, a potential drawback is their implicit assumption of a priori exchangeability of the probes. Consequently, these techniques cannot accommodate serial correlation in methylation data. Infinite HMMs, such as the hierarchical Dirichlet process hidden Markov model (HDP-HMM) (Teh et al., 2006) and Sticky HDP-HMM (Fox et al., 2011), could be utilized to fill this gap. Although these models are a step in the right direction, they have several undesirable features for differential analysis. First, the degree of first order dependence is uniform irrespective of the inter-probe distances. This is unrealistic in methylation datasets where the correlation typically decreases with inter-probe distance (Hansen et al., 2012; Jaffe et al., 2012; Hebestreit et al., 2013). Second, an ad hoc exploratory analysis of the GI cancer dataset reveals that the serial correlation in the treatment-probe effects is weaker than the serial dependence between the differential state variables in equation (2.2). Although there may not be a biological explanation for this phenomenon, this makes sense from a statistical perspective because the differential states are binary functions of the treatment-probe interactions; the differential states are more sensitive in detecting first order dependence even when the higher-dimensional (and noisier) treatment-probe interactions show negligible correlation. This suggests that a hypothetical two-group Markov model, rather than an infinite-group Markov model such as HDP-HMM or Sticky HDP-HMM, would provide a better fit for the data. Third, the range of allocation patterns supported by infinite HMMs is relatively limited. In particular, realistic allocation patterns, such as power law decays in the cluster sizes and large numbers of small-sized clusters, a common feature of cancer datasets (Lijoi et al., 2007b), are assigned relatively small prior probabilities by infinite HMMs.

2.1. Sticky PYP: A Two-restaurant, Two-cuisine Franchise (2R2CF) for Differential Analysis

The proposed Sticky PYP comprises a cohort of regular PYPs producing the probe-specific random effects by switching the generative PYP at random locations along the probe sequence. Alternatively, the Chinese restaurant franchise (CRF) metaphor for HDP-HMMs and Sticky HDP-HMMs can be generalized to the two-restaurant two-cuisine franchise (2R2CF) to give an equivalent representation of Sticky PYPs appropriate for differential analysis. We first present a descriptive overview of 2R2CF.

Imagine a franchise with two restaurants labeled 1 and 2. Each restaurant consists of two sections, labeled section 1 and 2. Each section serves a single cuisine and the section-cuisine menu consists of infinite dishes. Section 1 of both restaurants exclusively serves cuisine 1. The cuisine 1 menus of restaurant 1 and 2, along with the selection probabilities of the dishes, are identical. Similarly, section 2 of both restaurants exclusively serves cuisine 2, and the cuisine 2 menus of the two restaurants are identical.

A succession of $p$ customers, representing the CpG sites or probes, arrive at the franchise. The waiting times between successive customers correspond to the inter-probe distances, $e_1,\dots ,e_{p-1}$. Each customer first selects a restaurant and then a section (equivalently, cuisine) in that restaurant. Each restaurant section has an infinite number of tables, and a customer either sits at a table already occupied by the previous customers or sits at a new table. All customers at a table are served the same dish chosen from the section-cuisine menu by the first customer who sat at that table. The first customer at a table independently picks a dish from the infinite cuisine menu with a cuisine-specific probability associated with each dish. As a result, multiple tables at a restaurant section may serve the same dish.

Restaurant 1 specializes in cuisine 1. Consequently, section 1 is more popular with the restaurant 1 patrons. Similarly, restaurant 2 specializes in cuisine 2, and so, restaurant 2 customers tend to favor section 2 over section 1. By design, if a customer has eaten a cuisine 1 (2) dish, then the next customer is more likely to visit restaurant 1 (2), where cuisine 1 (2) is more popular. In this manner, each customer tends to select the same cuisine as the previous customer.

In the metaphor, cuisine 1 symbolizes the non-differential state and cuisine 2 symbolizes the differential state. The dish that franchise customer $j$ eats represents the probe-specific random effect, ${\mathit{\theta }}_j$. Since cuisine 1 represents the non-differential state, its dishes are characterized by $T$-variate random vectors with all equal elements; see equation (2.2). In contrast, cuisine 2 (differential state) dishes are characterized by $T$-variate random vectors with at least two unequal elements.

The dependence in the restaurant and cuisine choices of consecutive customers account for the long runs of differential or non-differential states seen in DNA methylation data. However, a customer’s influence on the next customer diminishes as the time interval between the two customers increases; the differential statuses of two adjacent probes are statistically independent in the limit as the inter-probe distance grows.

The 2R2CF process is illustrated in Figure 2 and discussed below in greater detail. The following specification conditions on $G$, an unknown distribution in $ℛ$ that is assigned a Dirichlet process prior with mass parameter $\beta >0$ and univariate normal base distribution, $G_0=N\left({\mu }_G,{\tau }_G^2\right)$. The stick-breaking representation of the Dirichlet process implies that distribution $G$ is almost surely discrete because it has the infinite mixture distribution:

$$G\stackrel{d}{=}\sum _{v=1}^{\infty }{\varpi }_v{\delta }_{{\zeta }_v},\text{where}\sum _{v=1}^{\infty }{\varpi }_v=1\text{and}{\zeta }_v\stackrel{\text{i}.\text{i}.\text{d}.}{~}{G}_0\text{.}$$ (2.5)

Figure 2:

Cartoon representation of the two-restaurant two-cuisine franchise for differential analysis, showing the progressive choice of restaurant, cuisine section, and table by customer $j$, for $j>1$. The numbered circles represent table numbers. See the text in Section 2.1 for a detailed description of the 2R2CF process.

The distribution of the random probabilities, ${\varpi }_v$, which depend on mass parameter $\beta$, was derived in Sethuraman (1994); see also Ishwaran and James (2003) and Lijoi and Prünster (2010). In the sequel, we condition on distribution $G$; equivalently on the probabilities, ${\varpi }_v$, and univariate atoms, ${\zeta }_v$, for $v\in 𝒩$, the natural numbers.

Cuisine 1 menu.

Recall that cuisine 1 represents the non-differential state, for which the $T$-variate random vectors (i.e., dishes in the metaphor) has all equal elements. Cuisine 1 menu, with its countably infinite dishes and their associated probabilities, is modeled as a discrete menu distribution, $W_1$, in ${ℛ}^T$. With ${\mathbf{1}}_T$ denoting the column vector of $T$ ones, let

$$\mathit{\vartheta }\mid \psi =\psi {\mathbf{1}}_T\text{where}\psi \sim G.$$ (2.6)

Cuisine 1 menu distribution $W_1$ is defined as the law of random vector $\mathit{\vartheta }$. Then ${𝒮}_1=\left\{{\zeta }_v{\mathbf{1}}_T:v\in 𝒩\right\}$ represents the available cuisine 1 dishes and the support of $W_1$. The selection probability associated with dish ${\zeta }_v{\mathbf{1}}_T$ is ${\varpi }_v.$

The continuity of base distribution $G_0$ in equation (2.5) guarantees that the menu $W_1$ dishes are unique. On the other hand, the discreteness of distribution $G$ has practical implications for 2R2CF: (a) cuisine 1 consists of discrete dishes, as required, rather than a continuous spectrum, and (b) since section 1 at both restaurants serve the same menu, two section 1 customers may eat the same dish even if they select different restaurants.

Cuisine 2 menu.

As mentioned, cuisine 2 depicts the differential state and its dishes represent $T$-variate random vectors with at least two unequal elements. Its menu comprises countably infinite cuisine 2 dishes along with associated probabilities. The menu is therefore modeled by a $T$-variate distribution, $W_2$, satisfying two conditions: (i) it has a countably infinite support, and (ii) each $T$-variate atom of $W_2$ has at least two unequal elements. For any given $\mathit{\varphi }={\left({\varphi }_1,\dots ,{\varphi }_T\right)}^{\prime }\in {ℛ}^T$, a probability mass function for $W_2$ that satisfies these two conditions is

$$W_2(\mathit{\varphi })=\left\{\begin{array}{ll}\prod _{t=1}^T G\left({\varphi }_t\right)/\left(1-\sum _{v=1}^{\mathrm{\infty }} {\varpi }_v^T\right)& \text{if}{\varphi }_t\ne {\varphi }_{t^{\prime }}\text{for}\text{some}t\ne t^{\prime },\\ 0& \text{otherwise},\end{array}\right.$$ (2.7)

where $G\left(\varphi \right)$ denotes the mass function of distribution $G$ evaluated at $\varphi \in ℛ$. For a graphical summary of the parameters in equations (2.5) – (2.7), refer to the “Cuisine menus” block of Figure 11 of Supplementary Material. In line 1 of expression (2.7), normalizing constant $\left(1-{\sum }_{v=1}^{\mathrm{\infty }} {\varpi }_v^T\right)$ is the total probability that a $T$-variate random vector whose elements are i.i.d. $G$ has at least two distinct elements. Referring back to the atoms, ${\zeta }_v$, of distribution $G$ in (2.5), ${𝒮}_2=\left\{\mathit{\varphi }:{\varphi }_t={\zeta }_v\right.\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{s}\mathrm{o}\mathrm{m}\mathrm{e}v\in 𝒩,\mathrm{b}\mathrm{u}\mathrm{t}\mathrm{n}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{l}{\varphi }_t\mathrm{a}\mathrm{r}\mathrm{e}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{f}\mathrm{o}\mathrm{r}t=1,...,T\}$ is the list of cuisine 2 dishes and the support of $W_2$. The selection probability associated with dish $\mathit{\varphi }$ is $W_2(\mathit{\varphi })$.

Restaurant, section, table, and dish choices of the 2R2CF customers

Let the restaurant chosen by franchise customer $j$ be denoted by $g_j$ and the chosen cuisine (i.e., section) be denoted by $s_j$. Suppose he or she sits at table $v_j$ in that restaurant section and eats dish ${\mathit{\theta }}_j$.

Customer 1.

At time 0, suppose the first customer selects restaurant $g_1=1$ with probability ${\rho }_1>0$ and selects restaurant $g_1=2$ with probability ${\rho }_2=1-{\rho }_1>0$. For reasons that will become clear, we refer to ${\rho }_1$ as the baseline non-differential proportion and ${\rho }_2$ as the baseline differential proportion. Typically, the differential state is less frequent, and so ${\rho }_2<{\rho }_1$ (i.e., ${\rho }_2<1/2$). Proportion ${\rho }_1$ is given a uniform prior on the interval (1/2, 1).

Choice of cuisine $s_1$

Next, customer 1 selects a section in restaurant $g_1$. Since each restaurant specializes in its namesake cuisine, the cuisine is more popular with the restaurant customers. This is modeled as follows. Within restaurant $g_1$ (with $j=1$ for customer 1), customer $j$ selects cuisine 1 with probability

$${𝒬}_{g_j}(1)=\left\{\begin{array}{ll}{\rho }_1+{\rho }_2\gamma & \text{if}{g}_j=1,\\ {\rho }_1-{\rho }_1\gamma & \text{if}{g}_j=2,\end{array}\right.$$ (2.8)

for a speciality cuisine popularity parameter, $\gamma \in (0,1)$, determining the degree to which a restaurant’s patrons favor its namesake cuisine. For instance, if $\gamma$ is nearly 1, then a restaurant 1 (2) customer almost always (never) chooses cuisine 1. At the other extreme, if $\gamma$ is nearly 0, then the customer chooses cuisine 1 with approximate probability ${\rho }_1$ irrespective of the restaurant. Parameter $\gamma$ is assigned an independent uniform prior on the unit interval. The probability that a restaurant $g_j$ customer chooses cuisine 2 is then ${𝒬}_{g_j}(2)=1-{𝒬}_{g_j}(1)$.

Choice of table $v_1$ and dish ${\mathit{\theta }}_1$

Within section $s_1$ of restaurant $g_1$, we assume without loss of generality (since the table identifiers are arbitrary) that customer 1 sits at table $v_1=1$. At table 1, customer 1 randomly orders a cuisine $s_1$ dish from menu distribution $W_{s_1}$. The dish he or she eats represents the random effect of the first probe. That is, ${\mathit{\theta }}_1\mid s_1\sim W_{s_1}$. As the 2R2CF process evolves as more patrons arrive, the tables in a restaurant’s section are sequentially assigned new labels as they are occupied.

Table 1:

True parameter values used to generate the artificial datasets.

${\alpha }_1$	${\alpha }_2$	$d_2$	$\beta$	$\gamma$	${\rho }_2$	${\mu }_G$	${\tau }_G^2$	${\tau }_{\chi }^2$
20	20	0.33	20	0.9	0.1	0	1	0.1225

Customer $\mathit{j}$, for $\mathit{j}>1$.

The restaurant choice of a subsequent customer is influenced by the previous customer’s cuisine and waiting time. Suppose customer $j$ arrives at the franchise after a time interval of $e_{j-1}$ following the $(j-1)$th customer. Without loss of generality, $e_1,\dots ,e_{p-1}$ can be scaled so that their total equals 1. Since the probes in differential analysis typically represent CpG sites on a chromosome or gene, it has a scaled length of 1.

To model the dependencies between the franchise customers, we define a non-negative dependence parameter $\eta$ that transforms waiting time $e_{j-1}$ to an affinity measure between customer $(j-1)$ and customer $j$:

$$r_j=\mathrm{e}\mathrm{x}\mathrm{p}\left(-e_{j-1}/\eta \right),j>1.$$ (2.9)

The affinity measure belongs to the unit interval when $\eta >0$. If $\eta =0,r_j$ is defined as 0 irrespective of waiting time $e_{j-1}$. The affinity influences the restaurant choice through assumption (2.10) below.

Choice of restaurant $g_j$

The cuisine $s_{j-1}$ of the $(j-1)$th customer influences the restaurant choice of the $j$th customer through affinity $r_j$ and popularity parameter $\gamma$:

$$g_j\mid s_{j-1},{\rho }_1,\eta ,\gamma \sim {ℱ}_j.$$

Specifically, the probability that customer $j$ selects restaurant 1 is assumed to be

$${ℱ}_j(1)\stackrel{def}{=}P\left(g_j=1\mid s_{j-1},{\rho }_1,\eta ,\gamma \right)=\left\{\begin{array}{ll}{\rho }_1+{\rho }_2{r}_j/\gamma & \text{if}{s}_{j-1}=1,\\ {\rho }_1-{\rho }_1{r}_j/\gamma & \text{if}{s}_{j-1}=2,\end{array}\right.$$ (2.10)

and ${ℱ}_j\left(2\right)=1-{ℱ}_j(1)$. The idea is illustrated in the top panel of Figure 2, where customer $j$ chooses restaurant 1 with probability ${ℱ}_j(1)$ and restaurant 2 with probability ${ℱ}_j\left(2\right)$. If dependence parameter $\eta =0$, then the restaurant choices of the customers are independent; specifically, ${ℱ}_j(1)={\rho }_1$ irrespective of the cuisine $s_{j-1}$.

It can be verified that ${ℱ}_j$ is a probability mass function if and only if $r_j/\gamma <1$. Since the scaled waiting times are bounded above by 1, a globally sufficient condition is $\eta <-1/\mathrm{l}\mathrm{o}\mathrm{g}\gamma$. We therefore assume a mixture prior for dependence parameter $\eta$:

$$\eta \left|\gamma \sim \frac{1}{2}{\delta }_0+\frac{1}{2}\mathscr{H}\cdot ℐ(\eta <-1/\mathrm{l}\mathrm{o}\mathrm{g}\gamma )\right.,$$ (2.11)

where the second mixture component involves a continuous distribution, $\mathscr{H}$, restricted to the interval $[0,-1/\mathrm{l}\mathrm{o}\mathrm{g}\gamma )$, enforcing the globally sufficient condition. In our experience, posterior inferences on $\eta$ are relatively robust to the continuous prior $\mathscr{H}$ provided the prior is not highly concentrated on a small part of interval $[0,-1/\mathrm{l}\mathrm{o}\mathrm{g}\gamma )$. Refer to the “Restaurant and cuisine of customer $j$” block of Figure 11 of Supplementary Material for a DAG depicting the relationships of these parameters.

When $\eta =0$, we have a zero-order Sticky PYP; when $\eta >0$, we obtain a first order Sticky PYP. Some interesting consequences of specification (2.10) are:

1. Zero-order Sticky PYP: When $\eta =0$, each customer independently chooses restaurant 1 (or 2) with a baseline probability of ${\rho }_1$ (or ${\rho }_2$). The $p$ customers act identically.

2. First order Sticky PYP with $e_{j-1}/\eta$ large: At large relative distances, customer $j$ acts approximately independently of the history. Somewhat similarly to customer 1, customer $j$ chooses restaurant 1 (2) with a probability approximately, but not exactly, equal to ${\rho }_1\left({\rho }_2\right)$.

3. First order Sticky PYP with $e_{j-1}/\eta$ small: In the limit as $e_{j-1}/\eta \to 0$ (e.g., for a small inter-probe distance $e_{j-1}$), the restaurant choice of customer $j$ follows a hidden Markov model.

Since it drives the dependence characteristics of DNA methylation data, parameter $\eta$ is of interest. Prior specification (2.11) allows the data to direct the model order through posterior probability, $P[\eta =0|\mathbf{X}]$, an MCMC estimate of which is readily available; see Section 3.

Choice of cuisine $s_j$

In restaurant $g_j$, customer $j$ selects cuisine-section $s_j$ with distribution ${𝒬}_{g_j}$ defined in expression (2.8). For bookkeeping purposes, among franchise customers $1,...,j$, let $p_{gs}^{\left(j\right)}$ be the number of customers that choose section $s$ in restaurant $g$; that is, $p_{gs}^{\left(j\right)}={\sum }_{l=1}^j ℐ\left(g_l=g,s_l=s\right)$ for $g,s=1,2$.

For a graphical depiction of cuisine selection by the $j$th customer, see the middle panel of Figure 2, where $g_j=1$. That is, customer $j$, having already chosen restaurant 1, now chooses a cuisine-section. Restaurant 2 has been greyed out because it is no longer accessible to this customer. In the lower panel of Figure 2, we find that the customer picked cuisine-section 1, and so $s_j=1$.

Choice of table $v_j$

Applying the above notation, among customers $1,...,(j-1)$, there are $p_{g_j{s}_j}^{(j-1)}$ customers in the same restaurant and section as the $j$th customer. Suppose these customers have occupied tables $1,\dots ,M_{g_j{s}_j}^{(j-1)}$, and that there are $p_{g_j{s}_{j}k}^{(j-1)}$ customers seated at the $k$th table. Let ${ℳ}_{j-1}=\{p_{g_j{s}_{j}k}^{(j-1)}:k=1,\dots ,M_{g_j{s}_j}^{(j-1)}\}$ comprise these aggregated table occupancies.

Recall that the newly arrived jth customer may sit at any of the $M_{g_j{s}_j}^{(j-1)}$ occupied tables or a new $(M_{g_j{s}_j}^{(j-1)}+1)$th table. Two possibilities are illustrated in the lower panel of Figure 2. For a PYP with mass parameter ${\alpha }_{s_j}$ and cuisine-specific discount parameter $d_{s_j}\in [0,1)$, the predictive distribution of table $v_j$ of customer $j$ is related to the table occupancies as follows:

$$P\left(v_j=k\mid {ℳ}_{j-1}\right)\propto \left\{\begin{array}{ll}{p}_{g_j{s}_{j}k}^{(j-1)}-d_{s_j}& \text{if}k=1,\dots ,M_{g_j{s}_j}^{\left(j-1\right)},\\ {\alpha }_{s_j}+M_{g_j{s}_j}^{(j-1)}{d}_{s_j}& \text{if}k=(M_{g_j{s}_j}^{(j-1)}+1),\end{array}\right.$$ (2.12)

where the second line corresponds to customer $j$ sitting at a new table, in which case the new number of occupied tables is $M_{g_j{s}_j}^{\left(j\right)}=M_{g_j{s}_j}^{(j-1)}+1$ and table index $v_j=M_{g_j{s}_j}^{\left(j\right)}$. Otherwise, if customer $j$ sits at a previously occupied table, then table index $v_j\le M_{g_j{s}_j}^{(j-1)}$ and the number of occupied tables remains unchanged: $M_{g_j{s}_j}^{\left(j\right)}=M_{g_j{s}_j}^{(j-1)}$. See the DAG in Figure 11 of Supplementary Material.

The above predictive distribution implies that customer $j$ is more likely to choose tables with several occupants, positively reinforcing that table’s popularity for future customers. The number of occupied tables stochastically increases with the PYP mass and discount parameter.

For section $s$, if the PYP discount parameter $d_s=0$, we obtain the well-known Pòlya urn scheme for Dirichlet processes (?). PYPs act as effective dimension reduction devices because the random number of occupied tables is much smaller than the number of customers. In general, as the number of patrons in section $s$ of restaurant $g$ grows as more customers arrive at the franchise, that is, as $p_{gs}^{\left(j\right)}\to \mathrm{\infty }$, the number of occupied tables, $M_{gs}^{\left(j\right)}$, is asymptotically equivalent to

$$\left\{\begin{array}{ll}{\alpha }_s\mathrm{l}\mathrm{o}\mathrm{g}{p}_{gs}^{\left(j\right)}& \text{if}{d}_s=0\\ T_{d_s{\alpha }_s}{(p_{gs}^{\left(j\right)})}^{d_s}& \text{if}0<d_s<1\end{array}\right.$$ (2.13)

for a positive random variable $T_{d_s{\alpha }_s}$ (Lijoi and Prünster, 2010). The asymptotic order of the number of occupied tables increases with discount parameter $d_s$.

Choice of dish ${\mathit{\theta }}_j$

As discussed, all customers seated at a given table of section $s_j$ are served the same dish, chosen from the cuisine $s_j$ menu by the first customer to sit at that table. Let ${\mathit{\varphi }}_{g_j{s}_{j}k}$ denote the common dish eaten by customers at the kth table, $k=1,\dots ,M_{g_j{s}_j}^{(j-1)}$. The dish that customer $j$ eats represents the probe-specific random effect ${\mathit{\theta }}_j$, and

$${\mathit{\theta }}_j\left\{\begin{array}{ll}={\mathit{\varphi }}_{g_j{s}_j{v}_j}& \text{if}{v}_j=1,\dots ,M_{g_j{s}_j}^{\left(j-1\right)},\\ \sim W_{s_j}& \text{if}{v}_j=(M_{g_j{s}_j}^{(j-1)}+1).\end{array}\right.$$ (2.14)

In the latter case (line 2), the dish ${\mathit{\theta }}_j$ randomly selected by customer $j$ is registered as ${\mathit{\varphi }}_{g_j{s}_j{M}_{g_j{s}_j}^{\left(j\right)}}$, where $M_{g_j{s}_j}^{\left(j\right)}=M_{g_j{s}_j}^{(j-1)}+1$, and is served to all future customers who sit at table $M_{g_j{s}_j}^{\left(j\right)}$. Assumptions (2.12) and (2.14) imply that although the restaurants serve the same menus, the overall relative popularity of each dish is restaurant-specific. Refer to the block entitled “Cuisine menus” in Figure 11 of Supplementary Material.

The aforementioned process continues for the remaining 2R2CF customers. Expressions (2.8) and (2.10) guarantee that a cuisine is more popular at its namesake restaurant and the cuisine selected by a customer influences the restaurant choice of the next customer, making the next customer likely to select the same cuisine. This accounts for the lengthy runs of differential or non-differential probes seen in methylation data. In addition to achieving dimension reduction, the proposed Sticky PYP models the serial dependencies of adjacent probes as a decreasing function of the inter-probe distances.

Latent clusters and their differential states

Latent clusters, introduced earlier in expression (2.3), comprise probes with identical random effects and form the basis of the dimension reduction strategy. Returning to the 2R2CF metaphor, we identify a cluster as the set of customers who eat the same dish. However, in addition to the customers seated at a table, multiple tables in both restaurants may serve the same dish because of the shared cuisine menu. Therefore, irrespective of the restaurant, aggregating customers eating the same dishes, we obtain the probe-cluster allocation variables $c_1,\dots ,c_p$, and the number of latent clusters, $q$. The collection of customers eating the same cuisine 2 (differential state) dishes corresponds to a distinct differential cluster in $𝒟$, defined in equation (2.4).

From expression (2.13), we expect the number of occupied tables in the franchise to be much smaller than the number of customers, $p$. Furthermore, since multiple tables may serve the same dish, we expect the number of latent clusters, $q$, to be smaller than the number of occupied tables. With high probability, this implies that $q$ is much smaller than $p$.

PYP discount parameter ${\mathit{d}}_2$.

Consider the differential state cuisine menu, $W_2$, defined in (2.7). It can be shown that as the number of treatments, $T$, and the number of probes, $p$, increase, the differential clusters are not only asymptotically identifiable but consistently detectable in the posterior; refer to Section 4 of Guha and Baladandayuthapani (2016) for a detailed discussion of this remarkable phenomenon in standard PYP settings. Since the differential clusters can be inferred with high accuracy when $T$ and $p$ are large, discount parameter $d_2$ is given the mixture prior:

$$d_2\sim \frac{1}{2}{\delta }_0+\frac{1}{2}U(0,1)$$ (2.15)

where $d_2=0$ corresponds to a Dirichlet process. This provides 2R2CF the posterior flexibility to choose between a Dirichlet process and a more general PYP for a suitable clustering pattern of the differential probes. An allocation pattern typical of Dirichlet processes, such as exponentially decaying cluster sizes dominated by a few large clusters, results in a high posterior probability that $d_2$ equals 0. By contrast, an allocation pattern characteristic of non-Dirichlet PYPs, such as slower-than-exponential power law decays in the cluster sizes and relatively large numbers of smaller-sized clusters, causes the posterior of discount parameter $d_2$ to concentrate near 1 and exclude 0. A proof of the intrinsically different cluster patterns of Dirichlet processes and PYPs is given in Theorem 2.1 of Guha and Baladandayuthapani (2016).

Since distribution $G$ is discrete, all atoms of $T$-variate distribution $W_2$ may not be unique. Indeed, this is common for $T=2$ treatments. However, as $T$ grows, and provided the number of probes, $p$, grows at a slower-than-exponential rate as $T$, the probability that two atoms allocated to the probes are identical rapidly decays to 0. In regression problems unrelated to differential analysis, Section 2.3 of Guha and Baladandayuthapani (2016) derived a similar result for a simpler zero-order stochastic process. We have verified this phenomenon in simulation studies on differential analysis datasets. In several hundred artificial datasets generated from the Sticky PYP, for $p=1,500$ probes and $T$ as small as four, no two allocated atoms of $W_2$ were identical.

PYP discount parameter ${\mathit{d}}_1$.

Consider again the (non-differential) cuisine menu $W_1$ defined in (2.6). In general, the flexibility provided by PYP allocation patterns is not necessary for non-differential probes. This is because the allocation patterns of $W_1$ are driven by univariate parameter $\psi$ in (2.6) and mixture allocations of univariate objects are unidentifiable (e.g., Frühwirth-Schnatter, 2006). Consequently, we set PYP discount parameter $d_1=0$, reducing the two PYPs associated with the non-differential state (i.e., section 1 in both restaurants) to Dirichlet processes.

Other model parameters

Depending on the specifics of the application, an appropriate model is assumed for the subject-specific parameters ${\xi }_1,\dots ,{\xi }_n$. For example, we may assume ${\xi }_i\stackrel{\text{i.i.d.}}{\sim }N\left(0,{\tau }_{ϵ}^2\right)$. In other applications, it may be more appropriate to assume non-zero means and flexible error distributions: ${\xi }_i=b_i+{ϵ}_i$, where $b_i$ represents lane or batch effects in methylation data, and the i.i.d. ${ϵ}_i$ follow a random distribution with a Dirichlet process prior. Similarly, appropriate models for the probe-specific parameters ${\chi }_1,\dots ,{\chi }_p$ may include i.i.d. zero-mean normal distributions, and finite mixtures or HMMs with state-specific normal distributions. Inverse-gamma priors are assigned to ${\sigma }^2$ and ${\tau }_{ϵ}^2$. Suitable priors are assumed for mass parameters $\beta ,{\alpha }_1$, and ${\alpha }_2$ in expressions (2.6) and (2.12). Mean ${\mu }_G$ and variance ${\tau }_G^2$ of base distribution $G_0$ in expression (2.6) are given a joint normal-inverse gamma prior. The DAG in Figure 11 of Supplementary Material summarizes the complex relationships between the different model parameters.

3. Posterior Inference

Due to the analytical intractability of the BayesDiff model, we rely on MCMC methods for posterior inference and detection of differential probes.

3.1. MCMC Strategy

The model parameters are initialized using naïve estimation techniques and iteratively updated by MCMC techniques until the chain converges. We split the MCMC updates into three blocks. An outline of the MCMC procedure is as follows. Further details can be found in Section 7 of Supplementary Material.

1. Restaurant-cuisine-table-dish $\left(g_j,s_j,v_j,{\mathit{\theta }}_j\right)$ of customer $j$: For each probe $j=1,...,p$, we sample vector $\left(g_j,s_j,v_j,{\mathit{\theta }}_j\right)$ given the vectors of the other $(p-1)$ probes. This is achieved by proposing a new value of $\left(g_j,s_j,v_j,{\mathit{\theta }}_j\right)$ from a carefully constructed approximation to its full conditional, and by accepting or rejecting the move in a Metropolis-Hastings step. As discussed in Section 2.1, probe-cluster allocations $c_1,\dots ,c_p$ are immediately available from the restaurant-cuisine-table allocations $\left(g_j,s_j,v_j\right)$ of the $p$ probes, as are available the $q$ latent clusters with their allocated probes and the set of differential clusters, $𝒟$.

2. Latent vectors ${\mathit{\lambda }}_1,\dots ,{\mathit{\lambda }}_q$: The $Tq$ latent vector elements are not necessarily distinct because of the Dirichlet process prior on distribution $G$. Although the latent vector elements are known from the aforementioned block 1 updates, MCMC mixing is considerably improved by updating the latent vector elements conditional on the $p$ probe-cluster allocations. As the calculation in Supplementary Material shows, this is achieved by Gibbs sampling.

3. Remaining model parameters: Generated by standard MCMC techniques.

We discarded an initial burn-in of 10,000 MCMC samples and used the subsequent 50,000 draws for posterior inferences. Convergence was informally assessed by trace plots of various hyperparameters to validate the MCMC sample sizes. For the proposed moves (in discrete parameter space) described in Step 1, the average Metropolis-Hastings acceptance rate exceeded 90% in all our analyses.

3.2. Detection of Differential Probes with FDR Control

Post-processing the MCMC sample, a Bayesian approach for controlling the false discovery rate (FDR) (Newton et al., 2004) is applied to detect the probes $j$ with differential state $s_j=2$. Specifically, let $q_0$ be the nominal FDR level and ${\omega }_j$ be the posterior probability that probe $j$ is differential, so that ${\omega }_j=P\left[s_j=2\mid \mathbf{X}\right]$. An empirical average estimate, ${\hat{\omega }}_j$, is available from the MCMC sample. To achieve the desired FDR level in calling the differential probes, we first rank all the probes in decreasing order of ${\hat{\omega }}_j$. Let ${\hat{\omega }}_{(1)}>{\hat{\omega }}_{\left(2\right)}>\cdots >{\hat{\omega }}_{\left(p\right)}$ denote the ordered posterior probability estimates. For each $b=1,...,p$, we evaluate the posterior expected FDR of calling differential the first $b$ probes in the sorted sequence:

$${\widehat{\mathrm{F}\mathrm{D}\mathrm{R}}}_b=\frac{\sum _{j=1}^p \left(1-{\hat{\omega }}_j\right)ℐ\left({\hat{\omega }}_j\ge {\hat{\omega }}_{\left(b\right)}\right)}{\sum _{j=1}^p ℐ\left({\hat{\omega }}_j\ge {\hat{\omega }}_{\left(b\right)}\right)}=\frac{\sum _{j=1}^b \left(1-{\hat{\omega }}_{\left(j\right)}\right)}{b},$$ (3.1)

where the simplification occurs because the ${\hat{\omega }}_{\left(j\right)}$ are sorted. Finally, we pick the largest value of $b$, denoted by $b^{\mathrm{*}}$, for which ${\widehat{\mathrm{F}\mathrm{D}\mathrm{R}}}_b<q_0$. A nominal FDR level of $q_0$ is achieved by labeling the first $b^{\mathrm{*}}$ probes, arranged in decreasing order of ${\hat{\omega }}_j$, as differential.

4. Simulation Studies

Using artificial datasets with $T=5$ treatments, we analyzed the accuracy of BayesDiff in detecting differentially methylated probes. We compared the results with established differential methylation procedures and general statistical techniques for multiple treatment comparisons. We also evaluated the ability of the BayesDiff procedure in discovering the complex dependence structures of DNA methylation data.

Generation strategy

Proportions representing DNA methylation data were generated using the logit transformation as in equation (2.1). The inter-probe distances were the actual distances from the motivating TCGA dataset, scaled to add to 1. In order to capture the complexity of methylation data, such as the existence of multiple latent methylation states (e.g,. CpG islands and shores), different read depths across CpGs, and the incomplete conversion of bisulphite sequencing, the generation strategy was partly based on techniques implemented in WGBSSuite, a flexible stochastic simulation tool for generating single-base resolution methylation data (Rackham et al., 2015). However, the generation procedure differed from WGBSSuite in some respects. Specifically, it allowed more than two treatments $(T=5)$. Additionally, as in actual methylation datasets, the generation procedure incorporated serial dependence not only in the methylation levels but also in the differential states of the probes.

The probe-specific read depths were generated as $n_j\stackrel{\text{i.i.d.}}{\sim }\mathrm{P}\mathrm{o}\mathrm{i}\mathrm{s}\mathrm{s}\mathrm{o}\mathrm{n}(50)$. Unlike assumption (2.1), there were no subject-specific random effects in the generation mechanism. Instead, the normal means incorporated additive probe-specific random effects, ${\chi }_1^{\left(0\right)},\dots ,{\chi }_p^{\left(0\right)}$, that were generated using the following steps:

1. Generate the true methylation status of the probes, denoted by $h_1^{\left(0\right)},\dots ,h_p^{\left(0\right)}$, using the 4-state distance-based HMM of Rackham et al. (2015), with the states respectively representing the methylated, first transit, demethylated, and second transit states.

2. Set the baseline methylation levels for the methylated, (first or second) transit, and demethylated states as $p_{\text{methylated}}=0.8,p_{\text{transit}}=0.5$, and $p_{\text{un-methylated}}=0.2$.

3. For $j=1,...,p$, compute the probe-specific means:

   $${\tilde{\chi }}_j^{\left(0\right)}=\left\{\begin{array}{ll}\log (\frac{p_{\text{methylated}}}{\left(1-p_{\text{methylated}}\right.})& \text{if}{h}_j=1\text{(i.e.,}\text{methylated}\text{state)},\\ \log (\frac{p_{\text{transit}}}{1-p_{\text{transit}}})& \text{if}{h}_j=2,4\text{(first}\text{or}\text{second}\text{transit}\text{state)}\\ \log (\frac{p_{\text{demethylated}}}{1-p_{\text{demethylated}}})& \text{if}{h}_j=3\text{(i.e.,}\text{demethylated}\text{state).}\end{array}\right.,$$

4. Generate ${\chi }_j^{\left(0\right)}\stackrel{\text{indep}}{\sim }N\left({\tilde{\chi }}_j^{\left(0\right)},{\tau }_{\chi }^2\right)$ for $j=1,\dots ,p$.

Noise and dependence levels

We investigated four scenarios corresponding to the combinations of two noise levels and two dependence levels. For each scenario, 20 datasets were independently generated, with each dataset consisting of $p=500$ probes and four samples associated with each of $T=5$ treatments, i.e., a total to $n=20$ samples. The low noise setting corresponded to true variance parameter ${\sigma }_0^2=0.36$; equivalently, to a signal-to-noise of $R_0^2\approx 70\mathrm{\%}$. The high noise setting corresponded to ${\sigma }_0^2=1$ or $R_0^2\approx 40\mathrm{\%}$. The true between-probe dependencies comprised two levels: no serial correlation (i.e., a zero-order Sticky PYP) with ${\eta }_0=0$, and positive serial correlation (i.e., a first order Sticky PYP) with ${\eta }_0=0.004$. Although ${\eta }_0=0.004$ may appear to be small, its value is calibrated to the scaled inter-probe distances and represents fairly high inter-probe dependence. For example, when the distance between two adjacent probes is equal to the standardized average distance of $\bar{e}=1/(p-1)=1/499,{\eta }_0=0.004$ gives an affiliation of $r_0=0.6$ in equation (2.9). For convenience, we will refer to the two dependence levels as “no-correlation” and “high correlation.” True values of the other model parameters were common to the four scenarios and are displayed in Table 1. Setting a true baseline differential proportion of ${\rho }_2=0.1$ resulted in approximately 10% true differentially methylated CpGs in each dataset.

Posterior inferences

Assuming all model parameters to be unknown, each artificial dataset was analyzed using a BayesDiff model that differed in key respects from the true generation mechanism. For example, unlike the 4-state HMM of the generation strategy, the probe-specific random effects ${\chi }_j$ were analyzed using a BayesDiff model that ignored the first order dependence, and instead, relied on a 3-state finite mixture model representing the methylated, transit, and unmethylated states. Additionally, in contrast to the zeroed-out subject-specific random effects during data generation, BayesDiff assumed i.i.d. normal random effects with zero means.

To assess BayesDiff’s accuracy in detecting the absence or presence of inter-probe serial correlation, in the no-correlation $\left({\eta }_0=0\right)$ situation, we evaluated $\mathrm{l}\mathrm{o}\mathrm{g}\left(\frac{P[\eta =0\mid \mathbf{X}]}{P[\eta >0\mid \mathbf{X}]}\right)$, the log-Bayes factor comparing zero order to first order Sticky PYPs. In the high correlation $\left({\eta }_0=004\right)$ situation, we evaluated $\mathrm{l}\mathrm{o}\mathrm{g}\left(\frac{P[\eta >0\mid \mathbf{X}]}{P[\eta =0\mid \mathbf{X}]}\right)$, the log-Bayes factor comparing first order to zero order Sticky PYPs. Thus, in any scenario, a large positive value of this measure provides strong evidence that BayesDiff detects the correct model order.

Although conceptually straightforward, the estimation of Bayes factors requires multiple MCMC runs even for relatively simple parametric models (Chib, 1995). Basu and Chib (2003) extended the estimation strategy to infinite dimensional models such as Dirichlet processes. However, the computational costs are prohibitively high for big datasets, and multiple MCMC runs stretch present-day computational resources beyond their limits. Faced with these challenges, we relied on an alternative strategy for estimating the lower bounds of log-Bayes factors as by-products of the Section 3.1 algorithm. As it turns out, this is often sufficient to infer Sticky PYP model orders. Let ${\mathrm{\Theta }}^{-}$ denote all BayesDiff model parameters except $\eta$. In the high correlation situation, applying Jensen’s inequality, a lower bound for the corresponding log-Bayes factor is $E\left[\mathrm{l}\mathrm{o}\mathrm{g}\left(\frac{P\left[\eta >0\mid \mathbf{X},{\mathrm{\Theta }}^{-}\right]}{P\left[\eta =0\mid \mathbf{X},{\mathrm{\Theta }}^{-}\right]}\right)|\mathbf{X}\right]$. Unlike log-Bayes factors, this lower bound can be easily estimated using a single MCMC run. In the no-correlation situation, a lower bound for the log-Bayes factor, $\mathrm{l}\mathrm{o}\mathrm{g}\left(\frac{P[\eta =0\mid \mathbf{X}]}{P[\eta >0\mid \mathbf{X}]}\right)$, is similarly estimated.

In the four simulation scenarios, box plots of these estimated lower bounds for the 20 datasets are presented in Figure 3. Except for the high noise–no-correlation scenario, for which the results were inconclusive, the estimated lower bounds of the log-Bayes factors in favor of the true correlation structure were all positive and large. In the low noise–no-correlation scenario, BayesDiff decisively favored zero-order models, and the smallest lower bound among the 20 datasets was 13.9, corresponding to Bayes factors exceeding $e^{13.9}=1,088,161$. The 25^th percentile of these lower bounds was 30.9, corresponding to Bayes factors exceeding $e^{30.9}=2.63\times {10}^{13}$. This is strong evidence that the BayesDiff approach is reliable in this scenario. For the high-correlation scenarios, the estimated lower bounds were even higher, indicating that BayesDiff overwhelmingly favors first order models when the data are serially correlated.

Figure 3:

Simulation study box plots for estimated lower bounds of the log-Bayes factors in favor of the true model order.

Comparisons with other methods

We evaluated the success of the BayesDiff procedure in detecting disease genomic signatures relative to six well-known statistical methods. These included generic multiple comparison techniques, namely, one-way analysis of variance (ANOVA) and the Kruskal-Wallis test. Also included were specially developed methods for detecting differential methylation in more than two treatments: COHCAP (Warden et al., 2013), methylKit (Akalin et al., 2012), BiSeq (Hebestreit et al., 2013), and RADMeth (Dolzhenko and Smith, 2014). The ANOVA and Kruskal-Wallis test procedures were performed separately on each probe after applying the inverse-logit transform to the data. The COHCAP method was directly applied to the synthetic data. The remaining three methods are designed for bisulfite sequencing, which consists of total methylation reads for each CpG site. For these methods, the methylation reads were calculated by multiplying the methylation proportions by the total reads. As recommended, the bandwidth smoothing parameter of the BiSeq method was tuned to optimize overall detection. For all six competing methods, probe-specific p-values were evaluated and adjusted for multiplicity using the FDR control procedure of Benjamini and Hochberg (1995).

Like most nonparametric Bayes techniques, the computational times of BayesDiff are considerably higher than frequentist methods, but negligible compared to the time frames over which the experimental data are collected. Furthermore, as we demonstrate, the substantially greater accuracy of BayesDiff more than compensates for its computational costs. On a personal computer with an Intel Core i7–4770 processor with 3.40 GHz frequency and 8 GB RAM, the average run time for the Section 3.1 MCMC algorithm, applied to the synthetic datasets with $n=20$ samples, $T=5$ treatments, and $p=500$ probes, was 0.60 seconds per iteration. However, the computational times are greatly reduced by running the datasets in parallel across multiple cores of a research computing cluster. Analyzing datasets of various sizes, we found that the computational cost is $O\left(T{p}^2\right)$ but does not appreciably depend on $n/T$. This is reasonable because the mixture model primarily focuses on ${\mathit{\theta }}_1,\dots ,{\mathit{\theta }}_p\in {ℛ}^T$. Due to the intensive nature of the one-parameter-at-a-time Gibbs sampling updates in Block 2, the Metropolis-Hastings algorithm of Guha (2010) can be applied to significantly speed up the updates and make the calculations more scalable. As part of ongoing work developing a fast R package, we find that ten- to hundred-fold speedups are possible with this fast MCMC strategy, which can also accelerate the block 1 parameter updates of Section 3.1.

We computed the receiver operating characteristic (ROC) curves for differential probe detection for all seven methods. For a quantitative assessment, we calculated the area under curve (AUC), declaring the method with the largest AUC as the most reliable in each scenario. The ROC curves, averaged over the 20 datasets under each simulation scenario, are shown with the AUCs in Figure 7 of Supplementary Material. In all except the high-noise–no-correlation scenario, BayesDiff uniformly outperformed the other methods. Even in the high-noise–no-correlation scenario, BayesDiff performed better in the low FPR region. As expected, all seven methods had lower accuracies for higher noise levels. BayesDiff did significantly better than the competing methods in the high correlation scenarios, suggesting that the incorporation of between-probe dependencies improves its accuracy in situations typical of DNA methylation data.

Since researchers typically focus on small false positive rates (FPRs), that is, small significance levels, we also calculated the measures, AUC₂₀ and AUC₁₀. AUC₂₀ (AUC₁₀) is defined as the area under the ROC curve multiplied by 5 (10) when the FPR does not exceed 0.2 (0.1). The multiplicative factors ensure that the areas potentially vary between 0 and 1. The three versions of AUC are presented in Table 3 in Supplementary Material. As also seen in Figure 7, Table 3 reveals that in three of the four scenarios, BayesDiff had the largest overall AUC. Furthermore, BayesDiff had vastly improved reliability for low FPRs. For example, consider the low noise–high correlation scenario. The overall AUC for BayesDiff was 0.035 greater than that for ANOVA. In contrast, the gains for BayesDiff, relative to ANOVA, were +0.107 for AUC₂₀ and +0.146 for AUC₁₀. The advantages of BayesDiff were even greater relative to the other competing methods. In the high noise–low-correlation scenario, BayesDiff had a relatively low AUC, as mentioned. However, even in this scenario, it had the greatest AUC₂₀ and AUC₁₀ among all the methods. Additionally, for a nominal FDR of $q_0=0.05$, the achieved FDR of BayesDiff was between 0 and 0.03 in every dataset and simulation scenario. These results demonstrate the ability of BayesDiff to accurately detect differential probes, even in challenging situations in which the FPR is small.

5. Data Analysis

We returned to the motivating DNA methylation data consisting of the upper GI cancers: stomach adenocarcinoma (STAD), liver hepatocellular carcinoma (LIHC), esophageal carcinoma (ESCA), and pancreatic adenocarcinoma (PAAD). Applying the BayesDiff procedure, we detected the differentially methylated CpG loci among the cancer types.

Data processing

The dataset was obtained from The Cancer Genome Atlas project, publicly available through The Genomic Data Commons (GDC) portal (Grossman et al., 2016). The measurements on 485,577 probes located at CpG sites were made using the Illumina Human Methylation 450 platform. At the time of analysis, the dataset consisted of 1,224 tumor samples. We analyzed the data on a gene-by-gene basis, selecting 443 genes with mutations in at least 5% of the samples. To ensure that all CpG sites potentially linked to a gene were included in the analysis, we selected sites located within 50K base pairs outside the gene body, upstream from the 5’ end as well as downstream from the 3’ end. The number of gene-specific CpG sites ranged from 1 to 769, and are displayed in Figure 8(a) of Supplementary Material. As a final preprocessing step, since the methylation patterns of short genes are less informative in cancer investigations, we eliminated the 25 genes mapped to 20 or fewer CpG sites.

Inference procedure

The data were analyzed using the proposed BayesDiff approach. Exploratory analyses indicated that eliminating the probe-specific random effects ${\chi }_j$ in expression (2.1) produces a satisfactory model fit. Since experimental batch information is not available in the TCGA dataset, we assumed that the parameters ${\xi }_1,\dots ,{\xi }_n$ in (2.1) are i.i.d. with a random distribution having a Dirichlet process prior. The MCMC procedure of Section 3.1 was applied to obtain posterior samples for each gene. For detecting differential CpG sites, we applied the Section 3.2 procedure with a nominal FDR of $q_0=0.05$.

Results

Among the differentially methylated CpG sites detected by our approach, approximately 40.6% of the sites were located outside the gene body. Figure 4 displays the associations between detected methylation status and position of the CpG sites. We defined “near the 5’ (3’) end” as CpG sites located within one-fourth length of the gene body, either inside or outside the gene boundary, and closer to the transcription start (termination) site. Our results indicate that the proportion of differential methylation is higher for CpG sites inside the gene body and most differentially methylated loci are situated within the gene body, as is well known from numerous previous studies. However, our analysis also revealed significant amounts of differential methylation outside the gene body. Despite the common belief that DNA methylation analysis should focus on the 5’ end region, we found that CpG sites near the 3’ ends also displayed considerable differential methylation. These findings support the recommendations of Irizarry et al. (2009) that studies of DNA methylation alteration should be conducted on a higher resolution, epigenome-wide basis.

Figure 4:

Associations of detected methylation status and position of CpG sites.

Among the differentially methylated sites detected by BayesDiff, we estimated the pairwise differences between random effects associated with the four cancer types. Site-wise summaries of the largest pairwise differences of the cancer-specific effects are displayed in Figure 5. None of the four cancer types displayed consistent hypermethylation or hypomethylation across all genes or over entire chromosomes. However, we found that LIHC is frequently differentially methylated relative to one of the other cancer types, implying that it is the most volatile disease with respect to DNA methylation.

Figure 5:

Site-wise summary of the largest pairwise differences of differentially methylated loci among the four upper GI cancer types.

For each gene, Figure 8(b) of Supplementary Material displays 95% credible intervals for the lower bounds of log-Bayes factors of a first versus zero-order 2R2CF model, i.e., $\eta =0$ versus $\eta >0$ in expression (2.9). Models with first order dependence are overwhelmingly favored for a majority of the genes, suggesting that statistical techniques that fail to account for dependence between neighboring CpG sites are less effective for these data. Figure 9 of Supplementary Material displays the detailed differential methylation pattern for the top two mutated genes, TP53 and TTN. An obvious feature of both genes is that the differential methylation patterns are strongly serially correlated. For gene TP53, there are almost no differentially methylated loci within the gene body. The 3’ end region outside the gene body has a cluster of differentially methylated loci, for which cancer type STAD is mostly hypermethylated. The results for gene TTN tell a quite different story: most of the differentially methylated loci are inside the gene body and near the 5’ end. Cancer type LIHC is hypomethylated compared to PAAD around the 5’ end region, but it is hypermethylated compared to STAD near the 3’ end. Genes with at least 90% differentially methylated sites detected by BayesDiff are listed in Table 4 of Supplementary Material, along with the largest pairwise difference between the four cancer types among the differentially methylated loci. The number of CpG sites within each segment is listed in Table 5 of Supplementary Material.

Existing medical literature both supports and complements our findings. For example, hypermethylation of the EDNRB and SLIT2 genes have been found in STAD (Tao et al., 2012). Gene FBN2 was hypermethylated in ESCA (Tsunoda et al., 2009). While several studies have found that the gene and protein expressions of ABC transporter genes, such as ABCC9, are useful for understanding the prognosis of esophageal cancer (Vrana et al., 2018), we find that hypermethylation of ABCC9 is a major difference between cancer types ESCA and LIHC. Gene FLRT2 is a potential tumor suppressor that is hypermethylated and downregulated in breast cancer (Bae et al., 2017). Our results indicate that this gene is also hypermethylated in cancer type STAD versus LIHC. Mutations in SPTA1 gene has been linked with PAAD (Murphy et al., 2013); our results indicate that hypermethylation of this gene distinguishes PAAD from LIHC.

Finally, we compared our findings with those of ANOVA for multiple treatment comparisons. Table 6 lists the common set of genes with at least 90% differentially methylated sites identified by both BayesDiff and ANOVA. Table 7 displays the genes identified by only ANOVA, whereas Table 8 displays the large number of genes detected by only BayesDiff. Cross-referencing with the medical literature, we find that genes FLRT2 and FBN2 were detected by both methods. However, genes EDNRB, SLIT2, ABCC9, and SPTA1 were only identified by BayesDiff, revealing the benefits of the proposed Bayesian nonparametric method.

Accounting for data characteristics

To avoid making misleading biological interpretations, a statistical model must account for the observed biomarker means and variances, especially in multiple-testing approaches where the first two sample moments are important (Subramaniam and Hsiao, 2012). From this perspective, certain aspects of the BayesDiff model, such as variance ${\sigma }^2$ being a priori unrelated to the mean in expression (2.1), may appear to be unduly restrictive. However, even though it was not specifically designed to match data summaries such as sample moments, in practice, the nonparametric nature of the Sticky PYP allows the posterior to flexibly adapt to unique data characteristics, such as sample moments, and account for mean-variance relationships in a robust manner. For example, consider again the top mutated genes, TP53 and TTN, discussed in Figure 9 of Supplementary Material. The ability of BayesDiff to match the sample moments of the gene-specific probes can be demonstrated as follows. Given the inter-probe distances, the joint posterior of the BayesDiff parameters induces predictive distributions on the $n$ measurements for each probe. Functionals of these predictive distributions, such as probe-specific sample moments, are easily estimated by post-processing the MCMC sample. For these two genes, Figure 10 of Supplementary Material reveals that the sample moments predicted by BayesDiff are a close match to the actual first and second sample moments with correlations exceeding 99% in each plot. Similar results were observed in other datasets.

6. Discussion

DNA methylation data exhibit complex structures due to biological mechanisms and distance-dependent correlations between adjacent CpG sites or probes. The identification of the differential signatures of multiple sets of tumor samples is crucial for developing targeted treatments for disease. This paper formulates a flexible approach applicable to multiple treatments called BayesDiff. The technique relies on a novel Bayesian mixture model called the Sticky PYP or the two-restaurant two-cuisine franchise. In addition to facilitating simultaneous inferences on the probes, the model accommodates distance-based serial dependence and accounts for the complex interaction patterns commonly observed in cancer data. An effective MCMC strategy for detecting the differential probes is developed. The success of the BayesDiff procedure in differential DNA methylation, relative to well-established methodologies, is exhibited via simulation studies. The new technique is applied to the motivating TCGA dataset to detect the differential genomic signatures of four upper GI cancers. The results both support and complement known facts about epigenomic differences between these cancer types, while identifying genes with high proportions of differentially methylated CpG sites.

In addition to providing a good fit to the data, a statistical model must be able to account for features such as sample moments. The success of the BayesDiff model in this regard is demonstrated in Section 5 using the upper GI dataset. It must be emphasized, however, that BayesDiff may be less successful in accounting for the characteristics of some other datasets, possibly due to slow asymptotic convergence of the posterior to the underlying generative process. In such situations, more flexible global transformations (Li et al., 2016) or variance-stabilizing transformations (Durbin et al., 2002) may be utilized. Alternatively, local Laplace approximations of exponential family likelihoods through link functions (Zeger and Karim, 1991; Chib and Winkelmann, 2001) may extend the BayesDiff model to better explain the data characteristics.

Like most Bayesian models comprising several latent parameters, the proposed 2R2CF may be marginalized over different parameter sets to obtain equivalent versions of the same model. For example, we could marginalize over restaurants to obtain an equivalent “sticky cuisine” version in which there is just one restaurant with two cuisine-sections and a customer more likely to favor the cuisine selected by the previous customer. Alternatively, we could marginalize over sections to obtain an equivalent “sticky restaurant franchise” in which each restaurant comprises a single section with restaurant-specific probabilities ensuring that Cuisine 1 or 2 dishes are more popular at their namesake restaurant; a customer is then more likely to favor the restaurant selected by the previous customer. In all equivalent versions, however, a probe’s differential state is determined by the customer’s dish in the metaphor.

The 2R2CF perspective offers the twin advantages of parameter interpretability and generalizability. Section 9 of Supplementary Material presents the generalized form of the Sticky PYP, revealing the full potential of the proposed method in analyzing not only DNA methylation datasets, but other types of omics datasets, such as gene expression, RNASeq, and copy-number alteration data. Beyond biomedical applications, the generalized formulation offers a diverse palette of parametric and nonparametric models for capturing the distinctive features of datasets. These Bayesian mixture models are special cases of Sticky PYPs for particular choices of a countable group parameter (e.g., two “restaurants” in the 2R2CF metaphor for differential methylation problems) and countable state parameter (e.g., two “cuisines” in 2R2CF) with the state of a customer influencing the group of the next customer. In addition to extending PYPs to discrete time series-type data, the range of models includes Dirichlet processes, PYPs, infinite HMMs, hierarchical Dirichlet processes (Teh et al., 2006), hierarchical Pitman-Yor processes (Teh et al., 2006; Camerlenghi et al., 2019), finite HMMs, nested Chinese restaurant processes (Blei and Jordan, 2005), nested Dirichlet processes (Rodriguez et al., 2008), and analysis of densities models (Tomlinson and Escobar, 2003).

Ongoing work involves extending the correlation structure to model more sophisticated forms of inter-probe dependence in DNA methylation data. Commented R code implementing the BayesDiff method is available on GitHub at https://github.com/cgz59/BayesDiff. Using high-performance Rcpp subroutines, we are developing a fast R package for detecting differential genomic signatures in a wide variety of omics datasets. Initial results indicate that order-of-magnitude speedups will allow the fast analyses of high-dimensional datasets on personal computers.