Flexible copula model for integrating correlated multi-omics data from single-cell experiments

Summary:

With recent advances in technologies to profile multi-omics data at the single-cell level, integrative multi-omics data analysis has been increasingly popular. It is increasingly common that information such as methylation changes, chromatin accessibility, and gene expression are jointly collected in a single-cell experiment. In biomedical studies, it is often of interest to study the associations between various data types and to examine how these associations might change according to other factors such as cell types and gene regulatory components. However, since each data type usually has a distinct marginal distribution, joint analysis of these changes of associations using multi-omics data is statistically challenging. In this paper, we propose a flexible copula-based framework to model covariate-dependent correlation structures independent of their marginals. In addition, the proposed approach could jointly combine a wide variety of univariate marginal distributions, either discrete or continuous, including the class of zero-inflated distributions. The performance of the proposed framework is demonstrated through a series of simulation studies. Finally, it is applied to a set of experimental data to investigate the dynamic relationship between single-cell RNA-sequencing, chromatin accessibility, and DNA methylation at different germ layers during mouse gastrulation.

1. Introduction

Recent advances in high-throughput technologies have enabled the profiling of different molecular layers in living cells. It is increasingly common to collect multi-omics data, such as genome, epigenome, transcriptome, and proteome in single-cell studies. Oftentimes these data are of different modalities and contain information complement to each other. Analyzing them together may bring insights that cannot be revealed by analyzing each data type separately. The recent work of Lee et al. (2020) provides a comprehensive review on the multi-omics studies at the single-cell resolution, including the technologies and data analysis methods. In the article, the authors listed three major directions of statistical analysis on single-cell multi-omics data: (1) correlation analysis of various data modalities, for example, between DNA methylation and RNA expression (Hu et al., 2016); (2) analyzing each -omics layer separately then combining corresponding results (for instance Cao et al., 2018) and (3) integrate all data types to generate an overall map (Argelaguet et al., 2019).

This article aims to follow the first path listed above and to study various sources of associations in multi-omics data from single-cell experiments. Specifically, we intend to model how the association between two types of -omics data can change depending on other data modalities and cell types information. We named this covariate-dependent correlation “dynamic association” in this paper. The statistical challenge for studying dynamic association in this context is that different data types often have distinct marginal distributions. For instance, raw counts of single-cell RNA-sequencing data (scRNA-seq) are usually non-negative integers with zero-inflation. After being normalized, scRNA-seq is transformed into non-negative real numbers. DNA methylation and chromatin accessibility data are often measured as proportions. Due to the characteristics such as zero-inflation, skewness, and proportion/ratio of various data types, the conventional multivariate Gaussian distribution assumption is often not adequate in this setting.

In recent years, there has been increased interest in studying dynamic association (Wang et al., 2017; Kinzy et al., 2019; Ma et al., 2020; Yang and Ho, 2021). In his ground-breaking work, Li (2002) proposed a three-product-moment measure called the liquid association. Li et al. (2004) extended this notion to multivariate Y₁ and Y₂ and a third variable X, and derived the liquid association of linear combinations of Y₁ and linear combinations of Y₂ given X. In Ho et al. (2011) and Chen et al. (2011), the authors proposed model-based approaches to study covariate-dependent correlation structures. In both papers, the marginal distributions of Y₁ and Y₂ given X are both assumed to be Gaussian.

Our motivating data was generated from a single-cell multi-omics profiling study of mouse gastrulation (Argelaguet et al., 2019). At the formation stage of primary germ layers during gastrulation, it is unclear how gene expression, chromatin accessibility, and DNA methylation coordinate with each other. This data provides exciting opportunities to study transcription machinery in embryonic stem cells at various stages via the information contained in the genome, epigenome and transcriptome. In this paper, we set out to study how DNA methylation in genes encoding transcription factors (TFs) modulates the association patterns between scRNA-seq and chromatin accessibility in various germ layers during mouse development stages.

From the statistical perspective, this challenge involves modeling the joint distribution of $Y={\left(Y_1,Y_2\right)}^{\prime }$ with different marginal distribution, where both the marginal means and the correlation between Y₁ and Y₂ are functions of a set of covariates X = x, It is of particular interest to study the dynamic association between Y₁ and Y₂ and how it changes given different values of X. In the motivating example, Y₁ and Y₂ may represent scRNA-seq and chromatin accessibility, respectively, and the set of covariates x could include methylation level, gastrulation stage, and germ layer type.

In this article, we propose a general framework for studying the dynamic association between Y₁ and Y₂ given x where the marginal distributions are not limited to the Gaussian distribution and can be adapted to various univariate distributions. We construct copula-based models to study the joint distribution of (Y₁, Y₂) and develop a flexible approach to incorporate covariate-dependent correlation structures in the proposed framework. This approach can be adapted to many practical situations where the marginal distributions can be either continuous or discrete, including zero-inflated distributions. An adaptive Markov chain Monte Carlo algorithm is proposed to fit the model and estimate the parameters.

Tracing its origin to Sklar’s Theorem (Sklar, 1973), copula-based models offer a flexible way of constructing multivariate distribution functions by combining univariate marginal distribution via copulas (Nelsen, 2006). While historically fitting copula models has largely been implemented with frequentist approaches, with the advances in Bayesian statistics, a substantial number of literature in recent years have described model fitting from the Bayesian paradigm (e.g. Pitt et al., 2006; Smith and Khaled, 2012). The majority of works on copula regression have been on modeling the marginal means as a function of covariates. In order to study dynamic association using single-cell multi-omics data, we develop a novel framework to incorporate covariate-dependent correlation structures in the analysis.

The rest of this article is organized as follows: Section 2 presents the framework of the copula-based model. It also contains examples for specific marginal distributions that frequently considered in single-cell data analysis. Section 3 presents the details of estimation and model comparison procedures. Section 4 demonstrates the performance of the proposed model through simulation studies. We examine the coverage probability and mean squared error of the parameter estimates and compare the proposed model and a simpler model without dynamic association. Section 5 presents the results of the analysis on the single-cell mouse gastrulation data set. Finally, Section 6 concludes the article with a discussion.

2. Model Framework

2.1. Joint distribution

Let Y₁, … , Y_n be n independent bivariate random vectors and $Y_i={\left(Y_{i1},Y_{i2}\right)}^{\prime }$ for i = 1, … , n. For j = 1, 2, we assume that the marginal cumulative distribution function (CDF) of Y_ij is given by $F_j\left(\cdot ;{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)$, where θ_j represents a set of parameters associated with F_j, and ${\mathbf{\text{x}}}_i={\left(1,x_{i1},\dots ,x_{ip}\right)}^{\prime }$ a set of covariates for the i^th subject. We construct the joint CDF of Y_i via Gaussian copula with covariate-dependent correlation as follows. Let $Z_i={\left(Z_{i1},Z_{i2}\right)}^{\prime }$ be such that

$$Z_i\sim N_2\left(0=\left[\begin{array}{c}0\\ 0\end{array}\right],R_i=\left[\begin{array}{cc}1& {\rho }_i\\ {\rho }_i& 1\end{array}\right]\right)$$ (1)

with

$${\rho }_i=\text{corr}\left(Z_{i1},Z_{i2}\right)=\frac{\exp ({\mathbf{\text{x}}}_i^{\prime }\tau )-1}{\exp ({\mathbf{\text{x}}}_i^{\prime }\tau )+1},$$ (2)

where $\tau ={({\tau }_0,{\tau }_1,\dots ,{\tau }_p)}^{\prime }$. This formulation incorporates dynamic association between Y_i1 and Y_i2, i.e. association that depends on covariates, through the correlation between Z_i1 and Z_i2. We denote the joint CDF of Z_i by Φ_τ to reflect its dependence on the parameter τ. For both discrete and continuous marginals, the general form of the joint CDF of Y is given by

$$F_Y\left({\text{y}}_i;{\mathit{\theta }}_1;{\mathit{\theta }}_2,\mathit{\tau },{\mathbf{\text{x}}}_i\right)={\Phi }_{\tau }\left({\text{Φ}}^{-1}\left[F_1\left(y_{i1};{\mathit{\theta }}_1,{\mathbf{\text{x}}}_i\right)\right],{\text{Φ}}^{-1}\left[F_2\left(y_{i2},{\mathit{\theta }}_2,{\mathbf{\text{x}}}_i\right)\right]\right),$$ (3)

where Φ⁻¹ represents the inverse CDF of N(0,1) (Sklar, 1973). Note that the marginal distribution F_j can be either continuous or discrete. Further, apart from regressing the association on x_i given in (2), each of the two marginal means can also be covariate-dependent, as in Masarotto and Varin (2012).

2.2. Specific cases for marginal distributions

We now present two marginal distributions and their associated marginal mean regression models used in Sections 4 and 5, for the zero-inflated gamma distribution and the beta distribution, respectively. The discussion on the zero-inflated negative binomial distribution is provided in Appendix A in the Supporting Information. All of these distributions appear frequently as parametric assumptions in analyses of data arising from single-cell experiments.

Zero-inflated gamma distribution

If the scRNA-seq data are normalized (Lytal et al., 2020), the positive integer-valued counts are transformed to positive reals while genes with zero counts remain zero. In this case, a zero-inflated gamma distribution can be useful because its support is over the non-negative reals and it could easily accommodate the skewness exhibited in real data. In this distribution, Y_i degenerates at 0 with probability π, and follows a gamma(μ_iϕ, ϕ) distribution otherwise, with marginal density as follows:

$$f\left(y_i;{\mu }_i,\varphi ,\pi \right)=\left(1-\pi \right)\frac{{\varphi }^{{\mu }_i\varphi }}{\text{Γ}\left({\mu }_i\varphi \right)}{y}_i^{{\mu }_i\varphi -1}{e}^{-\varphi y_i}\cdot 1_{\left(y_i>0\right)}+\pi \cdot 1_{\left(y_i=0\right)}.$$ (4)

Regression model (Ding et al., 2015) can be constructed by regressing the mean μ_i onto covariates via the log-link function $\log \left({\mu }_i\right)={\mathbf{\text{x}}}_i^{\prime }\mathit{\beta }$. Optionally, it is sometimes of interest to consider covariates for the zero-inflation parameter π as well, for instance using a logit link via $\log \left[{\pi }_i/\left(1-{\pi }_i\right)\right]={\mathbf{\text{x}}}_i^{\prime }\eta$. The vector of all parameters in the marginal distribution is $\mathit{\theta }={({\mathit{\beta }}^{\prime },{\mathit{\eta }}^{\prime },\varphi )}^{\prime }$ when π is associated with covariates x_i, or $\mathit{\theta }={({\mathit{\beta }}^{\prime },\pi ,\varphi )}^{\prime }$ otherwise.

Beta distribution

The second type of data is proportion-valued within the interval (0, 1), which often arises from experiments such as measuring the DNA methylation levels (e.g. Weinhold et al., 2016; Liu et al., 2021) or chromatin accessibility (e.g. Smallwood et al., 2014; Clark et al., 2018). In this case, a beta distribution of the form

$$f\left(y;{\mu }_i,\varphi \right)=\frac{\text{Γ}\left(\varphi \right)}{\text{Γ}\left({\mu }_i\varphi \right)\text{Γ}\left[\left(1-{\mu }_i\right)\varphi \right]}{y}_i^{{\mu }_i\varphi -1}{\left(1-y_i\right)}^{\left(1-{\mu }_i\right)\varphi -1},0<y_i<1$$ (5)

is useful in beta regression (Cribari-Neto and Zeileis, 2010). Under this parameterization ${\mu }_i\in \left(0,1\right)$ is the mean of Y_i, and ϕ > 0 is the dispersion. The mean μ_i could be covariate-dependent via $\log \left[{\mu }_i/\left(1-{\mu }_i\right)\right]={\mathbf{\text{x}}}_i^{\prime }\mathit{\beta }$. The set of all parameters is $\mathit{\theta }={({\mathit{\beta }}^{\prime },\varphi )}^{\prime }$.

Finally, we present a simple simulated data set to illustrate this framework described by Expressions (1) to (3). For i = 1, … , 1000, let the covariates x_i ~ uniform(−1, 1). For ${\mathbf{\text{x}}}_i={\left(1,x_i\right)}^{\prime }$, set $\tau ={\left(-1,2.5\right)}^{\prime }$ in (2) so that log[(1 + ρ_i)/(1 − ρ_i)] = x_i. A total of 1000 pairs of (Y_i1, Y_i2) are simulated, where Y_i1 given x_i follows a gamma distribution according to (5) and (6) with ${\mathit{\beta }}_1={\left(1,1\right)}^{\prime }$, π = 0, and ϕ₁ = 10, while Y_i2 given x_i follows a beta distribution according to (7) and (8) with ${\mathit{\beta }}_2={\left(-0.5,1\right)}^{\prime }$ and ϕ₂ = 10. In Figure 1, the interval (−1, 1) is divided into eight sub-intervals of equal length. Slices of {(y_i1, y_i2)} are plotted for x_i falling within each interval. The superimposed contour on each plot represents the joint copula density evaluated at the midpoint of each interval for the given parameters. The Pearson correlation of (y_i1, y_i2)’s in each subset is calculated. As x increases, in addition to the change of marginal means, there is a gradual increase in the correlation between Y₁ and Y₂.

Figure 1.

The profile plot of {(y_i1, y_i2)} for x_i’s in eight intervals in increasing order. Conditional on x_i, y_i1 follows a gamma distribution, and y_i2 follows a beta distribution. The correlation ρ_i depends on x_i following log[(1 + ρ_i)/(1 − ρ_i)] = x_i. In addition, the marginal means increase with the magnitude of x_i too. The superimposed contours represent the copula density of (Y₁ , Y₂) evaluated at the midpoint of each interval.

3. Estimation and Model Selection

In this section, we describe the parameter estimation procedure from the Bayesian perspective via the Markov chain Monte Carlo (MCMC) sampling scheme. Let Y₁, … , Y_n be independent random variables described by (3), and ${\left\{y_i={\left(y_{i1},y_{i2}\right)}^{\prime }\right\}}_{i=1}^n$ an observed sample. Under the framework in the previous section, the set of all parameters is {θ₁, θ₂, τ}. For j = 1, 2, θ_j represents all parameters related to the j^th marginal distribution, and τ represents the regression parameter on the correlation in (2). Following Pitt et al. (2006) and Smith and Khaled (2012), we link each Y_i by a Gaussian-distributed latent variable Z_i. In the general form, the likelihood function is given by

$$L({\mathit{\theta }}_1,{\mathit{\theta }}_2,\mathit{\tau }|\text{y},\mathbf{\text{x}})=\prod _{i=1}^n{f}_Y(y_i;{\mathit{\theta }}_1,{\mathit{\theta }}_2,\mathit{\tau },{\mathbf{\text{x}}}_i),$$ (6)

where each f_Y(y_i; θ₁, θ₂, τ, x_i) can be expressed in terms of the latent Z_i and the marginal distributions F₁ and F₂. When the marginal distribution F_j is continuous, there is a one-to-one correspondence between Y_ij and Z_ij, and hence z_ij is completely known once y_ij is observed. On the other hand, when F_j is discrete, Z_ij is unknown, and needs to be generated during sampling. Further, calculating the joint f_Y (y_i; θ₁, θ₂, τ, x_i) may be computationally intense, e.g. having to evaluate n double integrals of bivariate Gaussian density functions. Instead, we adopt a one-margin-at-a-time approach. Under this approach, each MCMC iteration goes through updating (θ₁, z₁₁, … , z_n1), (θ₂, z₁₂, … , z_n2), and τ.

Consider first the case that the marginal distribution F_j is continuous. Conditional on the observed ${\left\{y_i={\left(y_{i1},y_{i2}\right)}^{\prime }\right\}}_{i=1}^n$ and θ_j, there is a one-to-one correspondence between z_ij and y_ij through $z_{ij}={\text{Φ}}^{-1}\left[F_j\left(y_{ij};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)\right]$, and hence (z₁₁, … , z_n1) is completely known. Equation (4) in Song (2000) provides the joint pdf of Y_i when both margins are continuous with pdf f₁ and f₂, given by

$$f_Y\left(y_i;{\mathit{\theta }}_1,{\mathit{\theta }}_2,\mathit{\tau },{\mathbf{\text{x}}}_i\right)={\left|R_i\right|}^{-1/2}\exp \left[\frac{1}{2}{z}_i^{\prime }\left(\mathbf{\text{I}}-R_i^{-1}\right)z_i\right]f_1\left(y_{i1};{\mathit{\theta }}_1,{\mathbf{\text{x}}}_i\right)f_2\left(y_{i2};{\mathit{\theta }}_2,{\mathbf{\text{x}}}_i\right),$$ (7)

where $z_i={\left({\text{Φ}}^{-1}\left[F_1\left(y_{i1};{\mathit{\theta }}_1\right)\right],{\text{Φ}}^{-1}\left[F_2\left(y_{i2};{\mathit{\theta }}_2\right)\right]\right)}^{\prime }$, and I is the identity matrix. This is readily verified based on the transformation $Y_{ij}=F_j^{-1}\left[\text{Φ}\left(Z_{ij}\right);{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right]$ for j = 1, 2 from ${\mathbf{Z}}_i={\left(Z_{i1},Z_{i2}\right)}^{\prime }\sim N_2\left(0,R_i\right)$. Based on this observation, the full conditional of θ_j is given by

$$\begin{array}{l}f\left({\mathit{\theta }}_j|\cdot \right)\propto \left[\prod _{i=1}^n{f}_Y\left(y_i;{\theta }_1,{\theta }_2,\mathit{\tau },{\mathbf{\text{x}}}_i\right)\right]\times f\left({\theta }_j\right)\\ \propto \exp \left[\frac{1}{2}\sum _{i=1}^n{z}_i^{\prime }\left(I-R_i^{-1}\right)z_i\right]\times \left[\prod _{i=1}^n{f}_j\left(y_{ij};{\theta }_j,{\mathbf{\text{x}}}_i\right)\right]\times f\left({\theta }_j\right),\end{array}$$ (8)

where f (θ_j) is the prior distribution of θ_j. Note that even though conditional on ${\left\{z_{i(-j)}\right\}}_{i=1}^n$, each $z_i^{\prime }\left(I-R_i^{-1}\right)z_i$ involves θ_j, through the nonlinear transformation $z_{ij}={\text{Φ}}^{-1}\left[F_j\left(y_{ij};{\mathit{\theta }}_j,x_i\right)\right]$.

On the other hand, when F_j is discrete, there is no longer a one-to-one correspondence between z_ij and y_ij. Instead, conditional on y_ij, z_i(−j) and θ_j, Z_ij follows $N\left({\rho }_i{z}_{i(-j)},1-{\rho }_i^2\right)$ truncated between ${\text{Φ}}^{-1}\left[F_j\left(y_{ij}^{-};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)\right]$ and ${\text{Φ}}^{-1}\left[F_j\left(y_{ij};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)\right]$, where $F_j\left(y_{ij}^{-};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)$ is the left-limit of F_j (y_ij; θ_j, x_i). The full conditional of θ_j is given by

$$\begin{array}{l}f\left({\mathit{\theta }}_j|\cdot \right)\propto \prod _{i=1}^n\{\text{Φ}\left[\frac{{\text{Φ}}^{-1}\left[F_j\left(y_{ij};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)\right]-{\rho }_i{z}_{i(-j)}}{1-{\rho }_i^2}\right]-\\ \text{Φ}\left[\frac{{\text{Φ}}^{-1}\left[F_j\left(y_{ij}^{-};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)\right]-{\rho }_i{z}_{i(-j)}}{1-{\rho }_i^2}\right]\}\times f\left({\mathit{\theta }}_j\right).\end{array}$$ (9)

Each likelihood of y_ij in the curly brackets is the probability of $Z_{ij}\sim N\left({\rho }_i{z}_{i(-j)},1-{\rho }_i^2\right)$ falling between ${\text{Φ}}^{-1}\left[F_j\left(y_{ij}^{-};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)\right]$ and ${\text{Φ}}^{-1}\left[F_j\left(y_{ij};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)\right]$.

Let the support of F_j be denoted as ${\mathcal{C}}_j$ for continuous and ${\mathcal{D}}_j$ for discrete distributions, respectively. The posterior sampling algorithm is described as follows.

1. Initiate θ₁, θ₂, τ, and ${\left\{z_i\right\}}_{i=1}^n$ . Set t = 1.

2. For j = 1, 2,

   1. sample ${\mathit{\theta }}_j^{(t)}$ from $f\left({\mathit{\theta }}_j|\cdot \right)$, where $f\left({\mathit{\theta }}_j|\cdot \right)$ is given by (8) for $y_{ij}\in {\mathcal{C}}_j$, and (9) for $y_{ij}\in {\mathcal{D}}_j$;
   2. for i = 1, … , n, update z_ij via

      1. $z_{ij}={\text{Φ}}^{-1}\left[F_j\left(y_{ij};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)\right]$ if $y_{ij}\in {\mathcal{C}}_j$
      2. $z_{ij}\sim N\left({\rho }_i{z}_{i(-j)},1-{\rho }_i^2\right)$ truncated at $\left({\text{Φ}}^{-1}\left[F_j\left(y_{ij}^{-};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)\right],{\text{Φ}}^{-1}\left[F_j\left(y_{ij};{\mathit{\theta }}_j,{\mathbf{\text{x}}}_i\right)\right]\right)$ if $y_{ij}\in {\mathcal{D}}_j$.
3. Sample τ^(t) from

   $$f(\mathit{\tau }|\cdot )\propto \left[\prod _{i=1}^{n}f\left(z_i\mid \mathit{\tau }\right)\right]\times f(\mathit{\tau }),$$ (10)

   where $f\left(z_i|\mathit{\tau }\right)$ is the density function of N₂(0, R_i) given τ, and f(τ) the prior density of τ.

4. Set t = t + 1 and return to Step 2.

Steps 2(a) and 3 require a Metropolis scheme. Here we adopt an adaptive random-walk Metropolis sampling algorithm proposed in Harrio et al. (2001). Suppose ${\mathit{\theta }}_j\in {\mathbb{R}}_d$. By the adaptive Metropolis sampling algorithm, we sample ${\mathit{\theta }}_j^{\ast }\sim N_d\left({\mathit{\theta }}_j^{(t)},C_t\right)$, where

$$C_t=\{\begin{array}{ll}{C}_0& t⩽t_0,\\ s_d\cdot \mathrm{cov}({\mathit{\theta }}_j^{(0)},\dots ,{\mathit{\theta }}_j^{(t)})+s_d\cdot ϵ\cdot I_d& t>t_0\end{array}$$ (11)

for some pre-specified covariance C₀ over the initial t₀ iterations. To prevent a degenerate C_t, we set ϵ = 0.0001 in practice. The scaling parameter s_d is set to be s_d = (2.4)²/d. Then the acceptance ratio is given by

$$r=\frac{f\left({\mathit{\theta }}^{\ast }|\cdot \right)}{f\left({\mathit{\theta }}^{(t)}|\cdot \right)},$$ (12)

and θ* is accepted with probability min{1, r}. If the parameter space is bounded, appropriate transformations need to be considered before sampling from the multivariate proposal distribution.

In the simulation studies and real data analysis, we have used weakly informative prior distributions for θ_j and τ. For instance, N_d(0, 100²I) for τ, or gamma(0.001, 0.001) for the dispersion parameter ϕ > 0. Unlike the strictly uninformative priors such as the Jeffreys’ prior, these priors are proper, but with a broad support still carry little prior information about the parameters.

4. Simulation Studies

4.1. Random variate generation

In this section, we describe the mechanism used to generate zero-inflated data in the simulation studies. For i = 1, 2, … , n, given covariates x_i and τ, first generate (z_i1, z_i2) with correlation ρ_i according to (1) and (2). Let $u_{ij}=\text{Φ}\left(z_{ij}\right)$, for j = 1, 2, where $\text{Φ}(\cdot )$ represents the CDF of N(0, 1). Finally, generate the response

$$y_{ij}=F_j^{-1}\left(u_{ij};{\mathit{\theta }}_j,{\pi }_j\right)=\inf \left\{y:F_j\left(y;{\mathit{\theta }}_j,{\pi }_j\right)⩾u_{ij}\right\},$$ (13)

where F_j denotes the j^th marginal CDF.

To simulate from a zero-inflated distribution, suppose y_ij = 0 with probability π_j, and comes from F_j with probability (1 − π_j). Then, given $u_{ij}=\text{Φ}\left(z_{ij}\right)$,

$$y_{ij}=\{\begin{array}{ll}{F}_j^{-1}\left(\frac{u_{ij}}{1-{\pi }_j}\right)& \mathrm{if}{u}_{ij}<\left(1-{\pi }_j\right)F_j\left(0;{\mathit{\theta }}_j,x_i\right)\\ 0& \mathrm{if}\left(1-{\pi }_j\right)F_j\left(0;{\mathit{\theta }}_j,x_i\right)⩽u_{ij}<\left(1-{\pi }_j\right)F_j\left(0;{\mathit{\theta }}_j,x_i\right)+{\pi }_j\\ F_j^{-1}\left(\frac{u_{ij}-{\pi }_j}{1-{\pi }_j}\right)& \mathrm{if}{u}_{ij}⩾\left(1-{\pi }_j\right)F_j\left(0;{\mathit{\theta }}_j,x_i\right)+{\pi }_j\end{array}.$$ (14)

4.2. Simulation studies and results

This subsection presents two simulation scenarios. The two simulations share the similarity that the marginal distributions are zero-inflated gamma and beta in both cases. In the first simulation, only one covariate is considered. In the second simulation study, data are simulated according to the mouse gastrulation data described in Section 5. In both simulation scenarios, we compare the performance of our proposed method to that of existing approaches in terms of statistical power. An additional simulation scenario where the marginal distribution is discrete as zero-inflated negative binomial is provided in Appendix B.3 in the Supporting Information.

Scenario 1: Zero-inflated gamma and beta distribution

For i = 1, 2, … , n, let (y_i1, y_i2) be the bivariate response, and $x_i={\left(1,x_i\right)}^{\prime }$ where x_i ~ beta(2, 8) is the covariate. Latent variables ${\left\{\left(z_{i1},z_{i2}\right)\right\}}_{i=1}^n$ are generated with $\mathit{\tau }=(1,1{)}^{\prime }$. Marginally, ${\left\{y_{i1}\right\}}_{i=1}^n$ is generated from a zero-inflated gamma distribution given by (4) with $\log \left({\mu }_{i1}\right)=x_i^{\prime }{\mathit{\beta }}_1$ for ${\mathit{\beta }}_1=(2,1{)}^{\prime }$ and ϕ₁ = 10. We also regress the zero-inflation parameter π_i on x_i according to $\log \left[{\pi }_i/\left(1-{\pi }_i\right)\right]=x_i^{\prime }\mathit{\eta }$ for $\mathit{\eta }=(-1,0.5{)}^{\prime }$. At the other margin, ${\left\{y_{i2}\right\}}_{i=1}^n$ is generated from a beta distribution given by (5) with $\log \left[{\mu }_{i2}/\left(1-{\mu }_{i2}\right)\right]=x_i^{\prime }{\mathit{\beta }}_2$ for ${\mathit{\beta }}_2=(1,1{)}^{\prime }$ and ϕ₂ = 10. These parameters are set to these values so that the simulated responses ${\left\{\left(y_{i1},y_{i2}\right)\right\}}_{i=1}^n$ are similar to the observed response from the real data analysis in Section 5.

For sample sizes n ∈ {100, 200, 500, 1000}, a total of 1000 random samples were generated using this setting. The proposed model is fitted for each sample, and posterior 95% equal-tail credible intervals (CI) were obtained from the MCMC output, where the total number of iterations was 10000 with the first 1000 as burn-ins. We then calculated the proportion of CIs covering the true value of each parameter averaged over these 1000 replicates as the coverage probability. Results are shown in Table 1. As the sample size increases, the coverage probabilities are all about the nominal level and the MSEs gradually decrease towards 0.

Table 1.

Empirical coverage probability (95% CP) of posterior 95% credible intervals and mean squared error (MSE) of parameters in simulation scenario 1 based on 1000 random samples. For each parameter and sample size, the number on top represents the 95% CP, and the number on the bottom in parentheses represents the MSE.

	n = 100	n = 200	n = 500	n = 1000		n = 100	n = 200	n = 500	n = 1000
β 01	0.932	0.935	0.942	0.948	β 11	0.934	0.941	0.951	0.952
	(0.196)	(0.156)	(0.075)	(0.041)		(0.092)	(0.061)	(0.016)	(0.001)
η 0	0.931	0.938	0.949	0.952	η 1	0.930	0.938	0.950	0.950
	(0.198)	(0.150)	(0.068)	(0.046)		(0.012)	(0.008)	(0.037)	(0.000)
β 02	0.931	0.941	0.945	0.952	β 12	0.937	0.948	0.951	0.949
	(0.215)	(0.162)	(0.109)	(0.054)		(0.019)	(0.011)	(0.004)	(0.000)
τ 0	0.932	0.939	0.952	0.948	τ 1	0.933	0.936	0.948	0.951
	(0.141)	(0.105)	(0.067)	(0.023)		(0.013)	(0.009)	(0.005)	(0.000)
ϕ 1	0.933	0.937	0.949	0.953	ϕ 2	0.932	0.938	0.951	0.949
	(0.998)	(0.403)	(0.176)	(0.063)		(0.960)	(0.342)	(0.218)	(0.110)

Further, we compare the proposed method and several existing methods in terms of statistical power in detecting dynamic association. Fixing the sample size at n = 500, pairs of ${\left\{\left(z_{i1},z_{i2}\right)\right\}}_{i=1}^n$ were simulated with $\mathit{\tau }={\left({\tau }_0,{\tau }_1\right)}^{\prime }={\left(0,{\tau }_1\right)}^{\prime }$ for τ₁ = 0, 0, 3, 0.6, … , 2.7, 3. For i = 1, 2, … , n, the covariates is given by $x_i={\left(1,x_i\right)}^{\prime }$ with x_i ~ beta(2, 8), according to the data analysis results in Section 5. Note that when τ₁ = 0, ρ_i is not dependent on the covariate x_i and hence there is no dynamic association. On the other hand, the covariate x_i affects ρ_i when τ₁ ≠ 0. Therefore, testing for dynamic association in this example is the same as testing H₀ : τ₁ = 0 against H₁ : τ₁ ≠ 0. For each true value of τ₁, 10000 random samples were simulated. For the proposed method, a 95% credible interval of τ₁ was calculated from the MCMC output. The power was the proportion of CIs excluding 0 out of the 10000 replicates.

The proposed method was compared to the liquid association (LA) in Li (2002) and a standardized version of LA (sLA), and the full and simple version of the conditional normal model (CNM (full), CNM (simple)) in Ho et al. (2011). Under the assumptions that Y₁, Y₂ are standardized with mean 0 and variance 1, X follows a standard normal distribution, and the marginal means of Y₁ and Y₂ are free of X, Li (2002) used $\mathbb{E}\left(Y_1{Y}_{2}X\right)$ to measure the dynamic association between Y₁ and Y₂ according to X. In this simulation, the sample mean $\widehat{\mathbb{E}}\left(Y_1{Y}_{2}X\right)$ was used a statistic for the liquid association, and p-values were obtained using a permutation test. A standardized version of the LA statistic, $\frac{\widehat{\mathbb{E}}\left(Y_1{Y}_{2}X\right)}{SE\left[\widehat{\mathbb{E}}\left(Y_1{Y}_{2}X\right)\right]}$, was also considered in the comparison.

In terms of the two versions of the conditional normal model in Ho et al. (2011), a key assumption of the model is that conditional on X, Y₁ and Y₂ follow a bivariate normal distribution. The full model includes covariate-dependent mean, variances and correlation while the CNM (simple) only considers covariate-dependent correlation. P-values for sLA, CNM (full) and CNM (simple) were obtained from the functions provided in the LiquidAssociation R package.

The comparison of statistical power for the methods described above is presented in Figure 2. At τ₁ = 0, our proposed flexible copula model (FC) and CNM (full) are the only two methods that maintain the 5% nominal significance level. Strikingly, the original liquid association measure (LA) has a Type I error rate of almost 100%. CNM (simple) and sLA also have fairly high Type I error of around 30%. This is partly due to these models do not account for covariate-dependent means. On the other hand, as the signal of τ₁ increases, the power to detect the signal for FC and CNM (full) both increase toward 1. However, the conditional normal model fails to keep up with the proposed model, mainly because of its assumption that Y₁ and Y₂ given X are normally distributed.

Figure 2.

Comparison of statistical power in testing H₀ : τ₁ = 0 against H₁ : τ₁ ≠ 0 for the proposed flexible copula model (FC), full and simple version of conditional normal model (CNM (full), CNM (simple)), unstandardized liquid association (LA) and standardized liquid association (sLA). For each true value of τ₁, the power was calculated based on 10000 random samples.

Scenario 2: Data simulated according to real data analysis

In this scenario, we perform power analyses to compare the proposed copula model versus the full conditional normal model in Ho et al. (2011) according to the experimental data analysis presented in Section 5 (Equations (14) to (16)). Specifically, for i = 1, 2 … , n = 627, let x_i0 = 1, x_i1 ~ beta(1.5, 7.5), (w_i, x_i2, x_i3, x_i4, x_i5) ~ categorical(0.2 · 1₅), i.e. a categorical distribution of five categories with equal probability, and x_ij = x_i1 · x_i(j−4) for j = 6, 7, 8, 9. The covariate vector for the i^th observation is given by $x_i={\left(x_{i0},x_{i1},\dots ,x_{i9}\right)}^{\prime }$. Note that when simulating from the categorical distribution, w_i represents the baseline category, hence not included in the covariates. In terms of the multi-omics profiling data in Section 5, x_i1 represents the proportion of methylation at the promoter region of a particular gene for the i^th cell, (w_i, x_i2, x_i3, x_i4, x_i5) the germ layer of the cell, and x_i6, … , x_i9 the interaction between the methylation and the germ layer. Similar to Scenario 1, y_i1 was generated from a zero-inflated gamma distribution according to (4) and y_i2 was generated from a beta distribution according to (5). We regressed both marginal means as well as the zero-inflation parameter π in (4). Most parameters were set to be the posterior estimates from the results in Section 5 related to the genes Mrpl58 and Sox17. Importantly, in terms of the dynamic association parameter τ in (2), the posterior estimate for τ₁ was ${\widehat{\tau }}_1\approx -12$ (See Table 2 for detail.) In this simulation, the true value of τ₁ is varied from −15 to 0 with increments of 3. For each true value of τ₁, we simulated 10000 random samples, and tested H₀ : τ₁ = 0 against H₁ : τ₁ ≠ 0 using the proposed method and the conditional normal model in Ho et al. (2011). The power of the two methods was calculated as the proportion of H₀ rejected out of the 10000 replicates.

Table 2.

Top twenty genes with the largest dynamic association estimates between scRNA-seq and chromatin accessibility associated with the DNA methylation at the promoter region of Sox17 at each germ layer. Point estimates (posterior mean) represent the estimated dynamic association. Corresponding 95% credible intervals are shown in parentheses.

		Epiblast (4.5-5.5 days)		Epiblast (6.5-7.5 days)		Endoderm		Mesoderm		Ectoderm
	Gene	${\widehat{\tau }}_1$	Gene	$\widehat{{\tau }_1+{\tau }_6}$	Gene	$\widehat{{\tau }_1+{\tau }_7}$	Gene	$\widehat{{\tau }_1+{\tau }_8}$	Gene	$\widehat{{\tau }_1+{\tau }_9}$
1	Mrpl58	−11.33 (−15.04,−1.16)	Rps23	12.73 (5.87,17.92)	Fh1	−22.41 (−32.54,−0.03)	Mrpl54	−14.93 (−20.87,−4.08)	Adat1	29.79 (2.5,41.49)
2	Otulin	−10.9 (−13.96,−1.2)	Abl1	11.53 (5.13,17.01)	Dcaf11	21.03 (4.76,29.77)	Pnisr	13.42 (1.5,21.68)	Ptk7	−28.41 (−39.35,−0.01)
3	Pusll	−10.47 (−14.32,−2.19)	Cit	−11.38 (−15.16,−2.07)	Dhx15	−20.41 (−31,−0.91)	Kmt2b	−13.33 (−20.85,−4.33)	Btf3l4	−25.21 (−38.67,−4.53)
4	Ergic3	−10.34 (−15.74,−0.12)	Ebna1bp2	9.15 (2.34,13.39)	Fasn	20.25 (2.14,28.95)	Atxn10	−13.33 (−19.62,−3.15)	Ppib	23.76 (4.59,33.91)
5	Thoc5	−9.78 (−13.57,−1.03)	Zgpat	9.14 (2.75,13.29)	Taco1	19.37 (0.48,31.89)	Rer1	12.57 (1.24,19.52)	Dnajc17	22.74 (4.35,34.64)
6	Vmp1	−9.32 (−13.84,−1.53)	Atmin	−9.07 (−13.62,−1.45)	Cycs	19.2 (0.95,30.35)	Eif2b2	12.12 (3.26,18.43)	Rcc1	21.74 (0.87,37)
7	Tom1l1	9.17 (0.4,13.38)	Chd1l	−8.85 (−12.8,−2.18)	Gtf2a1	−18.51 (−24.96,−3.56)	Tm9sf1	−11.98 (−17.18,−2.44)	Ppp3cb	21.54 (1.3,32.85)
8	Snrpd2	9.09 (0.62,13.29)	Ephb4	−8.46 (−13.08,−1.03)	Mipep	−18.21 (−25.32,−3.4)	Zc3h4	−11.92 (−19.43,−0.44)	Chd1l	−21.47 (−34.37,−3.84)
9	Rtl8b	9.08 (3.04,12.8)	Pi4k2b	8.25 (1.56,12.48)	Zfp740	−17.9 (−27.44,−1.26)	Mthfs	11.81 (5.98,17.58)	Bop1	20.69 (0.1,35.51)
10	Prr3	−8.56 (−14.18,−0.12)	Cbx3	8.24 (2.85,11.95)	Ube2f	17.85 (3.45,24.07)	Gpatch11	11.8 (1.65,18.55)	Smg1	−20.67 (−32.88,−2.18)
11	Mrpl20	8.36 (1.4,11.83)	Tcp11l1	−8.17 (−12.57,−0.94)	Chaf1a	−17.59 (−26.49,−1.98)	Tia1	−11.79 (−18.22,−1.2)	Pmpca	19.61 (0.34,30.26)
12	Calu	−7.99 (−13.21,−1.26)	Abhd4	8.1 (1.35,13.81)	Tusc2	17.47 (1.87,29.05)	Mtx1	−11.71 (−18.38,−0.49)	Kxd1	−19.58 (−28.52,−0.51)
13	Nxn	7.9 (0.75,11.51)	Mrpl54	8.07 (0.16,12.22)	Pafah1b2	−17.24 (−26.9,−1.42)	Nup205	−11.46 (−17.72,−2.83)	Hmmr	19.43 (2.55,28.1)
14	Oga	7.9 (0.06,11.76)	Gsta4	−7.94 (−12.62,−0.13)	Chchd3	16.18 (4.94,25.27)	Pgm2	−11.34 (−17.7,−3.36)	Gcn1	−19.24 (−30.05,−4.88)
15	Ccdc186	7.83 (1.26,11.52)	Ufm1	−7.91 (−12.74,−0.13)	Kat5	15.6 (4.03,22.69)	Far1	10.97 (0.06,17.09)	Usp25	−19.12 (−33.32,−0.77)
16	Tspan4	−7.79 (−12.89,−0.73)	Tmed7	7.82 (1.59,12.68)	Ctsd	15.1 (0.97,25.92)	Ldha	−10.75 (−16.72,−0.93)	Dhx36	−18.94 (−28.81,−0.07)
17	Clcn5	−7.78 (−10.87,−0.07)	Ap1s1	7.72 (1.06,12.33)	Fcf1	−15.08 (−21.91,−1.55)	Srebf2	−10.75 (−16.27,−3.16)	Tmem192	18.76 (2.45,27.33)
18	Mcmbp	−7.52 (−11.4,−1.16)	Echdc2	−7.65 (−11.8,−1.92)	Klf9	14.64 (2.06,22.23)	Mtif2	10.62 (1.5,17.44)	Pgs1	18.51 (2.44,31.89)
19	Pim1	−7.33 (−10.65,−2.27)	Zfp511	7.63 (0.57,11.98)	Lin28b	14.57 (2.03,22.06)	Cenph	−10.61 (−15.54,−3.65)	Ciao1	18.32 (1.15,32.6)
20	Ikbip	−7.21 (−11.29,−1.92)	Ndufa12	7.6 (0.77,12.21)	Noct	−14.11 (−21.64,−4.61)	Phf10	−10.46 (−16.22,−1.4)	Tns3	−18.04 (−30.65,−0.48)

The result of the comparison is presented in Figure A.2 in Appendix B.1. Both methods maintain the 5% nominal Type I error rate. As the magnitude of τ₁ increases, the proposed method outperforms the CNM (full) in detecting dynamic association. We note here that unlike Scenario 1, we did not compare the proposed method to Liquid Association in Li (2002) in scenario 2, since Li (2002)’s approach could not consider multiple covariates.

5. Multi-omics Profiling of Mouse Gastrulation

Argelaguet et al. (2019) studied the multi-omics profiling of mouse gastrulation at single-cell resolution. In this study, n = 627 single cells were taken from different germ layers of embryonic cells between 4.5 and 7.5 days. During early stages of development (4.5-5.5 days), samples solely came from the epiblast. At 6.5-7.5 days, samples were taken from epiblast, endoderm, mesoderm, and ectoderm. For each cell, parallel chromatin accessibility, DNA methylation, and gene expression profile were sequenced under the single-cell scNMT-seq (single-cell nucleosome, methylation, and transcription sequencing) protocol of Clark et al. (2018). On average, 37.9 cells per embryo were sequenced for scNMT-seq; 54.6 cells per embryo were sequenced for scRNA-seq. According to Argelaguet et al. (2019); Mohammed et al. (2017), the cells from the same germ layer at the same developmental stage are assumed to be homogeneous. All sequencing were performed on a NextSeq 500 instrument with a raw sequencing depth of 1 million paired-end reads per cell for scRNA-seq.

Raw scRNA-seq data were normalized via the deconvolution strategy described in Lun et al. (2016). Methylation calling and separation of endogenous methylation and chromatin accessibility was performed using Nucleosome Occupancy and Methylation sequencing (NOMe-seq; Kelly et al., 2012; Pott, 2017). Following Smallwood et al. (2014), the CpG methylation or GpC accessibility for each site in each cell is measured as binary (0 or 1). Then the CpG methylation or GpC accessibility rate at the promoter (or genebody) region were computed as the average of the methylated CpG (or GpC for accessibility) sites over all observed sites within the region. Hence DNA methylation and chromatin accessibility are both recorded as rates (proportions).

Among the 18,345 genes sequenced, we filtered out genes with more than 80% 0’s and with small variances in scRNA-seq and chromatin accessibility (less than the medians of all genes) and kept the remaining 1,260 genes in the following analysis. For each of the 1, 260 genes and the i^th sample, the scRNA-seq and the gene-body chromatin accessibility measurements were paired as Y_i1 and Y_i2, respectively. We denote this set of 1,260 genes as ${\mathcal{G}}_1$. In Argelaguet et al. (2019), the authors identified 15 genes that are transcription factors at different germ layers whose methylation at the promoter regions may affect how scRNA-seq interacts with chromatin accessibility during mouse gastrulation. We call the set of these 15 transcription factors ${\mathcal{G}}_2$.

The study by Duren et al. (2017) reported that the association between transcription activities and cis-regulatory elements (REs) can be modulated by trans-acting transcription factors (TFs). The interplays between cis-REs, TFs, and the expression levels of target genes could vary in different cells at various developmental stages. To study these intricate transcription regulatory mechanisms, for each pair of genes in ${\mathcal{G}}_1$ and ${\mathcal{G}}_2$ (1,260 × 15 = 18, 900 gene pairs in total), we fit the proposed copula model by considering the methylation at the promoter region, different germ layers, and their interactions. For i = 1, … , n, we assume that Y_i1 follows a zero-inflated gamma distribution with marginal mean regression performed via (4), and that Y_i2 follows a beta distribution with marginal mean regression performed via (5). The marginal mean of gamma distribution, μ_i1, is covariate-dependent and is modeled as:

$$\begin{array}{l}\hfill \log \left({\mu }_{i1}\right)={\beta }_{01}+{\beta }_{11}\times \text{methylation}+{\beta }_{21}\times \text{I}(\text{epiblast},\mathrm{6.5-7.5}\text{days})+{\beta }_{31}\times \text{I}(\text{endoderm})+\\ {\beta }_{41}\times \text{I}(\text{mesoderm})+{\beta }_{51}\times \text{I}(\text{ectoderm})+{\beta }_{61}\times \text{methylation}\times \text{I}(\text{epiblast},\mathrm{6.5-7.5}\text{days})+\\ {\beta }_{71}\times \text{methylation}\times \text{I}(\text{endoderm})+{\beta }_{81}\times \text{methylation}\times \text{I}(\text{mesoderm})+\\ {\beta }_{91}\times \text{methylation}\times \text{I}(\text{ectoderm}),\end{array}$$ (15)

where the variable methylation refers to the percentage of methylation at the promoter region, and I(·) is an indicator specifying the germ layer for the i^th observation. Note that in this case, epiblast (4.5-5.5 days) is the baseline. Further, we allow the zero-inflation parameter π to vary across different germ layers; i.e.

$$\begin{array}{l}\log \left(\frac{{\pi }_i}{1-{\pi }_i}\right)={\eta }_0+{\eta }_1\times \text{I}(\text{epiblast},\mathrm{6.5-7.5}\text{days})+{\eta }_2\times \text{I}(\text{endoderm})+\\ {\eta }_3\times \text{I}(\text{mesoderm})+{\eta }_4\times \text{I}(\text{ectoderm}).\end{array}$$ (16)

The dispersion parameter for Y_i1 is given by ϕ₁. The marginal mean regression for Y_i2 is similar to (14), but with a logit link for the marginal mean μ_i2. The dispersion at this margin is given by ϕ₂. Regression on ρ_i is performed according to

$$\begin{array}{l}\log \left(\frac{1+{\rho }_i}{1-{\rho }_i}\right)={\tau }_0+{\tau }_1\times \text{methylation}+{\tau }_2\times \text{I}(\text{epiblast},\mathrm{6.5-7.5}\text{days})+{\tau }_3\times \text{I}(\text{endoderm})+\\ {\tau }_4\times \text{I}(\text{mesoderm})+{\tau }_5\times \text{I}(\text{ectoderm})+{\tau }_6\times \text{methylation}\times \text{I}(\text{epiblast},\mathrm{6.5-7.5}\text{days})+\\ {\tau }_7\times \text{methylation}\times \text{I}(\text{endoderm})+{\tau }_8\times \text{methylation}\times \text{I}(\text{mesoderm})+\\ {\tau }_9\times \text{methylation}\times \text{I}(\text{ectoderm}).\end{array}$$ (17)

Note that by this setting, τ₁ represents the dynamic association between RNA-seq and chromatin accessibility of gene 1 due to the methylation of gene 2 for epiblast (4.5-5.5 days). The coefficients τ₆ to τ₉ represent the change in the dynamic association from the baseline to their respective germ layer. To guarantee convergence, posterior draws are based on Markov chains of 20,000 iterations, and 5,000 burn-ins in each of the 18,900 models (gene pairs).

We examine the results from several different aspects. From a broad perspective, we are interested in how the two groups of genes are connected to each other in terms of dynamic association. For instance, τ₁ quantifies the dynamic association for epiblast (4.5-5.5 days) according to Equation (16). Based on the posterior draws of τ₁, for each of the 15 transcription factors whose methylation may lead to significant dynamic association between RNA-seq and chromatin accessibility of genes in ${\mathcal{G}}_1$, we kept the top 5% genes with the largest ${\widehat{\tau }}_1$ in absolute value, provided the corresponding credible intervals exclude 0. A network plot in Figure 3 summarizes the results. In the plot, the 15 transcription factors are represented as diamonds. Each of these 15 genes are linked to a small number of RNA-seq/chromatin genes (in circles), forming small “local” clusters. Eight out of these fifteen local clusters were isolated from others. The other seven formed larger clusters through other genes.

Figure 3.

Network plot of ${\mathcal{G}}_1$ (RNA-seq/chromatin accessibility, in circles) and ${\mathcal{G}}_2$ (transcription factors, in diamonds) based on the magnitude of posterior draws of τ₁ for epiblast (4.5-5.5 days). Links are established if |${\widehat{\tau }}_1$| is among the 5% largest, provided the credible interval excludes 0. Edges with positive estimates (${\widehat{\tau }}_1>0$) are represented as dash lines, negative estimates (${\widehat{\tau }}_1<0$) as solid lines.

Biologically, Sox17 is the transcription factor essential for vertebrate endoderm development (Kanai-Azuma et al., 2002; Hudson et al., 1997), hence we choose Sox17 as an example. We rank the gene pairs with significant dynamic associations (95% CI excluding 0) by the magnitude of point estimates at each germ layer, and find the top gene pairs having the largest dynamic association. Table 2 presents the top twenty genes with the largest dynamic association estimates between scRNA-seq and chromatin accessibility due to the DNA methylation at the promoter region of Sox17 at each germ layer.

Lastly, for each specific pair of genes, we wish to examine how the dynamic association changes according to the methylation level across different germ layers. We demonstrate this perspective by considering the gene pairs Sox17 and Mrpl58 from Figure 3. According to (16) and the posterior mean of τ, Figure A.2 in Appendix C.1 presents an illustration of the estimated ρ as a function of the promoter methylation of Sox17 at different germ layers. The methylation of Sox17 influences $\widehat{\rho }$ in similar manners in Epiblast (4.5-5.5 days) and Ectoderm. When Sox17 is more methylated in these cell types, the correlation between RNA-seq and chromatin accessibility in Mrpl58 tends to decrease. This decreasing pattern is also present in Mesoderm, but to a lesser degree. On the other hand, in Epiblast (6.5-7.5 days) and Endoderm, when Sox17 is more methylated in these cell types, the correlation between RNA-seq and chromatin accessibility in Mrpl58 tends to increase as shown in Figure A.2.

As pluripotent epiblast cells differentiate into the three primary germ layers, lineage specific transcription factors, such as Sox17, activate gene regulatory networks necessary for lineage specification. The opposite profiles of ρ for Mrpl58 in epiblast and endoderm as Sox17 methylation changes (Figure A.2) might imply that Sox17 acts to repress Mrpl58 during endoderm formation. Among the genes where $\widehat{\tau }$ is negative in Table 2, which is suggestive of transcriptional activation by Sox17, are Dhx15, Mipep, and Noct, which all have cis-regulatory modules bound by Sox17 during Xenopus gastrulation and may be part of the Sox17 gene regulatory network (Mukherjee et al., 2020).

To further demonstrate this dynamic relationship, Figure 4 provides the contour plots of the posterior predictive density of (Y₁, Y₂). Each row represents the estimated density in each germ layer at the 10^th, 50^th, and 90th sample percentile of the promoter methylation of Sox17, which corresponds to 0.04, 0.09, and 0.14, respectively. The points superimposed on each plot represent the observed data in each germ layer for the promoter methylation falling within a 20% equal-tail neighborhood of the corresponding sample percentiles. When the methylation level is small, the association between scRNA-seq and chromatin accessibility of Mrpl58 is mildly positive. As the methylation level increases, the association gradually decreases towards being negatively associated. For instance, at the baseline (epiblast, 4.5-5.5 days), the association between scRNA-seq and chromatin accessibility of Mrpl58 is positive when the methylation of Sox17 is low. As the methylation level of Sox17 increases, the association pattern becomes negative, and the strength of the negative association increases with the methylation level. Network plots similar to Figure 3 for other germ layers are provided in Appendix C.2. Convergence and posterior diagnostics results for this model are provided Appendix C.3. In addition, results of a ZINB model using the raw scRNA-seq counts is provided in Appendix C.4 in the Supporting Information.

Figure 4.

Contour plots of posterior predictive joint density of scRNA-seq (zero-inflated gamma distribution) and chromatin accessibility (beta distribution) of Mrpl58. The rows correspond to the promoter methylation of Sox17 at its 10^th, 50^th, and 90^th sample percentile, respectively. Each column represents a specific germ layer. Points on each plot represent the observed data in each germ layer within a 20% equal-tail neighborhood of the sample percentiles of the promoter methylation of Sox17.

6. Discussion

We proposed a general framework for analyzing the dynamic association between two random variables given a set of covariates. In order to account for various marginal distributions observed in single-cell multi-omics data, we constructed copula-based models and incorporated flexible covariate-dependent correlation structures in the proposed approaches. Simultaneously, the marginal means can be covariate-dependent as well. An MCMC sampling algorithm is introduced to estimate the model parameters. The usefulness of the proposed framework is demonstrated through a series of simulation studies and an analysis of a set of single-cell multi-omics data for mouse gastrulation.

When analyzing the co-expression pattern across different genes or modalities, most existing approaches assume normality on the marginal distributions and directly use the Pearson correlation coefficient for inference. However, despite its convenience, this assumption may not be appropriate for single-cell multi-omics analysis, due to the nature of the data such as zero-inflated discrete count data, and proportion-valued data. In the paper, we consider several specific choices for marginal distributions for their popularity in analyzing multi-omics data. Furthermore, our proposed framework could accommodate a wide range of marginal distributions, continuous or discrete. This flexibility lends itself to situations where the normality assumption and the Pearson correlation may be inadequate.

In our proposed model, we used the Fisher’s transformation for ρ to ensure −1 ⩽ ρ ⩽ 1. Other sigmoid functions can also be easily adapted into our model framework. Our model-based framework can also be readily applied to study coregulators in transcriptional activities as reported by Zhou et al. (2007) by considering multiple co-regulators as covariates.

In this paper, we followed the Bayesian approach for several reasons. First, model fitting using the Bayesian method via an MCMC scheme is fairly straightforward. Second, we can obtain the full posterior probability distribution of the parameters, which makes inference readily achievable. To calculate the standard error of parameters when modeling the dynamic association, frequentist approaches, such as Ho et al. (2011), often use resampling schemes such as jackknifing.

The Bayesian framework in this paper also offers a method for model comparison using the log-pseudo marginal likelihood (LPML, Gelfand and Dey, 1994), which we have explored in Appendix B.2 through a simulation study that compares the dynamic association model to a simpler model without dynamic association. In addition, the LPML is useful to compare different candidate marginal distributions, e.g. to test whether the distributional assumptions are appropriate for the data.

Alternatively, the Kolmogorov-Smirnov test can be used to test the goodness-of-fit of a specific marginal null distribution. In the experimental data analysis in Section 5, for each of the 1,260 genes, we performed a Kolmogorov-Smirnov test on the non-zero part of the scRNA-seq with the gamma distribution being the null hypothesis. Shape and scale parameters of the gamma distribution were estimated using MLE assuming a gamma likelihood. With multiple comparison correction (Benjamini and Hochberg, 1995), none of the null distribution of gamma was rejected under a 5% false discovery rate.

To correct for multiple testings, the approaches described in Muller et al. (2004) and Wang and Dunson (2010) can be implemented in conjunction with our analytical framework. Briefly, the posterior densities can be used to calculate Bayes factors and the expected false discovery rate to determine the optimal threshold on the Bayes factors for type I error control under multiple hypotheses. In terms of model fitting, we set the tuning parameters of the adaptive MCMC algorithm according to the suggestions given by Harrio et al. (2001) and Harrio et al. (2005). We considered uninformative priors on all parameter in this paper. Nevertheless, when fitting the marginal means using a (generalized) linear model, informative priors such as the g-prior (Zellner, 1986) on the regression coefficients can be considered. In the experimental data analysis, we considered DNA methylation statuses for genes encoding transcription factors as the covariate “methylation” in Equations (15) and (17). Databases such as ELMER (Yao et al., 2015), FunGenES (Schulz et al., 2009), and Mouse TF Atlas (Zhou et al., 2017) could potentially provide useful prior information for transcription factors and their target genes in mouse embryonic stem cells.

In the simulation studies as well as the real data analysis, convergence of the Markov chains was verified using the Gelman-Rubin convergence diagnostic based on four parallel chains. In terms of the computational performance, the proposed model was implemented solely using the statistical software R. It takes about two minutes on a personal laptop (Intel Core i5-7200U CPU) to complete a Markov chain of 10000 iterations in Simulation Scenario 1 of Section 4.

Data Availability Statement

The data that support the findings in this paper are openly available at Gene Expression Omnibus, accession number GSE121708 (https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE121708, Argelaguet et al. (2019)).