A double Pólya-Gamma data augmentation scheme for a hierarchical Negative Binomial - Binomial data model

Abstract

A double Pólya-Gamma data augmentation scheme is developed for posterior sampling from a Bayesian hierarchical model of total and categorical count data. The scheme applies to a Negative Binomial - Binomial (NBB) hierarchical regression model with logit links and normal priors on regression coefficients. The approach is shown to be very efficient and in most cases out-performs the Stan program. The hierarchical modeling framework and the Pólya-Gamma data augmentation scheme are applied to human mitochondrial DNA data.

1. Introduction

One of the classical hierarchical statistical model settings involve a hierarchy of what we will call total count and category count data. Typically, category count data $Y_i,i=1,\dots ,N$ can be modelled as $Y_i~\text{Binomial}\left(m_i,p_i\right)$ where $m_i$ is the total count for each observation. When the total counts are observed they are usually treated as known and fixed. Even if there is interest in modeling both the category counts $\left(Y_i\right)$ and the total counts $\left(m_i\right)$ it is usually not necessary to model them jointly. In fact, if the data distributions of a total count, $p\left(\mathbf{m}\mid {\mathit{\theta }}_1\right)$, and of a categorical count, $p\left(\mathbf{y}\mid \mathbf{m},{\mathit{\theta }}_2\right)$, do not have any common parameters and if the parameters ${\mathit{\theta }}_1$ and ${\mathit{\theta }}_2$ are a priori independent, the posteriors are conditionally independent as well since

$$p\left({\mathit{\theta }}_1,{\mathit{\theta }}_2\mid \mathbf{m},\mathbf{y}\right)=\frac{p\left(\mathbf{m}\mid {\mathit{\theta }}_1\right)p\left({\mathit{\theta }}_1\right)}{p(\mathbf{m})}\frac{p\left(\mathbf{y}\mid \mathbf{m},{\mathit{\theta }}_2\right)p\left({\mathit{\theta }}_2\right)}{p(\mathbf{y}\mid \mathbf{m})}.$$ (1)

That is, we get the same posterior inferences by simply modeling the categorical count separately and treating the total count data as fixed. However, if the data are not conditionally independent of the other parameters, i.e. if we have $p\left(\mathbf{m}\mid {\mathit{\theta }}_1,{\mathit{\theta }}_2\right)$ and/or $p\left(\mathbf{y}\mid \mathbf{m},{\mathit{\theta }}_2,{\mathit{\theta }}_1\right)$ the posteriors of ${\mathit{\theta }}_1$ and ${\mathit{\theta }}_2$ will not be independent. For example, if there is reason to believe that the proportion of a category $\left(p_i\right)$ is related to the expected total count then a joint (hierarchical) data model is preferable to two separate models. In this paper we consider a particular modeling framework for this situation where total counts are modelled via the Negative Binomial distribution and categorical counts, conditional on total counts, are modelled as Binomials. Furthermore, we develop a double Pólya Gamma data augmentation scheme to efficient posterior sampling for this model type.

Our motivation for this work comes from analyzing human mitochondrial DNA data. In addition to the 23 pairs of chromosomes, each cell in the human body has a collection of mitochondrial DNA (mtDNA) located outside of the cell nucleus. MtDNA is in the form of a circular chromosome, of which there are generally hundreds of copies in a single cell. The total number of these chromosomes within a cell is called its copy number and can vary e.g. over time, through disease treatments, between tissue types, and between healthy and unhealthy cells. Each mtDNA chromosome is comprised of approximately 16,569 nucleotides (the A’s, C’s, G’s and T’s that comprise all DNA). By convention, each of these nucleotides is given a number, representing its position. The positions are numbered sequentially in a standard manner, beginning with 1 at a specific nucleotide and moving in a defined direction around the circle, ending at position 16,569 adjacent to position 1. At each position one specific nucleotide type (A, C, G, or T) is defined as a wild type, roughly considered as the “normal” type at that position. If any other type is found at that position it is called a mutant. Within one tissue sample, and even within one cell, the mtDNA chromosomes may be a mixture of wild types and mutants at a given position. The mutant proportion refers to the proportion of the mtDNA chromosomes that have a mutant instead of the wild type at the position under consideration. At the vast majority of positions, a tissue sample will be 0% mutation proportion. Data on mtDNA is usually collected in the aggregate (i.e. a pool of mtDNA from thousands or millions of cells), so we observe the average copy number and average mutant proportion per cell in a sample. Both copy number (Filograna et al., 2021) and mutant proportion (Picard et al., 2014) has been linked to human disease, including cancer (Kopinski et al., 2021). However, no studies to date have attempted to model the relationships between and dynamics of mtDNA copy numbers and mutant proportions.

There are a few examples in the literature of joint modeling of total counts and category counts with two or more categories. Terza and Wilson (1990) modelled households’ choices among types of trips and frequency of trips with a multinomial Poisson hurdle model where regression parameters for total counts are restricted to be proportional to the regression parameters of the category counts. Illian et al. (2013) modelled the number of T. carnosus plants and their health status with a Poisson-Binomial model and forge an association between the total counts and category counts by making the spatial effect in one layer proportional to the spatial effect in the other layer. Chiou and Fu (2013) used a similar approach in the setting of number and severity of car crashes. Schmidt et al. (2022) model Aedes borne diseases with a Bayesian Poisson-multinomial model that includes random effects which are correlated between the two model layers. Qu and Bradley (2023) present a similar modeling approach for US Census data. In our setting the interest is in directly describing the association between total counts and category counts rather than accounting for it via covariance structures. Both Terza and Wilson (1990) and Chiou and Fu (2013) apply maximum likelihood to estimate parameters. Illian et al. (2013) and Schmidt et al. (2022) apply mostly normal priors on regression parameters and random effects and utilize general purpose programs to obtain posterior samples (INLA and Stan). Qu and Bradley (2023) apply a Gibbs sampler by applying multivariate logit-beta prior distributions.

We will present a double Pólya-Gamma data augmentation scheme for posterior sampling in this paper. In general, data augmentation schemes for posterior sampling have a long and successful history in Bayesian statistics (van Dyk and Meng, 2001). In a nutshell, data augmentation schemes re-parameterize (augment) the model by introducing latent variables that do not change the original (now marginal) model but makes it possible to apply a Gibbs sampler for posterior sampling which is usually more efficient and easier to use than other sampling methods such as the Metropolis-Hastings algorithm. Perhaps the most famous example is the Albert and Chib (1993) algorithm for the binary probit model $Y_i~\text{Bernoulli}\left({\Phi }^{-1}\left({\mathbf{x}}_i^{\prime }\mathit{\beta }\right)\right),i=1,\cdots ,N$, where $\mathit{\beta }$ is a vector of unknown regression parameters with a multivariate normal prior, ${\mathbf{x}}_i$ is a vector of known covariates, and $\Phi$ is the standard Gaussian cdf. They introduced $N$ latent normally distributed variables and the Gibbs sampler consists of a multivariate normal (for $\mathit{\beta }$) and $N$ truncated normals for the latent variables. This approach has been widely used, extended e.g. to correlated and multivariate settings (Chib and Greenberg, 1998), and extensively studied e.g. in the context of spatial data (Berrett and Calder, 2016). However, applying this data augmentation scheme to category count data, i.e. $Y_i~\text{Binomial}\left(m_i,{\Phi }^{-1}\left({\mathbf{x}}_i^{\prime }\mathit{\beta }\right)\right)$, can be computationally ineffective since it requires $m_i$ latent variables for each of the $N$ observations.

The success of the Albert and Chib (1993) algorithm is no doubt the reason for the popularity of the probit link in Bayesian statistics. However, the logit link is usually preferred by domain scientists due to easier interpretation of the regression parameters. Unfortunately, the straightforward extension of the algorithm i.e. introducing $N$ latent logistically distributed variables does not result in closed form full conditional distributions. Various authors have proposed other sampling schemes for the Logistic regression model. For example, Held and Holmes (2006) re-wrote the logistic latent variables in terms of a Kolmogorov-Smirnov normal mixture. This approach introduces $2N$ latent variables and the full conditional distributions are multivariate normal (for $\mathit{\beta }$) and truncated normals for $N$ latent variables, but rejection sampling is required for the other $N$ latent variables. Extending this approach to category count data requires $2m_i$ latent variables for each observation. The Held and Holmes (2006) approach gives exact posterior inference but can be computationally slow. In a series of papers Frühwirth-Schnatter and Frühwirth (2007); Frühwirth-Schnatter et al. (2009); Frühwirth-Schnatter and Frühwirth (2010) the logistic regression model is replaced by an approximation that facilitates efficient sampling. This approach introduces $2N$ latent variables that have to be sampled as well as choosing the number of normal mixtures used for approximations. Fussl et al. (2013) extend this approach to category count data.

Polson et al. (2013) introduced a data augmentation scheme for the logistic regression model that involves only $N$ latent variables and does not require model approximations. The latent variables are Pólya-Gamma distributed and with normal priors on regression parameters the full conditionals are multivariate normal and Póly-Gamma. The number of latent variables is the same for category counts as binary data, instead the total counts $\left(m_i\right)$ are parameters in the latent distributions. This data augmentation scheme has been shown to be very effective. In fact, Choi and Hobert (2013) showed that the algorithm is uniformly ergodic when the (normal) prior on the regression parameters is proper and Wang and Roy (2018) proved geometric ergodicity for a flat prior. We describe the algorithm of Polson et al. (2013) in more detail in Section 2.

Bayesian regression models for total count data usually don’t offer convenient and conjugate priors for regression coefficients. For Poisson regression models, some work has been done on developing a multivariate Log-Gamma distribution that is a conjugate prior in that setting (Demirhan and Hamurkaroglu, 2011; Bradley et al., 2018). This prior has many hyper-parameters and is not straightforward to use. However, Pillow and Scott (2012) and Zhou et al. (2012) show how the Pólya-Gamma data augmentation scheme from Polson et al. (2013) can be applied to the Negative-Binomial regression model with logit link and normal priors on regression coefficients and that is the approach we will use here.

In this paper we focus on a particular hierarchical data model of the form

$$\begin{gathered} M_i~\text{NB}\left(r,q_i\right),i=1,\dots ,N \\ Y\mid M_i=m_i~\text{Binomial}\left(m_i,p_i\left(r,q_i\right)\right), \end{gathered}$$ (2)

where $\text{NB}\left(r,q_i\right)$ denotes the Negative Binomial Distribution and the success probability $p_i$ is dependent on $r$ and/or $q_i$ in some way. We develop a double Pólya-Gamma data augmentation scheme for posterior sampling of parameters in the case of logit links and normal priors on regression parameters. Neelon (2019) also apply the Pólya-Gamma data augmentation scheme twice for one model that includes both a Bernoulli regression part and a Negative-Binomial regression part but for a zero-inflated Negative Binomial regression model. The hierarchical data model in Eq. (2) presents new challenges, particularly due to the interdependence of parameters in the two layers. Other data augmentation approaches to zero inflated total count data can be found in Neelon and Chung (2017) and Zhang and Yi (2023).

The remainder of this paper is organized as follows. In Section 2 we describe the data augmentation scheme of Polson et al. (2013) for both Binomial and Negative-Binomial regression models. In Section 3 we extend that approach to a Negative-Binomial - Binomial hierarchical data model, first with linear regression terms only and then a model that includes spatial effects. We illustrate the efficiency of the scheme with simulation studies in Section 4. Two applications of the hierarchical model and the double Pólya-Gamma data augmentation scheme to human mitochondrial DNA data are presented in Section 5 and conclusions are in Section 6.

2. Data augmentation for Binomial and Negative Binomial Logit models

In this section we summarize the Pólya-Gamma data augmentation scheme for Binomial and Negative-Binomial Logit models described in Polson et al. (2013) and Pillow and Scott (2012).

2.1. Data augmentation for posterior sampling

Data augmentation is a technique that can be used to facilitate a Gibbs sampler for a Bayesian model with non-conjugate priors. Suppose we have a data model $p(\mathbf{y}\mid \mathit{\beta })$, a prior $p(\mathit{\beta })$, and we want samples from the posterior. A data augmentation approach introduces a latent data variable $\mathit{\omega }$ and a conditional distribution $p(\mathit{\omega }\mid \mathit{\beta },\mathbf{y})$, augmenting the data model to the joint pdf $p(\mathbf{y},\mathit{\omega }\mid \mathit{\beta })=p(\mathbf{y}\mid \mathit{\beta })p(\mathit{\omega }\mid \mathit{\beta },\mathbf{y})$. It is easy to see that the marginal data model is still the same as the original model and the desired posterior $p(\mathit{\beta }\mid \mathbf{y})$ is the marginal of the joint posterior $p(\mathit{\beta },\mathit{\omega }\mid \mathbf{y})\propto p(\mathbf{y}\mid \mathit{\beta })p(\mathit{\omega }\mid \mathit{\beta },\mathbf{y})p(\mathit{\beta })$. If the full conditional distributions $p(\mathit{\beta }\mid \mathit{\omega },\mathbf{y})$ and $p(\mathit{\omega }\mid \mathit{\beta },\mathbf{y})$ are tractable, a Gibbs sampler can be applied to sample $\mathit{\beta }$ and $\mathit{\omega }$, of which only the $\mathit{\beta }$ samples are of interest for inference. The challenge of course is to choose the augmentation distribution $p(\mathit{\omega }\mid \mathit{\beta },\mathbf{y})$ in such a way that the full conditional distributions are easily sampled.

A well known example of this is the aforementioned Albert and Chib (1993) algorithm for Bayesian binary probit models $Y_i~\text{Bernoulli}\left({\Phi }^{-1}\left({\mathbf{x}}_i^{\prime }\mathit{\beta }\right)\right)$. The traditional presentation of Albert and Chib (1993) data augmentation splits the joint (augmented) data model the other way, i.e. $p(\mathbf{y},\mathit{\omega }\mid \mathit{\beta })=p(\mathbf{y}\mid \mathit{\omega },\mathit{\beta })p(\mathit{\omega }\mid \mathit{\beta })$ where $p(\mathit{\omega }\mid \mathit{\beta })$ is multivariate normal and $p(\mathbf{y}\mid \mathit{\omega },\mathit{\beta })=\prod _{i=1}^N\left(I\left\{y_i=1,{\omega }_i>0\right\}+I\left\{y_i=0,{\omega }_i\le 0\right\}\right)$ where $I\{\cdot \}$ is the indicator function. This is equivalent to introducing the latent data variables ${\omega }_i\mid \mathit{\beta },y_i~{\text{TrN}}_{y_i}\left({\mathbf{x}}_i^{\prime }\mathit{\beta },1\right)$ where ${\text{TrN}}_{y_i}(\cdot ,\cdot )$ denotes a normal distribution that is truncated above at zero if $y_i=0$ and below at zero if $y_i=1$.

2.2. Pólya-Gamma distribution

The Pólya-Gamma family of distributions, denoted $\text{PG}(b,c)$, is a subset of the class of infinite convolutions of gamma distributions (Barndorff-Nielsen et al., 1982). Polson et al. (2013) define $\text{PG}(b,0)$ with $b>0$ as the distribution of the random variable $\omega$ that has Laplace transform

$$E[\text{exp}(-\omega t)]={\text{cosh}}^{-b}\left(\sqrt{\frac{t}{2}}\right)=\prod _{k=1}^{\infty }{\left(1+\frac{t}{2{\pi }^2(k-1/2{)}^2}\right)}^{-b},$$ (3)

where the second equality follows from the Weierstrass factorization theorem. Inverting the Laplace transform we see that if $\omega ~\text{PG}(b,0)$ it is equal in distribution to an infinite convolution of gamma distributed random variables:

$$\omega \stackrel{D}{=}\frac{1}{2{\pi }^2}\sum _{k=1}^{\infty }\frac{g_k}{(k-1/2{)}^2}\text{where}{g}_k\stackrel{\text{ind}}{\sim }\text{Gamma}\left(b,1\right).$$

The general case of Pólya-Gamma distribution can be obtained through an exponential tilting of the $\text{PG}(b,0)$ distribution as follows

$$p(\omega \mid b,c)=\frac{\text{exp}\left(-\frac{1}{2}\omega c^2\right)p(\omega \mid b,0)}{{\int }_0^{\infty }\text{exp}\left(-\frac{1}{2}\omega c^2\right)p(\omega \mid b,0)d\omega },$$ (4)

where $p(\omega \mid b,0)$ is the pdf of the $\text{PG}(b,0)$ distribution.

Polson et al. (2013) describe a method to sample from the Pólya-Gamma distribution using the accept-reject algorithm, extending the work of Devroye (2009). The algorithm exploits the additive property of Pólya-Gamma distributions by drawing $b$ samples from $\text{PG}(1,c)$ and adding them together to obtain a sample from $\text{PG}(b,c)$. Each sample from $\text{PG}(1,c)$ is obtained via an accept-reject step using a proposal distribution that is a mixture of an inverse-Gaussian and an exponential density. Recognizing that the target density can be written as an infinite but alternating sum of decreasing terms the accept-reject step can be executed very efficiently. This method is implemented in the R package BayesLogit (Polson et al., 2019).

Polson et al. (2013) also show the following identity that relates the kernel of a Binomial data model with logit link to the expectation of a $\text{PG}(b,0)$ random variable and is instrumental in making the data augmentation scheme work:

Theorem 2.1. For $b>0,a\in \mathbb{R}$, and $\psi \in \mathbb{R}$ we have

$$\frac{{\left(e^{\psi }\right)}^a}{{\left(1+e^{\psi }\right)}^b}=2^{-b}{e}^{\kappa \psi }\underset{0}{\overset{\infty }{\int }}{e}^{-\omega {\psi }^2/2}p(\omega \mid b,0)d\omega ,$$ (5)

where $\kappa =a-b/2$ and $p(\omega \mid b,0)$ is the pdf of $\text{PG}(b,0)$.

2.3. Pólya-Gamma data augmentation for Binomial and Negative Binomial logit models

Consider a Binomial regression model with a logit link,

$$Y_i\stackrel{\text{ind}}{\sim }\text{Binomial}\left(n_i,p_i\right),\text{where logit}\left(p_i\right)={\mathbf{x}}_i^{\prime }\mathit{\beta },$$

or a Negative-Binomial regression model with logit link

$$Y_i\stackrel{\text{ind}}{\sim }\text{NB}\left(r,p_i\right),\text{where logit}\left(p_i\right)={\mathbf{x}}_i^{\prime }\mathit{\beta },$$

for $i=1,\dots ,N$. Here we parameterize the Negative-Binomial so that the pdf is proportional to $p_i^{y_i}{\left(1-p_i\right)}^r$ so in both cases the likelihood is proportional to

$$\prod _{i=1}^N\frac{{\left(e^{{\psi }_i}\right)}^{a_i}}{{\left(1+e^{{\psi }_i}\right)}^{b_i}},$$ (6)

where ${\psi }_i={\mathbf{x}}_i^{\prime }\mathit{\beta }$ and $a_i=y_i$. For the Binomial model we have $b_i=n_i$ but $b_i=r+y_i$ for the Negative-Binomial model. For now we assume that $r$ and $n_i$ are fixed.

The data augmentation scheme of Polson et al. (2013) introduces latent variables

$${\omega }_i\mid \mathit{\beta },y_i\stackrel{\text{ind}}{\sim }\text{PG}\left(b_i,{\mathbf{x}}_i^{\prime }\mathit{\beta }\right)i=1,\dots ,N,$$ (7)

where again, $b_i=n_i$ or $b_i=r+y_i$. As in the general case (see Section 2.1) the full conditional distribution of ${\omega }_i$ is simply the data augmentation distribution in (7). Using Theorem 2.1 and the pdf of $\text{PG}\left(b_i,{\mathbf{x}}_i^{\prime }\mathit{\beta }\right)$ from (4) the full conditional distribution of $\mathit{\beta }$ becomes

$$p(\mathit{\beta }\mid \mathit{\omega },\mathbf{y})\propto p(\mathbf{y}\mid \mathit{\beta })\times p(\mathit{\omega }\mid \mathit{\beta },\mathbf{y})\times p(\mathit{\beta })\propto \prod _{i=1}^N{e}^{{\kappa }_i{\mathbf{x}}_i^{\prime }\mathit{\beta }}\underset{0}{\overset{\infty }{\int }}{e}^{-\frac{1}{2}\omega {\left({\mathbf{x}}_i^{\prime }\mathit{\beta }\right)}^2}p\left(\omega \mid b_i,0\right)d\omega \times \prod _{i=1}^N\frac{e^{-\frac{1}{2}{\omega }_i{\left({\mathbf{x}}_i^{\prime }\mathit{\beta }\right)}^2}p\left({\omega }_i\mid b_i,0\right)}{{\int }_0^{\infty }{e}^{-\frac{1}{2}\omega {\left({\mathbf{x}}_i^{\prime }\mathit{\beta }\right)}^2}p\left(\omega \mid b_i,0\right)d\omega }\times p(\mathit{\beta }),$$ (8)

where $p\left(\omega \mid b_i,0\right)$ is the pdf of $\text{PG}\left(b_i,0\right)$. Since the integrals cancel we are left with

$$p(\mathit{\beta }\mid \mathit{\omega },\mathbf{y})\propto p(\mathit{\beta })\prod _{i=1}^N\text{exp}\left({\kappa }_i{\mathbf{x}}_i^{\prime }\mathit{\beta }-\frac{1}{2}{\omega }_i{\left({\mathbf{x}}_i^{\prime }\mathit{\beta }\right)}^2\right)\propto p(\mathit{\beta })\text{exp}\left(-\frac{1}{2}(\mathbf{X}\mathit{\beta }-\mathbf{Z}{)}^{\prime }𝛀(\mathbf{X}\mathit{\beta }-\mathbf{Z})\right),$$ (9)

where $𝛀$ is a diagonal matrix with diagonal values ${\omega }_1,\dots ,{\omega }_N$,

$$\mathit{\kappa }={\left(a_1-\frac{b_1}{2},\cdots ,a_N-\frac{b_N}{2}\right)}^{\prime },\mathbf{Z}={\left(\frac{{\kappa }_1}{{\omega }_1},\cdots ,\frac{{\kappa }_N}{{\omega }_N}\right)}^{\prime },\text{and}\mathbf{X}={\left({\mathbf{x}}_1^{\prime },\cdots ,{\mathbf{x}}_n^{\prime }\right)}^{\prime }.$$

The second part of (9) is proportional to the likelihood for linear regression $\mathbf{Z}=\mathbf{X}\mathit{\beta }+\mathit{ϵ}$, where $\mathit{ϵ}~N\left(0,{𝛀}^{-1}\right)$ and if the prior for $\mathit{\beta }$ is set as $p(\mathit{\beta })~N(\mathbf{b},\mathbf{B})$, the Gibbs sampler is summarized as follows (Polson et al., 2013; Pillow and Scott, 2012):

$${\omega }_i\mid \mathit{\beta },\mathbf{y}~\text{PG}\left(b_i,{\mathbf{x}}_i^{\prime }\mathit{\beta }\right)i=1,\dots ,N$$ (10)

$$\mathit{\beta }\mid \mathbf{y},\mathit{\omega }~N\left({\mathit{\mu }}_{\omega },{\mathbf{V}}_{\omega }\right),$$ (11)

where $b_i=n_i$ (Binomial model) or $b_i=r+y_i$ (Negative Binomial model), and

$${\mathbf{V}}_{\omega }={\left({\mathbf{X}}^{\prime }𝛀\mathbf{X}+{\mathbf{B}}^{-1}\right)}^{-1}\text{and}{\mathit{\mu }}_{\omega }={\mathbf{V}}_{\omega }\left({\mathbf{B}}^{-1}\mathbf{b}+\mathbf{X}\mathit{\kappa }\right).$$ (12)

3. Bayesian Negative-Binomial – Binomial (NBB) hierarchical data model

We now discuss a Negative-Binomial – Binomial (NBB) hierarchical data model inspired by analysis of human mitochondrial DNA (mtDNA) data (Section 3.1). We then expand the Pólya-Gamma data augmentation scheme to this NBB model, first with regression terms only (Section 3.2) and then adding spatial effects (Section 3.3).

3.1. NBB hierarchical model

We now define the general NBB regression model. Let $M_i$ be a total count for subject $i$ and let $Y_i$ be the corresponding category count for $i=1,\dots ,N$. In our application, $M_i$ is the total number of mtDNA chromosomes per cell (copy number) and $Y_i$ is the count of how many of those mtDNA copies have a mutation at a given position. Also, let ${\mathbf{x}}_{1i}$ and ${\mathbf{x}}_{2i}$ be vectors of covariates for the total counts and category counts, respectively. We propose the following logistic NBB model

$$M_i\mid r,\mathit{\alpha }~\text{NB}\left(r,q_i\right)i=1,\dots ,N$$ (13)

$$\text{where logit}\left(q_i\right)={\mathbf{x}}_{1i}^{\prime }\mathit{\alpha },$$ (14)

$$Y_i\mid M_i=m_i,\mathit{\beta },\mathit{\alpha }~\text{Binomial}\left(m_i,p_i\right)$$ (15)

$$\text{where logit}\left(p_i\right)={\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1+\eta \left(q_i,r\right)\delta \equiv {\mathbf{x}}_{3i}^{\prime }\mathit{\beta },$$ (16)

$$\mathit{\alpha }~N(\mathbf{a},\mathbf{A}),\mathit{\beta }~N(\mathbf{b},\mathbf{B}),r~p(r),$$ (17)

where $p(r)$ is an unspecified prior distribution with support on ${\mathbb{R}}_{+},{\mathbf{x}}_{3i}={\left[{\mathbf{x}}_{2i}^{\prime },\eta \left(q_i,r\right)\right]}^{\prime }$, and $\mathit{\beta }={\left[{\mathit{\beta }}_1^{\prime },\delta \right]}^{\prime }$.

In this setup the dependence of the category probability $p_i$ on the mean total count $\frac{r{q}_i}{1-q_i}$ is modelled as a linear additive effect on the logit scale, i.e. $\eta \left(q_i,r\right)\delta$, where $\eta$ is some function of $q_i$ and $r$. This keeps the linear regression model form for $\text{logit}\left(p_i\right)$ and the Pólya-Gamma data augmentation scheme can be applied directly to sample $\mathit{\beta }$. Posterior sampling of $\mathit{\alpha }$ is a bit trickier since $\eta \left(q_i,r\right)$ depends on $\mathit{\alpha }$. As will be shown in Section 3.2 the Pólya-Gamma data augmentation scheme can also be used for $\mathit{\alpha }$ if $\eta \left(q_i,r\right)$ is a linear function of $\mathit{\alpha }$. In that case the $\delta$ parameter can be interpreted in terms of multiples of the expected total count $E\left(M_i\right)=r\text{exp}\left({\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)$. In particular, if $E\left(M_i\right)$ is multiplied by a constant $c$ the category odds are multiplied by $c^{\delta }$ since

$$\frac{p_i}{1-p_i}=\text{exp}\left({\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1+\delta {\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)=\text{exp}\left({\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1\right){\left(\frac{E\left({\mathit{M}}_i\right)}{r}\right)}^{\delta }.$$ (18)

Therefore, if $\delta$ is positive the category odds increase with increasing total counts, a negative $\delta$ indicates decreasing odds with increasing total counts, and $\delta =0$ indicates no relationship.

3.1.1. Some modeling considerations

While the Binomial distribution is a natural choice for categorical counts there are of course other options for modeling total counts, including the Poisson distribution. We argue that the Negative-Binomial distribution is flexible enough for most situations, has the benefit of accounting for over dispersion, and has the added bonus of lending itself to an efficient Pólya-Gamma data augmentation scheme.

The joint modeling of total counts and category counts provides a mechanism to directly estimate the association between them (represented here as the $\delta$ parameter) and it allows for borrowing of strength between model layers. However, if one of the model layers is severely misspecified it does not only affect the estimation of parameters in that layer but both. For example, if the model for categorical counts $\left(Y_i\right)$ is misspecified it can affect the estimate of the regression coefficients $(\mathit{\alpha })$ in the model for total counts in unpredictable ways. One way to counter this is through modularization. For example, one could cut this feedback by letting the marginal posterior of $(\mathit{\alpha })$ only depend on the total count data, or simply fit the model for total counts separately and plug-in an estimate for $\mathit{\alpha }$ to fit the model for category counts. Either way, if different modularization gives very different results than fitting the full model that could help identify flaws in the model. For more detailed discussions of modularization and cutting feedback see for example Jacob et al. (2017), Bayarri et al. (2009) and Plummer (2014).

Another thing to keep in mind when employing the NBB hierarchical model is the potential for non-identifiability of some parameters. Note for example, that if there are no covariates for the total count data, i.e. if (14) is simply $\text{logit}\left(q_i\right)={\alpha }_0$, then an intercept in (16) and $\delta$ are not identifiable since (16) becomes $\text{logit}\left(p_i\right)={\beta }_0+\sum _j{\beta }_j{x}_{2,ji}+{\alpha }_0\delta$. Furthermore, if the covariates are all categorical and same covariates are used for both data levels, then the intercept ${\beta }_0$ and $\delta$ are not identifiable. To see this, suppose we have a categorical covariate with 3 levels and 2 observations per level. Then, using the typical indicator variables, the design matrix for $\text{logit}\left(p_i\right)$ is

$$X_3=\left[\begin{array}{cccc}1& 0& 0& {\alpha }_0\\ 1& 0& 0& {\alpha }_0\\ 1& 1& 0& {\alpha }_0+{\alpha }_1\\ 1& 1& 0& {\alpha }_0+{\alpha }_1\\ 1& 0& 1& {\alpha }_0+{\alpha }_2\\ 1& 0& 1& {\alpha }_0+{\alpha }_2\end{array}\right]\text{and}\mathit{\beta }=\left[\begin{array}{c}{\beta }_0\\ {\beta }_1\\ {\beta }_2\\ \delta \end{array}\right],$$ (19)

and denoting the column vectors of $X_3$ as ${\mathbf{X}}_j$ we see that ${\mathbf{X}}_4={\alpha }_0{\mathbf{X}}_1+{\alpha }_1{\mathbf{X}}_2+{\alpha }_2{\mathbf{X}}_3$.

3.2. Data augmentation scheme for the NBB model

We now present a double Pólya-Gamma data augmentation scheme for the NBB model defined in (13) – (17). We define two latent random vectors ${\mathit{\omega }}_1$ and ${\mathit{\omega }}_2$ where

$${\omega }_{1i}\mid \mathit{\alpha },r,m_i\stackrel{\text{ind}}{\sim }\text{PG}\left(r+m_i,{\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)i=1,\dots ,N$$ (20)

$$\text{and}{\omega }_{2i}\mid \mathit{\beta },\mathit{\alpha },r,m_i\stackrel{\text{ind}}{\sim }\text{PG}\left(m_i,{\mathbf{x}}_{3i}^{\prime }\mathit{\beta }\right)i=1,\dots ,N.$$ (21)

Note that ${\omega }_{2i}$ may depend on $\mathit{\alpha }$ and $r$ since ${\mathbf{x}}_{3i}$ is a yet to be identified function of $\mathit{\alpha }$ and/or $r$. The joint posterior distribution is

$$p\left(\mathit{\beta },\mathit{\alpha },r,{\mathit{\omega }}_1,{\mathit{\omega }}_2\mid \mathbf{y},\mathbf{m}\right)\propto p(\mathbf{m}\mid \mathit{\alpha },r)p\left({\mathit{\omega }}_1\mid \mathit{\alpha },r,\mathbf{m}\right)p(\mathit{\alpha })p(r)\times p(\mathbf{y}\mid \mathbf{m},\mathit{\beta },\mathit{\alpha },r)p\left({\mathit{\omega }}_2\mid \mathit{\beta },\mathit{\alpha },r,\mathbf{m}\right)p(\mathit{\beta }).$$ (22)

As in Section 2.3 the full conditional distributions of ${\mathit{\omega }}_1$ and ${\mathit{\omega }}_2$ are given in (20) and (21). Also as in Section 2.3, the full conditional distribution of $\mathit{\beta }$ is multivariate normal with mean and covariance matrix defined in (12) with $a_i=y_i$ and $b_i=m_i$.

The full conditional distribution of $\mathit{\alpha }$ is

$$p(\mathit{\alpha }\mid \cdot )\propto p(\mathbf{m}\mid \mathit{\alpha },r)p\left({\mathit{\omega }}_1\mid \mathit{\alpha },r,\mathbf{m}\right)p(\mathit{\alpha })p(\mathbf{y}\mid \mathbf{m},\mathit{\beta },\mathit{\alpha },r)p\left({\mathit{\omega }}_2\mid \mathit{\beta },\mathit{\alpha },r,\mathbf{m}\right)\propto p(\mathit{\alpha })\prod _{i=1}^N\text{exp}\left(\frac{m_i-r}{2}{\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)\text{exp}\left(-\frac{{\omega }_{1i}}{2}{\left({\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)}^2\right)\times \prod _{i=1}^N\text{exp}\left(\left(y_i-\frac{m_i}{2}\right)\left({\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1+\eta \left(q_i,r\right)\delta \right)\right)\text{exp}\left(-\frac{{\omega }_{2i}}{2}{\left({\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1+\eta \left(q_i,r\right)\delta \right)}^2\right),$$ (23)

where similar to the derivation in Section 2.3 we used Theorem 2.1 to write the likelihoods $p(\mathbf{m}\mid \mathit{\alpha },r)$ and $p(\mathbf{y}\mid \mathbf{m},\mathit{\beta },\mathit{\alpha },r)$ in the form of (5) and the integrals cancel against the integrals in the Pólya-Gamma densities $p\left({\mathit{\omega }}_1\mid \mathit{\alpha },r,\mathbf{m}\right)$ and $p\left({\mathit{\omega }}_2\mid \mathit{\beta },\mathit{\alpha },r,\mathbf{m}\right)$. We are left with linear and quadratic terms involving $\mathit{\alpha }$, i.e. an expression proportional to a normal density, but recall that $\eta \left(q_i,r\right)$ also depends on $\mathit{\alpha }$. If we set $\eta \left(q_i,r\right)=\text{logit}\left(q_i\right)={\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }$ we get

$$p(\mathit{\alpha }\mid \cdot )\propto p(\mathit{\alpha })\prod _{i=1}^N\text{exp}\left\{\left(\frac{m_i-r}{2}+\left(y_i-\frac{m_i}{2}\right)\delta -{\omega }_{2i}{\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1\delta \right){\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }-\left(\frac{{\omega }_{1i}}{2}+\frac{{\omega }_{2i}}{2}{\delta }^2\right){\left({\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)}^2\right\}\propto p\left(\mathit{\alpha }\right)\text{exp}\left\{-\frac{1}{2}{\left({\mathbf{X}}_1\mathit{\alpha }-{\mathbf{Z}}_1\right)}^{\prime }{𝛀}_1\left({\mathbf{X}}_1\mathit{\alpha }-{\mathbf{Z}}_1\right)\right\},$$ (24)

where ${\mathbf{Z}}_1={\left(Z_{1i},\dots ,Z_{1N}\right)}^{\prime }$ with

$$Z_{1i}=\frac{\frac{m_i-r}{2}+\left(y_i-\frac{1}{2}{m}_i\right)\delta -{\omega }_{2i}{\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1\delta }{{\omega }_{1i}+{\omega }_{2i}{\delta }^2},$$

$$\begin{array}{cc}& {𝛀}_1=\left[\begin{array}{ccc}{\omega }_{11}+{\omega }_{21}{\delta }^2& & \\ & \ddots & \\ & & {\omega }_{1N}+{\omega }_{2N}{\delta }^2\end{array}\right]\text{and}{\mathbf{X}}_1=\left[\begin{array}{c}{\mathbf{x}}_{11}^{\prime }\\ ⋮\\ {\mathbf{x}}_{1N}^{\prime }\end{array}\right]\end{array}.$$ (25)

As before, we now have an expression of the form of a linear regression model and with the normal prior on $\mathit{\alpha }$ we get the usual normal-normal update.

To summarize, the Gibbs sampler for a fixed $r$ is as follows

$$\begin{gathered} {\omega }_{1i}\mid \cdot \stackrel{\text{ind}}{\sim }\text{PG}\left(r+m_i,{\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)i=1,\dots ,N, \\ \mathit{\alpha }\mid \cdot ~N\left({\mathit{\mu }}_{{\omega }_1},{\mathbf{V}}_{{\omega }_1}\right), \end{gathered}$$

$$\begin{gathered} {\omega }_{2i}\mid \cdot \stackrel{\text{ind}}{\sim }\text{PG}\left(m_i,{\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1+\left({\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)\delta \right)i=1,\dots ,N, \\ \mathit{\beta }\mid \cdot ~N\left({\mathit{\mu }}_{{\omega }_2},{\mathbf{V}}_{{\omega }_2}\right), \end{gathered}$$ (26)

where

$${\mathbf{V}}_{{\omega }_1}={\left({\mathbf{X}}_1^{\prime }{𝛀}_1{\mathbf{X}}_1+{\mathbf{A}}^{-1}\right)}^{-1},{\mathit{\mu }}_{{\omega }_1}={\mathbf{V}}_{{\omega }_1}\left({\mathbf{A}}^{-1}\mathbf{a}+{\mathbf{X}}_1^{\prime }{𝛀}_1{\mathbf{Z}}_1\right),$$

$${\mathbf{V}}_{{\omega }_2}={\left({\mathbf{X}}_2^{\prime }{𝛀}_2{\mathbf{X}}_2+{\mathbf{B}}^{-1}\right)}^{-1},{\mathit{\mu }}_{{\omega }_2}={\mathbf{V}}_{{\omega }_2}\left({\mathbf{B}}^{-1}\mathbf{b}+{\mathbf{X}}_2^{\prime }{𝛀}_2{\mathbf{Z}}_2\right).$$ (27)

The values of ${𝛀}_1,{\mathbf{Z}}_1$, and ${\mathbf{X}}_1$ are given in and (25), and

$${\mathbf{Z}}_2=\left[\begin{array}{c}\frac{y_1-m_1/2}{{\omega }_{21}}\\ ⋮\\ \frac{y_N-m_N/2}{{\omega }_{2N}}\end{array}\right],{𝛀}_2=\left[\begin{array}{lll}{\omega }_{21}& & \\ & \ddots & \\ & & {\omega }_{2N}\end{array}\right],\text{and}{\mathbf{X}}_2=\left[\begin{array}{cc}{\mathbf{x}}_{21}^{\prime }& {\mathbf{x}}_{11}^{\prime }\mathit{\alpha }\\ ⋮& \\ {\mathbf{x}}_{2N}^{\prime }& {\mathbf{x}}_{1N}^{\prime }\mathit{\alpha }\end{array}\right].$$ (28)

Finally, in the case that $r$ is not fixed, a Metropolis-Hastings step can be added where the full conditional distribution is $p(r\mid \cdot )\propto p(\mathbf{m}\mid \mathit{\alpha },r)p(r)$.

3.3. NBB model in spatial setting

The NBB model in Section 3.1 can be utilized for spatially correlated count data by adding random spatial effects to the logit-scale regression expressions in (14) and (16) in the same way we would do for a NB regression model and a Binomial regression model separately. Denoting the spatial effect for total counts and category counts with $v_{1i}$ and $v_{2i}$, respectively we set

$$\text{logit}\left(q_i\right)={\mathbf{x}}_{i1}^{\prime }\mathit{\alpha }+v_{1i}i=1,\dots ,N$$

$$\text{logit}\left(p_i\right)={\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1+\eta \left(q_i,r\right)\delta +v_{2i}={\mathbf{x}}_{3i}^{\prime }\mathit{\beta }+v_{2i}$$

$$\text{where}{\mathit{v}}_1={\left(v_{11},v_{12},\cdots ,v_{1N}\right)}^{\prime }~{\text{N}}_N\left(\mathbf{0},𝚺\left({\mathit{\theta }}_1\right)\right)$$

$$\text{and}{\mathit{v}}_2={\left(v_{21},v_{22},\cdots ,v_{2N}\right)}^{\prime }~{\text{N}}_N\left(\mathbf{0},𝚺\left({\mathit{\theta }}_2\right)\right).$$ (29)

Since the spatial effects have normal priors it follows that with the Pólya-Gamma data augmentation scheme the full conditional distribution of $v_1$ and $v_2$ are multivariate normal, in the same way as the regression coefficients. We present the details of the full conditional distributions below. The covariance matrices $𝚺\left({\mathit{\theta }}_1\right)$ and $𝚺\left({\mathit{\theta }}_2\right)$ are determined by the choice of spatial model and will typically depend on unknown parameters ${\mathit{\theta }}_1$ and ${\mathit{\theta }}_2$. In our simulations studies in Section 4.1 we consider two spatial models, one applicable to areal data and one for point referenced data. Firstly, a CAR model with $𝚺\left({\mathit{\theta }}_j\right)={\sigma }_j^2{\left(\mathbf{D}-{\rho }_j\mathbf{W}\right)}^{-1},j=1,2$, where $\mathbf{W}$ is an adjacency matrix and $\mathbf{D}$ is a diagonal matrix with the $i$th diagonal value equal to sum of the $i$th row in $\mathbf{W}$, i.e. the total number of neighbors for the $i$th location. In this case ${\sigma }_j^2>0$ are variance parameters and ${\rho }_j\in [-1,1]$ are correlation parameters. Secondly, a squared exponential covariance function, $𝚺\left({\mathit{\theta }}_j\right)={\sigma }_j^2\mathbf{R}\left({\psi }_j\right)$ where ${\mathbf{R}}_{m,n}\left({\psi }_j\right)=\text{exp}\left(-\frac{1}{{\psi }_j}{‖{\mathbf{s}}_m-{\mathbf{s}}_n‖}^2\right)$ where ${\mathbf{s}}_m$ and ${\mathbf{s}}_n$ are the spatial locations of two observations, ${\sigma }_j^2>0$ are variance parameters, and ${\psi }_j>0$ are correlation length parameters (see e.g. Banerjee et al., 2014).

Similar to before, we set $\eta \left(q_i,r\right)=\text{logit}\left(q_i\right)={\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }+v_{1i}$ which now includes both the regression part and the spatial effect. There are of course many other options, e.g. only include the regression part or allow different $\delta$ parameters for the fixed and random effects. As long as $\eta \left(q_i,r\right)$ is linear in $\mathit{\alpha }$ and $v_{1i}$ the data augmentation scheme works.

The Pólya-Gamma data augmentation scheme for the spatial NBB model starts with defining two latent random vectors ${\mathit{\omega }}_1$ and ${\mathit{\omega }}_2$ that now also include the random effects, i.e.

$${\omega }_{1i}\mid \mathit{\alpha },r,m_i,v_{1i}\stackrel{\text{ind}}{\sim }\text{PG}\left(r+m_i,{\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }+v_{1i}\right)i=1,\dots ,N$$

$$\text{and}{\omega }_{2i}\mid \mathit{\beta },\mathit{\alpha },m_i,v_{1i},v_{2i}\stackrel{\text{ind}}{\sim }\text{PG}\left(m_i,{\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1+\left({\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }+v_{1i}\right)\delta +v_{2i}\right)i=1,\dots ,N.$$ (30)

The joint posterior distribution is as follows

$$p\left(\mathit{\beta },\mathit{\alpha },r,{\mathit{v}}_1,\right.\left.{\mathit{v}}_2,{\mathit{\theta }}_1,{\mathit{\theta }}_2,{\mathit{\omega }}_1,{\mathit{\omega }}_2\mid \mathbf{y},\mathbf{m}\right)\propto p\left(\mathbf{m}\mid \mathit{\alpha },r,{\mathit{v}}_1\right)p\left({\mathit{\omega }}_1\mid \mathit{\alpha },r,\mathbf{m},{\mathit{v}}_1\right)p\left({\mathit{v}}_1\mid {\mathit{\theta }}_1\right)p(\mathit{\alpha })p\left({\mathit{\theta }}_1\right)p(r)\times p\left(\mathbf{y}\mid \mathbf{m},\mathit{\beta },\mathit{\alpha },{\mathit{v}}_1,{\mathit{v}}_2\right)p\left({\mathit{\omega }}_2\mid \mathbf{m},\mathit{\beta },\mathit{\alpha },{\mathit{v}}_1,{\mathit{v}}_2\right)p\left({\mathit{v}}_2\mid {\mathit{\theta }}_2\right)p\left(\mathit{\beta }\right)p\left({\mathit{\theta }}_2\right).$$ (31)

To increase the efficiency of the Gibbs sampler we sample the parameters $\mathit{\alpha }$ and ${\mathit{v}}_1$ jointly, while similarly, $\mathit{\beta }$ and ${\mathit{v}}_2$ can also be sampled jointly. The full conditional distribution for $\mathit{\alpha },{\mathit{v}}_1$ is

$$\begin{gathered} p(\alpha ,v_1\mid \cdot )\propto p(\alpha ,v_1)\prod _{i=1}^N\text{exp}(\frac{m_i-r}{2}[x_{1i}^{\prime }\alpha +v_{1i}])\text{exp}(-\frac{{\omega }_{1i}}{2}{[x_{1i}^{\prime }\alpha +v_{1i}]}^2)\times \prod _{i=1}^N\text{exp}((y_i-\frac{m_i}{2})[x_{2i}^{\prime }{\beta }_1+(x_{1i}^{\prime }\alpha +v_{1i})\delta +v_{2i}])\times \prod _{i=1}^N\text{exp}(-\frac{{\omega }_{2i}}{2}{[x_{2i}^{\prime }{\beta }_1+(x_{1i}^{\prime }\alpha +v_{1i})\delta +v_{2i}]}^2)\propto p(\alpha ,v_1)\prod _{i=1}^N\text{exp}(-\frac{1}{2}({\omega }_{1i}+{\omega }_{2i}{\delta }^2){(x_{1i}^{\prime }\alpha +v_{1i}-z_{1i})}^2) \\ =p(\alpha ,v_1)\text{exp}(-\frac{1}{2}{(X_1^{*}[\begin{array}{c}\alpha \\ v_1\end{array}]-Z_1)}^{\prime }{\Omega }_1(X_1^{*}[\begin{array}{c}\alpha \\ v_1\end{array}]-Z_1)), \end{gathered}$$ (32)

where ${𝛀}_1$ is a diagonal matrix with ${\omega }_{1i}+{\omega }_{2i}{\delta }^2$ on the diagonal, ${\mathbf{Z}}_1={\left(z_{11},\cdots ,z_{1N}\right)}^{\prime }$ where

$$z_{1i}=\frac{\frac{m_i-r}{2}+\left(y_i-\frac{m_i}{2}\right)\delta -{\omega }_{2i}{\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1\delta -{\omega }_{2i}\delta v_{2i}}{{\omega }_{1i}+{\omega }_{2i}{\delta }^2},$$ (33)

and the design matrix ${\mathbf{X}}_1^{*}=\left({\mathbf{X}}_1,{\mathbf{I}}_N\right)$ where ${\mathbf{X}}_1={\left({\mathbf{x}}_{11},\cdots ,{\mathbf{x}}_{1N}\right)}^{\prime }$ and $\mathbf{I}$ is the $N\times N$ dimensional identity matrix. Since the joint prior of $\mathit{\alpha }$ and ${\mathit{v}}_1$ can be written as $N\left({\mathbf{c}}_1,{\mathbf{C}}_1\right)$ where

$${\mathbf{c}}_1=\left[\begin{array}{l}\mathbf{a}\\ \mathbf{0}\end{array}\right]\text{and}{\mathbf{C}}_1=\left[\begin{array}{cc}\mathbf{A}& \mathbf{0}\\ \mathbf{0}& 𝚺\left({\mathit{\theta }}_1\right)\end{array}\right],$$ (34)

the joint full conditional is the usual normal-normal update, i.e.

$$\mathit{\alpha },{\mathit{v}}_1\mid \cdot ~N\left({\mathit{\mu }}_{{\mathit{\omega }}_1},{\mathit{V}}_{{\mathit{\omega }}_1}\right),$$ (35)

where ${\mathit{V}}_{{\mathit{\omega }}_1}={\left({\mathbf{X}}_1^{{*}^{\prime }}{𝛀}_1{\mathbf{X}}_1^{*}+{\mathbf{C}}_1^{-1}\right)}^{-1}$ and ${\mathit{\mu }}_{{\mathit{\omega }}_1}={\mathbf{V}}_{{\mathit{\omega }}_1}\left({\mathbf{C}}_1^{-1}{\mathbf{c}}_1+{\mathbf{X}}_1^{{*}^{\prime }}{𝛀}_1{\mathbf{Z}}_1\right)$.

Similarly, the full conditional distribution for $\mathit{\beta }$ and ${\mathit{v}}_2$ is

$$p\left(\mathit{\beta },{\mathit{v}}_2\mid \cdot \right)\propto p\left(\mathit{\beta },{\mathit{v}}_2\right)\prod _{i=1}^N\text{exp}\left(-\frac{{\omega }_{2i}}{2}{\left({\mathbf{x}}_{3i}^{\prime }\mathit{\beta }+v_{2i}-\frac{y_i-m_i/2}{{\omega }_{2i}}\right)}^2\right)\propto p\left(\mathit{\beta },{\mathit{v}}_2\right)\text{exp}\left(-\frac{1}{2}{\left({\mathbf{X}}_2^{*}\left[\begin{array}{c}\mathit{\beta }\\ {\mathit{v}}_2\end{array}\right]-{\mathbf{Z}}_2\right)}^{\prime }{\mathbf{\Omega }}_2\left({\mathbf{X}}_2^{*}\left[\begin{array}{c}\mathit{\beta }\\ {\mathit{v}}_2\end{array}\right]-{\mathbf{Z}}_2\right)\right),$$ (36)

where ${𝛀}_2$ is a diagonal matrix with diagonal terms ${\omega }_{2i},{\mathbf{X}}_2^{*}=\left({\mathbf{X}}_2,\mathbf{I}\right)$,

$${\mathbf{X}}_2=\left[\begin{array}{cc}{\mathbf{x}}_{21}^{\prime }& {\mathbf{x}}_{11}^{\prime }\mathit{\alpha }+v_{11}\\ {\mathbf{x}}_{22}^{\prime }& {\mathbf{x}}_{12}^{\prime }\mathit{\alpha }+v_{12}\\ ⋮& \\ {\mathbf{x}}_{2N}^{\prime }& {\mathbf{x}}_{1N}^{\prime }\mathit{\alpha }+v_{1N}\end{array}\right]\text{and}{\mathbf{Z}}_2=\left[\begin{array}{c}\frac{y_1-m_1/2}{{\omega }_{21}}\\ \frac{y_2-m_2/2}{{\omega }_{22}}\\ ⋮\\ \frac{y_N-m_N/2}{{\omega }_{2N}}\end{array}\right].$$ (37)

So the full conditional distribution of $\mathit{\beta }$ and ${\mathit{v}}_2$ follows:

$$\mathit{\beta },{\mathit{v}}_2\mid \cdot ~N\left({\mathit{\mu }}_{{\mathit{\omega }}_2},{\mathbf{V}}_{{\mathit{\omega }}_2}\right),$$ (38)

where ${\mathbf{V}}_{{\mathit{\omega }}_2}={\left({\mathbf{X}}_2^{{*}^{\prime }}{𝛀}_2{\mathbf{X}}_2^{*}+{\mathbf{C}}_2^{-1}\right)}^{-1},{\mathit{\mu }}_{{\mathit{\omega }}_2}={\mathbf{V}}_{{\mathit{\omega }}_2}\left({\mathbf{C}}_2^{-1}{\mathbf{c}}_2+{\mathbf{X}}_2^{*\prime }{𝛀}_2{\mathbf{Z}}_2\right)$ and

$${\mathbf{c}}_2=\left[\begin{array}{l}\mathbf{b}\\ \mathbf{0}\end{array}\right]{\mathbf{C}}_2=\left[\begin{array}{cc}\mathbf{B}& \mathbf{0}\\ \mathbf{0}& 𝚺\left({\mathit{\theta }}_2\right)\end{array}\right].$$ (39)

The full conditional distribution of ${\mathit{\theta }}_1$ and ${\mathit{\theta }}_2$ depends on the type of spatial model used. Here we assume that for both levels of the data model hierarchy we have either ${\mathit{\theta }}_j=\left({\mathit{\sigma }}_j^2,{\mathit{\rho }}_j\right)$ and $𝚺\left({\mathit{\theta }}_j\right)={\sigma }_j^2\mathbf{R}\left({\rho }_j\right)={\sigma }_j^2{\left(\mathbf{D}-{\rho }_j\mathbf{W}\right)}^{-1}$ for $j=1,2$, or ${\mathit{\theta }}_j=\left({\sigma }_j^2,{\psi }_j\right)$ and $𝚺\left({\mathit{\theta }}_j\right)={\sigma }_j^2\mathbf{R}\left({\psi }_j\right)$ with ${\mathbf{R}}_{m,n}\left({\psi }_j\right)=\text{exp}\left(-\frac{1}{{\psi }_j}{‖{\mathbf{s}}_m-{\mathbf{s}}_n‖}^2\right)$. For both spatial models we can assign a conjugate prior on ${\sigma }_j^2$, i.e. ${\sigma }_j^2~\text{InverseGamma}\left({\gamma }_j,{\xi }_j\right)$ resulting in the full conditional distributions

$${\sigma }_j^2\mid \cdot ~\text{InverseGamma}\left({\gamma }_j+\frac{N}{2},{\xi }_j+\frac{1}{2}{\mathit{v}}_j^{\prime }\mathbf{R}{\left({\rho }_j\right)}^{-1}{\mathit{v}}_j\right),j=1,2$$ (40)

for the CAR model. The full conditional distribution for the squared exponential model is the same except that $\mathbf{R}\left({\rho }_j\right)$ is replaced by $\mathbf{R}\left({\psi }_j\right)$. For both spatial models the other parameters, ${\rho }_j$ or ${\psi }_j$, require a Metropolis-Hastings step where for the CAR model we have

$$p\left({\rho }_j\mid \cdot \right)\propto p\left({\rho }_j\right)\frac{1}{{\left|{\sigma }_j^2\mathbf{R}\left({\rho }_j\right)\right|}^{\frac{1}{2}}}\text{exp}\left(-\frac{1}{2{\sigma }_j^2}{\mathit{v}}_j^{\prime }\mathbf{R}{\left({\rho }_j\right)}^{-1}{\mathit{v}}_j\right),$$ (41)

and $p\left({\rho }_j\right)$ is the chosen prior for ${\rho }_j$. The squared exponential model is handled similarly.

4. Simulation studies

To illustrate the performance of the proposed Gibbs sampler with the data augmentation scheme we conducted extensive simulation studies. We generated synthetic data from the NBB model both with and without a spatial effect and used the Gibbs sampler described in Section 3.2 and Section 3.3 to obtain samples from the posterior. For comparison we also fitted the same models using the Stan program (Stan Development Team, 2023b) which utilizes Hamiltonian Monte Carlo methods to obtain posterior samples. The Gibbs sampler was programmed in R (R Core Team, 2023) and we used the rstan package (Stan Development Team, 2023a) for Stan. Example codes are available on GitHub https://github.com/xxm115/NBB_sim. While other MCMC approaches and programs can also be used (e.g. Metropolis-Hastings algorithm, Jags, NIMBLE, etc.) Stan is currently the go-to software for general-purpose Bayesian inference (see for example Štrumbelj et al., 2024) and is computationally very efficient (see e.g. Beraha et al., 2021). Integrated nested Laplace approximation (INLA; Rue et al., 2009) would also be a natural competitor in general, but here the focus is on sampling methods and Stan gives a more direct comparison.

For comparison of the efficiency of the two samplers, we calculated the effective sample size (ESS) and the effective sampling rate (ESR) for the samples from both methods. The ESS provides an estimate of how many independent samples would contain the same information as the original correlated chain with the total number of iterations. It is calculated as $\text{ESS}=M{\lambda }^2/{\sigma }^2$ where $M$ is the sample size, ${\lambda }^2$ is the sample variance, and ${\sigma }^2$ is the estimate of the spectral density at frequency zero. We used the coda package in R (Plummer et al., 2015) to calculate the ESS for our output (after burn-in) and ESR was calculated as the ratio of the effective sample size to the total time (including burn-in) taken to generate the whole MCMC chain, i.e. $\text{ESR}=\text{ESS}/T$ where $T$ is the total run time in seconds.

4.1. Simulation study for NBB regression model

We generated 20 data sets from the following NBB model:

$$M_i\mid \mathit{\alpha },r~\text{NB}\left(r,q_i\right)i=1,\dots ,100$$

$$\text{where logit}\left(q_i\right)={\alpha }_0+{\alpha }_1{x}_{11,i}+{\alpha }_2{x}_{12,i},$$

$$Y_i\mid M_i=m_i,\mathit{\beta },\delta ,\mathit{\alpha }~\text{Binomial}\left(m_i,p_i\right)$$

$$\text{where logit}\left(p_i\right)={\beta }_0+{\beta }_1{x}_{21,i}+{\beta }_2{x}_{22,i}+\delta \left({\alpha }_0+{\alpha }_1{x}_{11,i}+{\alpha }_2{x}_{12,i}\right),$$ (42)

where the covariates $x_{11},x_{12},x_{21}$, and $x_{22}$ were generated from a normal distribution. The $r$ parameter was set to 100 and the values chosen for other parameters are listed in Table 1. We found that the efficiency of both Gibbs and Stan varied depending on the specific parameter values chosen in Eq. (42). Therefore we devised eight scenarios, tweaking the values of $\mathit{\alpha },\mathit{\beta }$, and $\delta$ as well as the distribution used to generate covariates, to obtain all combinations of low and high values of three data characteristics. Firstly, data with low (around 200) or high (around 5700) total counts $m_i$ by varying $\mathit{\alpha }$ and ${\mathbf{x}}_{1i}$. Secondly, data with low (around 0.2) or high (around 0.9) category proportions $p_i$ by varying $\beta ,{\mathbf{x}}_{2i}$, and $\delta$. And thirdly, data with low or high association between total counts and category proportions, i.e. the value of $\delta$. The specific parameter values selected are shown in Table 1.

Table 1.

Parameter settings for the NBB regression model simulation study (Section 4.1). Eight simulation scenarios were used, named in triples of “high” or “low”, referring to the resulting values of total counts $\left(m_i\right)$, mutant proportion $\left(p_i\right)$, and their correlation $(\delta )$. Columns two and three display the mean and variance of the normal distribution used for generating the covariates. Columns four, five, and six show the values of the regression coefficients.

Scenario			$x_{11},x_{12}$		$x_{21},x_{22}$		${\alpha }_0,{\alpha }_1,{\alpha }_2$			${\beta }_0,{\beta }_1,{\beta }_2$			$\delta$
${m_i-p}_i-\delta$			mean	variance	mean	variance
low	low	low	0.5	0.5	0.5	0.5	1.0	−0.1	0.1	−0.7	0.2	0.2	0.01
low	low	high	0.5	2.0	0.5	0.5	0.1	−0.1	0.1	−0.1	−0.1	−0.1	0.90
high	low	low	5.0	0.5	5.0	0.5	1.0	−0.1	0.7	0.5	−0.1	−0.7	0.01
high	low	high	5.0	0.5	5.0	0.5	1.0	−0.1	0.7	−0.5	−0.1	0.2	−0.80
low	high	low	0.5	0.5	0.5	0.5	1.0	−0.1	0.1	0.8	0.5	0.8	0.01
low	high	high	0.5	2.0	1.0	0.5	0.5	−0.1	0.1	0.5	0.2	0.2	1.00
high	high	low	5.0	0.5	3.0	1.0	1.0	−0.1	0.7	1.0	−0.1	0.7	0.01
high	high	high	5.0	0.5	5.0	2.0	1.0	−0.1	0.7	0.1	−0.1	−0.1	1.00

In all cases the regression parameters ($\mathit{\alpha },\mathit{\beta }$, and $\delta$) were assigned vague normal priors, $N\left(0,{100}^2\right)$. In stead of assigning a prior for the $r$ parameter we assumed that the correct value (100) is known. While this is of course never the case in practice, these simulation studies are focused on assessing the performance of the proposed Gibbs sampler. Posterior sampling of $r$ will depend on the prior distribution that is chosen as well as the chosen proposal density for a Metropolis-Hastings step. For both the Gibbs sample and Stan we ran one MCMC chain of length 10,000 per dataset and discarded the first 5,000 as burn-in. Convergence was assessed via examination of traceplots and the Gelman-Rubin statistics and estimated posterior densities were compared to ensure that the two methods gave the same posterior distributions.

Fig. 1 shows boxplots of the resulting 20 ESS and ESR values for each parameter in the low-high-high scenario. Results for other scenarios were similar and are omitted for brevity. Stan generally gives larger ESS than the Gibbs sampler which is not surprising since the Stan algorithm is known to usually produce samples with lower auto-correlation than Gibbs samplers. However, the Gibbs sampler gives better (or similar) ESR than Stan, which is not surprising either since the Gibbs sampler is tailored to the NBB model and Stan uses a multi-purpose algorithm. In fact, the Gibbs sampler generally ran 3 times faster than Stan, usually between 5 and 10 seconds per chain compared to around 25 seconds for Stan. The median ESS and ESR and their range, over both the 20 data sets and all 7 parameters, for all eight scenarios are shown in Fig. 2. In all but one scenarios Stan performs better in terms of ESS but the Gibbs sample performs better in terms of ESR in all but one scenario. We also observe that the Gibbs sampler performs much better when the total count $m_i$ is low, regardless of the other two settings. Stan is less affected by this. This is perhaps not surprising since our Gibbs sampler includes samples from $\text{PG}\left(r+m_i,{\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)$ and $\text{PG}\left(m_i,{\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1+\left({\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)\delta \right)$ which are obtained by summing up $r+m_i$ and $m_i$ samples of $\text{PG}\left(1,{\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)$ and $\text{PG}\left(1,{\mathbf{x}}_{2i}^{\prime }{\mathit{\beta }}_1+\left({\mathbf{x}}_{1i}^{\prime }\mathit{\alpha }\right)\delta \right)$, respectively. Therefore, when $m_i$ is large we can expect the Gibbs sampler to take longer to execute, but since Stan does not rely on additive samples it is not affected in the same way. Performance of both methods is less affected by the values of $p_i$ and $\delta$ but is generally slightly better when these values are low, but it is unclear why this is the case.

Fig. 1.

Effective sample size (ESS, on the left) and effective sampling rate (ESR, on the right) for the “low-high-high” simulation scenario. The blue boxes represent the results obtained using the STAN method, while the red boxes represent the results obtained using the proposed Gibbs algorithm. The boxplot summarizes the median and interquartile range of 20 ESS and ESR values calculated for each parameter from 20 datasets.

Fig. 2.

Median and range of effective sample size (ESS, first two graphs) and effective sampling rate (ESR, last two graphs) for all eight simulation scenarios in Section 4.1. The $x$-axis on each graph indicates the low or high value of total counts $(m)$, while the red or blue colors indicate the low or high values of mutant proportion ($p$), and the solid or dotted line type indicate the low or high values of the correlations between the two ($\delta$). For each scenario the median and range is across all 7 parameters and across all the 20 data sets.

4.2. Simulation study for spatial NBB model

Here we simulate 20 data sets from a NBB model with spatial effects and apply the Gibbs sampler described in Section 3.3. Assuming either a CAR model or Squared Exponential model for the spatial effects the specific model considered here is the following:

$$M_i\mid \mathit{\alpha },r,{\mathit{v}}_1~\text{NB}\left(r,q_i\right)i=1,\dots ,100$$

$$\text{where}\text{logit}\left(q_i\right)={\alpha }_0+x_{11,i}{\alpha }_1+x_{12,i}{\alpha }_2+v_{1i}$$

$$Y_i\mid M_i=m_i,\mathit{\beta },\delta ,\mathit{\alpha },{\mathit{v}}_1,{\mathit{v}}_2~\text{Binomial}\left(m_i,p_i\right)$$

$$\text{where}\text{logit}\left(p_i\right)={\beta }_0+x_{21,i}{\beta }_1+x_{22,i}{\beta }_2+\left({\alpha }_0+x_{11,i}{\alpha }_1+x_{12,i}{\alpha }_2+v_{1i}\right)\delta +v_{2i}$$

$$\text{and}{\mathit{v}}_j={\left(v_{j1},v_{j2},\cdots ,v_{j100}\right)}^{\prime }~\text{N}\left(\mathbf{0},𝚺\left({\mathit{\theta }}_j\right)\right),j=1,2,$$ (43)

where the fixed effects (covariates) are simulated from a normal distribution, specifically $x_{11,i}~N(1,1),x_{12}~N(2,1),x_{21}~N(1,2)$ and $x_{22}~N(2,4)$. The regression parameters are set as ${\alpha }_0=1,{\alpha }_1=-0.1,{\alpha }_2=0.1,{\beta }_0=0.1,{\beta }_1=-0.1,{\beta }_2=0.1,\delta =0.01$. Again, we fix $r$ at $r=100$ and assume it is known. For the CAR model, we assume a context of a ten by ten grid where each node has neighbors on the left, right, above and below and the relationship is described by the adjacency matrix. We set ${\rho }_1=0.8$ and ${\rho }_2=0.5$ to mimic a rather strong spatial correlation. For the Squared Exponential model, we randomly generated a set of locations with an average distance of 129 (range: 1.51 – 504). We set ${\psi }_1=250$ and ${\psi }_2=200$, i.e. about half the range of the data, to obtain a rather strong spatial correlation. For both the CAR and Squared Exponential model, we chose the values of the variance parameters such that the fixed effects and random effects have similar range (on either log or logit scales). In particular, we set ${\sigma }_1^2=2.25$ and ${\sigma }_2^2=4$ for the CAR model and we set ${\sigma }_1^2=0.5$ and ${\sigma }_2^2=0.8$ for the squared exponential model.

We assigned normal priors on regression parameters, $N\left(0,{100}^2\right)$, and InverseGamma(0.1, 0.1) priors on ${\sigma }_j^2$ and ${\rho }_j$ or ${\psi }_j$. We applied the proposed algorithm derived in Section 3.3 as well as Stan and obtained converging result with one MCMC chain of size 10,000 and discarded the first 5,000 as burn-in. Fig. 3 shows boxplots of the ESS and ESR for each of the parameters for the NBB CAR model and NBB Squared Exponential Model. As before we see that the Gibbs sampler out performs Stan in terms of ESR. The difference in running times is more substantial than before, with our Gibbs sampler taking around 200 seconds per chain but Stan usually takes an average of 6 times longer. But unlike before, Gibbs gives much better ESS for the regression parameters $\mathit{\alpha },\mathit{\beta }$, and $\delta$. This is partly due to the joint sampling of regression parameters and spatial effects.

Fig. 3.

Boxplots of effective sample size (ESS, left column) and effective sampling rate (ESR, right column) for NBB CAR model (first row) and NBB Squared Exponential Model (second row) simulation studies in Section 4.2. The blue boxes represent the results obtained using the STAN model, while the red boxes represent the result obtained using the proposed Gibbs algorithm. The boxplots summarizes the median and interquartile range of 20 ESS and ESR values calculated for each parameters from 20 data sets.

5. NBB model for mitochondrial DNA data

We present two applications of the NBB model and use the Pólya-Gamma data-augmentation scheme presented in Section 3 to obtain samples from the posterior distributions. In Section 5.1 we have mtDNA copy number and mutation data from acute myeloid leukemia (AML) patients at three different time points as well as some demographic and clinical data. In Section 5.2 we fit an NBB model with spatial effects to mtDNA data from a tissue sample from a cutaneous squamous cell carcinoma patient. In both cases we ran four MCMC chains and checked convergence using the Gelman-Rubin statistic (Gelman and Rubin, 1992).

5.1. AML mtDNA data

We have data (see Fig. 4) from 27 AML patients collected at three stages of the disease; diagnosis, remission, and relapse (Farrar et al., 2016). At each stage a blood sample is taken and the average copy number and mutation proportion at various base positions is estimated. In this example we chose position 2487 which lies within the ribosomal RNA gene 16S. Mutations in this gene have been associated with Rett syndrome (Cardaioli et al., 1999), myopathy (Coulbault et al., 2007), and other diseases. Other data for each patient included gender and whether the patient carries the internal tandem duplication (ITD) mutation in the FLT3 gene (FLT/ITD). The FLT3/ITD mutation is the second most common among adult AML patients (Liu et al., 2019).

Fig. 4.

Mitochondrial DNA data from 27 acute myeloid leukemia patients (Section 5.1). Top row: Boxplots of copy number (left) and mutant proportion at position 2487 (right) at the three decease stages: diagnosis, remission, and relapse. The graphs also show the original observations, connected with a line for each patient and color coded according to whether the values increase or decrease between stages. Bottom left: Boxplots of mtDNA mutant proportions separated by gender and FLT3/ITD status. Bottom right: logit of mtDNA mutant proportions versus log of copynumber, which indicates a positive relationship between the two.

Here we let $M_{ij}$ and $Y_{ij}$ be the mean copy number per cell and mean number of mutations for position 2487 for patient $i$ at stage $j$ for $i=1,\dots ,27$ and $j=1,2,3$. We see from Fig. 4 that copy numbers tend to increase between diagnosis and remission and then decrease again when the cancer relapses. Based on this we include a stage effect in the total count model setting ${\text{R}}_{ij}$ and ${\text{P}}_{ij}$ as indicator variables of whether an observation was taken at the remission and relapse stage, respectively. We also observe from Fig. 4 that there is a possible interaction between gender and FLT3/ITD for mutation proportion so we included both main effects and interaction effects in the category count model, where ${\text{Gender}}_i$ and ${\text{FLT}}_i$ denote indicator variables for whether patient $i$ is male and FLT/ITD positive, respectively. The model specification is the following:

$$M_{ij}\mid r,\mathit{\alpha }\stackrel{\text{ind}}{\sim }\text{NB}\left(r,q_{ij}\right)i=1,\dots ,27,j=1,2,3$$

$$\text{where logit}\left(q_{ij}\right)={\alpha }_0+{\alpha }_1{\text{R}}_{ij}+{\alpha }_2{\text{P}}_{ij}$$ (44)

$$Y_{ij}\mid M_{i,j},\mathit{\beta },\delta ,\mathit{\alpha }\stackrel{\text{ind}}{\sim }\text{Binomial}\left(M_{ij},p_{ij}\right)$$

$$\text{where logit}\left(p_{ij}\right)={\beta }_0+{\beta }_1{\text{Gender}}_i+{\beta }_2{\text{FLT}}_i+{\beta }_3{\text{Gender}}_i*{\text{FLT}}_i+\delta \text{logit}\left(q_{ij}\right).$$ (45)

We assigned independent $N\left(0,{100}^2\right)$ priors for all regression parameters ($\mathit{\alpha },\mathit{\beta }$, and $\mathit{\delta }$) and a Uniform prior over the interval (0, 1000) for $r$. Posterior means and 90% credible intervals for model parameters are shown in Table 2. The average copy number is significantly different (i.e. the 90% credible interval does not include zero) at the three treatment stages. In particular, the average copy number at remission stage is about exp(0.34) = 1.40 (90% CI: 1.34 – 1.48) times the copy number at diagnosis and the mean copy number at relapse is about 0.66 (90% CI: 0.63 – 0.69) times of the copy number at diagnosis. This result suggests that mtDNA copy number falls below normal levels during the disease course, as the number of healthy myeloid cells decreases as a consequence of the malignancy. Indeed, it has recently been shown that myeloid and monocytic cells have substantially higher copy number than erythroid cells (Campbell et al., 2023). When the patient responds to treatment, there is a rebound in healthy myeloid cells and the mtDNA copy number recovers. However, relapse disease is typically more severe than at diagnosis, and therefore the mtDNA copy number falls to levels even below that at diagnosis.

Table 2.

Posterior means and 90% posterior credible intervals for NBB regression model in Section 5.1 (on the left); Posterior means and 90% posterior credible intervals for NBB spatial model in Section 5.2 (on the right).

NBB regression model, Section 5.1			NBB spatial model, Section 5.2
Parameter	Post. mean	90% post. CI	Parameter	Post. mean	90% post. CI
${\alpha }_0$	0.06	[0.02, 0.10]	${\alpha }_0$	−0.03	[−0.10, 0.04]
${\alpha }_1$	0.34	[0.29, 0.39]	${\beta }_0$	−0.01	[−0.34, 0.33]
${\alpha }_2$	−0.42	[−0.47, −0.37]	$\delta$	−0.79	[−1.35, −0.26]
$\delta$	0.15	[0.02, 0.27]	$r$	92.13	[90.62, 93.64]
$r$	154.44	[151.73, 157.13]	${\sigma }_1$	0.48	[0.44, 0.53]
${\beta }_0$	−1.59	[−1.67, −1.52]	${\sigma }_2$	2.13	[1.92, 2.37]
${\beta }_1$	0.03	[−0.06, 0.12]	$\sqrt{{\psi }_1}$	18.06	[15.76, 20.49]
${\beta }_2$	0.10	[−0.03, 0.22]	$\sqrt{{\psi }_2}$	17.34	[15.12, 19.44]
${\beta }_3$	−0.15	[−0.31, 0.01]

As expected, the interaction term in the mutant count model level $\left({\beta }_3\right)$ is negative, but zero fits rather comfortably within the posterior density. However, the $\delta$ parameter is significantly different from zero indicating a significant association between copy number and mutant proportion. Here we find that when the average copy number is multiplied by a constant $c$ the odds of mutant DNA is multiplied by $c^{0.15}$, given other covariates are fixed. For example, if the mean count is doubled, the odds would be multiplied by $2^{0.15}\approx 1.11$ (90% CI: 1.01 – 1.21). Copy number increase in response to mutant copies is likely the result of the cell attempting to compensate for the deficiencies in mutant mtDNA copies.

In order to assess potential undesired feedback between model layers (modules) we also applied a simple “plug-in” modularization approach. That is, we fit the model for total counts in (44) and the model for category counts in (45) separately, plugging-in the posterior means of $\text{logit}\left(q_{ij}\right)$ in the latter case. It turned out that posterior means and 90% posterior credible intervals were almost identical to those estimated via the full model. Our interpretation of this is that no undesired feedback is present in this example.

5.2. Spatial mtDNA data

We have spatial mtDNA data from a tumor slice taken from a cutaneous squamous cell carcinoma patient, see Fig. 5. A grid of over 1900 spatial locations, called spots, was sampled and mean copy number estimated at each grid point. However, due to the way the DNA sequencing was applied, not all base positions were observed at all spots. We use data from the base position 2129 which was observed at $N=238$ locations. As in the previous example, this position is within the 16S ribosomal RNA gene. For a location $i$, let $M_i$ and $Y_i$ be copy number per cell and the number of mutations per cell, respectively. We fit the following squared exponential spatial NBB model:

$$M_i\mid r,{\alpha }_0,{\mathit{v}}_{\mathbf{1}}~\text{NB}\left(r,q_i\right)i=1,\dots ,N$$

$$\text{where}\text{logit}\left(q_i\right)={\alpha }_0+v_{1i}$$

$$Y_i\mid {\beta }_0,\delta ,{\alpha }_0,{\mathit{v}}_{\mathbf{1}},{\mathit{v}}_{\mathbf{2}}~\text{Binomial}\left(M_i,p_i\right)$$

$$\text{where}\text{logit}\left(p_i\right)={\beta }_0+\left({\alpha }_0+v_{1i}\right)\delta +v_{2i}$$

$$\text{and}{\mathit{v}}_j={\left(v_{j1},v_{j2},\cdots ,v_{jN}\right)}^{\prime }~{\text{N}}_N\left(\mathbf{0},{\sigma }_j^2\mathbf{R}\left({\psi }_j\right)\right)$$

$$\text{where}{\mathbf{R}}_{m,n}\left({\psi }_j\right)=\text{exp}\left(-\frac{1}{{\psi }_j}{‖x_m-x_n‖}^2\right),j=1,2.$$ (46)

We selected proper but vague priors for all parameters, in particular ${\alpha }_0~N\left(0,{10}^4\right),{\beta }_0~N\left(0,{10}^4\right),\delta ~N\left(0,{10}^4\right),r~\text{Uniform}(0,1000),{\sigma }_j^2~\text{InverseGamma}(\text{shape}=0.1,\text{scale}=0.1)$, and ${\psi }_j~\text{Gamma}(\text{shape}=0.01,\text{scale}=1000),j=1,2$. Note that the prior for ${\psi }_j$ has mean 10 and variance 100² which allows for any reasonable values in the context of the distances observed in our data.

Fig. 5.

Spatial Plot for mtDNA mean copy number (on the left) and the mutant proportions (on the right) for Position 2129. Only spots where position 2129 was observed are shown.

The posterior means with the 90% credible intervals are shown in Table 2. We see that the $\delta$ parameter is significantly different from zero and is negative. Specifically, after taking into account the spatial variations, if the average copy number is multiplied by a constant $c$ the odds of a mutation is multiplied by $c^{-0.79}$. For example, if the mean copy number is doubled the odds of mutant mtDNA is multiplied by $2^{-0.79}\approx 0.58(90\%\text{CI}:0.39-0.84)$. This result could point to this particular mutation being “gain-of-function”, i.e. conferring a growth advantage to the cell. If the mutation present at a higher proportion, the cell requires fewer mtDNA copies to proliferate.

The square root of ${\psi }_1$ and ${\psi }_2$ give some idea of the spatial correlation structure, specifically, the correlation length. It determines how quickly the correlation between the spatial locations decreases as the distance between them increases. The posterior mean lengths are around 18 and compared to the distance matrix which has a mean of 297.91 (range: 13.58–751.66), this is relatively small, indicating the correlation between two spatial locations decreases quickly with distance.

6. Conclusions

We have presented a hierarchical joint model for total counts and category count data and illustrated its use in a novel application. While most statistical literature treats total counts and category counts separately we have shown that there are important applications where understanding is gained from modeling them jointly, in this case the analysis of mitrochondrial DNA data. The specific way we modelled the dependence of category proportion and the mean total count can of course be further expanded and developed. Here we chose to add a linear effect to model the logit category proportion that is a particular function of the total count mean, i.e. $\text{log}\left(E\left(M_i\right)/r\right)$ where $E\left(M_i\right)$ is the mean total count and $r$ is the dispersion parameter of the Negative Binomial. This lends itself to a nice interpretation of the odds of category success ($p$): When the mean total count is multiplied with a constant, the odds of success are multiplied with that constant to the power of the regression parameter ($\delta$). Therefore, the sign of $\delta$ indicates whether the success probability increases (positive) or decreases (negative) with increasing total count mean and $\delta =0$ indicates no relation. The main reason for our choice of connection between levels, however was the that it facilitated the efficient data augmentation scheme presented in Section 3.2.

We showed through simulation studies that the double Pólya-Gamma data augmentation scheme we developed for posterior sampling from the joint model has as-good or better efficiency as the Stan program. In the case of regression effects only the Stan sampler gives better effective sample sizes, which is not surprising since that is generally the strength of Stan and Gibbs samplers often produce samples with high auto-correlation. However, the double Pólya-Gamma data augmentation scheme has better effective sampling rate in all our regression scenarios. When adding spatial effects, our double Pólya-Gamma data augmentation scheme outperformes Stan both in terms of effective sample size and in terms of effective sampling rate.

Possible further developments of this work include extending the hierarchical model to include multivariate categorical counts that have the same total counts. This is particularly relevant for mtDNA data where the mutant proportion at multiple positions may be of interest.