A default prior distribution for contingency tables with dependent factor levels

Abstract

A default prior distribution is proposed for the Bayesian analysis of contingency tables. The prior is specified to allow for dependence between levels of the factors. Different dependence structures are considered, including conditional autoregressive and distance correlation structures. To demonstrate the prior distribution, a dataset is considered which involves estimating the number of injecting drug users in the eleven National Health Service board regions of Scotland using an incomplete contingency table where the dependence structure relates to geographical regions.

1. Introduction

Contingency tables (e.g.  [1]) are formed when a population is cross-classified according to a series of categories (or factors). Each cell count of the table gives the number observed under each cross-classification. The aim of forming such a table is to summarise the data, and typically, with a view to identifying interactions or relationships between the factors.

The standard statistical practice to model such interactions is the log–linear model (e.g.  [1, Chapter 7]). In this case the logarithm of the expected cell count is proportional to a linear predictor depending on the main effect terms and interaction terms between the factors. Each combination of interaction terms defines its own log–linear model so that the identification of the non-zero interaction terms translates to an exercise in model comparison. Additionally incomplete contingency tables with missing cell counts can be used to estimate closed populations  [4] where some of the factors correspond to sources that have either observed or not observed individuals in the population.

In this paper, we consider the case where the levels of one or more of the factors may be dependent on one another. An obvious example is when one of the factors has levels corresponding to geographical regions or locations which may be dependent due to their geographical proximity. In these cases, we may expect the parameters of the log–linear model to have some dependence structure. Bayesian analysis of contingency tables is common (e.g.  [3], [13], [5]) and is the approach taken here. One feature of the Bayesian approach is that prior information on the interaction terms can be incorporated through the prior distribution. We take the position of having weak prior information on the magnitude of the log–linear parameters but wish to incorporate the information provided by the dependence structure mentioned above. In the case of weak prior information and model uncertainty, care must be taken when specifying prior distributions due to Lindley’s paradox (e.g.  [16, pp. 77–79]). There have been several attempts in the literature (e.g.  [3], [15], [18]) to specify “default” prior distributions that can be applied for log–linear models under model uncertainty. We extend these approaches by developing a default prior that can take account of the dependence structure between the factor levels and can be seen as a generalisation of the above mentioned priors. The proposed prior is constructed by conditioning on the constraints on the parameters which are introduced in contingency table analysis to maintain identifiability of the parameters.

This paper is organised as follows. In Section  2 we set out our notation and briefly describe log–linear models. In Section  3 we derive our proposed default prior distribution including descriptions of different dependence structures. Finally, we apply our proposed prior to a real data application in Section  4, which involves estimating the number of injecting drug users in Scotland. Here, one of the factors corresponds to geographical regions, and we wish to take account of the possible dependence structure that may exist for the regions.

2. Notation and log–linear models

2.1. Notation

We assume that there are a total of $c$ factors such that each factor $k=1,\dots ,c$ has $l_k$ levels. The corresponding contingency table has $n={\prod }_{k=1}^c{l}_k$ cells. Let $y$ be the $n\times 1$ vector of cell counts with elements denoted as $y_i$ and where $i=(i_1,\dots ,i_c)$ identifies the combination of factor levels that cross-classify the cell $i$. Let $\mathcal{S}$ be set of all $n$ cross-classifications so that

$$\mathcal{S}=\{(i_1,\dots ,i_c):i_l\in \{1,\dots ,l_k\}\}\text{.}$$

Finally, let $N={\sum }_{i\in \mathcal{S}}{y}_i$ be the total population size. In the case of an incomplete contingency table, $N$ is unknown, since elements of $y$ are unknown.

As a pedagogic example that we use for illustrative purposes throughout, suppose that there are three factors used to cross-classify a population of hospital patients: age (2 levels: young; old), hypertension (2 levels: no; yes) and region (3 levels: A; B; C). In this example, $c=3$, where $l_1=2,l_2=2$ and $l_3=3$, and the three factors (age, hypertension and region) have been labelled 1, 2 and 3, respectively. It follows that there are $n=2\times 2\times 3=12$ cells.

2.2. Log–linear models

We now briefly describe log–linear models and initially assume that the form of the log–linear model is known, i.e. it is known which interactions are present. We extend to the case of model uncertainty later in this section. Let ${\eta }_i$ denote the linear predictor associated with cell $i\in \mathcal{S}$, where

$${\eta }_i=\varphi +z_i^T\theta \text{,}$$

with $\varphi \in \mathbb{R}$ denoting the intercept term, $\theta$ the $q\times 1$ vector of log–linear parameters (i.e. the main effects and interaction terms) and $z_i$ the $q\times 1$ vector of zeros and ones identifying which elements of $\theta$ are applicable to cell $i\in \mathcal{S}$.

For identifiability, certain elements of $\theta$ are constrained, e.g. by sum-to-zero, or corner-point constraints, so we can rewrite ${\eta }_i$ as

$${\eta }_i=\varphi +x_i^T\beta \text{,}$$

where $\beta \in {\mathbb{R}}^p$ is the $p\times 1$ vector of unconstrained regression parameters, and $x_i$ is the $p\times 1$ vector which identifies which elements of $\beta$ correspond to cell $i\in \mathcal{S}$, with $p<q$.

Finally, let $\eta$ be the $n\times 1$ vector with elements ${\eta }_i$, and let $X$ be the $n\times p$ model matrix with rows $x_i$. Then we can write

$$\eta =\varphi 1_n+X\beta \text{,}$$

where $1_n$ denotes the $n\times 1$ vector of ones.

For the statistical analysis of contingency tables, it is common to assume that

$$y_i|\varphi ,\beta \sim \text{Poisson}\left({\lambda }_i\right)\text{,}$$ (1)

independently, where $log{\lambda }_i={\eta }_i$.

In practice, we typically do not know the form of the log–linear model. This is equivalent to not knowing the elements of $z_i$ and $x_i$, or the columns of $X$. Let $\mathcal{M}$ be the set of competing log–linear models which are indexed by $m\in \mathcal{M}$. Associated with each log–linear model are $z_i^{\left(m\right)},x_i^{\left(m\right)},X^{\left(m\right)},{\theta }^{\left(m\right)}$ and ${\beta }^{\left(m\right)}$, where $z_i^{\left(m\right)}$ and ${\theta }^{\left(m\right)}$ are $q^{\left(m\right)}\times 1$ vectors, $x_i^{\left(m\right)}$ and ${\beta }^{\left(m\right)}$ are $p^{\left(m\right)}\times 1$ vectors, and $X^{\left(m\right)}$ is an $n\times p^{\left(m\right)}$ matrix.

In the next section, we derive a default prior distribution for ${\beta }^{\left(m\right)}|m$. For the intercept, $\varphi$, we assume a prior given by $\pi \left(\varphi \right)\propto 1$. Although this prior is improper, the resulting posterior is still proper  [5]. This prior will not cause a problem under Lindley’s paradox since it is present for all models in $\mathcal{M}$[16, p. 174].

3. A default prior distribution for ${\beta }^{\left(m\right)}|m$

3.1. Derivation

In this section we develop a default prior distribution for ${\beta }^{\left(m\right)}|m$. For notational simplicity, we drop the dependency on the model $m$ by removing the superscript $\left(m\right)$.

Suppose that there are a total of $T$ log–linear terms and $\beta =({\beta }_1,\dots ,{\beta }_T)$ where ${\beta }_t$, for $t=1,\dots ,T$, is the $p_t\times 1$ vector corresponding to the regression parameters for the main effect or interaction term $t$. Similarly let ${\theta }_t$ denote the corresponding $q_t\times 1$ vector of log–linear parameters, for $t=1,\dots ,T$.

Let ${\mathcal{R}}_t$ be the set of $f$ main effect terms that define the $f$-way interaction ${\beta }_t$. Dellaportas and Forster  [3] refer to ${\mathcal{R}}_t$ as the constituent terms of the interaction. Note that $q_t={\prod }_{j\in {\mathcal{R}}_t}{q}_j$ and if ${\beta }_t$ corresponds to a main effect then ${\mathcal{R}}_t$ has only one element, i.e. $t$. Consider the pedagogic example, from Section  2.1, and $t$ corresponding to the 2-way interaction between age and region so that $q_t=6$ and $p_t=2$. The constituent terms, ${\mathcal{R}}_t$, have two elements: the terms corresponding to age and region.

We initially consider deriving the default prior distribution under sum-to-zero constraints. We describe how the prior can be extended to any system of constraints in Section  3.4. Following Dellaportas and Forster  [3] we assume that $\beta$ has a multivariate normal distribution with mean zero, where ${\beta }_r$ and ${\beta }_t$ are independent for $r,t=1,\dots ,T$ and $r\ne t$. Thus, all that remains is to specify the $p_t\times p_t$ covariance matrix for each ${\beta }_t$, for $t=1,\dots ,T$.

The elements of ${\theta }_t$ are subject to constraints and can be written in the form

$${\theta }_t=A_t{\beta }_t\text{,}$$ (2)

where $A_t$ is a $q_t\times p_t$ matrix defining the constraints. Under sum-to-zero constraints, $A_t$ can be written as

$$A_t=P_t\left(\begin{array}{c}\hfill I_{p_t}\hfill \\ \hfill C_t\hfill \end{array}\right)\text{,}$$ (3)

where $I_{p_t}$ is the $p_t\times p_t$ identity matrix, $C_t$ is a $(q_t-p_t)\times p_t$ matrix and $P_t$ is a $q_t\times q_t$ permutation matrix. For $t$ corresponding to the age and region interaction in the pedagogic example,

$${\theta }_t=\left(\begin{array}{c}\hfill {\theta }_{t1}\hfill \\ \hfill {\theta }_{t2}\hfill \\ \hfill {\theta }_{t3}\hfill \\ \hfill {\theta }_{t4}\hfill \\ \hfill {\theta }_{t5}\hfill \\ \hfill {\theta }_{t6}\hfill \end{array}\right)\text{,}{A}_t=\left(\begin{array}{cc}\hfill 1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \\ \hfill -1\hfill & \hfill -1\hfill \\ \hfill -1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill -1\hfill \\ \hfill 1\hfill & \hfill 1\hfill \end{array}\right)\text{,}{C}_t=\left(\begin{array}{cc}\hfill -1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill -1\hfill \\ \hfill 1\hfill & \hfill 1\hfill \\ \hfill -1\hfill & \hfill -1\hfill \end{array}\right)\text{,}$$

$$P_t=\left(\begin{array}{cccccc}\hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \end{array}\right)\text{.}$$

The elements of ${\theta }_t$ are ordered so that the factor levels of region vary the fastest.

Initially, ignoring the constraints that are applied to ${\theta }_t$, we assume that the distribution of ${\theta }_t$ is

$${\theta }_t|{\sigma }_t^2,D_t\sim N(0,{\sigma }_t^2{D}_t)\text{,}$$

where ${\sigma }_t^2>0$ and $D_t$ is a $q_t\times q_t$ positive-definite scale matrix. The off-diagonal elements of $D_t$ control the dependence structure or correlation between the elements of the constrained parameters, ${\theta }_t$, corresponding to different factor levels.

It follows from (2), (3) that

$$P_t^T{\theta }_t=\left(\begin{array}{c}\hfill {\beta }_t\hfill \\ \hfill C_t{\beta }_t\hfill \end{array}\right)\text{.}$$ (4)

Let

$${\gamma }_t=\left(\begin{array}{c}\hfill {\gamma }_t^{\left(1\right)}\hfill \\ \hfill {\gamma }_t^{\left(2\right)}\hfill \end{array}\right)=P_t^T{\theta }_t$$

be the permuted elements of ${\theta }_t$ according to the inverse permutation $P_t^{-1}=P_t^T$, so that ${\gamma }^{\left(1\right)}={\beta }_t$ and ${\gamma }^{\left(2\right)}=C_t{\beta }_t$. The prior distribution for ${\beta }_t$ is the conditional distribution of ${\gamma }^{\left(1\right)}$ (which is ${\beta }_t$) given that ${\gamma }^{\left(2\right)}=C_t{\beta }_t$, i.e. we find the distribution of ${\beta }_t$ from (4) subject to the constraints. It can be shown (see Appendix A) that

$${\beta }_t|{\sigma }_t^2,D_t\sim N(0,{\sigma }_t^2{\Sigma }_t)\text{,}$$ (5)

where

$${\Sigma }_t={\left(A_t^T{D}_t^{-1}{A}_t\right)}^{-1}\text{.}$$ (6)

In the next two sections we consider $D_t$. It may be that $D_t$ is completely specified a priori. The most plausible situation for this is when we assume independence between the levels of this term and $D_t=I_{q_t}$. We consider this case in Section  3.2. In Section  3.3 we also consider where $D_t$ is unknown due to its dependence on some unknown hyperparameter which controls the strength of correlation between the elements of ${\theta }_t$.

3.2. Independent correlation structure

Suppose we assume that the factor levels are independent, i.e. $D_t=I_{q_t}$, so that

$${\Sigma }_t={\left(A_t^T{A}_t\right)}^{-1}\text{.}$$

Denote by $X_t$ the $n\times p_t$ matrix formed by the columns of $X$ corresponding to ${\beta }_t$. Since $X_t$ is a permutation of the matrix formed by stacking $A_t$ to form an $n\times p_t$ matrix, it follows that

$$X_t^T{X}_t=\frac{n}{q_t}{A}_t^T{A}_t\text{,}$$

and therefore ${\Sigma }_t=(n/q_t){\left(X_t^{T}X\right)}^{-1}$. The corresponding prior distribution for ${\beta }_t$ is

$${\beta }_t|{\sigma }_t^2\sim N(0,\frac{{\sigma }_t^{2}n}{q_t}{\left(X_t^T{X}_t\right)}^{-1})\text{.}$$

If ${\sigma }_t^2=g{q}_t/n$, then since (under sum-to-zero constraints) $X_t^T{X}_r\ne 0$, for all $t\ne r$   [14], it follows that the prior distribution for $\beta =({\beta }_1,\dots ,{\beta }_T)$ is

$$\beta |g\sim N(0,g{\left(X^{T}X\right)}^{-1})\text{.}$$ (7)

If $g>0$ is unknown and given a prior distribution, then (7) is a hierarchical prior distribution that is identical to the generalised hyper-g prior proposed by Sabanes-Bové and Held  [18] for generalised linear models (GLMs) when applied to log–linear models. If, instead, $g$ is fixed then (7) is the default prior distribution considered by Dellaportas and Forster  [3] who advocate setting $g=kn$ for some constant $k$, which represents the number of units of prior information. Ntzoufras et al.  [15] use $k=1$ under their unit information prior for GLMs when applied to log–linear models.

3.3. General correlation structure

We now consider terms, $t$, whose constituent terms, ${\mathcal{R}}_t$, contain factors with correlated levels and $D_t$ depends on some unknown hyperparameter $\tau$. This hyperparameter, $\tau$, controls the strength of correlation through some structure imposed on $D_t$. Initially consider a main effect term $t$. In this paper we focus on the case where the factor levels correspond to geographical regions or locations and propose two structural forms for $D_t$. However there exist many possible applications with correlated factor levels and other correlation structures that can be used depending on the nature of the factor levels.

• 1.Conditional autoregressive structure
• Suppose that the $q_t$ levels correspond to regions. Let $G$ be the $q_t\times q_t$ neighbourhood matrix with $ij$th element

  $$G_{ij}=\{\begin{array}{cc}1\hfill & \text{if regions }i\ne j\text{ are neighbours ,}\hfill \\ 0\hfill & \text{if otherwise ,}\hfill \end{array}$$

  for $i,j=1,\dots ,q_t$. Then for the conditional autoregressive (CAR) structure (e.g.  [2]),

  $$D_t={(I_{q_t}-\tau G)}^{-1}\text{,}$$

  where $\tau$ determines the strength of spatial correlation for the constrained parameters. To ensure that $D_t$ is positive-definite, the hyperparameter $\tau$ must lie in the interval $({\tau }_{\min },{\tau }_{\max })=(e_{q_t}^{-1},e_1^{-1})$, where $e_1$ and $e_{q_t}$ are the maximum and minimum eigenvalues of $G$, respectively.
• 2.Distance correlation structure
• Suppose the $q_t$ levels correspond to locations such as cities. Then the $ij$th element of $D_t$ is given by a correlation function that depends on the distance, $d_{ij}$, between locations $i$ and $j$, and $\tau$. For example, the Gaussian correlation function gives

  $$D_{t,ij}=exp(-\frac{d_{ij}^2}{2{\tau }^2})\text{,}$$

  where, again, $\tau >0$ controls the strength of correlation.

Note that in both examples, the hyperparameter, $\tau$, is not actually a correlation coefficient; it merely controls the strength of correlation. We need to specify a prior distribution for $\tau$. This will depend on the application.

For a term $t$ that corresponds to an interaction term, we propose

$$D_t=\underset{r\in {\mathcal{R}}_t}{⨂}{D}_r\text{.}$$ (8)

The form given by (8) has been chosen for its consistency. Suppose that the correlation between two levels of a main effect term is $d$. Then, for an interaction involving this main effect, the correlation between the two levels will be $d$ if and only if the factor levels of the other constituent terms are identical. To demonstrate this we return to our pedagogic example where the regions A and B, and B and C are neighbours, but A and C are not neighbours. A CAR structure is specified. In this example, the neighbourhood matrix is

$$G=\left(\begin{array}{ccc}\hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill 1\hfill & \hfill 0\hfill & \hfill 1\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \end{array}\right)\text{,}$$

so that $D_t$ for the main effect of region is

$$D_{\mathrm{region}}=\frac{1}{1-2{\tau }^2}\left(\begin{array}{ccc}\hfill 1-{\tau }^2\hfill & \hfill \tau \hfill & \hfill {\tau }^2\hfill \\ \hfill \tau \hfill & \hfill 1\hfill & \hfill \tau \hfill \\ \hfill {\tau }^2\hfill & \hfill \tau \hfill & \hfill 1-{\tau }^2\hfill \end{array}\right)\text{.}$$

The eigenvalues of $G$ are $(-\sqrt{2},0,\sqrt{2})$, so, therefore, $\tau \in ({\tau }_{\min },{\tau }_{\max })=(-1/\sqrt{2},1/\sqrt{2})$. If an independent correlation structure is specified for the main effect of age, then

$$D_{\mathrm{age}:\mathrm{region}}=\frac{1}{1-2{\tau }^2}\left(\begin{array}{cccccc}\hfill 1-{\tau }^2\hfill & \hfill \tau \hfill & \hfill {\tau }^2\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill \tau \hfill & \hfill 1\hfill & \hfill \tau \hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill {\tau }^2\hfill & \hfill \tau \hfill & \hfill 1-{\tau }^2\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 1-{\tau }^2\hfill & \hfill \tau \hfill & \hfill {\tau }^2\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill \tau \hfill & \hfill 1\hfill & \hfill \tau \hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill {\tau }^2\hfill & \hfill \tau \hfill & \hfill 1-{\tau }^2\hfill \end{array}\right)\text{.}$$ (9)

The correlation between A and B for the main effect of region is $\tau {(1-{\tau }^2)}^{-1/2}$. For the age and region interaction, the correlation between levels involving A and B is $\tau {(1-{\tau }^2)}^{-1/2}$ if and only if they have the same level for age. It now follows from (6), (9) that the scale matrix for the prior distribution is

$${\Sigma }_{\mathrm{age}:\mathrm{region}}=\frac{1}{3+4\tau }\left(\begin{array}{cc}\hfill 1+\tau \hfill & \hfill -1/2\hfill \\ \hfill -1/2\hfill & \hfill 1\hfill \end{array}\right)\text{.}$$

If we denote the regression parameters for this term as ${\beta }_t=({\beta }_{t1},{\beta }_{t2})$, where $t=\mathrm{age}:\mathrm{region}$, then the prior correlation between ${\beta }_{t1}$ and ${\beta }_{t2}$ is

$$\mathrm{corr}({\beta }_{t1},{\beta }_{t2})=-\frac{1}{2\sqrt{1+\tau }}\text{.}$$

If $\tau =0$, corresponding to independence between the regions, i.e. $D_t=I_{q_t}$, and thus we have the Sabanes-Bové and Held  [18] prior, then $\mathrm{corr}({\beta }_{t1},{\beta }_{t2})=-1/2$. The function $\mathrm{corr}({\beta }_{t1},{\beta }_{t2})$ is increasing in $\tau$ but the correlation is always negative. This is caused by the sum-to-zero constraints. As $\tau$ increases, the magnitude of the negative correlation decreases.

A further advantage of using the structure defined by (8) is computational. If we assume that the independence model, containing only the main effect terms, is the simplest model we wish to consider then we will always have the same set of hyperparameters in each model.

3.4. Alternative constraint systems

We now consider alternative constraint systems to sum-to-zero constraints, e.g. corner-point or Helmert constraints. Let ${\beta }_A$ and $\beta$ denote the vectors of regression parameters under the alternative and sum-to-zero constraints, respectively. Since, under the sum-to-zero constraints, each component, ${\beta }_t$, of $\beta$ has a normal distribution, then $\beta$ has a normal distribution with mean zero and variance matrix $\Psi =\mathrm{diag}\{{\sigma }_1^2{\Sigma }_1,\dots ,{\sigma }_T^2{\Sigma }_T\}$. It can be shown (see Appendix B) that

$${\beta }_A={\left(X_A^T(I_n-\frac{1}{n}{J}_n)X_A\right)}^{-1}{X}_A^T(I_n-\frac{1}{n}{J}_n)X\beta ,=R_{A}X\beta \text{,}$$ (10)

where $X_A$ and $X$ are the model matrices under the alternative and sum-to-zero constraints, respectively, $J_n$ is the $n\times n$ matrix of ones and

$$R_A={\left(X_A^T(I_n-\frac{1}{n}{J}_n)X_A\right)}^{-1}{X}_A^T(I_n-\frac{1}{n}{J}_n)\text{.}$$

Therefore ${\beta }_A\sim N(0,{\Psi }_A)$, where the prior variance matrix, ${\Psi }_A$, is given by

$${\Psi }_A=R_{A}X\Psi X^T{R}_A^T\text{.}$$

Note that, under the alternative constraints, ${\beta }_t$ and ${\beta }_r$ may no longer, necessarily, be independent. This is equivalent to the fact that ${\Psi }_A$ (given by the above expression) may no longer, necessarily, be block diagonal.

Under the independence structure described in Section  3.2, where $D_t=I_{q_t}$, for $t=1,\dots ,T$, then

$${\Psi }_A=g{R}_{A}X{\left(X^{T}X\right)}^{-1}{X}^T{R}_A^T\text{.}$$

The matrix $H=X{\left(X^{T}X\right)}^{-1}{X}^T$ is called the hat matrix and is invariant to the type of constraint system used, i.e. $H=H_A=X_A{\left(X_A^T{X}_A\right)}^{-1}{X}_A^T$ and therefore

$${\Psi }_A=g{\left(X_A^T{X}_A\right)}^{-1}\text{.}$$

Therefore the proposed prior distribution is a generalisation of the default prior distribution of Sabanes-Bové and Held  [18] for any type of constraint system.

4. Example: estimating the number of injecting drug users (IDUs) in Scotland from capture–recapture data

In this section we apply our proposed default prior distribution to an incomplete contingency table which has six factors and 352 cells that involves estimating the number of injecting drug users (IDUs) in Scotland in 2006. These data have been previously analysed by King et al.  [12] and Overstall et al.  [17]. The six factors are social enquiry reports (2 levels: observed; unobserved); hospital records (2 levels: observed; unobserved); Scottish drug misuse database (2 levels: observed; unobserved); age (2 levels: ≤35 years; >35 years); gender (2 levels: male; female) and region (11 levels: National Health Service (NHS) board regions—see Fig. 1). The first three factors are sources and the 44 cells which correspond to not being observed by any of these sources for the different age/gender/region combinations have missing counts. Therefore the total population of IDUs, $N$, is unknown. We use Markov chain Monte Carlo (MCMC) methods to obtain posterior distributions for the missing cell entries and therefore a posterior distribution for the total population of IDUs.

Fig. 1.

Map showing the eleven regions of Scotland which correspond to National Health Service (NHS) board regions.

King et al.  [12] and Overstall et al.  [17] merged the eleven regions into just two levels: Greater Glasgow and Clyde, and the Rest of Scotland. Without merging, using all eleven distinct regions, there are small cell counts for many of the regions. For instance, in one region there are only 19 observed IDUs over all source, age and gender cross-classifications. This suggests that a prior distribution that involves smoothing (or borrowing of information), such as the prior proposed in Section  3, is required. We apply the proposed prior where the independence structure is specified for all of the factors except region where we use the CAR structure described in Section  3.3. By calculating the eigenvalues of the neighbourhood matrix, $G$, for this example, ${\tau }_{\min }=-0.457$ and ${\tau }_{\max }=0.247$. We place a uniform prior on $\tau$ in the interval $({\tau }_{\min },{\tau }_{\max })$. The prior distribution for each ${\beta }_t$ is

$${\beta }_t|{\sigma }_t^2,D_t\sim N(0,{\sigma }_t^2{\Sigma }_t)\text{,}$$

where ${\Sigma }_t$ is given by (6). Following from Section  3.2, we set ${\sigma }_t^2=g{q}_t/n$, with

$$g\sim \mathrm{IG}(\frac{a}{2},\frac{bn}{2})\text{,}$$

where IG denotes the inverse-gamma distribution, and $a=b=10^{-3}$, as suggested by Sabanes-Bové and Held  [18]. We only specify non-zero prior model probabilities for the log–linear models that contain at most two-way interactions and assume a discrete uniform prior over all of these models. It was found that this allowed enough complexity to obtain an adequate overall model when using the Bayesian $p$-value to assess model adequacy (see,  [8, Chapter 6]).

We use the data-augmentation MCMC approach proposed by King and Brooks  [13] with the reversible jump implementation for GLMs of Forster et al.  [6] to make moves between log–linear models and the weighted least squares Metropolis–Hastings implementation of Gamerman  [7] to make moves within the same log–linear model. We ran the algorithm for one million iterations (discarding the first 10% as burn-in).

For the total population size of IDUs, we obtain a posterior distribution for the total population size with a mean of 21 700 and a 95% highest posterior density interval (HPDI) of (18 900, 24 800). Overstall et al.  [17] obtained a posterior mean of 24 000 and a 95% HPDI of (19 500, 29 700) and King et al.  [12] a mean of 25 000 with a 95% HPDI of (20 700, 35 000). The advantage of our approach over the latter two analyses is that we are able to provide posterior distributions of the total population size in each NHS board region, broken down by age and gender. Our approach also results in a smaller credible interval for the total population size due to it allowing for correlated regions and not discarding information by merging the factor levels of region.

The posterior mean of $\tau$ is 0.108 with a 95% HPDI of $(-0.096,0.247)$. The posterior probability of $\tau$ being positive is 0.816. It follows that the Bayes factor in support of the hypothesis that $\tau >0$ is 8.205. Therefore there appears to be positive evidence  [11] in support of positive spatial correlation between the regions of Scotland.

5. Concluding remarks

In this paper we have proposed a default prior distribution for the regression parameters of a log–linear model that can take account of any dependence structure that may exist between the factor levels. This prior can be applied in situations of model uncertainty and can be seen as a generalisation of other default prior distributions applied to log–linear models including those of Dellaportas and Forster  [3], Ntzoufras et al.  [15] and Sabanes-Bové and Held  [18].

Appendix A. Justification of default prior distribution

In this appendix we give justification for the prior given in Section  3.1, given by (5), (6). The prior distribution for ${\beta }_t$ is the conditional distribution of ${\gamma }^{\left(1\right)}$ given that ${\gamma }^{\left(2\right)}=C_t{\gamma }^{\left(1\right)}$, where $\gamma ={({\gamma }^{\left(1\right)},{\gamma }^{\left(2\right)})}^T\sim N(0,{\sigma }_t^{2}M)$, and $M=P_t^T{D}_t{P}_t$. Define

$$\psi =\left(\begin{array}{c}\hfill {\psi }^{\left(1\right)}\hfill \\ \hfill {\psi }^{\left(2\right)}\hfill \end{array}\right)=\left(\begin{array}{cc}\hfill I\hfill & \hfill 0\hfill \\ \hfill -C_t\hfill & \hfill I\hfill \end{array}\right)\left(\begin{array}{c}\hfill {\gamma }^{\left(1\right)}\hfill \\ \hfill {\gamma }^{\left(2\right)}\hfill \end{array}\right)\text{,}$$

so that we now require the conditional distribution of ${\psi }^{\left(1\right)}$ given that ${\psi }^{\left(2\right)}=0$. It follows, from the properties of the multivariate normal distribution, that $\psi$ has a multivariate normal distribution with mean $0$ and covariance matrix ${\sigma }_t^{2}T$ where

$$T=\left(\begin{array}{cc}{T}_{11}\hfill & T_{12}\hfill \\ T_{21}\hfill & T_{22}\hfill \end{array}\right)\text{,}$$

and

$$T_{11}=M_{11}\text{,}$$

$$T_{12}=M_{12}-M_{11}{C}_t^T\text{,}$$

$$T_{21}=M_{21}-C_t{M}_{11}\text{,}$$

$$T_{22}=M_{22}-M_{21}{C}_t^T{C}_t{M}_{12}+C_t{M}_{11}{C}_t^T\text{.}$$

The partitioning of $M$ and $T$ follows from the partitioning of $\gamma$ into ${\gamma }^{\left(1\right)}$ and ${\gamma }^{\left(2\right)}$. Using the properties of the multivariate normal distribution the covariance matrix of ${\beta }_t$ is ${\sigma }_t^2{\Sigma }_t$, where

$${\Sigma }_t=M_{11}-(M_{12}-M_{11}{C}_t^T){(M_{22}-C_t{M}_{12}-M_{21}{C}_t^T{C}_t{M}_{11}{C}_t^T)}^{-1}(M_{21}-C_t{M}_{11})\text{.}$$

Consider the inverse of ${\Sigma }_t$. It can be shown using, e.g.,  [10], and after some matrix algebra, that

$${\Sigma }_t^{-1}=M_{11}^{-1}+M_{11}^{-1}{M}_{12}{S}_M^{-1}{M}_{21}{M}_{11}^{-1}-M_{11}^{-1}{M}_{12}{S}_M^{-1}{C}_t-C_t^T{S}_M^{-1}{M}_{21}{M}_{11}^{-1}+C_t^T{S}_M^{-1}{C}_t\text{,}$$

where $S_M=M_{22}-M_{21}{M}_{11}^{-1}{M}_{12}$ is the Schur complement (e.g.  [9, p. 95]) of $M_{11}$ in $M$. As

$$M^{-1}={\left(P_t^T{D}_t{P}_t\right)}^{-1}=L=\left(\begin{array}{cc}\hfill L_{11}\hfill & \hfill L_{12}\hfill \\ \hfill L_{21}\hfill & \hfill L_{22}\hfill \end{array}\right)\text{,}$$

then it can be shown that

$${\Sigma }_t^{-1}=L_{11}+L_{12}{C}_t+L_{21}{C}_t^T+C_t^T{L}_{22}{C}_t,=\left(I{C}_t^T\right)\left(P_t^T{D}_t^{-1}{P}_t\right)\left(\begin{array}{c}\hfill I\hfill \\ \hfill C_t\hfill \end{array}\right),=A_t^T{D}_t^{-1}{A}_t\text{.}$$

Therefore ${\Sigma }_t={\left(A_t^T{D}_t^{-1}{A}_t\right)}^{-1}$ as required.

Appendix B. Correspondence of parameters between different constraint systems

In this appendix we provide a justification of the correspondence between the regression parameters under any constraint system and sum-to-zero constraints, given by (10). Let $Z_A=(1_n,X_A)$ and $Z=(1_n,X)$ be the $n\times (p+1)$ matrices formed by appending the vector of ones to the model matrices under the alternative and sum-to-zero constraints. The vector $({\varphi }_A,{\beta }_A)$, where ${\varphi }_A$ is the intercept under the alternative constraints, is given by

$$\left(\begin{array}{c}\hfill {\varphi }_A\hfill \\ \hfill {\beta }_A\hfill \end{array}\right)={\left(Z_A^T{Z}_A\right)}^{-1}{Z}_A^{T}Z\left(\begin{array}{c}\hfill \varphi \hfill \\ \hfill \beta \hfill \end{array}\right),={\left(\begin{array}{cc}\hfill n\hfill & \hfill 1_n^T{X}_A\hfill \\ \hfill X_A^T{1}_n\hfill & \hfill X_A^T{X}_A\hfill \end{array}\right)}^{-1}\left(\begin{array}{c}\hfill n\varphi +1_n^{T}X\beta \hfill \\ \hfill \varphi X_A^T{1}_n+X_A^{T}X\beta \hfill \end{array}\right),=\left(\begin{array}{cc}\hfill \frac{1}{n}+\frac{1}{n^2}{1}_n^T{X}_A{U}_A^{-1}{X}_A^T{1}_n\hfill & \hfill -\frac{1}{n}{1}_n^T{X}_A{U}_A^{-1}\hfill \\ \hfill -\frac{1}{n}{U}_A^{-1}{X}_A^T{1}_n\hfill & \hfill U_A^{-1}\hfill \end{array}\right)\left(\begin{array}{c}\hfill n\varphi +1_n^{T}X\beta \hfill \\ \hfill \varphi X_A^T{1}_n+X_A^{T}X\beta \hfill \end{array}\right)\text{,}$$

where $U_A=X_A^T(I_n-\frac{1}{n}{J}_n)X_A$. The expression for ${\beta }_A$, given by (10), easily follows.