Modeling clustered binary data with excess zero clusters

Abstract

We establish a zero-inflated (random-effects) logistic-Gaussian model for clustered binary data in which members of clusters in one latent class have a zero response with probability one, and members of clusters in a second latent class yield correlated outcomes. Response probabilities in terms of random-effects models are formulated, and maximum marginal likelihood estimation procedures based on Gaussian quadrature are developed. Application to esophageal cancer data in Chinese families is presented.

I. Introduction

Correlated data arise in many application areas including studies of disease occurrence among family members, studies involving repeated measures of outcome on units, and studies involving group randomization. In such contexts, interest is often focused on the characterization of the dependence structure and the response probabilities. Likelihood-based approaches for analyzing correlated binary data are limited and include the works of Bonney,^1,2 Prentice,^3,4 Connolly and Liang,⁵ Bowman and George,⁶ George and Bowman,⁷ and Fitzmaurize and Laird.⁸ Developments for the regression analysis of correlated data have been concerned with methods based on specifying desired marginals and correlation structures, and fitting them using estimating equations. These approaches have attracted much attention owing to the works of Liang and Zeger,⁹ Zeger and Liang,¹⁰ Prentice and Zhao,¹¹ and Liang et al.¹² To account for correlation within clusters, random-effects models have also been proposed and have been used in various applications for correlated binary data.^13–19

The presence of binary or count data with excess zeros are also a common phenomenon in a wide variety of disciplines. A literature review²⁰ on this cited examples from a variety of research areas including agriculture, econometrics, epidemiology, public health, medicine, and social work. For example, in laboratory litter studies, it may frequently happen that some animals are unaffected by treatment—the so-called nonresponse phenomena. In correlated grouped-time survival data, some groups of individuals may be immune to the event of interest. Moreover, in genetic studies, it is often suspected that only a small subgroup of patients may have a disease gene that would be linked to a disease marker. Consequently, in studying rare, genetic, or familial diseases, data that are randomly sampled will lead to many families that are largely devoid of individuals with the attribute. In practice, the status of the excess zeros is often not observed and presents a latent nature that may complicate the data analysis. Mixture models²¹ provide a natural framework for unobserved heterogeneity in population studies and the overall distribution of disease occurrence in such data or similar outcomes should appropriately be a mixture.

Approaches for dealing with the excess zero phenomena, notably the zero-inflated and hurdle models, which are based on mixture distribution have been developed.^22–24 A zero-inflated model assumes that the zero outcomes come from two different processes, “structural” and “random.” Structural zeros can be expected in situations where a subgroup of individuals is not at risk for an outcome and thus will always provide a zero outcome. On the other hand, random zeros assume zero outcomes happen by chance due to sampling variability. A hurdle model assumes that all zero outcomes are from one structural source that is having zeros only from an at-risk subgroup of individuals. Zero-inflated models may be conceptualized as allowing zeroes to arise from at-risk and not-at-risk populations. In this article, we focus on zero-inflated models.

Hall²⁴ introduced zero-inflated binomial model for count data and incorporated random effect to accommodate correlation of outcomes in a repeated measures design. To address both excess zeros and overdispersion, Lewsey and Thomson²⁵ used zero-inflated negative binomial (ZINB) regression models in examining the effect of economic status on decay-missing-filled data in both cross-sectional and longitudinal setting and showed that they provide a better fit than ZIP models. Albert et al.²⁶ gave an excellent overview of ZINB and zero-inflated beta-binomial (ZIBB) models with discussion and application to dental caries. Hur and others²⁷ proposed a model for clustered count data with excess zeros for health outcomes research. Lee et al.²⁸ discuss multilevel zero-inflated Poisson regression modeling for correlated count data with an interesting application to bottle feeding in infants. Recently, Loeys et al.²⁹ gave a more general introduction, where Bayesian alternatives have been proposed. However, relevant references about zero-inflated models for (cross-sectional) hierarchical data are limited. Recently,Fulton et al.³⁰ proposed a mixed model and estimating equation approach for zero inflation in correlated binary response with application to dating violent data.

The current paper builds on works of zero-inflated models and introduces a zero-inflated variance component model for clustered binary data with excess zero clusters. We consider an application of disease occurrence among families for motivation and development of our methods. For most applications in family studies, the outcome is a disease status—affected or not affected, and excess zero cluster of families is a common phenomena. For example, diseases that are rare, genetic, or familial are more susceptible in certain families than others. In our case study of esophageal cancer in 2951 Chinese families,³¹ 1580 (53%) had no affected family members (see Table I). While models for handling excess zeros for count data have been studied, and while the case study presents excess zero clusters, models that accommodate such data structures for clustered binary data with covariate effects have not been well developed. Our approach considers zero inflation on the “cluster level,” meaning that all binary response of individuals in a cluster are structurally or randomly zero.

Table I.

Distribution of number of esophageal cancer cases by family size.

	Number with esophageal cancer
Family size	0	1	2	3	4	5	6	7	Total
3	435	151	34	3					623
4	536	215	52	16	6				819
5	335	203	81	34	8	0			659
6	159	155	59	27	1	4	0		412
7	74	95	46	12	5	3	0	1	232
8	30	54	25	11	2	3	1	0	129
9	6	21	10	3	1	1	0	0	43
10	4	8	6	2	1	1	1	0	23
11	0	1	1	0	1	1	1	0	8
12	0	1	0	1	0	0	0	0	2
13	1	0	0	0	0	0	0	0	1
Total	1580	906	314	110	24	13	3	1	2951

In this paper, we (i) introduce a finite mixture likelihood model for clustered binary data with excess zero clusters, (ii) model the response probabilities in terms of random effects based on Gaussian distributional assumption to allow for investigation between cluster and heterogeneity, and (iii) develop approximate maximum (marginal) likelihood procedure using Gaussian quadrature for estimation of parameters. We illustrate our approach with application to esophageal cancer data in Chinese families. Finally, we perform simulation studies to evaluate the effectiveness of the proposed model.

2. Model

Suppose data are composed of N clusters, each of size $n_i,i=1,\dots ,N$ and a vector of binary responses $Y_i={\left(Y_{i1},\dots ,Y_{i{n}_i}\right)}^T$ measured on it. If $Y={\left(Y_1,Y_2,\dots ,Y_N\right)}^T,$ then the $Y_i^{\prime }s$ are independently distributed vectors. Let п₀ and п₁ represent two latent classes where п₀ is the class (of families) whose members do not manifest the attribute under study and п₁, the class (of families) whose members are susceptible to the attribute under study. In other words, we consider a data situation whose distribution is characterized as resulting from mixing groups from two (latent) populations. We shall call п₀ the zero-vector state. Furthermore, given a cluster (or family) from the class п₁, the probability of an outcome on a member follows a Bernoulli distribution with probability of success, ${\delta }_{ij}.$ We shall call п₁ the Bernoulli state.

We define the unobserved random variable, $Z_i\left(Z_i=0,1\right),i=1,$ …, such that

$$Z_i=\{\begin{array}{ll}0,\hfill & \text{with}\text{probability}1-{\alpha }_i\hfill \\ 1,\hfill & \text{with}\text{probability}{\alpha }_i\hfill \end{array}$$

Suppose that Z_i = 0 when Y_i is generated from the zero-vector state and Z_i = 1 when Y_i comes from the Bernoulli state. Thus, 1 − α_i is the probability that a randomly choosing cluster (family) comes from the zero-vector state, п₀, and α_i, the probability that it comes from the Bernoulli state, п₁. Then for the jth outcome in the ith cluster,

$$Y_{ij}=\{\begin{array}{ll}0,\hfill & \text{with}\text{probability}1-{\alpha }_i\hfill \\ \mathrm{Ber}\left({\delta }_{ij}\right),\hfill & \text{with}\text{probability}{\alpha }_i\hfill \end{array}$$

so that

$${\delta }_{ij}=P\left(Y_{ij}=1|Z_i=1\right)$$ (1)

Hence, ${\delta }_{ij}$ is simply a conditional probability that describes an individual’s tendency to manifest the outcome given: the individual is from a susceptible family.

Also,

$$P\left(\sum _{j=1}^{n_i}{Y}_{ij}=0\right)=\left(1-{\alpha }_i\right)+{\alpha }_i{\left(1-{\delta }_{ij}\right)}^{n_i}\text{and}P\left(\sum _{j=1}^{n_i}{Y}_{ij}>0\right)={\alpha }_i\prod _{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left(1-{\delta }_{ij}\right)}^{1-y_{ij}}$$

Therefore, the joint distribution of the ith cluster is derived via a mixture formulation^24,29 and from partition theorem in conditional expectations as

$$\begin{gathered} P\left(Y_i\right)=P\left(Y_{i1}=y_{i1},\dots ,Y_{i{n}_i}=y_{i{n}_i}\right)=E\left[P\left(y_{i1},y_{i2}\dots ,y_{i{n}_i}|Z_i\right)\right] \\ =P\left(Z_i=0\right)P\left(Y_i|Z_i=0\right)+P\left(Z_i=1\right)P\left(Y_i|Z_i=1\right) \\ =\left(1-{\alpha }_i\right)1_{[\text{y}=0]}+{\alpha }_i{P}_1\left(Y_i\right) \end{gathered}$$ (2)

By assumption,

$${}_{1_{\left[Y_i=0\right]}}=\{\begin{array}{l}1,\text{if}\sum _{j=1}^{n_i}{y}_{ij}=0\hfill \\ 0,\text{if}\sum _{j=1}^{n_i}{y}_{ij}>0\hfill \end{array}=\prod _{j=1}^{n_i}\left(1-y_{ij}\right)$$ (3)

Thus $1_{\left[Y_i=0\right]}$ can be thought of as a degenerate (one-point) distribution whose values are localized at 0. In addition, we have by assumption

$$P_1\left(Y_i\right)=P_1\left(y_{i1},\dots ,y_{i{n}_i}\right)=\prod _{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left(1-{\delta }_{ij}\right)}^{1-y_{ij}}$$ (4)

Substituting equations (3) and (4) into equation (2), we establish the joint distribution for the ith cluster as

$$P\left(Y_{i1}=y_{i1},\dots ,Y_{i{n}_i}=y_{i{n}_i}\right)=\left(1-{\alpha }_i\right)\prod _{j=1}^{n_i}\left(1-y_{ij}\right)+{\alpha }_i\prod _{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left(1-{\delta }_{ij}\right)}^{1-y_{ij}}$$

Thus, the model we obtain is a mixture of a form of a degenerate distribution representing the class (of families) that does not manifest the attribute and an independent distribution representing the class (of families) whose members yield correlated outcomes. In effect, we have established a finite mixture model for clustered binary data in which members of clusters in one latent class have a zero-response vector with probability one, and clusters in the other latent class yield correlated outcomes.

2.1. Features of the joint distribution

1. From equation (5), we find the first moment, the mean of Y_ij, as

$${\mu }_{ij}=E\left(Y_{ij}\right)=P\left(Y_{ij}=1\right)={\alpha }_i{\delta }_{ij}$$

and the second moment, the variance of Y_ij, as

$${\sigma }_{ij}=\mathit{Var}\left(Y_{ij}\right)=P\left(Y_{ij}=1\right)\left\{1-P\left(Y_{ij}=1\right)\right\}={\alpha }_i{\delta }_{ij}\left(1-{\alpha }_i{\delta }_{ij}\right)={\mu }_{ij}\left(1-{\mu }_{ij}\right)$$

Therefore, an alternative parametrization of the joint distribution in terms of μ_ij and α_i will also provide interpretable parameters.

The covariance of Y_ij and $Y_{i{j}^{\prime }}\left(j\ne j^{\prime }\right),$ is given by

$$\begin{gathered} {\sigma }_{ij}=\mathrm{cov}\left(Y_{ij},Y_{ij}\right)=P\left(Y_{ij}=1,Y_{i{j}^{\prime }}=1\right)-P\left(Y_{ij}=1\right)P\left(Y_{ij}=1\right) \\ ={\alpha }_i\left(1-{\alpha }_i\right){\delta }_{ij}{\delta }_{ij}=\frac{1-{\alpha }_i}{{\alpha }_i}{\mu }_{ij}{\mu }_{ij},j\ne j^{\prime } \end{gathered}$$

It follows that the correlation between Y_ij and $Y_{ij\text{'}}$ is

$${\rho }_{j{j}^{\prime }}=\frac{{\sigma }_{j{j}^{\prime }}}{\sqrt{{\sigma }_{jj}{\sigma }_{j^{\prime }{j}^{\prime }}}}=\frac{{\alpha }_i\left(1-{\alpha }_i\right){\delta }_{ij}{\delta }_{i{j}^{\prime }}}{\sqrt{\left(1-{\alpha }_i{\delta }_{ij}\right)\left(1-{\alpha }_i{\delta }_{i{j}^{\prime }}\right)}}=\frac{1-{\alpha }_i}{{\alpha }_i}\frac{{\mu }_{ij}{\mu }_{i{j}^{\prime }}}{\sqrt{\left(1-{\mu }_{ij}\right)\left(1-{\mu }_{i{j}^{\prime }}\right)}}$$ (6)

This would allow us to write the joint distribution in terms of means and correlations, if desired. In the absence of cluster-level and individual-level covariates, we set α_i = α and μ_{ij =} μ₀ Let ρ₀ denote the correlation in the absence of individual-level covariates. Then substitution in equation (6) gives

$$\alpha ={\left\{1+\frac{{\rho }_0}{{\mu }_0/\left(1-{\mu }_0\right)}\right\}}^{-1}$$

So, α is determined by the ratio of the baseline intraclass correlation to the population odds. We note an inverse relationship between ρ₀ and α. In particular, as ${\rho }_0\to 0,\alpha \to 1.$

1. If ${\alpha }_i\to 1,{\delta }_{ij}\to {\mu }_{ij},$ the joint distribution, equation (5), of the ith cluster reduces to

$$P\left(Y_{i1}=y_{i1},\dots ,Y_{i{n}_i}=y_{n_i}\right)=\prod _{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left(1-{\delta }_{ij}\right)}^{1-y_{ij}}\to \prod _{j=1}^{n_i}{\mu }_{ij}^{y_{ij}}{\left(1-{\mu }_{ij}\right)}^{1-y_{ij}}$$

This is the joint distribution of uncorrelated Bernoulli random variables, which in most applications is the null hypothesis of independence of outcomes within cluster.

Moreover, it can easily be shown from equations (1) and (5) that for the ith cluster,

$$P\left(Y_{ij}=1,Y_{i{j}^{\prime }}=1\right)=\frac{1}{{\alpha }_i}P\left(Y_{ij}=1|Y_{i{j}^{\prime }}=1\right),j\ne j^{\prime }$$

Clearly, .α_{i =} 1, implies independence of outcomes. Thus, α_i may be interpreted as a measure of cluster dependence. We shall term the parameter, α_i, the relative cluster dependence parameter.

1. The joint distribution is interchangeable, that is, the order of the response does not matter. This property makes the models suitable for analyzing both ordered and unordered data structures.

1. The joint distribution is reproducible. That is, the marginal distribution of any subset of $\left(Y_1,\dots ,Y_n\right)$ has the same form as the joint distribution. In other words, the interpretation of the parameters is the same regardless of cluster size.³² Hence, the situation where a subset of the cluster is missing causes no problem should this approach be adopted, provided the missing process is ignorable.¹²

1. Let $A_i={\sum }_{j=1}^{n_i}{y}_{ij}$ represent the number of successes of the outcomes in a cluster of size $n_i.$ Then the distribution of $a_i={\sum }_{j=1}^{n_i}{y}_{ij},$ is given as

   $$P\left(A_i=a_i\right)=\sum _{A_i=a_i}\left\{\left(1-{\alpha }_i\right)\prod _{j=1}^{n_i}\left(1-y_{ij}\right)+{\alpha }_i\prod _{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left(1-{\delta }_{ij}\right)}^{1-y_{ij}}\right\},a_i=\sum _{j=1}^{n_i}{y}_{ij}$$

The mean and variance are, respectively,

$$E\left(A_i\right)={\alpha }_i\sum _{j=1}^{n_i}{\delta }_{ij}\text{and}var\left(A_i\right)={\alpha }_i\left\{\sum _{j=1}^{n_i}{\delta }_{ij}\left(1-{\delta }_{ij}\right)+\left(1-{\alpha }_i\right){\left(\sum _{j=1}^{n_i}{\delta }_{ij}\right)}^2\right\}$$

In the absence of individual specific covariates, or if they are not of interest, we set ${\delta }_{ij}=\delta =\frac{\mu }{\alpha },$ a constant. Then,

$$P\left(A_i=a_i\right)=\{\begin{array}{cc}1-\alpha +\alpha {(1-\delta )}^{n_i},& \text{if}{a}_i=0\\ \left(\begin{array}{c}{n}_i\\ a_i\end{array}\right)\alpha {\delta }^{a_i}{(1-\delta )}^{n_i-a_i},& \text{if}{a}_i>0\end{array}$$

Because μ = αδ, we can use an alternative parametrization in terms of μ and α as

$$P\left(A_i=a_i\right)=\{\begin{array}{cc}1-\alpha +\alpha {\left(1-\frac{\mu }{\alpha }\right)}^{n_i},& \text{if}{a}_i=0\\ \left(\begin{array}{c}{n}_i\\ a_i\end{array}\right)\alpha {\left(\frac{\mu }{\alpha }\right)}^a{\left(1-\frac{\mu }{\alpha }\right)}^{n_i-a_i},& \text{if}{a}_i>0\end{array}$$

The mean and variance of the number of successes of outcome becomes, respectively,

$$E\left(A_i\right)=n_i\alpha \delta =n_i\mu \mathrm{var}\left(A_i\right)=n_i\mu (1-\mu )+n_i\left(n_i-1\right){\mu }^2\left(\frac{1}{\alpha }-1\right)$$

The first term of the variance may be thought of as the binomial component (the independent component) and the second term the extra-binomial variation due to dependence within the clusters analogous to that described in the studies by Williams³³ and Moore.^34,35 If μ < α < 1, var(A) exceeds that of the binomial and so we have positive clustering. In the case of independence α = 1, δ = μ and so the equations reduce to

$$P\left(A_i=a_i\right)=\left(\begin{array}{c}{n}_i\\ a_i\end{array}\right){\mu }^{a_i}{(1-\mu )}^{n_i-a_i},\text{if}{a}_i=0,1,2\dots ,n_i$$

We obtain the classical binomial distribution with probability of success μ. The variance reduces to that of the binomial, so that α may be interpreted as a measure of overdispersion with respect to the corresponding binomial distribution.

1. Let the number of successes, $0<a_i={\sum }_{j=1}^{n_i}{y}_j<n_i.$ Then, the conditional distribution given at least one success is

$$P\left(Y_1=y_1,\dots ,Y_{n_i}=y_{n_i}|a_i>0\right)=\frac{\sum _{A_i=a_i}\prod _{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left(1-{\delta }_{ij}\right)}^{1-y_{ij}}}{1-\prod _{j=1}^{n_i}\left(1-{\delta }_{ij}\right)}$$

We obtain a form of a weighted distribution, where the relative dependence parameter, α, appropriately drops off. The conditional distribution is analogous to the conditional likelihood model first described by Cox^36,37 for binary outcomes.

2.2. Parametrization for regression analysis

Suppose the jth subject has a vector of p individual-specific covariates, $X_{ij}=\left(x_{ij1},\dots ,x_{ijp}\right).$ Let the ith cluster has q cluster-specific covariates $W_i=\left(w_{i1},\dots ,w_{iq}\right)\text{and}{Z}_i\left(Z_i=0,1\right),i=1,\dots ,$ an unobserved random variable. The scientific objective is to characterize the dependence of Y_ij on X_ij and W_i. The observations may be taken to be independent between clusters, but certain within-cluster and within-individual correlations are anticipated, which can be modeled explicitly through random effects attached to the linear predictors. Therefore, to account for excess unobserved variation, not accounted for by cluster-specific covariates, we introduce a random-effect term. The logit model with Gaussian random effects has been studied extensively^{18,19,27,38–40} and will be adopted. Thus, we model the logit of the parameter, δ_ij, as

$${\delta }_{ij}(X)=P\left(Y_{ij}=1|Z_i=1,X_{ij}\right)=\frac{1}{1+\exp \left[{\beta }_0+\beta X_{ij}+{\gamma }_i\right]}$$

where ${\gamma }_i~N\left(0,{\sigma }^2\right)$ is an unobservable random effect assumed to have a Gaussian distribution with mean zero, and variance σ², to account for excess variation across cluster. In a similar manner, we could model the parameter α_i via a logit link as a function of cluster-specific covariates, W_i, but without random effects. That is,

$${\alpha }_i(W)=\left[P\left(Z_i=1|W_i\right)\right]=\frac{1}{1+\exp \left[{\text{λ}}_0+\text{λ}{W}_i\right]}$$

This is necessary, because α_i has an embedded property to measure within-cluster dependence, and heterogeneity is accounted for in the modeling for δij Hall²⁵ suggested similar parametrization in a random-effects zero-inflated count model for a repeated measures design.

In the absence of cluster-specific covariates, W_i = 0, thus, we have

$${\alpha }_i(W)=\left[P\left(Z_i=1|W_i\right)\right]=\frac{1}{1+\exp \left[{\text{λ}}_0\right]},{\text{λ}}_0\text{constant}$$

In this case, we shall model α_i = α, a constant, without loss of generality. The constant relative dependence model can be regarded as the analog of the desirable homoscedastic model in general linear models and to the equal correlation assumption made by Prentice⁶ in regard to the derivation of the extended beta-binomial distribution. An alternative approach would be to leave α_i unrestricted (dependent on i) by simply imposing a distribution on it for estimation purposes.

3. Parameter estimation

Different methods for estimating parameters in random-effects models have been proposed by several authors.^40–43 Pinheiro and Bates¹⁹ gave a detailed description of approximate log-likelihood functions for mixed-effect models, comparing their computational and statistical properties and concluded the Gaussian quadrature with a reasonable number of abscissas are quite accurate and efficient. When the dimension of the random effects is one or two, numerical integration techniques can be implemented reasonably easily and will be used.^21,44,45 Our specific application does not include cluster-specific covariates, and,therefore, we shall model α_i = α, a constant, without loss of generality. As indicated in the previous section, the constant relative dependence model can be regarded as the analog of the desirable homoscedastic model in general linear models.

Let $\theta =\left\{\alpha ,{\beta }_0,\beta ,{\sigma }^2\right\}$ be the parameters to be estimated, then the conditional distribution of $Y={\left(Y_1,\dots ,Y_N\right)}^T,\gamma =\left({\gamma }_1,{\gamma }_2,\dots {\gamma }_N\right)$ is

$$P(Y|\gamma ,\theta )=\prod _{i=1}^N\left\{\left(1-{\alpha }_i\right)\prod _{j=1}^{n_i}\left(1-y_{ij}\right)+{\alpha }_i\prod _{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left[1-{\delta }_{ij}\right]}^{1-y_{ij}}\right\},{\gamma }_i~N\left(0,{\sigma }^2\right)$$ (7)

The (marginal) distribution of Y is found by integrating out the conditional distribution (equation (7)) with respect to the unobserved random variable γ_i and is given by

$$P(Y|\theta )={\displaystyle \prod }_{i=1}^N\left\{\left(1-{\alpha }_i\right)\prod _{j=1}^{n_i}\left(1-y_{ij}\right)+{\alpha }_i{\int }_{-\infty }^{\infty }\left[\prod _{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left[1-{\delta }_{ij}\right]}^{1-y_{ij}}f\left({\gamma }_i;{\sigma }^2\right)\right]\text{d}{\gamma }_i\right\}$$

For convenience, we consider $V_i={\gamma }_i/\sigma ,\text{then}{V}_i~N(0,1),$ and so

$$P(Y|\sigma ,\theta )={\displaystyle \prod }_{i=1}^N\left\{\left(1-{\alpha }_i\right)\prod _{j=1}^{n_i}\left(1-y_{ij}\right)+{\alpha }_i{\int }_{-\infty }^{\infty }\left[\prod _{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left(1-{\delta }_{ij}\right)}^{1-y_{ij}}\varphi \left(V_i\right)\right]\text{d}{V}_i\right\}$$ (8)

where ϕ(.) is the standard normal density and

$${\delta }_{ij}\left({\beta }_0,\beta ,\sigma \right)=\frac{1}{1+\exp \left[-\left({\beta }_0+\beta X+\sigma V_i\right)\right]}$$

Since the integrals, ${\int }_{-\infty }^{\infty }\left[{\prod }_{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left(1-{\delta }_{ij}\right)}^{1-y_{ij}}\varphi \left(V_i\right)\right]\text{d}{V}_i,$ are over Gaussian densities, Gaussian quadrature would be appropriate and is used. Using an M-point Gaussian quadrature, an integral of the form ${\int }_{-\infty }^{\infty }f(v)\varphi (v)dv,wheref(v)=\prod _{j=1}^{n_i}{\delta }_{ij}^{y_{ij}}{\left(1-{\delta }_{ij}\right)}^{1-y_{ij}}\varphi \left(V_i\right)\text{and}\varphi (v)$ is the standard normal density approximated by the weighted sum

$${\int }_{-\infty }^{\infty }f(v)\varphi (v)dv\approx \frac{1}{\sqrt{\pi }}\sum _{m=1}^M{w}_{m}f\left(\sqrt{2}{v}_m\right)$$

where v_m are the Gaussian quadrature points and w_m, the associated weights. The terms v_m and w_m are available from Abramowitz and Stegun.⁴⁶ Applying this to the log-likelihood of equation (8), we have

$$\log L(\beta ,U|Y))\approx {\displaystyle \sum }_{i=1}^N\log \left\{\left(1-{\alpha }_i\right)\prod _{j=1}^{n_i}\left(1-y_{ij}\right)+\frac{{\alpha }_i}{\sqrt{\pi }}\sum _{m=1}^M{w}_m\left[\prod _{j=1}^{n_i}{\left({\delta }_{ijm}\right)}^{y_{ij}}{\left(1-{\delta }_{jim}\right)}^{1-y_{ij}}\right]\right\}$$

where

$${\delta }_{ijm}\left({\beta }_0,\beta ,\sigma \right)=\frac{1}{1+\exp \left\{{\beta }_0+\beta X_{ij}+\sqrt{2}\sigma v_m\right\}}$$

Maximum likelihood using Newton–Raphson algorithm can be used to estimate the parameters. Im and Gianola⁴⁴ recommend using a small number of quadrature points as possible. In principle, one could also use an EM algorithm combined with Gaussian quadrature,^44,47 but notably, the EM has the disadvantage of not readily providing standard errors⁴⁴ of the parameter estimates, and convergence is usually slow in the absence of close form solutions and therefore will not be discussed further in this application.

4. Model selection

Akaike’s information criterion⁴⁸ (AIC), $\text{AIC}=-2*log(\text{Likelihood})+2*$ (number of parameters), values was used to compare model performance. Smaller values of AIC indicate a better fit. For nested models, the likelihood-ratio test was used to compare whether one model was statistically superior to the other at the significant level of 0.05. For non-nested fitted models that had similar AIC, the Vuong⁴⁹ test was employed to further test for significant differences. The Vuong statistics⁴⁹ has been widely used for model performance in many fields^22,50 and is defined as

$$V=\frac{L{R}_n\left({\widehat{\theta }}_1,{\widehat{\theta }}_2\right)}{\sqrt{n}\varpi }$$

where n is the number of observations, $\widehat{{\theta }_1},\widehat{{\theta }_2}$ represent fitted parameter vectors from two comparable models, and ϖ is the standard deviation of the log-likelihood difference between two models, which is given by

$${\varpi }^2=\mathrm{var}\left[\log L^1\left({\widehat{\theta }}_1\right)-\log L^2\left({\widehat{\theta }}_2\right)\right]$$

where L¹ and L² are the two likelihood functions for the two comparable models and

$$L{R}_n=\sum _{i=1}^n\left(\log \frac{L_i^1}{L_i^2}\right)=\sum _{i=1}^n\left[\log L_i^1\left({\widehat{\theta }}_1\right)-\log L_i^2\left({\widehat{\theta }}_2\right)\right]$$

The Vuong statistics, V, asymptotically follows a standard normal distribution and is implemented in the R program. Therefore, at a 5% significance level, $|V|<1.96$ will suggest no significant difference between the two models. In this application, the ZIBB model is the baseline for comparison with the proposed models. Therefore, V < −1.96 will suggest the model is significantly better than the ZIBB, and V > 1.96 will favor ZIBB.

5. Application to esophageal cancer data in Chinese families

This application involves the study of esophageal cancer in 2951 nuclear families collected in Yangcheng County, Shanxi Province, Peoples Republic of China.³¹ Esophageal cancer, a gastointestinal cancer, is one of the deadliest cancers worldwide and more commonly seen in China than other parts of the world.^50–52 The main objective of the study was to assess the presence of familial aggregation of esophageal cancer. In this analysis, we consider as a cluster, the family unit, and assess aggregation of the disease adjusting for measured risk factors. Table I summarizes the distribution of the number of affected individuals by family size. Of the 2951 families, 1371(47%) had at least one affected member and 1580(53%) had no affected members—presenting a substantial number of “excess zero” clusters. The respondent within a family has correlated outcomes which are influenced in part or wholly by the cluster as well as the variables on the individual respondent. The following covariates are available: SEX is coded as 0 for female and 1 for male, Age is years centered at 50, Smoke is coded as 0 for nonsmoker and 1 for smoker, and Alcohol is coded 0 for nondrinker and 1 for drinker. The outcome variable, Y, is coded 1 if an individual is affected with esophageal cancer and 0 otherwise.

Assuming a single random effect, the general (full) model for predicting an individual’s response to esophageal cancer, accounting for potential family to family heterogeneity while adjusting for the excess zero clusters is given as:

$$\begin{array}{l}logit\left[{\delta }_{ij}|Z_i=1,X,{\text{λ}}_i\right]={\beta }_0+{\beta }_{1}Sex+{\beta }_{2}Age+{\beta }_{3}Smoke+{\beta }_{4}Alcohol+{\text{λ}}_i,\\ {\text{λ}}_i~N\left(0,{\sigma }^2\right),\text{and}0<\alpha \le 1.\end{array}$$ (9)

We shall call equation (9) the zero-inflated logistic-Gaussian (ZILG) model. Table 2 describes specific fitted derived models. In addition, we fit the data using a traditional zero-inflated approach, ZIBB,²⁶ and compare the results and conclusions to that obtained using the proposed models for clustered binary outcomes. The ZIBB model is appropriate because the total number of cancer cases for a family is bounded by the family size.

Table 2.

Derived models and description from the proposed ZILG model.

Model	Model expression	Model description
ZILG	$\begin{array}{l}\mathrm{Logit}\left({\delta }_{ij}|X,{\text{λ}}_i\right)={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\beta X_3+\beta X_4+{\text{λ}}_i\\ {\text{λ}}_i~N\left(0,{\sigma }^2\right)\\ 0<\alpha \le 1\end{array}$	This is a random-effects model that assumes correlation of outcomes within families and further tests for excess random variation across families, adjusting for excess zero clusters
SL	$\begin{array}{l}\mathrm{Logit}\left({\delta }_{ij}|X\right)={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\beta X_3+\beta X_4\\ \alpha =1\end{array}$	This is the complete independence model, which is the basic null hypothesis
SLG	$\begin{array}{l}\mathrm{Logit}\left({\delta }_{ij}|X,{\text{λ}}_i\right)={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\beta X_3+\beta X_4+{\text{λ}}_i\\ {\text{λ}}_i~N\left(0,{\sigma }^2\right)\\ \alpha =1\end{array}$	This is a random-effects model that assumes correlation of outcomes within families arises from unobserved random variation across families
ZIL	$\begin{array}{l}\mathrm{Logit}\left({\delta }_{ij}|X\right)={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\beta X_3+\beta X_4\\ 0<\alpha \le 1\end{array}$	This is a fixed-effect model that assumes correlation of outcomes within families, adjusting for excess zero clusters

For the application: X₁: Sex; X₂: Age; X₃: Alcohol; X₄: Smoke. SL: standard logistic; SLG: standard logistic Gaussian; ZIL: zero-inflated logistic; ZILG: zero-inflated logistic Gaussian.

Computations of the proposed models were performed using computer programs we developed which was linked with likelihood optimization routines using the R-package (Version 3.2.5). Appendix 1 provides the R-code. For our proposed models, the results did not change much for quadrature points, M>9 and so for computations, M=9 was employed to complete the analysis. In all of the models we have considered, the algorithm converged in less than 38 iterations.

Results of the fitted models are presented in Table 3. Within family dependence is described by the magnitude of the relative dependence parameter, α, and excess familial heterogeneity by the magnitude of, σ, the variance component parameter.

Table 3.

Estimates and standard errors of the regression analysis of esophageal cancer in Chinese families.

	Model I: SL	Model II: SLG	Model III: ZIL	Model IV: ZILG	Model V: ZIBB
Parameter	Estimate±SE	Estimate±SE	Estimate±SE	Estimate±SE	Estimate±SE
Mean risk: β0	−5.235±0.116	−4.494±0.109	−4.152±0.109	−4.272±0.141	−4.549±0.792
Individual covariates
Sex: β1	0.982±0.052	0.908±0.059	0.834±0.058	0.811±0.059	0.897±0.059
Age: β2	0.037±0.003	0.038±0.002	0.037±0.002	0.036±0.004	0.0381±0.002
Alcohol: β3	−1.175±0.171	−1.145±0.173	−1.046±0.169	−0.968±0.146	−1.1098±0.169
Smoke: β4	0.056±0.066	0.057±0.067	0.058±0.059	0.064±0.053	0.095±0.059
Dependence (mixing)	–	–	0.716±0.023	0.619±0.024	0.742±0.161
parameter: α
Dispersion parameter: ϕ					3.129
Variance component: σ		0.732±0.074		0.771±0.085	–
AIC	10,759.08	10,748.06	10,729.32	10,714.16	10,739.95
Vuong (V)a	−10.33	1.56	2.56	5.22
Likelihood ratio χ2(df)b		13.02(1)	31.76(1)	46.92(2)
p value		<0.001	<0.0001	<0.0001

AIC: Akaike’s information criterion; SE: standard error; SL: standard logistic; SLG: standard logistic Gaussian; ZIBB: zero-inflated beta-binomial; ZIL: zero-inflated logistic; ZILG: zero-inflated logistic Gaussian.

^aVuong test for comparing indicated model to ZIBB model (model V).

^bLikelihood ratio and p value for comparing the indicated models to the standard logistic regression model (model I), which represent the basic null hypothesis of independence.

Compared with the standard logistic (independence) model, the likelihood-ratio chi-square statistics show significance of the standard logistic-Gaussian (SLG) model (p<0.01), zero-inflated logistic (ZIL) (fixed effect) model (p<0.0001) and the ZILG model (p<0.0001). Both zero-inflated models (models III and IV) fit the data better than the SLG model (model II) based on the AIC. The likelihood-ratio test indicates that the ZILG (random-effects) model fits the data better than the ZIL (fixed effect) model (p<0.001), indicating significant variation of outcomes across families. It should be noted that the use of the likelihood-ratio test for testing variance components has been called into question, with some advocating halved p values for such testing of variance component parameters.^53,54 Therefore, we apply the correction of doubling the p values. In this analysis, we also notice that the difference in log-likelihood values between the zero-inflated fixed-effect model and random-effects models is large, relative to the number of degrees of freedom, so that the preference of the random-effects model is unequivocal.

The ZILG model was therefore the best fitted model which was also confirmed by the Vuong statistic (see Table 3) and thus selected. For this model, the maximum likelihood estimate of the relative dependence parameter, α, was 0.619 and a computed confidence interval (CI) of (0.582, 0.849): This suggests that the data were sampled from a population where the aggregation of esophageal cancer is higher than that from the general population. Similarly, the estimated, the variance component parameter, is 0.771 with CI of (0.605, 0.939) which excludes 0. Thus the data further suggests some degree of heterogeneity of outcomes across families.

At the individual covariate level, Sex and Age have positive significance while Alcohol has negative significance. Smoking was not significant at the 5% level. We conclude that, given a family member has esophageal cancer, the males were at a higher risk (more susceptible) of getting esophageal cancer than the females of the same family and that the older the family member, the more susceptible they become. The negative effect of Alcohol means that, it has a propensity to lower the risk of esophageal cancer. Although the amount (number of drinks) of alcohol drank was not available for this analysis, we can speculate that the significance could be due to moderation in drinking.

In comparison, the parameter estimate of the mean risk, β₀, is much larger in the logistic (independence) model and the logistic-Gaussian model than the zero-inflated models. The parameter estimates in both fixed- and random-effects zero-inflated models are relatively close for sex, age, and smoke. However, the parameter estimate for alcohol in the random-effects model is appreciably smaller (in absolute terms) compared to the estimate in the fixed-effects model. The estimates from ZIBB give estimates that are not much different from the standard logistic regression. The approach seems to underestimate the parameters in the model. This may be due to the fact that the data were not randomly sampled from the population but from families where there is a high risk of aggregation of the occurrence of esophageal cancer. In summary, we conclude that esophageal cancer aggregates in the families sampled.

6. Simulation studies

To evaluate the efficiency and robustness of the proposed method, we generate a correlated binary data that emulates the real data situation for simulation studies. We consider data comprise of N = 400 clusters each of cluster size, n_{i =} 5, i = 1,…,N. Similar to the approach by Fulton et al.,³⁰ we generated two continuous individual-level covariates, (X₁,X₂) from a standard normal distribution and two binary categorical covariates, (X₃,X₄). We assume absence of cluster-specific covariates, so that the zero-inflation probability, α_{i =} α, a constant. Consequently, the zero-inflation indicator, Z_i, was generated from Bernoulli(α), with α = 0.6. To induce a correlated data structure, the binary outcome, Y_ij, was generated from a probit randomeffects model³⁰:

$$P\left(Y_{ij}=1|Z_i=1,X_{ij},{\gamma }_i\right)=\Phi \{{\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+{\beta }_3{X}_3+{\beta }_4{X}_4)+{\gamma }_i\},{\gamma }_i~N\left(0,{\sigma }^2\right)$$

We fixed the regression parameters such that

$$\left({\beta }_0,{\beta }_1,{\beta }_2,{\beta }_3,{\beta }_4\right)=(0.0,0.5,-1.4,1.2,0.2).$$

The variance component σ² was taken to be 0.5² to reflect a moderate within-cluster correlation. The simulations were repeated 100 times to compare the performance of the ZILG, ZIL, SLG, and ZIBB models. We calculated the average (Mean) of the estimated parameters, and average of the estimated standard errors (SEs). Furthermore, the mean square error, MSE, of the estimated parameters for each model is evaluated. The simulation results are shown in Table 4. Both the ZILG and ZIL models perform reasonably well, in terms of small MSE compared to the ZIBB model. As expected, the ZILG was almost unbiased for estimating s. The SLG model over estimated s more than two-fold. The ZIBB model tend to understate the true parameter values.

Table 4.

Simulation statistics for average (Mean) parameter estimates, SE, and MSE across 100 simulated data sets.

	Parameter	True value $(\theta )$	Mean $(\widehat{\theta })$	SE	$(\theta -\widehat{\theta })$	MSE
ZILG	α	0.600	0.648	0.014.	−0.048	0.0029
	B0	0.000	0.041	0.001	−0.041	0.0159
	B1	0.500	0.482	0.042	0.018	0.0021
	B2	−1.425	−1.478	0.027	0.053	0.0050
	B3	1.233	1.346	0.046	−0.113	0.0149
	B4	0.200	0.189	0.029	0.011	0.0010
	σ	0.500	0.534	0.062	−0.034	0.0049
ZIL	α	0.600	0.764	0.231	−0.164	0.0275
	B0	0.000	0.004	0.041	−0.004	0.0102
	B1	0.500	0.558	0.063	−0.058	0.0051
	B2	−1.425	−1.493	0.034	0.068	0.0068
	B3	1.233	1.412	0.049	−0.179	0.0142
	B4	0.200	0.177	0.032	0.023	0.0014
SLG	B0	0.000	0.004	0.071	−0.004	0.0105
	B1	0.500	0.688	0.051	−0.188	0.0371
	B2	−1.425	−1.578	0.045	0.153	0.0256
	B3	1.233	1.239	0.061	−0.006	0.0163
	B4	0.200	0.197	0.034	0.003	0.0089
	σ	0.500	1.216	0.278	−0.716	0.5164
ZIBB	α	0.600	0.491	0.052	0.109	0.0125
	B0	0.000	−0.066	0.165	0.066	0.0146
	B1	0.500	0.713	0.049	−0.213	0.0471
	B2	−1.425	−1.278	0.075	−0.147	0.0238
	B3	1.233	1.095	0.146	0.138	0.0212
	B4	0.200	0.317	0.045	−0.117	0.0145

MSE: mean square error; SE: standard error; SLG: standard logistic-Gaussian model; ZIBB: zero-inflated beta-binomial model; ZIL: zero-inflated logistic model; ZILG: zero-inflated logistic-Gaussian model.

In estimating, α, the zero-inflated probability, the SE of ZILG is the smallest. The estimates of the ZILG model were closest to the true value and the most efficient as they make use of the full distributional assumptions of the observed data.

7. Conclusion

This article has been concerned with development of a finite mixture likelihood formulation for clustered binary data with excess zero clusters, a data structure in which all members of clusters in one latent class have a zero response with probability one, and clusters in the other latent class yield correlated outcomes. The development, albeit being straightforward and based on simple analytic formulation, is novel and well suited for areas of application including public health and biomedical research. For example, in correlated grouped-time survival data,⁵⁵ some groups of individuals may be immune to the event of interest. Through the simple simulation studies and real data analyses, we demonstrate that, the proposed ZILG random-effects model provides a useful tool for analyzing clustered binary data with excess zero clusters.

Often, because of the hierarchical study design or the data collection procedure, zero inflation and lack of independence may occur simultaneously, which render the standard methods inadequate. The relative errors incurred by ignoring the adjustment of excess zeros can be problematic even for a small number of groups with zeros, if traditional modeling methods are used.⁵⁶ The advantage of the proposed model is that it accounts for within-cluster (family) dependence and provides a good portrayal of cluster (family) differences while adjusting for excess zero clusters.

To our knowledge, the work we have established in this article is the first likelihood formulation for modeling clustered binary data with excess zero clusters where the zero-inflation is at the “cluster level” meaning that all binary responses of individuals in a cluster are structurally zero. It is possible, however, that closer scrutiny, practical considerations, and numerical studies for numerical stability would suggest modifications and/or refinements to the methods discussed. In this article, we only intend to accomplish the first stage of Fisher’s paradigm for the development of statistical methods, namely, the specification of the model which is the development of the joint distribution (or likelihood function) for the data. Future studies will include further simulation studies to evaluate the small sample properties of the likelihood-ratio test for evaluating the hypotheses of independence of outcome within a cluster, conduct sensitivity analyses of the models with different zero-inflation probabilities as well as when the zero inflation depends on covariates. Extensions and applications to polytomous outcomes, correlated survival outcomes, and hierarchical outcomes can be considered for future research. In conclusion, we remark that the proposed finite mixture model for correlated binary data is suitable and computationally tractable for the regression analysis of binary data with zero-inflated clusters with covariate effects.

APPENDIX A: R Code for likelihood evaluation and optimization

## PART A: HOUSEA. r Subroutine to read the datafile
Amat1=read.table(“c:/DATAESOC/esocg3.txt”,header=T,sep=““)
cmpa5<–function(rw1) {
uu1=1.0 – rw1[,2]
return(uu1)
}
rwt3=adply(Amat1,1,cmpa5)
prody1=ddply(rwt3,.(famid),summarise,pdy1=min(V1))
rm(rwt3)
##Reading Gaussian quadrature nodes and weights; nn = number of quadrature points
nn=9
nnp1=nn+1
mat1=matrix(NA,nrow=nrow(Amat1),ncol=nnp1)
mat1=data.frame(mat1)
xxx=gauss.quad(nn,”hermite”)
vvm=xxx$nodes
vvw=xxx$weights
mat1[,1]=Amat1[,1]
aa1=unique(Amat1[,1])
aa2=length(aa1)
mat2=matrix(NA,nrow=aa2,ncol=nnp1)
mat2=data.frame(mat2)
## PARTB: COMPUTEA. r – Subroutine to evaluation the likelihood function
loglike2 <– function(soln){
cmpwf<–function(rw5){
sum1=0.0
for(j in1:nn){
sum1=sum1+vvw[j]*rw5[,j+1]
}
return(sum1)
}
cmp1<–function(rw1){
y1=rw1[,2]
sum1=soln[1]+ soln[2]+soln[3]*rw1[,3]+soln[4]*rw1[,4]+soln[5]*rw1[,5]+soln[6]*vvm[j]*sqrt(2)
## rw1[,kk]reads variables from column kk; 2.To run the zero–inflated logistic model set vvm [j] to 0
if (y1 == 0)
{
sum2= −log(1.0 + exp(sum1))
} else
{
sum2= −log(1.0 + exp(sum1))
}
}
for(j in1:nn){
rwt2=adply(Amat1,1,cmp1)
mat1[,j+1]=rwt2$V1
}
famid=rwt2[,1]
for(j in1:nn){
uu20=mat1[,j+1]
xx99=data.frame(famid,uu20)
logv1=ddply(xx99,.(famid),summarise,logsum1=sum(uu20))
mat2[,j+1]=logv1$logsum1
}
rm(uu20) famid=unique(Amat1[,1]) mat2[,1]=famid for(j in1:nn){
mat2[,j+1]=exp(mat2[,j+1])
}
rwt9=adply(mat2,1,cmpwf)
famid=rwt9[,1]
nucsum=rwt9$V1
mat3=cbind(famid,nucsum)
# alpha
 alphac1<−function(rw1){
sumbb=1.0 + exp(−(soln[1]+soln[2]))
sumaa=1.0 + exp(−soln[1])
sumcc=sumbb/sumaa
if(sumcc>1)sumcc=1.0
return(sumcc)
}
# SET alpha = sumcc = 1,to run logistic Gaussian model
 alphav1=ddply(Amat1,.(famid),alphac1)
xx1=merge(mat3,alphav1,by=“famid”)
xx2=merge(xx1,prody1,by=“famid”)
cmpall<−function(rw1){
K0=1.0 – rw1[,3]
If(rw1[,3] > 1)K0=1
aa1= (rw1[,3]*rw1[,2])/sqrt(pi)
aa3=K0*rw1[,4] + aa1
if(aa3 <= 0.0){
return(NA)
}
aa2= log(aa3)
return(aa2)
}
logfamid=ddply(xx2,.(famid),cmpall) logvalue=sum(logfamid[,2])
return(logvalue)
}
# PART C: Main program WHOLEA.R. This links relevant subroutines to perform optimization.
rm(list=ls())
# source(“c:/DATAESOCC/wholea.r”,echo=T)
# main program wholea.r
# sub programs housea.r and computea.r
# COMPUTEA.r – likelihood function
# HOUSEA.r – read data file and weights of Gaussian Quadrature
# data file – esocg3.txt
library(maxLik)
library(plyr)
library(gee)
library(lme4)
library(statmod)
source(“c:/DATAESOC/HOUSEA.r”)
source(“c:/DATAESOC/COMPUTEA.r”)
initial1=c(0.0,0.0,0.0,0.0,0.0) res2=maxBFGS(loglike2,start=initial1,print.level=1)
aa1=solve(−res2$hessian)
aa2= sqrt(diag(aa1))