Modification of the generalized quasi-likelihood model in the analysis of the Add Health study

Abstract

The relationship between the mean and variance is an implicit assumption of parametric modeling. While many distributions in the exponential family have a theoretical mean-variance relationship, it is often the case that the data under investigation are correlated, thus varying from the relation. We present a generalized method of moments estimation technique for modeling certain correlated data by adjusting the mean-variance relationship parameters based on a canonical parameterization. The proposed mean-variance form describes overdispersion using two parameters and implements an adjusted canonical parameter which makes this approach feasible for all distributions in the exponential family. Test statistics and confidence intervals are used to measure the deviations from the mean-variance relation parameters. We use the modified relation as a means of fitting generalized quasi-likelihood models to correlated data. The performance of the proposed modified generalized quasi-likelihood model is demonstrated through a simulation study and we highlight the importance of accounting for overdispersion in the evaluation of adolescent obesity data collected from a U.S. longitudinal study.

1. Introduction

As a common statistical measure, the variance is often relied on to evaluate the model fit and to understand the differences between the responses through the construction of test statistics and confidence intervals. The form of the variance is often assumed based on the underlying distribution of the responses. In fact, the variance is related to the mean for most distributions in the exponential family. However, while the responses may be on a certain scale or resemble a certain distribution, extraneous variation can impact the mean-variance relationship. Extraneous variation, or so-called overdispersion, is often present in longitudinal or clustered data arising from a hierarchical data structure.

Ignoring overdispersion in the fit of correlated data results in summary statistics, including test statistics, with a larger variance than expected.¹ It often leads to a loss of efficiency in using statistics appropriate for the single-parameter family.² Studies have shown that ignoring overdispersion and thereby misspecifying the model biases the covariate effects and greatly impacts the standard error of the coefficients.^{3, 4} While underdispersion, the case when the variation is smaller than expected, may occur and also impacts the accuracy of the analysis when it is not appropriately specified, McCullagh and Nelder⁵ have suggested that overdispersion may be the norm. Various methods have been proposed to identify the underlying variation and provide corrections to improve estimates of the variance.^{6, 7}

Overdispersion or underdispersion is often identified by estimating the parameters in the mean-variance relationship and measuring the deviations from the theoretical values under the assumed distribution. Kukush et al.⁸ considered a pair of mean and variance functions with a common parameter vector θ estimated using an extended quasi-score function. Tsou⁹ considered two parameters $\left(\psi ,\lambda \right)$ in a parametric robust method of determining the mean-variance relationship through estimation of the power λ with an adjusted robust log likelihood method for fixed values of $\psi$. In addition, researchers have developed methods to test for overdispersion in proportions¹⁰ and score test statistics for overdispersed Poisson and binomial models.¹¹ Xiang et al.¹² provided a score test for overdispersion in a zero-inflated Poisson mixed regression model. Yang, Hardin, and Addy¹³ modified the score statistic to test overdispersion in the zero–inflated generalized Poisson mixed model. While these tests work well for identifying overdispersion, current parameterizations are limited to one parameter or are only applicable to distributions that have a particular form for the variance.

Overdispersed data are analyzed with appropriate statistical models such as generalized estimating equations, generalized linear mixed models, and joint modeling of the mean and dispersion.¹⁴ Generalized estimating equations account for correlation through the selection of a covariance structure for the correlated responses.¹⁵ Generalized linear mixed models have been used to model overdispersion in non-normal data.¹⁶ These models incorporate random effects, through random intercepts and random slopes, to account for correlation due to clustering.¹⁷ The joint modeling of the mean and the variance uses an additional dispersion submodel to address the overdispersion in a generalized linear model context.¹⁸ Joint modeling allows one to simultaneously model both the mean and the variance through submodels. This technique has been extended to consider joint modeling in hierarchical generalized linear model structures.^{19, 20}

Quasi-likelihood models are useful in cases where the underlying distribution is unspecified.²¹ This modeling technique relaxes the distributional assumption in the random component and instead relies on the specification of a mean-variance function. The regression parameter estimates and standard errors are obtained from the specified mean-variance relationship and estimates of the covariance matrix in a quadratic form. The quasi-likelihood approach possesses many good properties, including unbiased estimates and small standard errors as compared to alternative methods.²² While the quasi-likelihood method is appropriate for evaluating overdispersed data, the form of the variance has been limited to a single multiplicative overdispersion parameter.

This paper proposes a modified generalized quasi-likelihood (MGQL) model which utilizes a canonical two-parameter mean-variance relation. The proposed canonical parameterization is flexible and can be used to represent the form of the variance for any distribution in the exponential family. The incorporation of this mean-variance relationship in the MGQL extends quasi-likelihood models to describe a larger class of variance functions in the analysis of correlated data.

In Section 2, we review mean-variance relationships in the exponential family and the generalized quasi-likelihood approach to estimate the regression parameters and variance components in the analysis of clustered data.²³ In Section 3, we present the canonical parameterization of the mean-variance relationship. We provide a generalized method of moments (GMM) approach to estimate the mean-variance parameters and introduce a test to identify overdispersion or underdispersion. We propose a model that incorporates the mean-variance relationship in the form of a modified generalized quasi-likelihood. In Section 4, a simulation study is utilized to demonstrate the performance of the MGQL model. In Section 5, the MGQL model is used to analyze data collected through the National Longitudinal Study of Adolescent to Adult Health (Add Health).²⁴ This study collected health information on adolescents over four waves of interviews, and is highly correlated due to the nested structure of the longitudinal study. We demonstrate the use of the MGQL to appropriately account for overdispersion in the evaluation of risk factors associated with obesity.

2. Background

2.1. Mean-Variance Relation Parameters

Let the vector of observations $\mathit{y}={\left(y_1,\dots ,y_n\right)}^T$ be realizations of a set of independent random variables Y with means μ related to a set of k covariates $\mathit{X}=\left({\mathit{x}}_1^T,\dots ,{\mathit{x}}_k^T\right)$ through a function $g(\cdot )$ such that $E\left(\mathit{Y}\right)=\mathit{\mu }={\left({\mu }_1,\dots ,{\mu }_n\right)}^T$ and $g\left(\mathit{\mu }\right)=\mathit{\eta }=\mathit{X}\beta$. The estimates of the regression parameters $\mathit{\beta }={\left({\beta }_0,\dots ,{\beta }_k\right)}^T$ can be obtained using estimating equations from the likelihood. Let the joint probability function of Y_i with known functions a, b, and c and known dispersion parameter ϕ⁵ be

$$f\left(y;\theta ,\varphi \right)=exp\left[\frac{\left(y\theta -b\left(\theta \right)\right)}{a\left(\varphi \right)}+c\left(y,\varphi \right)\right].$$

If θ is unknown, we have a two-dimensional exponential family with log likelihood

$$l\left(\theta ,\varphi |y\right)=\frac{\left(y\theta -b\left(\theta \right)\right)}{a\left(\varphi \right)}+c\left(y,\varphi \right).$$

Using the expectation of the derivative of the likelihood

$$E\left(\frac{\partial l\left(\theta ,\varphi |y\right)}{\partial \theta }\right)=0$$

and the property

$$E\left(\frac{{\partial }^{2}l\left(\theta ,\varphi |y\right)}{\partial {\theta }^2}\right)+E{\left(\frac{l\left(\theta ,\varphi |y\right)}{\partial \theta }\right)}^2=0$$

yields the expected value $E\left(\mathit{Y}\right)=b^{\prime }\left(\theta \right)$ and the variance $var\left(\mathit{Y}\right)=b^{″}\left(\theta \right)a\left(\varphi \right)$ where $b^{\prime }(\cdot )$ denotes the first derivative and $b^{″}(\cdot )$ denotes the second derivative. Thus, both $b^{\prime }$ and $b^{″}$ are functions of the canonical parameter θ. The mean and the variance are related through the first derivative and second derivative of the function b(θ). The variance of the observations is a product of a function of the canonical parameter θ and a function of the dispersion parameter ϕ.

The Poisson distribution and the binomial distribution are members of the exponential family and are commonly used to analyze count data and binary data, respectively. The Poisson distribution has probability mass function

$$f\left(y;\alpha \right)=exp\left(y\hspace{0.17em}log\hspace{0.17em}\alpha -\alpha -\mathit{log}y\text{!}\right)$$

with $a\left(\varphi \right)=1$, $b\left(\theta \right)=\alpha =\exp \left(\theta \right)$, $c\left(y,\varphi \right)=-\log y\text{!}$, and canonical parameter $\theta =\log \left(\alpha \right)$. Thus, the expected mean and variance under the Poisson distribution are equal to α. The binomial distribution has probability distribution function

$$f\left(y;m,p\right)=\left(\begin{array}{c}m\\ y\end{array}\right)p^y{\left(1-p\right)}^{m-y},$$

with $a\left(\varphi \right)=1$, $b\left(\theta \right)=m\log \left(1+\exp \left(\theta \right)\right)$, $c\left(y,\varphi \right)=\log \left(\begin{array}{c}m\\ y\end{array}\right)$ and the canonical parameter $\theta =\log \left(\frac{p}{1-p}\right)$. Under the binomial distribution, the expected mean is $mp$ and the expected variance is $mp\left(1-p\right)$.^{5, 25}

2.2. Generalized Quasi-likelihood Models

Generalized quasi-likelihood models use the specification of the mean-variance relationship to evaluate correlated data. Consider vectors of correlated observations ${\mathit{y}}_{\text{1}},\dots {\mathit{y}}_{\mathit{n}}$ for $i=1,\dots n$ where ${\mathit{y}}_{\mathit{i}}=\left(y_{i1},\dots .y_{i{n}_i}\right)$. The correlated observations $y_{ij}$ for $j=1,\dots ,n_i$ where $n_i$ is the sample size of the ith vector comes from a distribution in the exponential family with link function $g(\cdot )$ such that

$$g\left({\mu }_{ij}\right)={\eta }_{ij}={\mathit{x}}_{ij}^{\text{T}}\mathit{\beta }+\sigma {\xi }_i$$

for ${\xi }_i~N\left(0,1\right)$ as ${\xi }_i={\alpha }_i/\sigma$ for k covariates and random effects ${\alpha }_i~N\left(0,{\sigma }^2\right)$ for cluster i. We estimate the parameters β and σ using GQL. Let the response vector be

$${\mathit{S}}_{\mathit{i}}=\left(y_i^{\text{T}},u_i^{\text{T}}\right)$$

for cluster i, where ${\mathit{y}}_{\mathit{i}}^{\text{T}}=\left(y_{i1},\dots ,y_{i{n}_i}\right)$ and ${\mathit{u}}_{\mathit{i}}^{\text{T}}=\left({\mathit{u}}_{\mathit{i}\text{1}}^{\text{T}},{\mathit{u}}_{\mathit{i}\text{2}}^{\text{T}}\right)$ contains the pairwise products for ${\mathit{u}}_{\mathit{i}\text{1}}=\left(y_{i1}^2,\dots ,y_{i{n}_i}^2\right)$ and ${\mathit{u}}_{\mathit{i}\text{2}}=\left(y_{i1}{y}_{i2},\dots ,y_{ij}{y}_{i{j}^{*}},\dots ,y_{i\left(n_i-1\right)}{y}_{i{n}_i}\right)$. Let $\mathit{\theta }={\left({\mathit{\beta }}^T,\sigma \right)}^T$ and ${\mathit{M}}_{\mathit{i}}\left(\theta \right)$ be the mean of the response vector S_i. Let ${\mathit{\Omega }}_{\mathit{i}}\left(\theta \right)$ be the covariance matrix for S_i with elements ${\omega }_{ij}$. Then, the set of generalized quasi-likelihood estimating equations,

$${\sum }_{i=1}^n\frac{\partial {\mathit{M}}_{\mathit{i}}^{\text{'}}\left(\theta \right)}{\partial \theta }{\mathit{\Omega }}_{\mathit{i}}^{-1}\left(\theta \right)\left[{\mathit{S}}_{\mathit{i}}-{\mathit{M}}_{\mathit{i}}\left(\theta \right)\right]=0$$ (2.1)

provides GQL estimates of β and σ.^{21, 23} The mean of the response vector, ${\mathit{M}}_{\mathit{i}}\left(\theta \right)$, is evaluated as

$${\mathit{M}}_{\mathit{i}}\left(\theta \right)=E\left({\mathit{S}}_{\mathit{i}}\right)=E\left(Y_{i1},\dots ,Y_i{n}_i,Y_{i1}^2,\dots ,Y_{i{n}_i}^2,Y_{i1}{Y}_{i2},\dots ,Y_{i\left(n_i-1\right)}{Y}_{n_i}\right)$$

and

$$E\left(Y_{ij}\right)={\mu }_{ij}\left(\theta \right)=E\left[g_{\left(1\right)}\left(x_{ij}^T\beta +\sigma \xi \right)\right]$$

$$E\left(Y_{ij}^2\right)=m_{ijj}\left(\theta \right)=E\left[g_{\left(2\right)}\left(x_{ij}^T\beta +\sigma \xi \right)\right]$$

$$E\left(Y_{ij}{Y}_{ik}\right)=m_{ijk}\left(\theta \right)=E\left[g_{\left(1\right)}\left(x_{ij}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{ik}^T\beta +\sigma \xi \right)\right]$$

where the functions $g_{\left(r\right)}\left({\eta }_{ij}\right)$ are the r^th finite moments of y_ij. The partial derivative matrix $\frac{\partial {\mathit{M}}_{\mathit{i}}^{\text{'}}\left(\theta \right)}{\partial \theta }$ has dimension $\left(p+1\right)\times \left\{n_i\left(n_i+1\right)/2\right\}$, with partial derivatives

$$\frac{\partial {\mu }_{ij}\left(\theta \right)}{\partial \beta }=E\left[{\tilde{g}}_{\left(1\right)}\left(x_{ij}^T\beta +\sigma \xi \right)\right]x_{ij}^T$$

$$\frac{\partial m_{ijj}\left(\theta \right)}{\partial \beta }=E\left[{\tilde{g}}_{\left(2\right)}\left(x_{ij}^T\beta +\sigma \xi \right)\right]x_{ij}^T$$

$$\begin{gathered} \frac{\partial m_{ijk}\left(\theta \right)}{\partial \beta }=E\left[{\tilde{g}}_{\left(1\right)}\left(x_{ij}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{ik}^T\beta +\sigma \xi \right)\right]x_{ij}^T+E[g_{\left(1\right)}\left(x_{ij}^T\beta +\sigma \xi \right){\tilde{g}}_{\left(1\right)}(x_{ik}^T\beta \\ +\sigma \xi )]x_{ik}^T \end{gathered}$$

$$\frac{\partial {\mu }_{ij}\left(\theta \right)}{\partial \sigma }=E\left[\xi {\tilde{g}}_{\left(1\right)}\left(x_{ij}^T\beta +\sigma \xi \right)\right]x_{ij}^T$$

$$\frac{\partial m_{ijj}\left(\theta \right)}{\partial \sigma }=E\left[\xi {\tilde{g}}_{\left(2\right)}\left(x_{ij}^T\beta +\sigma \xi \right)\right]x_{ij}^T$$

$$\frac{\partial m_{ijk}\left(\theta \right)}{\partial \sigma }=E\left[\xi \left\{{\tilde{g}}_{\left(1\right)}\left(x_{ij}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{ik}^T\beta +\sigma \xi \right)+g_{\left(1\right)}\left(x_{ij}^T\beta +\sigma \xi \right){\tilde{g}}_{\left(1\right)}\left(x_{ik}^T\beta +\sigma \xi \right)\right\}\right]$$

where ${\tilde{g}}_{\left(r\right)}(\cdot )$ as the first derivative of $g_{\left(r\right)}(\cdot )$. Then the covariance matrix is

$${\mathit{\Omega }}_{\mathit{i}}=\left[\begin{array}{cc}{\mathit{\Sigma }}_{\mathit{i}}& {\mathit{P}}_{\mathit{i}}\\ {\mathit{P}}_{\mathit{i}}^{\mathbf{\text{'}}}& {\mathit{Q}}_{\mathit{i}}\end{array}\right]$$

where ${\text{Σ}}_i=cov\left(Y_i\right)$, $P_i=cov\left(Y_i,U_i^{\text{T}}\right)$ and $Q_i=cov\left(U_i\right)$. The diagonal elements of ${\Sigma }_i$ are the variances of Y_i such that ${\sigma }_{ijj}=Var\left(Y_{ij}\right)=m_{ijj}\left(\theta \right)-{\mu }_{ij}^2\left(\theta \right)$ with off diagonal elements ${\sigma }_{ijk}=Cov\left(Y_{ij},Y_{ik}\right)=m_{ijk}\left(\theta \right)-{\mu }_{ij}\left(\theta \right){\mu }_{ik}\left(\theta \right)$. The matrix $P_i$ is of dimension $n_i\times \left\{n_i\left(n_i+2\right)/2\right\}$ and contains $cov\left(Y_{ij},Y_{ij}^2\right)$, $cov\left(Y_{ij},Y_{ij}{Y}_{il}\right)$ and $cov\left(Y_{ij},Y_{ik}{Y}_{il}\right)$. For $j=k=l$,

$$cov\left(Y_{ij},Y_{ij}^2\right)=p_{ijjj}\left(\theta \right)=E\left[g_{\left(3\right)}\left(x_{ij}^T\beta +\sigma \xi \right)\right]-{\mu }_{ij}\left(\theta \right)m_{ijj}\left(\theta \right).$$

For $j=k\ne l$ and $j=l\ne k$, the covariance elements are

$$cov\left(Y_{ij},Y_{ij}{Y}_{il}\right)=E\left(Y_{ij}^2{Y}_{il}\right)-{\mu }_{ij}\left(\theta \right)m_{ijl}\left(\theta \right)$$

$$=E\left[g_{\left(2\right)}\left(x_{ij}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{il}^T\beta +\sigma \xi \right)\right]-{\mu }_{ij}\left(\theta \right)m_{ijl}\left(\theta \right).$$

For $j\ne k\ne l$,

$$cov\left(Y_{ij},Y_{ik}{Y}_{il}\right)=p_{ijkl}\left(\theta \right)$$

$$=E\left[g_{\left(1\right)}\left(x_{ij}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{ik}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{il}^T\beta +\sigma \xi \right)\right]-{\mu }_{ij}\left(\theta \right)m_{ikl}\left(\theta \right).$$

In the covariance matrix, ${\mathit{Q}}_{\mathit{i}}$ contains $cov\left(Y_{ij}{Y}_{ik},Y_{il}{Y}_{iw}\right)$ with dimension $\left\{n_i\left(n_i+1\right)/2\right\}\times \left\{n_i\left(n_i+1\right)/2\right\}$. For $j=k=l=m$,

$$cov\left(Y_{ij}^2,Y_{ij}^2\right)=q_{ijjjj}\left(\theta \right)=E\left[g_{\left(4\right)}\left(x_{ij}^T\beta +\sigma \xi \right)\right]-m_{ijj}^2\left(\theta \right).$$

For $j=k\ne l\ne w$,

$$cov\left(Y_{ij}^2,Y_{il}{Y}_{iw}\right)=q_{ijjlw}\left(\theta \right)$$

$$=E\left[g_{\left(2\right)}\left(x_{ij}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{il}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{iw}^T\beta +\sigma \xi \right)\right]-m_{ijj}\left(\theta \right)m_{ilw}\left(\theta \right).$$

For $j\ne k\ne l\ne w$,

$$cov\left(Y_{ij}{Y}_{ik},Y_{il}{Y}_{iw}\right)=q_{ijklw}$$

$$=E\left[g_{\left(1\right)}\left(x_{ij}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{ik}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{il}^T\beta +\sigma \xi \right)g_{\left(1\right)}\left(x_{iw}^T\beta +\sigma \xi \right)\right]-m_{ijk}\left(\theta \right)m_{ilw}\left(\theta \right).$$

The quasi-likelihood estimate ${\widehat{\mathit{\theta }}}_{QL}={\left({\widehat{\mathit{\beta }}}_{QL}^T,{\widehat{\sigma }}_{QL}\right)}^T$ is obtained using Newton-Raphson iteration as ${\widehat{\mathit{\theta }}}_{QL}\left(t+1\right)={\widehat{\mathit{\theta }}}_{QL}\left(t\right)+{\left[{\sum }_{i=1}^n\frac{\partial {\mathit{M}}_{\mathit{i}}^{\mathbf{\text{'}}}\left(\theta \right)}{\partial \theta }{\mathit{\Omega }}_{\mathit{i}}^{-1}\left(\theta \right)\frac{\partial {\mathit{M}}_{\mathit{i}}\left(\theta \right)}{\partial \theta }\right]}_{\left(t\right)}^{-1}\left[{\sum }_{i=1}^n\frac{\partial {\mathit{M}}_{\mathit{i}}^{\mathbf{\text{'}}}\left(\theta \right)}{\partial \theta }{\mathit{\Omega }}_{\mathit{i}}^{-1}\left(\theta \right)\left[{\mathit{S}}_{\mathit{i}}-{\mathit{M}}_{\mathit{i}}\left(\theta \right)\right]\right]$ with covariance V where $\widehat{V}\left({\widehat{\mathit{\theta }}}_{QL}\right)={\left[{\sum }_{i=1}^n\frac{\partial {\mathit{M}}_{\mathit{i}}^{\mathbf{\text{'}}}\left(\theta \right)}{\partial \theta }{\mathit{\Omega }}_{\mathit{i}}^{-1}\left(\theta \right)\frac{\partial {\mathit{M}}_{\mathit{i}}\left(\theta \right)}{\partial \theta }\right]}^{-1}$. These GQL estimators are consistent and efficient.²³ A specification of the GQL model is important as consistency of the regression parameter estimates depends on correctly specifying the link function and the efficiency depends on a correctly specified variance function.

3. Modified Generalized Quasi-likelihood Models using the Canonical Mean-Variance Parameterization

3.1. Canonical Parameterization

For a random variable Y, we consider describing the variance in terms of two parameters including a dispersion parameter $\psi$ and power parameter λ. Tsou⁹ suggested the mean-variance relationship, $\psi {\mu }^{\lambda }$, which introduced additional flexibility compared to the one parameter form. However, this form is limited to distributions with a variance power relationship such as the Poisson or gamma distributions ($\lambda =1$ or $\lambda =2$, respectively). We generalize this power parameterization through the canonical parameter. Consider the canonical parameter θ through its derivative of the inverse link function h, where $h=g^{-1}$. Then, we propose the general mean-variance relationship as

$$var\left(Y\right)=\psi {\left[h^{\prime }\left(g\left(\mu \right)\right)\right]}^{\lambda }=\psi {\left[h^{\prime }\left(\theta \right)\right]}^{\lambda }$$

where $h^{\prime }\left(\theta \right)$ is the first derivative of the inverse of the canonical link and $\psi$ and λ are parameters measuring the overdispersion, $\psi >0$. This form of the mean-variance relationship is distinct as it is applicable to members of the exponential family of distributions and provides general flexibility in describing the mean-variance relationship.

For example, if Y follows a Poisson distribution with natural parameter α, then the canonical parameter is $\theta =log\left(\alpha \right)$ and the corresponding mean-variance parameter relation is

$$var\left(Y\right)=\psi {\left[h^{\prime }\left(\theta \right)\right]}^{\lambda }=\psi {\alpha }^{\lambda }.$$

If Y follows a binomial distribution with natural parameters m and p and canonical parameter $\theta =logit\left(p\right)$, then the canonical mean-variance parameter relation is

$$var\left(Y\right)=\psi {\left[h^{\prime }\left(\theta \right)\right]}^{\lambda }=\psi {\left[\frac{exp\left(\theta \right)}{{\left(1+exp\left(\theta \right)\right)}^2}\right]}^{\lambda }=\psi {\left[p\left(1-p\right)\right]}^{\lambda },$$

as $p=e^{\theta }{\left(1+e^{\theta }\right)}^{-1}$. Thus, the dispersion parameter $\psi$ and power parameter λ are a means of adjusting for deviation from the assumed distributional properties. Values of the overdispersion parameters $\psi$ and λ different than 1 indicate violations in the variance assumption. While the binomial does not have a natural power relationship between the mean and variance, the relationship is tractable in its power form when based on the canonical parameter. We measure the deviations and consider distributions that are not fully identified but belong to the quasi-exponential family. Such distributions are identified based on the scale of the responses which is robust to misspecification.

3.2. Estimation of $\psi$ and λ

Let ${\widehat{\mathit{\gamma }}}_{GMM}$ be an estimator for a vector of parameters $\mathit{\gamma }={\left(\psi ,\lambda \right)}^T$ that minimizes the quadratic objective function $f_n{\left(\mathit{\gamma }\right)}^T{W}_n{f}_n\left(\mathit{\gamma }\right)$ where $f_n\left(\mathit{\gamma }\right)$ is a function of the vector of the sample moment conditions, and W_n is a symmetric, positive definite weight matrix of dimension n.^{26, 27} Then,

$${\widehat{\mathit{\gamma }}}_{GMM}=argmi{n}_{\mathit{\beta }}\left\{f_n{\left(\mathit{\gamma }\right)}^T{\mathit{W}}_n{f}_n\left(\mathit{\gamma }\right)\right\}$$ (3.1)

is a generalized method of moments estimator for γ which minimizes the objective function. Thus, the GMM estimators of the parameters $\psi$ and λ, are obtained from the population moment conditions

$$E\left(h^{\prime }\left({\theta }_i\right)\left(var\left(y_i\right)-\psi {\left[h^{\prime }\left({\theta }_i\right)\right]}^{\lambda }\right)\right)=0$$ (3.2a)

$$E\left(h^{\prime }{\left({\theta }_i\right)}^2\left(var\left(y_i\right)-\psi {\left[h^{\prime }\left({\theta }_i\right)\right]}^{\lambda }\right)\right)=0$$ (3.2b)

where $h^{\prime }\left({\theta }_i\right)$ is the first derivative of the inverse link function and $var\left(y_i\right)$ is an empirical estimate of the variance based on the squared residual, ${\left(y_i-{\mu }_i\right)}^2$. Equating the moment conditions and an empirical estimate of $f_n\left(\mathit{\gamma }\right)$ results in

$$\frac{1}{n}\sum _{i=1}^{n}f\left(y_i,\psi ,\lambda \right)=\left[\begin{array}{c}\frac{1}{n}\sum _{i=1}^n{h}^{\prime }\left({\theta }_i\right)\left(var\left(y_i\right)-\psi {\left[h^{\prime }\left({\theta }_i\right)\right]}^{\lambda }\right)\\ \frac{1}{n}\sum _{i=1}^n{h}^{\prime }{\left({\theta }_i\right)}^2\left(var\left(y_i\right)-\psi {\left[h^{\prime }\left({\theta }_i\right)\right]}^{\lambda }\right)\end{array}\right]=\left[\begin{array}{c}0\\ 0\end{array}\right]$$

We make use of a two-step GMM approach, with an identity weight matrix in the first step. In the second step, the weights are selected as an estimate of the optimal weight matrix for GMM as

$${\widehat{\mathit{W}}}_n={\left[\frac{1}{n}{\sum }_{i=1}^{n}f\left(y_i,\widehat{\psi },\widehat{\lambda }\right)f{\left(y_i,\widehat{\psi },\widehat{\lambda }\right)}^T\right]}^{-1}$$

where $\widehat{\psi }$ and $\widehat{\lambda }$ are mean-variance relationship parameter estimates from the first step.²⁸ Thus, the vector of GMM estimates for the mean-variance relation parameters ${\widehat{\mathit{\gamma }}}_{GMM}$ minimizes the quadratic objective function $f_n{\left(\mathit{\gamma }\right)}^T{\mathit{W}}_n{f}_n\left(\mathit{\gamma }\right)$. The generalized method of moments approach is flexible and estimates both parameters $\left(\psi ,\lambda \right)$ simultaneously.

An alternative approach is to fix one parameter at a time and estimate the second parameter using one moment condition. Thus, an extension of GMM is to make use of additional moment conditions, such as

$$f_3\left(y_i,\psi ,\lambda \right)=h^{\prime }{\left({\theta }_i\right)}^3\left(var\left(y_i\right)-\psi {\left[h^{\prime }\left({\theta }_i\right)\right]}^{\lambda }\right)$$ (3.3)

to estimate the parameters. The additional moment condition improves the asymptotic efficiency, although there is a possibility of small sample bias.²⁹

To identify the mean-variance relation in clustered data, consider y_ij, the j^th observation in the i^th cluster, $j=1,\dots ,n_i$ and $i=1,\dots ,n$, with mean ${\mu }_{ij}$ which is related to k covariates and random effect α_i through the link function g such that

$$E\left(Y_{ij}\right)={\mu }_{ij}$$

and

$$g\left({\mu }_{ij}\right)={\eta }_{ij}=x_{ijk}^T\beta +{\alpha }_i$$

where the random effect α_i represents the variation between clusters such that ${\alpha }_i~N\left(0,{\sigma }^2\right)$. Further, let ${\xi }_i={\alpha }_i/\sigma$, such that ${\xi }_i~N\left(0,1\right)$ so the linear predictor reduces to

$$g\left({\mu }_{ij}\right)=x_{ijk}^T\beta +\sigma {\xi }_i.$$

The general mean-variance relationship, obtained for the data across all the clusters, is

$$var\left(Y\right)=\psi {\left[h^{\prime }\left(\theta \right)\right]}^{\lambda },$$

where θ is the canonical parameter, $h^{\prime }\left(\theta \right)$ is the first derivative of the inverse canonical link, and $\psi$ and λ are the variance parameters estimated from the data, (3.2a) and (3.2b). The estimates of $\psi$ and λ indicate the strength of the clustering (variance of the random effect).

The parameters $\psi$ and λ are essential in defining the variance and identifying deviations from the theoretical values in a known distributional mean-variance relation. We obtain GMM estimators using the first and second moments based on the fact that the distribution is a member of the quasi-exponential family.³⁰ We do not require complete distributional assumptions, as is required with maximum likelihood estimators, and the estimates are obtainable even when likelihood methods are computationally burdensome.²⁶ The GMM estimators for $\psi$ and λ are consistent and asymptotically normal.³¹

3.3. Inference for $\psi$ and λ

Assume that the data come from a quasi-exponential family. The sample moments are asymptotically normally distributed, so we have

$$\sqrt{n}\left(f_n\left(\widehat{\mathit{\gamma }}\right)\right)\stackrel{d}{\to }N\left(0,\mathit{\Delta }\right),$$

with the asymptotic variance $\mathit{\Delta }=E\left[f\left(y,{\mathit{\gamma }}^{\ast }\right)f{\left(y,{\mathit{\gamma }}^{\ast }\right)}^T\right]$ where ${\mathit{\gamma }}^{\ast }$ is the estimate of the parameters of interest.³⁰ For the mean-variance relationship parameters $\psi$ and λ, the GMM estimator ${\widehat{\mathit{\gamma }}}_{GMM}=\left({\widehat{\psi }}_{GMM},{\widehat{\lambda }}_{GMM}\right)$ has the asymptotic covariance

$$var\left({\widehat{\mathit{\gamma }}}_{GMM}\right)={\mathit{V}}_{GMM}=\frac{1}{n}{\left[{\mathit{\Gamma }}^T\mathit{W}\Gamma \right]}^{-1}{\mathit{\Gamma }}^T\mathit{W}\Delta W\Gamma {\left[{\mathit{\Gamma }}^T\mathit{W}\Gamma \right]}^{-1}$$

where W is a specified weight matrix and $\mathit{\Gamma }$ is the expected value of the Jacobian of population moment conditions found as

$$\mathit{\Gamma }=E\left[\frac{\partial f\left(y,\mathit{\gamma }\right)}{\partial \mathit{\gamma }}\right]=E{\left[\frac{\partial f\left(y,\lambda ,\psi \right)}{\partial \psi },\frac{\partial f\left(y,\lambda ,\psi \right)}{\partial \lambda }\right]}^T.$$

In the optimal case, the weight matrix is selected as $\mathit{W}={\left(\mathit{\Delta }\right)}^{-1}$, so that

$${\mathit{V}}_{GMM}=\frac{1}{n}{\left[{\mathit{\Gamma }}^{\prime }W\Gamma \right]}^{-1}$$

resulting in asymptotically efficient GMM estimators, ${\widehat{\psi }}_{GMM}$ and ${\widehat{\lambda }}_{GMM}$.²⁶ In practice, the covariance matrix is evaluated using $\widehat{\mathit{\gamma }}$,

$$\widehat{\mathit{\Gamma }}=\frac{1}{n}{\sum }_{i=1}^n\frac{\partial f\left(y,\widehat{\mathit{\gamma }}\right)}{\partial \widehat{\mathit{\gamma }}}.$$

Significant overdispersion is identified through two hypothesis tests of the overdispersion parameters $\psi$ and λ,

$$H_0:\psi =1,H_a:\psi >1$$

and

$$H_0:\lambda =1,H_a:\lambda >1.$$

Then the z-test statistics

$$Z_{\psi }=\frac{\widehat{\psi }-1}{\sqrt{\mathrm{var}\left(\widehat{\psi }\right)}}$$

and

$$Z_{\lambda }=\frac{\widehat{\lambda }-1}{\sqrt{\mathrm{var}\left(\widehat{\lambda }\right)}}\text{,}$$

follow the standard normal distribution under the null hypothesis. Thus, a measure of the overdispersion is given based on the joint 100(1-α)% confidence intervals for $\psi$ and λ,

$$\left({\widehat{\psi }}_{GMM}-z_{1-\frac{\alpha }{2}}\sqrt{V_{GMM,\psi }},{\widehat{\psi }}_{GMM}+z_{1-\frac{\alpha }{2}}\sqrt{V_{GMM,\psi }}\right)$$

$$\left({\widehat{\lambda }}_{GMM}-z_{1-\frac{\alpha }{2}}\sqrt{V_{GMM,\lambda }},{\widehat{\lambda }}_{GMM}+z_{1-\frac{\alpha }{2}}\sqrt{V_{GMM,\lambda }}\right)$$

where z_α is the α^th quantile from the standard normal distribution.²⁸

3.4. Modified Generalized Quasi-likelihood Models

In this section, we propose a modified generalized quasi-likelihood model for correlated data based on the canonical parameterization. As correlated data necessitate dealing with extravariation, we rely on our two-parameter mean-variance relation. The GQL approach relies on the specification of the mean-variance relationship rather than a distributional assumption. We address the correlation through the empirical mean-variance estimates of $\psi$ and λ.

The generalized quasi-likelihood estimating equation (2.1), with M_i, $\frac{\partial {\mathit{M}}_{\mathit{i}}^{\mathbf{\text{'}}}\left(\theta \right)}{\partial \theta }$, and the covariance matrix ${\mathit{\Omega }}_{\mathit{i}}\left(\theta \right)$, is used to estimate the regression parameters β and the variance of the random effect σ. Though GQL performs well and produces consistent and efficient estimators,²³ it relies on the estimate of the covariance ${\mathit{\Omega }}_{\mathit{i}}\left(\theta \right)$. We update this estimate to incorporate extravariation based on the canonical parameterization $\psi {\mathit{\Omega }}_{\mathit{i}}{\left(\theta \right)}^{\lambda }$ such that the modified quasi-likelihood estimate ${\widehat{\mathit{\theta }}}_{MGQL}$ is obtained iteratively as

$${\widehat{\mathit{\theta }}}_{MGQL}\left(t+1\right)={\widehat{\mathit{\theta }}}_{MGQL}\left(t\right)+{\left[{\sum }_{i=1}^n\frac{\partial {\mathit{M}}_{\mathit{i}}^{\mathbf{\text{'}}}\left(\theta \right)}{\partial \theta }{\left(\psi {\mathit{\Omega }}_i^{\lambda }\left(\theta \right)\right)}^{-1}\frac{\partial {\mathit{M}}_i\left(\theta \right)}{\partial \theta }\right]}_{\left(t\right)}^{-1}\left[{\sum }_{i=1}^n\frac{\partial {\mathit{M}}_{\mathit{i}}^{\mathbf{\text{'}}\left(\theta \right)}}{\partial \theta }{\left(\psi {\mathit{\Omega }}_i^{\lambda }\left(\theta \right)\right)}^{-1}\left[{\mathit{S}}_{\mathit{i}}-{\mathit{M}}_{\mathit{i}}\left(\theta \right)\right]\right].$$

with covariance

$$\widehat{V}\left({\widehat{\mathit{\theta }}}_{MGQL}\right)={\left[{\sum }_{i=1}^n\frac{\partial {\mathit{M}}_{\mathit{i}}^{\mathbf{\text{'}}}\left(\theta \right)}{\partial \theta }{\left(\psi {\mathit{\Omega }}_i^{\lambda }\left(\theta \right)\right)}^{-1}\frac{\partial {\mathit{M}}_i\left(\theta \right)}{\partial \theta }\right]}^{-1}.$$

This modification makes use of the GMM estimates of $\psi$ and λ. For given $\psi$ and λ, the MGQL estimates of β and σ in (2.1) are unbiased. The MGQL estimators are consistent and efficient as ${\mu }_{ij}$ is the mean of $y_{ij}$.

4. Simulation Study

We simulate hierarchical binary data and evaluate the estimation of the regression parameters using the MGQL model which incorporates GMM estimates of the mean-variance parameters into the quasi-likelihood model framework, a GQL model, and a generalized linear mixed model (GLMM) over 1000 iterations. The two-level binary data contain 50 clusters with 10 observations in each cluster, with the linear predictor $\log it\left({\mu }_i\right)={\eta }_i={\beta }_1{X}_{i1}+{\beta }_2{X}_{i2}+{\alpha }_i$ where ${\beta }_1={\beta }_2=1$ and X₁ and X₂ are generated from standard normal distributions. The canonical mean-variance parameter relation under the Bernoulli distribution is $\mathrm{var}\left(Y\right)=\psi {\left[p\left(1-p\right)\right]}^{\lambda }$. To evaluate the performance of these regression methods under the true mean-variance relation and an overdispersed form, we consider cases where the random effects are generated under the normal distribution and the t-distribution with 4 degrees of freedom. The GLMM is fit using the default optimization techniques in the R statistical software, which utilizes Nelder-Mead for the preliminary optimization of the random effects parameters and the bobyqa optimizer from the minqa package for the final estimation of the random and fixed effects parameters.

4.1. Normally Distributed Random Effects

We evaluate hierarchical binary data with normally distributed random effects. The random intercept α_i associated with each cluster is generated from $N\left(0,{\sigma }^2\right)$ with σ = 0.6, 0.8, 1, 1.2, 1.4. The results are reported in Table 1. The generalized linear mixed model estimates for the standard error of $\widehat{\sigma }$ were obtained using a profile likelihood approach.

Table 1:

Model Fit Simulation Results for Normally Distributed Random Effects

		β1		β2		σ		Iterations
		Est	SE	Est	SE	Est	SE
σ = 0.6	MGQL	1.0083	0.1228	1.0181	0.1231	0.5687	0.2177	5.5
	GQL	1.0053	0.1319	1.0120	0.1320	0.5837	0.2019	5.2
	GLMM	1.0040	0.1316	1.0110	0.1318	0.5706	0.1872	-
σ = 0.8	MGQL	1.0093	0.1243	1.0241	0.1247	0.7822	0.1880	4.5
	GQL	1.0045	0.1344	1.0159	0.1348	0.7948	0.1814	4.0
	GLMM	1.0035	0.1327	1.0149	0.1331	0.7790	0.1869	-
σ = 1	MGQL	1.0115	0.1281	1.0237	0.1284	0.9841	0.1924	4.5
	GQL	1.0047	0.1372	1.0137	0.1375	1.0014	0.1916	4.0
	GLMM	1.0041	0.1286	1.0131	0.1288	0.9832	0.1943	-
σ = 1.2	MGQL	1.0103	0.1333	1.0225	0.1336	1.1837	0.2067	4.6
	GQL	1.0025	0.1401	1.0126	0.1404	1.2044	0.2096	4.1
	GLMM	1.0023	0.1220	1.0125	0.1222	1.1835	0.2108	-
σ = 1.4	MGQL	1.0148	0.1402	1.0253	0.1405	1.3951	0.2280	4.7
	GQL	1.0078	0.1437	1.0146	0.1439	1.4120	0.2326	4.2
	GLMM	1.0080	0.1137	1.0147	0.1139	1.3874	0.2335	-

The simulation results demonstrate that the MGQL approach performs well and suggests that the MGQL model recovers the true values when relying on the estimated mean-variance relationship in the covariance matrix. While the parameter estimates are similar across the three methods, the standard errors for the MGQL estimates of β₁ and β₂ are lower than the standard errors of the GQL approach across all values of σ. The MGQL approach requires slightly more iterations, on average, than the GQL approach to achieve convergence, although the two approaches require similar computation time. As expected, the GLMM performs the best among the three methods as the data are generated under this model.

4.2. t-Distributed Random Effects

We evaluate the performance of the MGQL, GQL, and GLMM for non-normally distributed random effects. The random effects α_i are generated under the t-distribution with 4 degrees of freedom, which has heavier tails than the normal distribution. The model parameter estimates and standard errors are reported in Table 2.

Table 2:

Model Fit Simulation Results for t-Distributed Random Effects

		β1		β2		σ		Iterations
		Est	SE	Est	SE	Est	SE
σ = 0.6	MGQL	1.0185	0.1245	1.0141	0.1243	0.7653	0.1922	5.0
	GQL	1.0148	0.1347	1.0099	0.1345	0.7831	0.1818	4.0
	GLMM	1.0136	0.1324	1.0089	0.1320	0.7671	0.1883	-
σ = 0.8	MGQL	1.0150	0.1287	1.0071	0.1283	1.0047	0.1953	4.7
	GQL	1.0143	0.1379	1.0041	0.1374	1.0214	0.1935	4.1
	GLMM	1.0143	0.1281	1.0041	0.1275	1.0075	0.1977	-
σ = 1	MGQL	1.0134	0.1354	1.0119	0.1352	1.2435	0.2136	4.9
	GQL	1.0118	0.1414	1.0111	0.1413	1.2542	0.2151	4.2
	GLMM	1.0127	0.1190	1.0120	0.1191	1.2417	0.2194	-
σ = 1.2	MGQL	1.0104	0.1427	1.0090	0.1425	1.4770	0.2385	4.7
	GQL	1.0102	0.1450	1.0093	0.1449	1.4821	0.2414	4.3
	GLMM	1.0119	0.1207	1.0110	0.1205	1.4698	0.2466	-
σ = 1.4	MGQL	1.0128	0.1507	1.0050	0.1503	1.7140	0.2705	4.8
	GQL	1.0135	0.1490	1.0076	0.1487	1.7109	0.2716	4.4
	GLMM	1.0155	0.1328	1.0096	0.1327	1.6969	0.2770	-

As seen in the previous simulation, MGQL tends to be more efficient than the GQL approach for estimates of β. The simulation results also highlight the advantage of implementing the MGQL model for overdispersed data. For small values of σ, the simulated data reflect the true mean-variance parameterization for the Bernoulli distribution and thus we see similar performance across the three methods. However, for larger values of σ where overdispersion is present (for σ = 1.2 and σ = 1.4, there was significant overdispersion in 60.0% and 62.3% of simulations, respectively), we find that the MGQL model produces improved estimates of β₁ and β₂ compared to the GQL model and GLMM. In addition, for values of σ > 1, we see that MGQL produces more efficient variance estimates. Thus, when the random effects are not normally distributed, the MGQL approach has many advantages for modeling overdispersed data compared to the GQL model and GLMM.

5. Numerical Example

The Add Health Study is a longitudinal study in the United States of adolescents in 7^th through 12^th grade, with information collected over four waves of interviews between 1994 and 2008.²⁴ The data are available on the Add Health website (http://www.cpc.unc.edu/addhealth). We fit a modified generalized quasi-likelihood model to evaluate the binary variable adolescent obesity for 2,712 adolescents in the United States. The factors associated with obesity include activity scale and feeling scale, ratings of physical activity and emotional health. The mean-variance parameter estimates are $\widehat{\psi }=3.16$ and $\widehat{\lambda }=1.80$ with standard errors 0.27 and 0.06, respectively. The estimates indicate a significant deviation from the distributional form of the mean-variance relationship (test statistics $Z_{\psi }=7.94$ and $Z_{\lambda }=13.57$). Thus, making use of the true mean-variance form accounts for the clustering. The model parameter estimates and standard errors of the MGQL, GQL, and GLMM are provided (Table 3).

Table 3:

Parameter estimates and standard errors for adolescent obesity data

		${\beta }_{ActivityScale}$	${\beta }_{FeelingScale}$	σ
MGQL	Estimate	−1.0921	−0.6758	2.6078
	Std. Error	0.0326	0.0650	0.0794
GQL	Estimate	−1.3458	−0.5041	2.7022
	Std. Error	0.0449	0.0718	0.0974
GLMM	Estimate	−1.3438	−0.6527	2.6900
	Std. Error	0.0471	0.0726	0.0959

The covariates activity scale and feeling scale as well as the random effect are found to be significant across all three models. The regression parameter estimates for activity scale are positive, indicating that increased physical activity is associated with a lower probability of obesity. Similar estimates are produced for the GQL and GLMM approaches, while the MGQL estimate is slightly smaller $\left({\beta }_{ActivityScale}=-1.0921\right)$. Similarly, the parameter estimates for feeling scale vary slightly although all three estimates are negative, indicating that larger values of the computed emotional health measure is associated with a lower probability of obesity. The estimates of the standard error of the random effect σ vary slightly among the three models, with ${\widehat{\sigma }}_{MGQL}=2.6078$, ${\widehat{\sigma }}_{GQL}=2.7022$, and ${\widehat{\sigma }}_{GLMM}=2.6900$. The random effect variance for the MGQL model is found to be the lowest of the three estimates.

6. Conclusions

It is common to assume that the variance of a random variable is a function of the mean, although it is often the case that the true variance in the data may be inflated due to underlying correlation or the hierarchical data structure. While the presence of overdispersion impacts the accuracy of statistical evaluations, the MGQL is a modeling approach that appropriately fits correlated data. The MGQL approach is flexible as it accounts for correlation through an extended representation of the covariance. The canonical parameterization is tractable in the power form for any distribution in the exponential family. Moreover, deviations in the variance can be readily identified using the proposed GMM estimators of the mean-variance parameters $\psi$ and λ which are consistent and asymptotically normal. A simulation study demonstrated that the MGQL addresses correlation through the use of the mean-variance relationship and performs as well or better than existing methods including GQL models and GLMM, particularly for non-normally distributed random effects. The study confirmed that the MGQL retains good properties of quasi-likelihood approaches including unbiased estimates and small standard errors. In addition, we consider a numerical example to evaluate obesity data from the Add Health study. We verified that the MGQL model produced comparable results to existing models. Factors including activity scale and feeling scale were found to be negatively associated with obesity, and the MGQL model produced a lower variance estimate of the random effect.