Bivariate Random Effects Models for Meta-Analysis of Comparative Studies with Binary Outcomes: Methods for the Absolute Risk Difference and Relative Risk

Abstract

Multivariate meta-analysis is increasingly utilized in biomedical research to combine data of multiple comparative clinical studies for evaluating drug efficacy and safety profile. When the probability of the event of interest is rare or when the individual study sample sizes are small, a substantial proportion of studies may not have any event of interest. Conventional meta-analysis methods either exclude such studies or include them through ad-hoc continuality correction by adding an arbitrary positive value to each cell of the corresponding 2 by 2 tables, which may result in less accurate conclusions. Furthermore, different continuity corrections may result in inconsistent conclusions. In this article, we discuss a bivariate Beta-binomial model derived from Sarmanov family of bivariate distributions and a bivariate generalized linear mixed effects model for binary clustered data to make valid inferences. These bivariate random effects models use all available data without ad hoc continuity corrections, and accounts for the potential correlation between treatment (or exposure) and control groups within studies naturally. We then utilize the bivariate random effects models to reanalyze two recent meta-analysis data sets.

1. Introduction

The growth of evidence-based medicine has led to an increase in attention to meta-analysis.¹ Meta-analysis, also known as systematic overview, is a statistical process commonly used in biomedical research of combining the information from several independent studies concerned with the same clinical question including the treatment or exposure effect, with the aim of being able to resolve contradictory issues that cannot be concluded from a single study alone.

In meta-analysis of a set of N clinical trials with a binary outcome comparing an experimental treatment with a placebo, data can be represented as a series of 2 by 2 tables. The standard fixed and random meta-analysis methods for providing an overall estimate of the treatment effect across all studies rely on some assumptions.² Specifically, the fixed effect model assumes homogeneous treatment effects across all studies. Let θ_i be the value of a chosen measure (e.g., risk difference or log relative risk) of treatment effect in the i^th study (i = 1, 2, …, N), the homogeneity requires θ_i = θ (i = 1, 2, …, N). Let θ̂_i be an estimate of θ_i, and w_i denote the weight, which is often taken to be the reciprocal of the estimated variance ν̂_i of θ̂_i (i.e., ŵ_i = 1/ν̂_i),³ then the overall treatment effect based on the fixed effect model is estimated as a weighted average of the individual study estimated treatment effects, that is, ${\widehat{\theta }}_w={\displaystyle \sum _i}{\widehat{w}}_i{\widehat{\theta }}_i/{\displaystyle \sum _i}{\widehat{w}}_i$. Under the combined null hypothesis H₀: θ_i = 0 (i = 1, 2, …, N), the test statistic $U={\left({\displaystyle \sum _i}{\widehat{w}}_i{\widehat{\theta }}_i\right)}^2/{\displaystyle \sum _i}{\widehat{w}}_i$ follows a χ² distribution with 1 degree of freedom. A formal test of homogeneity can be performed using the Cochran’s Q statistic, defined by $Q={\displaystyle \sum _i}{\widehat{w}}_i{\left({\widehat{\theta }}_i-{\widehat{\theta }}_w\right)}^2$, which has approximately a ${\chi }_{N-1}^2$ distribution under the null hypothesis H₀: θ_i= θ (i = 1, 2, …, N).

Through a random-effects model, DerSimonian and Laird⁴ provided a way of incorporating heterogeneity into the overall estimate by including a between-study variance component ${\sigma }_b^2$. It basically assumes that θ̂_i ~ N (θ_i, v̂_i) and ${\theta }_i\sim N\left(\theta ,{\sigma }_b^2\right)$.² The overall treatment effect is once again obtained as a weighted average with the weight being estimated as ${\widehat{w}}_i^{\ast }=1/\left({\widehat{v}}_i+{\widehat{\sigma }}_b^2\right)$, i.e., ${\widehat{\theta }}_w^{\ast }={\displaystyle \sum _i}{\widehat{w}}_i^{\ast }{\widehat{\theta }}_i/{\displaystyle \sum _i}{\widehat{w}}_i^{\ast }$. Under the combined null hypothesis H₀: θ_i = 0 (i = 1, 2, …, N), the test statistic $U^{\ast }={\left({\displaystyle \sum _i}{\widehat{w}}_i^{\ast }{\widehat{\theta }}_i\right)}^2/{\displaystyle \sum _i}{\widehat{w}}_i^{\ast }$ follows a χ² distribution with 1 degree of freedom. The method of moments estimate of ${\sigma }_b^2$ is given by ${\widehat{\sigma }}_b^2=max\left\{{\left[{\displaystyle \sum _i}{\widehat{w}}_i-\left({\displaystyle \sum _i}{\widehat{w}}_i^2\right)/{\displaystyle \sum _i}{\widehat{w}}_i\right]}^{-1}[Q-(N-1)],0\right\}$.⁴

A concern on these conventional two-step meta-analysis methods is that they require estimating study- specific treatment effect θ̂_i (commonly expressed by log relative risk, log odds ratio or risk difference) and its variance ν̂_i based on the normal approximation. When the probability of the event of interest is rare or if the individual study sample sizes are small, this normality assumption might not hold. Furthermore, a substantial proportion of studies may not have any event of interest. To circumvent the issues of zero cells, the conventional meta-analysis methods either exclude such studies⁵ or add an arbitrary positive value to each cell of the corresponding 2 by 2 tables in the analysis, which may lead to less accurate conclusions. For example, different continuity corrections may result in different conclusions.⁶ An interesting yet challenging methodology question is how to use all available data without assigning an arbitrary number to the empty cells in meta-analysis.^7–10 Furthermore, it has been noted that these weighting-according-to-the-variance methods may introduce biases in meta-analyses of binary outcomes because this weighting scheme favors studies with certain frequencies of outcome events.¹¹ The relative weights for the individual studies in a meta-analysis may change considerably among different choices of effect measurements, which may lead to contradictory conclusions. This is particularly true for the sparse data scenario. Specifically, a study with zero event in both treatment and placebo groups, which would be excluded on a relative scale, would be included and even be given a large weight on a risk difference scale.⁶

Recently, multivariate random effects models for meta-analyses have become increasingly popular in biomedical research. For example, multivariate random effects models have been proposed for meta-analyses of diagnostic test studies^12–18 and correlated multiple outcomes.^{19, 20} Given the potential issues of applying conventional meta-analysis methods based on a univariate outcome, we discuss bivariate random effects models to deal with those challenges for meta-analyses of comparative clinical trials with binary outcomes in this article. Although the proposed methods were primarily presented for bivariate meta-analyses, they can be easily generalized to multivariate meta-analyses. Specifically, Section 2 shows the estimation of marginal treatment effects using the maximum likelihood methods under two models, i.e., a generalized linear mixed effects model and a bivariate Beta-binomial model. In Section 3, we reanalyze the data from two case studies: the study of type 2 diabetes mellitus after gestational diabetes ²¹ and the study of myocardial infarction with Rosiglitazone.⁵ Section 4 concludes this article with a brief discussion.

2. Bivariate Random Effects Models for Meta-Analysis of Comparative Studies

Let n_ki be the number of subjects, and p_ki be the probability of “success” for the i^th study (i = 1, 2, …, N) in the k^th treatment (or exposure) group with k = 1 denoting the placebo (or unexposed) group and k =2 denoting the treated (or exposed) group. Let Y_kij denote a Bernoulli random variable with value 1 denoting a “success” and value 0 denoting a “failure” for the j^th subject (j = 1, 2, …, n) of the i^th study in the k^th treatment group. Let $X_{ki}={\displaystyle \sum _{j=1}^{n_{ki}}}{Y}_{\mathit{kij}}$ be the total number of “success” in the k^th treatment group in the i^th study. In the first stage, conditional on the probability of events (i.e., p_ki) and the number of subjects (i.e., n_ki) of the k^th treatment group in the i^th study, the bivariate random effects model assumes that X_ki is independently binomially distributed as Bin (n_ki, p_ki) for k = 1, 2 and i = 1, 2, …, N, that is,

$$P(X_{1i}=x_{1i},X_{2i}=x_{2i}\mid n_{1i},n_{2i},p_{1i},p_{2i})=\prod _{k=1}^2\left(\begin{array}{c}{n}_{ki}\\ x_{ki}\end{array}\right){(p_{ki})}^{x_{ki}}{(1-p_{ki})}^{n_{ki}-x_{ki}}.$$ (1)

In the second stage, the joint distribution f(p_1i, p_2i), which is also denoted as f(p₁, p₂) for ease of notation, is specified. Specifically, we first review the bivariate generalized linear mixed effects models and then propose the bivariate Beta-binomial models as an alternative, for the evaluation of marginal treatment or exposure effect. Note that bivariate models are commonly used when there are two outcomes (e.g. the response to a treatment and the appearance of a side effect), in this article, we use bivariate models to jointly model the study-specific response rates in the placebo group and the treatment group in a meta-analysis with multiple studies.

2.1 Bivariate Generalized Linear Mixed Effects Models

In the second stage, the bivariate generalized linear mixed effects model (BGLMM) assumes a bivariate normal distribution of (p_1i, p_2i) in a transformed scale, which implies a linear relationship between p_1i and p_2i on a transformed scale. It is generally specified as follows,

$$\text{g}(p_{1i})={\nu }_1+{\nu }_{1i},\text{g}(p_{2i})={\nu }_2+{\nu }_{2i},\text{and}{({\nu }_{1i},{\nu }_{2i})}^T\sim N(\mathbf{0},{\mathbf{\sum }}_{\mathbf{v}}),$$ (2)

where g() is the link function such as the commonly used logit, probit and complementary log-log transformation functions, (ν₁,ν₂) are the fixed effects, and ${\mathbf{\sum }}_{\mathbf{v}}=\left(\begin{array}{cc}{\sigma }_1^2& \rho {\sigma }_1{\sigma }_2\\ \rho {\sigma }_1{\sigma }_2& {\sigma }_2^2\end{array}\right)$ is the variance-covariance matrix. To implement the natural constraint of −1≤ρ≤1, one can use the Fisher’s z transformation as ρ= [exp (2z) − 1]/[exp(2z)+1].

Based on the model in equation (2), the median success probability in the k^th treatment group for the population can be estimated as M(p_k)= g⁻¹ (ν_k), k =1, 2. And, its mean can be estimated as

$$E(p_k)={\int }_{-\infty }^{+\infty }{g}^{-1}({\nu }_k+z){\sigma }_k^{-1}\phi (z/{\sigma }_k)dz,k=1,2,$$ (3)

where φ() is the standard Gaussian density function. Based on the bivariate normality assumption of (ν_1i, ν_2i)^T, the expected success probability in group k (k = 1, 2) at a given success probability in group l (l = 1, 2) in the transformed scale is given by

$$\text{E}[\text{g}(p_k)\mid \text{g}(p_l)]={\nu }_k+\rho {\sigma }_k/{\sigma }_l[\text{g}(p_l)-{\nu }_l]=({\nu }_k-\rho {\nu }_l{\sigma }_k/{\sigma }_l)+\rho {\sigma }_k/{\sigma }_l\text{g}(p_l),k\ne l;k,l=1,2.$$ (4)

Thus, the BGLMM implies a linear relationship between p₁ and p₂ on a transformed scale.

2.2 Bivariate Beta-Binomial Models

As an alternative, beta-binomial distributions can be used to model the success probabilities of the treatment and control groups to account for the within-study correlation. To allow for the possible correlation between the success probabilities in the treatment and control groups, Lee²² introduced a class of bivariate Beta-binomial distributions using the framework introduced by Sarmanov.²³ Such bivariate Beta-binomial distributions can be used to model the success probabilities of (p_1i, p_2i) jointly as follows,

$$f(p_{1i},p_{2i})=f(p_1,p_2)=\left[1+\omega \prod _{k=1}^2(p_k-{\mu }_k)\right]\prod _{k=1}^2\left[\frac{{(p_k)}^{{\alpha }_k-1}{(1-p_k)}^{{\beta }_k-1}}{B({\alpha }_k,{\beta }_k)}\right],$$ (5)

where α_k, β_k > 0, $E(p_k)={\mu }_k=\frac{{\alpha }_k}{{\alpha }_k+{\beta }_k}$ and $B({\alpha }_k,{\beta }_k)={\int }_0^1{x}^{{\alpha }_k-1}{(1-x)}^{{\beta }_k-1}dx$. The bivariate beta-binomial distribution specified by equation (5) has several attractive features. First, the marginal distribution of p_k follows a Beta distribution f(p_k)= Beta(α_k, β_k). Secondly, a correlation between the success probabilities in the treatment and control groups is allowed and modeled by ρ= ωδ₁δ₂, where ${{\delta }_k}^2=\frac{{\alpha }_k{\beta }_k}{{({\alpha }_k+{\beta }_k)}^2({\alpha }_k+{\beta }_k+1)}$ is the variance of p_k. When ω = 0, equation (5) collapses to the product of two univariate Beta densities, corresponding to independent Beta distributions for p₁ and p₂. To ensure a valid joint probability density function, ω must satisfy the condition

$$-{[max({\alpha }_1{\alpha }_2,{\beta }_1{\beta }_2)]}^{-1}\prod _{k=1}^2({\alpha }_k+{\beta }_k)\le \omega \le {[max({\alpha }_1{\beta }_2,{\beta }_1{\alpha }_2)]}^{-1}\prod _{k=1}^2({\alpha }_k+{\beta }_k).$$ (6)

To ensure a valid joint probability density function for (p₁, p₂) and avoid computational difficulties, we re-parameterize ω by the unconstrained parameter η as follows,

$$\omega =\prod _{k=1}^2({\alpha }_k+{\beta }_k)\left\{\frac{exp(\eta )}{1+exp(\eta )}{[max({\alpha }_1{\beta }_2,{\beta }_1{\alpha }_2)]}^{-1}-\frac{1}{1+exp(\eta )}{[max({\alpha }_1{\alpha }_2,{\beta }_1{\beta }_2)]}^{-1}\right\}.$$ (7)

The conditional distribution of p_k for a chosen p_l is $f(p_k\mid p_l)=\frac{f(p_1,p_2)}{f(p_l)}=f(p_k)\left[1+\omega {\displaystyle \prod _{k=1}^2}(p_k-{\mu }_k)\right](k\ne l;k=1,2;l=1,2)$. Thus, the conditional mean of p_k for a chosen p_l is given by E(p_k | p_l) = μ_k + ρδ_k/δ_l(p_l − μ_l), and the conditional variance of p_k for a chosen p_l is given by

$$\mathit{Var}(p_k\mid p_l)={\delta }_k^2\left\{1-{\rho }^2\frac{{(p_l-{\mu }_l)}^2}{{\delta }_l^2}\right\}+\rho \frac{p_l-{\mu }_l}{{\delta }_k{\delta }_l}\left\{\frac{2{\beta }_k{\alpha }_k({\beta }_k-{\alpha }_k)}{({\alpha }_k+{\beta }_k+2)({\alpha }_k+{\beta }_k+1){({\alpha }_k+{\beta }_k)}^3}\right\}.$$ (8)

Thus, the bivariate Beta-Binomial model implies a linear relationship between p₁ and p₂ on the original scale. The unconditional joint probability density function for (X₁= x_1i, X₂= x_2i) is,

$$\begin{array}{l}P(X_1=x_{1i},X_2=x_{2i}\mid n_{1i},n_{2i},{\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2,\omega )\\ ={\int }_0^1{\int }_0^1\prod _{k=1}^2\left(\begin{array}{c}{n}_{ki}\\ x_{ki}\end{array}\right){(p_{ki})}^{x_{ki}}{(1-p_{ki})}^{n_{ki}-x_{ki}}\left[1+\omega \prod _{k=1}^2(p_{ki}-{\mu }_{ki})\right]\prod _{k=1}^2\left[\frac{{(p_{ki})}^{{\alpha }_k-1}{(1-p_{ki})}^{{\beta }_k-1}}{B({\alpha }_k,{\beta }_k)}\right]{dp}_{1i}{dp}_{2i}\\ =\prod _{k=1}^2\left(\begin{array}{c}{n}_{ki}\\ x_{ki}\end{array}\right)\frac{B({\alpha }_k+x_{ki},{\beta }_k+n_{ki}-x_{ki})}{B({\alpha }_k,{\beta }_k)}\times \left[1+\omega \prod _{k=1}^2\frac{x_{ki}-n_{ki}{\mu }_k}{{\alpha }_k+{\beta }_k+n_{ki}}\right],\end{array}$$ (9)

which leads to the following log-likelihood function for the observed 2 × 2 tables after ignoring some constant,

$$logL(\omega ,{\alpha }_k,{\beta }_k)=\sum _{i=1}^N\sum _{k=1}^2\left\{logB({\alpha }_k+x_{ki},{\beta }_k+n_{ki}-x_{ki})-logB({\alpha }_k,{\beta }_k)+log\left[1+\omega \prod _{k=1}^2\frac{x_{ki}-n_{ki}{\mu }_k}{{\alpha }_k+{\beta }_k+n_{ki}}\right]\right\},$$ (10)

where ω must satisfy the condition in equation (6) to ensure nonnegative probability.

2.3 Marginal Treatment Effects: Risk Difference and Risk Ratio

Although the issue of deciding which effect measure to use in a particular application is non-trivial,¹ we focus on the estimation of risk ratio (RR) and risk difference (RD) here for two reasons: 1) the interpretation of odds ratio (OR) as an estimate of RR often leads to exaggerated associations when the binary outcome of interest is common;^24–26 and 2) the well-known non-collapsibility issue related to OR makes it undesirable in interpretation and estimation.^{27, 28} For example, in the presence of effect modification, when an exposure increases risk but all risks are less than 0.5, it is possible for the relative risk and the risk difference to change in the same direction, but the odds ratio to change in the opposite direction.²⁹ In this article, we focus on the overall marginal (or population averaged) treatment (or exposure) effect, as suggested by McCullagh,^{30, 31} which is defined as the risk difference (RD) = E(p₁) − E(p₂) and the risk ratio (RR) = E(p₁)/E(p₂), where $E(p_k)={\int }_{-\infty }^{+\infty }{g}^{-1}({\nu }_k+z){\sigma }_k^{-1}\phi (z/{\sigma }_k)dz$ for the BGLMM and E(p_k) = α_k/(α_k + β_k) for the bivariate Beta-binomial model, k =1,2. Furthermore, although the computation of E(p_k)(k = 1,2) from BGLMM involves integration, there is a closed-form formula of $E(p_k)=\mathrm{\Phi }\left({\nu }_k/\sqrt{1+{\sigma }_k^2}\right)$ (k= 1,2) for the bivariate probit random effects model, and a well-established approximation formula of $E(p_k)\approx \text{expit}\left({\nu }_i/\sqrt{1+C^2{\sigma }_k^2}\right)$ (k= 1,2) for the bivariate logit random effects models,³² where $C=16\sqrt{3}/(15\pi )$. For the complementary log-log random effects models, E(p_k) can be easily computed by numerical integration, for example, by the trapezoidal rule with 1,000 equal space subintervals as implemented in this article.

2.4 Model Implementation

The bivariate Beta-binomial model and the bivariate generalized linear mixed model can be fitted using commonly used statistical software. We implement it through the SAS NLMIXED procedure (SAS Institute Inc., Cary, NC), which maximizes the likelihood function by dual quasi-Newton optimization techniques for the bivariate Beta-binomial model, and uses an adaptive Gaussian quadrature to approximate the likelihood integrated over the random effects for BGLMM.³³ Furthermore, the delta method built in SAS NLMIXED is used to compute the population averaged overall treatment effect estimates and their standard errors based on the normal approximation. To select a model that can give a better goodness of fit, either the finite sample corrected Akaike’s Information Criterion (AIC_C) or the Bayesian Information Criterion (BIC) can be used as the guideline.³⁴

3. Two Case Studies

To illustrate and compare the performance of the bivariate Beta-binomial model and the bivariate generalized linear mixed effects models discussed in this article, we apply them to two recently published meta-analyses.

3.1 Example 1: Meta-analysis of type 2 diabetes mellitus after gestational diabetes

Recently, Bellamy et al.²¹ presented an interesting comprehensive systematic review and meta-analysis to assess the strength of association between gestational diabetes and type 2 diabetes mellitus. In summary, 20 cohort studies were included in the meta-analysis with 675,455 women and 10,859 type 2 diabetic events. Table 1 shows the frequencies of the diabetic events for these 20 studies.

Table 1.

Example 1: Data from a Meta-analysis of Studies on Type 2 Diabetes Mellitus after Gestational Diabetes²¹

Study	Type 2 Diabetes Mellitus with Gestational Diabetes		Type 2 Diabetes Mellitus without Gestational Diabetes
	# events	# observations	# events	# observations
1	2874	21823	6628	637341
2	71	620	22	868
3	21	68	0	39
4	43	166	150	2242
5	53	295	1	111
6	405	5470	16	783
7	6	70	7	108
8	13	35	8	489
9	7	23	0	11
10	23	435	0	435
11	44	696	0	70
12	21	229	1	61
13	10	28	0	52
14	15	45	1	39
15	105	801	7	431
16	10	15	0	35
17	33	241	0	57
18	14	47	3	47
19	224	615	18	328
20	5	145	0	41

We fitted the bivariate Beta-binomial and the bivariate generalized linear mixed effects models as described in Section 2 to study the association between gestational diabetes and type 2 diabetes mellitus. Table 2 presents the parameter estimates and their standard errors, including the population averaged risk of type 2 diabetes mellitus for those with and without gestational diabetes, population averaged risk difference and risk ratio, and the goodness of fit measures including the finite sample corrected Akaike’s Information Criterion (AIC_C) and the Bayesian Information Criterion (BIC). As shown in Table 2, there is not enough evidence to support that the risks of type 2 diabetes mellitus for those with and without gestational diabetes are correlated within studies from both the bivariate Beta-binomial model and the bivariate generalized linear mixed models with three link functions, i.e., the models with ρ = 0 provide better goodness-of-fit for all four models considered. Note that the results from different models are very similar here. Based on AIC_C and BIC, the best fitted model is a bivariate logit generalized linear mixed effects model with ρ = 0 and ${\sigma }_1^2={\sigma }_2^2$. Based on this model, the population averaged risk of type 2 diabetes mellitus for those with and without gestational diabetes are estimated to be 0.200 (standard error = 0.031) and 0.025 (standard error = 0.006) respectively. The population averaged risk difference is estimated to be 0.175 (standard error = 0.031), where the population averaged risk ratio is estimated to be 7.948 (standard error = 2.167). It is interesting to note that the population averaged risk ratio estimates from all models are slightly higher than what Bellamy et al.²¹ reported (i.e., 7.43 with 95% confidence interval of 4.79 to 11.51) based on the random effects model by DerSimonian and Laird.⁴ The reason might be the fact that an ad hoc continuity correction was implemented for the studies with zero diabetic events in the group without gestational diabetes in Bellamy et al.,²¹ where our models do not.

Table 2.

Point Estimates (Standard Errors) for Meta-analysis of Studies on Type 2 Diabetes Mellitus after Gestational Diabetes²¹

Model Description	Modification from the standard model	Probability P̂1	Probability P̂ 2	Risk Difference RD	Relative Risk RR	BIC	AICC
Bivariate Probit GLMM
	No change	.2014 (.0306)	.0238 (.0061)	.1776 (.0292)	8.4556 (2.1305)	306.1	302.8
	Random effects v1i = v2i	.1934 (.0262)	.0254 (.0064)	.1680 (.0202)	7.6029 (0.9612)	341.6	339.3
	Correlation ρ = 0	.2036 (.0313)	.0226 (.0059)	.1810 (.0319)	9.0077 (2.7280)	305.1	302.2
	Correlation ρ = 0 and ${\sigma }_1^2={\sigma }_2^2$	.2002 (.0291)	.0237 (.0067)	.1765 (.0295)	8.4589 (2.6317)	302.5	300.1
Bivariate Logit GLMM
	No change	.1973 (.0307)	.0266 (.0068)	.1707 (.0293)	7.4154 (1.8853)	306.2	303.0
	Random effects v1i = v2i	.2012 (.0298)	.0233 (.0051)	.1779 (.0251)	8.6212 (0.8395)	356.3	354.0
	Correlation ρ = 0	.1991(.0313)	.0255 (.0066)	.1736 (.0320)	7.8038 (2.3676)	305.2	302.4
	Correlation ρ = 0 and ${\sigma }_1^2={\sigma }_2^2$	.1999 (.0312)	.0252 (.0059)	.1748 (.0314)	7.9476 (2.1669)	302.2	299.9
Bivariate Complementary log-log GLMM
	No change	.2037 (.0353)	.0224 (.0063)	.1813 (.0353)	9.0950 (2.8922)	306.2	303.0
	Random effects v1i = v2i	.2113 (.0361)	.0197 (.0055)	.1916 (.0355)	10.7288 (3.2038)	373.3	371.0
	Correlation ρ = 0	.2051 (.0359)	.0209 (.0059)	.1842 (.0375)	9.9088 (3.5280)	305.4	302.6
	Correlation ρ = 0 and ${\sigma }_1^2={\sigma }_2^2$	.2082 (.0364)	.0225 (.0057)	.1857 (.0376)	9.2608 (3.0130)	302.6	300.3
Bivariate Beta-binomial model
	No change	.2039 (.0308)	.0222 (.0056)	.1817 (.0313)	9.1890 (2.6929)	307.7	307.0
	Correlation ρ = 0	.2041 (.0308)	.0222 (.0054)	.1819 (.0313)	9.2114 (2.6571)	304.7	303.4

^*AIC_C = the finite sample corrected Akaike’s Information Criterion and BIC = the Bayesian Information Criterion. P̂₁= the risk of type 2 Diabetes Mellitus with gestational diabetes, P̂₂= the risk of type 2 Diabetes Mellitus without gestational diabetes. RD = P̂₁ − P̂₂; RR = P̂₁/P̂₂.

Because one of the studies has almost all the cases (9502 out of 10859 cases), we did a sensitivity meta-analysis by excluding that study. The results are presented in Appendix eTable 1. In summary, it suggests similar conclusions. Specifically, the best fitted model is a bivariate probit generalized linear mixed effects model with ρ = 0 and ${\sigma }_1^2={\sigma }_2^2$, and the population averaged risk of type 2 diabetes mellitus for those with and without gestational diabetes are estimated to be .205 (standard error = 0.031) and 0.025 (standard error = 0.007) respectively. The population averaged risk difference is estimated to be 0.181 (standard error = 0.032), where the population risk ratio is estimated to be 8.371 (standard error = 2.756).

3.2 Example 2: Meta-analysis of the risk of myocardial infarction with Rosiglitazone

To investigate whether rosiglitazone, a drug for treating type 2 diabetes mellitus, significantly increases the risk of myocardial infarction (MI) or cardiovascular disease (CVD)-related death, Nissen and Wolski⁵ performed a meta-analysis of 48 clinical trials that satisfied the inclusion criteria for their analysis. Among them, 10 studies have no MI events and 25 studies have no CVD-related deaths, which were simply excluded by Nissen and Wolski from their analysis. This meta-analysis data set has been reanalyzed by Shuster et al.,³⁵ Tian et al.⁸ and others,^36–38 and updated by Dahabreh.³⁹ For the illustration purpose, we will only focus on the association between rosiglitazone usage and the risk of myocardial infarction. In summary, 86 out of 16,856 in the rosiglitazone group and 72 out of 12,962 in the control group had MI event in the 48 clinical trials.

Similar to Section 3.1, we fitted the bivariate Beta-binomial and the bivariate generalized linear mixed effects models as described in Section 2 on those 48 clinical trials to study the association between rosiglitazone usage and the risk of MI in type 2 diabetes mellitus patients. Table 3 presents the parameter estimates and their standard errors, including the population averaged risk of MI event for those with and without rosiglitazone usage, the population averaged risk difference and risk ratio, and the goodness of fit measurements including the finite sample corrected Akaike’s Information Criterion (AIC_C) and the Bayesian Information Criterion (BIC). The BGLMM models assuming random effects v_1i = v_2i with any of the three link functions provide better model fit than the bivariate Beta-binomial model with either ρ ≠ 0 or ρ = 0. Based on AIC_C and BIC, the bivariate logit and complementary log-log generalized linear mixed effects models with random effects v_1i = v_2i provide similar best fit.

Table 3.

Point Estimates (Standard Errors) for Meta-analysis of the Risk of Myocardial Infarction with Rosiglitazone⁵

Model Description	Modification from the standard model	Probability P̂ 1	Probability P̂ 2	Risk Difference RD	Relative Risk RR	BIC	AICC
Bivariate Probit GLMM
	No change	.00493 (.00091)	.00359 (.00079)	.00133 (.00075)	1.3710 (.2455)	262.4	253.7
	Random effects v1i = v2i	.00494 (.00093)	.00364 (.00077)	.00130 (.00074)	1.3577 (.2325)	254.8	249.5
	Correlation ρ = 0	.00486 (.00095)	.00379 (.00914)	.00107 (.00132)	1.2811 (.3973)	272.4	265.3
	Correlation ρ = 0 and ${\sigma }_1^2={\sigma }_2^2$	.00491 (.00096)	.00377 (.00087)	.00115 (.00128)	1.3037 (.3884)	269.2	263.8
Bivariate Logit GLMM
	No change	.00624 (.00131)	.00484 (.00115)	.00141 (.00086)	1.2929 (.2023)	261.9	253.2
	Random effects v1i = v2i	.00627 (.00133)	.00480 (.00109)	.00146 (.00085)	1.3046 (.1933)	254.2	248.9
	Correlation ρ = 0	.00618 (.00140)	.00508 (.00141)	.00110 (.00199)	1.2164 (.4359)	272.7	265.7
	Correlation ρ = 0 and ${\sigma }_1^2={\sigma }_2^2$	.00627 (.00134)	.00496 (.00113)	.00131 (.00153)	1.2643 (.3441)	268.8	263.5
Bivariate Complementary log-log GLMM
	No change	.00501 (.00094)	.00353 (.00099)	.00148 (.00092)	1.4187 (.3389)	261.9	253.2
	Random effects v1i = v2i	.00499 (.00094)	.00373 (.00088)	.00126 (.00075)	1.3390 (.2399)	254.2	248.9
	Correlation ρ = 0	.00494 (.00097)	.00367 (.00106)	.00126 (.00133)	1.3423 (.4361)	272.7	265.6
	Correlation ρ = 0 and ${\sigma }_1^2={\sigma }_2^2$	.00497 (.00097)	.00384 (.00092)	.00113 (.00130)	1.2945 (.3885)	268.8	263.5
Bivariate Beta-binomial model
	No change	.00491 (.00090)	.00385 (.00138)	.00111 (.00126)	1.2915 (.3823)	277.8	269.9
	Correlation ρ = 0	.00491 (.00090)	.00380 (.00088)	.00111 (.00126)	1.2932 (.3826)	274.0	267.5

^*AIC_C = the finite sample corrected Akaike’s Information Criterion and BIC = the Bayesian Information Criterion. P̂₁ = the risk of myocardial infarction in the rosiglitazone group, P̂₂ = the risk of myocardial infarction in the control group, RD = P̂₁ − P̂₂; RR = P̂₁/P̂₂.

It is worthy to mention that for the logit BGLMM model, it seems that the approximation of $E(p_k)\approx \text{expit}\left({\nu }_k/\sqrt{1+C^2{\sigma }_k^2}\right)$, where $C=16\sqrt{3}/(15\pi )$, slightly overestimate the population averaged probability of MI for each group. For example, for the logit BGLMM model assuming random effects v_1i = v_2i, the estimated population averaged probabilities of MI in the rosiglitazone treatment and control groups are 0.00627 (standard error = 0.00133) and 0.00480 (standard error = 0.00109) using the approximation of $E(p_k)\approx \text{expit}\left({\nu }_k/\sqrt{1+C^2{\sigma }_k^2}\right)$. While, using the numerical integration by $E(p_k)={\int }_{-\infty }^{+\infty }1/[1+exp(-{\nu }_k-z)]{\sigma }_k^{-1}\phi (z/{\sigma }_k)dz$, the corresponding estimates are 0.00493 (standard error = 0.00140) and 0.00366 (standard error = 0.00114), which are consistent with the estimates from other models. However, we notice that the overestimation of the population averaged probability of MI using the approximation formula of the logit BGLMM does not seem to have any noticeable effects on the estimation of risk difference or risk ratio.

4. Discussion

In this article, we discussed bivariate Beta-binomial models derived from Sarmanov family of bivariate distributions and bivariate generalized linear mixed effects models using a general link function for meta-analysis of 2 by 2 tables in comparative clinical studies. Specifically, we have discussed logit, probit and complementary log-log link functions as special cases. These bivariate random effects models naturally account for the potential correlation between treatment (or exposure) and control groups within studies. Moreover, they can be used to make valid inferences using all available data without using ad hoc continuity corrections for the sparse data scenario. We illustrated the utilization of the bivariate random effects models in two recent meta-analysis data sets, which emphasizes the importance of model selection. In particular, based on AIC_C and BIC, in the example one, the best fitted model is a bivariate logit generalized linear mixed effects model with ρ = 0 and ${\sigma }_1^2={\sigma }_2^2$, which suggests that the study-specific risks of type 2 diabetes mellitus (in logit scale) for those with and without gestational diabetes are independent and have similar variations. In the example two, both the bivariate logit and complementary log-log generalized linear mixed effects models with random effects v_1i = v_2i provide similar best fit, which suggests that one can reasonably assume a fixed effect of rosiglitazone on the risk of myocardial infarction (in logit or complementary log-log scale). Furthermore, we provided methods to estimate the population averaged risk difference and relative risk. It is worth to noting that the bivariate Beta-binomial model and the bivariate generalized linear mixed effects models involves two different distributional assumptions, one would imagine that their performance would heavily depend on whether the distributional assumptions approximate the underline data generating mechanism. In particular, the bivariate generalized linear mixed effects models implies a linear relationship between p₁ and p₂ on a transformed scale, and after transforming back to the scale of probability, the relation between p₁ and p₂ is no longer linear. The Beta-binomial model implies a linear relationship between p₁ and p₂ on the original scale. So which model works better in a particular application depends on whether the relation between p₁ and p₂ is linear on the original scale or on the transformed scale. We suggest that fitting both models and comparing goodness-of-fit to select the best model to make inference in practice.

Alternative approaches using Bayesian methods can be fitted by free downloadable software such as WinBUGS, for example, by the Bayesian random effect models as in Warn, Thompson and Spiegelhalter.⁴⁰ However, Warn et al. ⁴⁰ focused on the conditional treatment effects. Here, we focus on the overall marginal (or population averaged) treatment effects, as suggested by McCullagh.^{30, 31} Remark that our parameterization of BGLMM is slightly different from the random effects models by Smith et al.⁴¹ and Warn et al.⁴⁰ Specifically, Smith et al. ⁴¹ considered a Bayesian logit random effects model assuming logit (p_1i)= μ_i − δ_i/2, logit (p_2i) = μ_i + δ_i/2, δ_i ~ N(δ_i, σ²), and non-informative priors for the average event rates, μ_i s, which are treated as the nuisance parameters. It implicitly restricts Var[logit(p_1i)] = Var[logit(p_2i)], i.e., restricting ${\sigma }_1^2={\sigma }_2^2$ as in the BGLMM model. Warn et al.⁴⁰ assumes that g(p_1i) = μ_i, g(p_2i) = μ_i + δ_i and δ_i ~ N(δ, σ²) where g() is a link function, which implicitly restricts Var[g(p_1i)] ≤ Var[g(p_2i)], i.e., restricting ${\sigma }_1^2\le {\sigma }_2^2$ in our BGLMM parameterization. It is worth pointing out that our purpose here is not to demonstrate the advantage of our approach over a Bayesian approach, because both BGLMM and Bivariate Beta-binomial models can be fitted using a Bayesian approach. For the general model that we considered in equation (2), we do not make any restrictions on the variances of ${\sigma }_1^2$ and ${\sigma }_2^2$. Furthermore, the bivariate Beta-binomial model and the bivariate generalized linear mixed models we proposed in this article do not include any study-level or individual level covariates. It is straightforward to include covariates when using the BGLMM through the SAS NLMIXED procedure.