Relative efficiency of unequal versus equal cluster sizes in cluster randomized trials using generalized estimating equation models

Abstract

There is growing interest in conducting cluster randomized trials (CRTs). For simplicity in sample size calculation, the cluster sizes are assumed to be identical across all clusters. However, equal cluster sizes are not guaranteed in practice. Therefore, the relative efficiency (RE) of unequal versus equal cluster sizes has been investigated when testing the treatment effect. One of the most important approaches to analyze a set of correlated data is the generalized estimating equation (GEE) proposed by Liang and Zeger, in which the “working correlation structure” is introduced and the association pattern depends on a vector of association parameters denoted by ρ. In this paper, we utilize GEE models to test the treatment effect in a two-group comparison for continuous, binary, or count data in CRTs. The variances of the estimator of the treatment effect are derived for the different types of outcome. RE is defined as the ratio of variance of the estimator of the treatment effect for equal to unequal cluster sizes. We discuss a commonly used structure in CRTs- exchangeable, and derive the simpler formula of RE with continuous, binary, and count outcomes. Finally, REs are investigated for several scenarios of cluster size distributions through simulation studies. We propose an adjusted sample size due to efficiency loss. Additionally, we also propose an optimal sample size estimation based on the GEE models under a fixed budget for known and unknown association parameter (ρ) in the working correlation structure within the cluster.

1. Introduction

In recent years, there has been growing interest in conducting cluster randomized trials (CRTs) (Campbell et al., 2000; Gulliford et al., 2014; Gravenstein et al., 2016; Kalfon et al., 2016; Mehring et al., 2016; Nagayama et al., 2016; Yamagata et al., 2016). CRTs are performed when randomization at the individual-level is practically infeasible or may lead to severe estimation bias of treatment effect. It is reasonable to expect correlation among individuals in the same cluster, since they may share similar characteristics or be exposed to common factors. The degree of such similarity is commonly quantified by the intracluster correlation coefficient (ICC). Even if the ICCs are pretty small for most CRTs (Murray et al., 2004; Turner et al., 2004), they must be considered in study design and data analysis to avoid underestimation of variance of the treatment effect and inflated type I error rates (Murray, Varnell, et al., 2004).

Methods of analyzing correlated data have been extensively developed. One of the most important approaches is the generalized estimating equation (GEE) proposed by Liang and Zeger (Liang et al., 1986), which fits a marginal model in the context of longitudinal studies. In order to describe the pattern of measures within the individual, the idea of “working correlation structure” is introduced and the pattern depends on a vector of association parameters denoted by ρ. The advantage of this approach is the consistent estimates provided that the marginal model is correctly specified, even if the working correlation matrix is incorrectly assumed. Therefore, GEEs have been commonly applied (Sanda et al., 2008; Lin et al., 2015; Park et al., 2015; Toriola et al., 2015; Jeffe et al., 2016).

Sample size calculation or power estimation is an important topic in study design for investigators. The equations of sample size for CRTs have been published as a function of ICC or the coefficient of variation (CV) of cluster sizes (Donner et al., 1981; Liu et al., 1997; Shih, 1997; Rosner et al., 2003; Eldridge et al., 2006; Austin, 2007; Candel et al., 2008; Van Breukelen et al., 2008; Teerenstra et al., 2010; Rosner et al., 2011; Van Breukelen et al., 2012; Amatya et al., 2013). Shih (Shih, 1997) provides a sample size formula based on the GEE methods for CRTs designed to test the treatment effect. The general formula for a two-group comparison is given. Specifically, the formulas for continuous and binary outcomes are demonstrated with the assumptions of equal cluster sizes n and the exchangeable working correlation R(ρ), i.e., 1 for diagonal entries and ρ otherwise. For count data, Bhaumik and Gibbons (Amatya, Bhaumik, et al., 2013) consider population-average estimators using GEE models for cluster-level randomized design. They assume half of the clusters are randomized to the treatment and the other half are randomized to the control. The asymptotic variance of the treatment effect estimator is derived and the required number of clusters is calculated with the same assumptions: equal cluster sizes n and the exchangeable working correlation R(ρ).

In the design of CRTs, the required sample sizes include the number of clusters m and cluster sizes n_i, i = 1, ⋯, m. For simplicity, the cluster sizes are assumed to be identical across clusters, i.e., n_i = n for all i, resulting in total sample size of N = nm. In general, the cluster size n is a pre-determined number, therefore the sample size calculation is for the number of clusters only. However, equal cluster sizes are not guaranteed in practice. The relative efficiency (RE) of unequal versus equal cluster sizes has been investigated when testing the treatment effect. For continuous outcomes in cluster randomized studies, Manatunga et al (Manatunga et al., 2001) derive sample size estimation while accounting for the variability due to cluster size and a lower bound for RE is given. For both CRTs and person randomization within centers, Breukelen et al (Van Breukelen et al., 2007) address the RE of unequal versus equal cluster sizes with continuous outcomes. They investigate RE based on maximum likelihood parameter estimation for a range of cluster size distributions. Additionally, they present an approximate formula for computing the RE as a function of the mean and variance of cluster size and the intraclass correlation (ρ). They conclude that the loss of efficiency due to variation of cluster sizes rarely exceeds 10 percent and can be compensated by sampling 11 percent more clusters. Candel and Breukelen (Candel et al., 2010) give the adjusted sample size formula for varying cluster sizes with a binary outcome in CRTs. They derive the asymptotic RE of unequal versus equal cluster sizes from first-order marginal quasi-likelihood (MQL) estimation with mixed effect logistic regression model. A simpler formula of sample size calculation is presented to estimate the efficiency loss due to variation in cluster sizes. They find that 14% more clusters are needed to cover the efficiency loss in many cases.

In this paper, we will utilize GEE models to test the treatment effect in a two-group comparison for continuous, binary, or count data in CRTs. The variances of the estimator of the treatment effect will be derived for different types of outcome. Obviously, they are dependent on which working correlation structure is chosen. As shown (Shih, 1997; Amatya, Bhaumik, et al., 2013), researchers often consider the exchangeable structure since the independent working correlation structure is a special case of exchangeable and first order auto-regressive (AR(1)) fits time-structured data more appropriately. Here, we will also discuss this commonly used structure. RE is defined as the ratio of variance of the estimator of the treatment effect for equal to unequal cluster sizes. We will derive a simpler formula of RE with continuous, binary, and count outcomes. Finally, REs are investigated for several scenarios of cluster sizes distribution through simulation studies. We will propose the adjusted sample size due to the efficiency loss. Additionally, cost is an important factor in study design of CRTs since it is associated with recruiting an additional cluster instead of an additional subject in an individual-level randomized trial. Under a fixed budget we also have the same proposal about the optimal sample size, which maximizes the power, based on the GEE models for known and unknown association parameter (ρ) in the working correlation structure within the cluster.

The outline of this paper is as follows. In section 2, we briefly summarize the GEE methods proposed by Liang and Zeger (Liang and Zeger, 1986) and derive the variance of the estimator of the treatment for three kinds of outcomes (continuous, binary, and count) in a two-group comparison. Section 3 introduces the REs of unequal versus equal cluster sizes for the treatment effect. The formula for RE is derived for the exchangeable working correlation structure. Section 4 presents the possible patterns of cluster size distribution and the results through simulation studies that investigate the REs. In section 5, we show the sample size formulas under two different situations- no cost constrains and given a fixed budget. The practical uses are discussed when planning a CRT, followed by discussion of the limitations and directions for future research.

2. Generalized estimating equation models

Let ${\mathit{y}}_i=(y_{i1}, y_{i2},\cdots , y_{i{n}_i}{)}^{\prime }$ be a vector of responses from the ith cluster, i = 1, ⋯, m. The responses are assumed to be independent across clusters but correlated within each cluster. The marginal model is

$$g\left({\mu }_{ij}\right)=C_{ij}^{\prime }\varnothing , i=1, \cdots , m; \text{j}=1,\cdots ,{\text{n}}_i,$$ (1)

where μ_ij = E(y_ij) and g(∙) is a linking function. C_ij is a vector of covariates, and ∅ is an unknown l × 1 vector of regression coefficients. This model specifies a relationship between μ_ij and covariates C_ij. The conditional variance of y_ij given C_ij is defined as var (y_ij|C_ij) = γ(μ_ij)θ, where γ is a known variance function of μ_ij nd θ is a scale parameter. The mean of y_i is denoted by μ_i = E(y_i) and the variance-covariance matrix for y_i is denoted by ${\mathit{V}}_i=\theta {\mathit{A}}_i^{1/2}{\mathit{R}}_i\left(\text{ρ}\right){\mathit{A}}_i^{1/2}$, where ${\mathit{A}}_i=\text{diag}\left\{\gamma \left({\mu }_{i1}\right),\cdots , \gamma \left({\mu }_{i{n}_i}\right)\right\}$ and working correlation structure R_i(ρ) describes the pattern of measures within the ith cluster. R_i(ρ) is an n_i × n_i matrix and depends on a vector of association parameters denoted by ρ. Both γ and θ are dependent on the distribution of responses. For instance, if y_ij is continuous, γ(μ_ij) = 1 and θ is the random error variance; if y_ij is binary, γ(μ_ij) = μ_ij(1 − μ_ij) and θ=1; If y_ij is count, γ(μ_ij) = μ_ij and θ = 1.

The estimate of ∅ is obtained by solving the following estimating equation

$$U(\varnothing )=\sum _{i=1}^m{\mathit{D}}_i^{\prime }{V}_i^{-\mathbf{1}}\left({\mathit{y}}_i-{\mathit{\mu }}_i\right)=0,$$ (2)

Where D_i = ∂μ_i/∂∅′. Let $\widehat{\varnothing }$ be the solution of Eq. (2). Liang and Zeger (Liang and Zeger, 1986) show that $\sqrt{m}(\widehat{\varnothing }-\varnothing )$ is asymptotically multivariate normal with covariance matrix

$${\mathit{V}}_{\mathit{R}}=\underset{m\to \infty }{\text{lim}}m\left({\text{Σ}}_1^{-1}{\text{Σ}}_0{\text{Σ}}_1^{-1}\right),$$ (3)

Where ${\text{Σ}}_1={\sum }_{i=1}^m{\mathit{D}}_i^{\prime }{V}_i^{-1}{\mathit{D}}_i$ and ${\text{Σ}}_0={\sum }_{i=1}^m{\mathit{D}}_i^{\prime }{V}_i^{-1}\text{cov}({\mathit{y}}_{\mathit{i}}|C_i)V_i^{-1}{\mathit{D}}_i$. As noted, GEE gives asymptotically consistent estimate $\widehat{\varnothing }$ even when the working correlation matrix is incorrectly assumed (Liang and Zeger, 1986).

For the purpose of sample size calculation at the design stage, we need to assume values for the variance and covariance of y_i. Therefore, Σ₀ = Σ₁ and ${\mathit{V}}_{\mathit{R}}={\text{lim}}_{m\to \infty }m{\text{Σ}}_1^{-1}$ when cov(y_i|C_i) is assumed to be same as V_i. Suppose we are interested in testing the treatment effect for a two-group comparison: the treated vs. control group. For simplicity, let us consider a special case with l = 2, where ∅ = (β₀, β₁)′ and the design matrices C_i. Specifically, the coefficient β₀ is the intercept, β₁ is the treatment effect, $C_{it}=\left({\mathbf{1}}_{n_i}, {\mathbf{1}}_{n_i}\right)$ for ith cluster assigned to the treated group, $C_{it}=\left({\mathbf{1}}_{n_i}, {\mathbf{0}}_{n_i}\right)$ for ith cluster assigned to the control group, and 1_n and 0_n an n × 1 vector of 1’s and 0’s, respectively. Let V_β denote the (2, 2)th element of V_R. Thus, $\sqrt{m}\left(\widehat{{\beta }_1}-{\beta }_1\right)$ has an asymptotically normal distribution N(0, V_β), equivalently, $\text{Var}\left(\widehat{{\beta }_1}\right)=V_{\beta }/m$. The cluster allocations of the treated and control groups are, m_t = mπ and m_c = m(1 – π), respectively. The hypotheses of interest are H₀:β₁ = 0 versus H₁:β₁ = β.

2.1. Continuous outcome

This section shows details of using an identity link function on the continuous outcome, that is, ${\mathit{\mu }}_i=C_i{}^{\prime }\varnothing$, and V_i = σ²R_i(ρ). With this setting, D_i = ∂μ_i/∂∅′ = C_i. Thus,

$$\begin{gathered} {\text{Σ}}_1=\frac{1}{{\sigma }^2}{\sum }_{i=1}^m{\mathit{C}}_i^{\prime }{R}_i^{-\mathbf{1}}{\mathit{C}}_i=\frac{1}{{\sigma }^2}{\sum }_{i\in \text{trt}}{\mathit{C}}_{it}^{\prime }{R}_i^{-\mathbf{1}}{\mathit{C}}_{it}+\frac{1}{{\sigma }^2}{\sum }_{i\in \text{cont}}{\mathit{C}}_{it}^{\prime }{R}_i^{-\mathbf{1}}{\mathit{C}}_{it} \\ =\frac{1}{{\sigma }^2}{\sum }_{i\in \text{trt}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)+\frac{1}{{\sigma }^2}{\sum }_{i\in \text{cont}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right)=\frac{1}{{\sigma }^2}\left(\begin{array}{cc}t+c& t\\ t& t\end{array}\right), \end{gathered}$$

Where i ∈ trt denotes all the clusters assigned to treated group, and i ∈ cont denotes all the clusters assigned to control group, respectively; $t={\sum }_{i\in \text{trt}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}$ and $c={\sum }_{i\in \text{cont}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}$. Then

$${\mathit{V}}_{\mathit{R}}={\text{lim}}_{m\to \infty }m{\text{Σ}}_1^{-1}={\sigma }^2\underset{m\to \infty }{\text{lim}}m\left(\begin{array}{cc}\frac{1}{c}& -\frac{1}{c}\\ -\frac{1}{c}& \frac{1}{t}+\frac{1}{c}\end{array}\right),\text{and}$$

$$\begin{gathered} {\mathit{V}}_{\beta }={\sigma }^2\underset{m\to \infty }{\text{lim}}m\left[\frac{1}{t}+\frac{1}{c}\right] \\ ={\sigma }^2\underset{m\to \infty }{\text{lim}}m\left[{\left({\sum }_{i\in \text{trt}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-1}{\mathbf{1}}_{n_i}\right)}^{-1}+{\left({\sum }_{i\in \text{cont}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}^{-1}\right] \\ ={\sigma }^2\underset{m\to \infty }{\text{lim}}\left[\frac{m}{\pi {\sum }_{i=1}^m{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}}+\frac{m}{\left(1-\pi \right){\sum }_{i=1}^m{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}}\right] \\ =\frac{{\sigma }^{2}m}{\pi \left(1-\pi \right)\left({\sum }_{i=1}^m{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}. \end{gathered}$$ (4)

If we assume equal cluster size n_i = n for all i, then

$$V_{\beta }=\frac{{\sigma }^2}{\pi \left(1-\pi \right){\mathbf{1}}_n^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_n}.$$ (5)

For the exchangeable working correlation structure, ${\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}=\frac{n_i}{1+\left(n_i-1\right)\text{ρ}}$. Thus,

$$V_{\beta }=\frac{{\sigma }^{2}m}{\pi \left(1-\pi \right)\left({\sum }_{i=1}^m\frac{n_i}{1+\left(n_i-1\right)\text{ρ}}\right)}$$

and

$$V_{\beta }=\frac{{\sigma }^2\left[1+\left(n-1\right)\text{ρ}\right]}{\pi \left(1-\pi \right)n},$$ (6)

for unequal and equal cluster sizes, respectively. Please note that Eq. (6) is same as (6) in Shih’s paper (Shih, 1997).

2.2. Binary outcome

For a binary outcome, we consider the logit model: ${\mathit{\mu }}_i={\mathit{p}}_i=\text{exp}\left(C_i{}^{\prime }\varnothing \right)/\left(1+\text{exp}\left(C_i{}^{\prime }\varnothing \right)\right)$. Here, p_ij = p₀ = exp(β₀)/(1 + exp(β₀)), for all the subjects in the control group and j = 1, ⋯, n_i; and p_ij = p₁ = exp(β₀ + β₁)/(1 + exp(β₀ + β₁)), for all the subjects in the treated group and j = 1, ⋯, n_i. We can show ${\beta }_1=\text{log}(\frac{p_1\left(1-p_0\right)}{\left(1-p_1\right)p_0})$, which is the logarithm of odds ratio for the response. Therefore, the null hypothesis H₀:β₁ = 0 is equivalent to p₀ = p₁ and the alternative hypothesis is p₀ ≠ p₁.

Under this scenario, ${\mathit{D}}_i=\partial {\mathit{\mu }}_i/\partial {\varnothing }^{\prime }=p_0\left(1-p_0\right)\left({\mathbf{1}}_{n_i}, {\mathbf{0}}_{n_i}\right)$ for the subjects in the control group, denoted by D_ic, and ${\mathit{D}}_i=p_1\left(1-p_1\right)\left({\mathbf{1}}_{n_i}, {\mathbf{1}}_{n_i}\right)$ for the subjects in the treated group, denoted by D_it, and V_i = p_i(1 − p_i)R_i(ρ). Thus,

$$\begin{gathered} {\text{Σ}}_1={\sum }_{i=1}^m\frac{1}{p_i\left(1-p_i\right)}{\mathit{D}}_i^{\prime }{R}_i^{-\mathbf{1}}{\mathit{D}}_i={\sum }_{i\in \text{trt}}\frac{1}{p_1\left(1-p_1\right)}{\mathit{D}}_{it}^{\prime }{R}_i^{-\mathbf{1}}{\mathit{D}}_{it}+{\sum }_{i\in \text{cont}}\frac{1}{p_0\left(1-p_0\right)}{\mathit{D}}_{ic}^{\prime }{R}_i^{-\mathbf{1}}{\mathit{D}}_{ic} \\ =p_1\left(1-p_1\right)\left({\sum }_{i\in \text{trt}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)+p_0\left(1-p_0\right)\left({\sum }_{i\in \text{cont}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right) \\ =\left(\begin{array}{cc}t+c& t\\ t& t\end{array}\right), \end{gathered}$$

where $c=p_0(1-p_0)({\sum }_{i\in \text{cont}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i})$ and $t=p_1\left(1-p_1\right)\left({\sum }_{i\in \text{trt}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)$. Then

$${\mathit{V}}_{\mathit{R}}=\underset{m\to \infty }{\text{lim}}m\left(\begin{array}{cc}\frac{1}{c}& -\frac{1}{c}\\ -\frac{1}{c}& \frac{1}{t}+\frac{1}{c}\end{array}\right)$$ (7)

And

$$\begin{gathered} V_{\beta }=\underset{m\to \infty }{\text{lim}}\left[\frac{m}{p_1\left(1-p_1\right)\left({\sum }_{i\in \text{trt}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}+\frac{m}{p_0\left(1-p_0\right)\left({\sum }_{i\in \text{cont}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}\right] \\ =\underset{m\to \infty }{\text{lim}}\left[\frac{m}{p_1\left(1-p_1\right)\pi \left({\sum }_{i=1}^m{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}+\frac{m}{p_0\left(1-p_0\right)\left(1-\pi \right)\left({\sum }_{i=1}^m{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}\right] \\ = \frac{m}{{\sum }_{i=1}^m{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}}\left(\frac{1}{\pi p_1\left(1-p_1\right)}+\frac{1}{\left(1-\pi \right)p_0\left(1-p_0\right)}\right). \end{gathered}$$ (8)

Assuming equal cluster size n_i = n for all i, we have

$$V_{\beta }=\frac{1}{{\mathbf{1}}_n^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_n}\left(\frac{1}{\pi p_1\left(1-p_1\right)}+\frac{1}{\left(1-\pi \right)p_0\left(1-p_0\right)}\right).$$ (9)

When the working correlation structure is exchangeable, Eq. (8) is equivalent to the one substituting Eq. (7) into (5) in Pan’s paper (Pan, 2001) and Eq. (8) derived by Shih (Shih, 1997). Additionally, Eq. (9) becomes $\frac{1+\left(n-1\right)\text{ρ}}{n}\left(\frac{1}{\pi p_1\left(1-p_1\right)}+\frac{1}{\left(1-\pi \right)p_0\left(1-p_0\right)}\right)$.

2.3. Count data

The log model is used to analyze count data, ${\mathit{\mu }}_i=\text{exp}\left(C_i{}^{\prime }\varnothing \right)$. ${\mathit{D}}_i=\partial {\mathit{\mu }}_i/\partial {\varnothing }^{\prime }=C_i\text{exp}\left(C_i{}^{\prime }\varnothing \right)$ and ${\mathit{V}}_i={\mathit{A}}_i^{1/2}{\mathit{R}}_i\left(\text{ρ}\right){\mathit{A}}_i^{1/2}$, where ${\mathit{A}}_{\mathit{i}}=\text{diag}\left\{\text{exp}\left(C_{i1}{}^{\prime }\varnothing \right),\cdots , \text{exp}\left(C_{i{n}_i}{}^{\prime }\varnothing \right)\right\}$. Specifically, ${\mathit{D}}_{ic}=C_{ic}\text{exp}\left(C_{ic}{}^{\prime }\varnothing \right)=e^{{\beta }_0}\left({\mathbf{1}}_{n_i}, {\mathbf{0}}_{n_i}\right)$ for the subjects in the control group, and ${\mathit{D}}_{it}=e^{{\beta }_0+{\beta }_1}\left({\mathbf{1}}_{n_i}, {\mathbf{1}}_{n_i}\right)$ for the subjects in the treated group. Thus,

$$\begin{gathered} {\text{Σ}}_1={\sum }_{i\in \text{trt}}{\mathit{D}}_{it}^{\prime }{V}_{it}^{-1}{\mathit{D}}_{it}+{\sum }_{i\in \text{cont}}{\mathit{D}}_{it}^{\prime }{V}_{it}^{-1}{\mathit{D}}_{it} \\ =e^{{\beta }_0+{\beta }_1}\left(\sum _{i\in \text{trt}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)+e^{{\beta }_0}\left(\sum _{i\in \text{cont}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right) \\ =\left(\begin{array}{cc}t+c& t\\ t& t\end{array}\right), \end{gathered}$$

where $c=e^{{\beta }_0}\left({\sum }_{i\in \text{cont}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)$ and $t=e^{{\beta }_0+{\beta }_1}\left({\sum }_{i\in \text{trt}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)$. Following (7),

$$\begin{gathered} V_{\beta }=\underset{m\to \infty }{\text{lim}}\left[\frac{m}{e^{{\beta }_0+{\beta }_1}\left({\sum }_{i\in \text{trt}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}+\frac{m}{e^{{\beta }_0}\left({\sum }_{i\in \text{cont}}{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}\right] \\ =\underset{m\to \infty }{\text{lim}}\left[\frac{m}{e^{{\beta }_0+{\beta }_1}\pi \left({\sum }_{i=1}^m{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}+\frac{m}{e^{{\beta }_0}\left(1-\pi \right)\left({\sum }_{i=1}^m{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}\right] \\ = \frac{m}{e^{{\beta }_0}\left({\sum }_{i=1}^m{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}\right)}\left(\frac{1}{\pi e^{{\beta }_1}}+\frac{1}{1-\pi }\right). \end{gathered}$$ (10)

If we assume equal cluster size n_i = n for all i, then

$$V_{\beta }=\frac{1}{e^{{\beta }_0}{\mathbf{1}}_n^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_n}\left(\frac{1}{\pi e^{{\beta }_1}}+\frac{1}{1-\pi }\right).$$ (11)

With the assumptions of exchangeable working correlation structure, equal cluster size n and equal allocation π = 0.5, we have $V_{\beta }=\frac{2\left[1+\left(n-1\right)\text{ρ}\right]}{n{e}^{{\beta }_0}}\left(\frac{1}{e^{{\beta }_1}}+1\right)$. It is Eq. (19) developed by Bhaumik and Gibbons (Amatya, Bhaumik, et al., 2013).

3. Relative efficiency of unequal versus equal cluster sizes

We will use relative variance of the treatment effect to define RE when comparing unequal to equal cluster sizes. Let Ω_equal denote a design with equal cluster sizes n, and let Ω_unequal denote a design with unequal cluster sizes n_i, i = 1, ⋯, m. Obviously $mn={\sum }_{i=1}^m{\text{n}}_i$. The RE of unequal versus equal cluster sizes for the treatment effect, RE$\left(\widehat{{\beta }_1}\right)$, is defined as

$$\text{RE}\left(\widehat{{\beta }_1}\right)=\frac{\text{Var}(\widehat{{\beta }_1}|{\Omega }_{\text{equal}})}{\text{Var}(\widehat{{\beta }_1}|{\Omega }_{\text{unequal}})}.$$ (12)

From equations (4) and (5) for a continuous outcome, (8) and (9) for a binary outcome, and (10) and (11) for count data, we have the same

$$\text{RE}\left(\widehat{{\beta }_1}\right)=\frac{{\sum }_{i=1}^m{\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}}{m{\mathbf{1}}_n^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_n} .$$ (13)

REs are surprisingly kept the same for all three types of outcomes. In other words, REs are independent of the outcome type provided that GEE models are used.

With the assumption of exchangeable working correlation structure,

$${\mathbf{1}}_{n_i}^{\prime }{R}_i^{-\mathbf{1}}{\mathbf{1}}_{n_i}=\frac{n_i}{1+\left(n_i-1\right)\text{ρ}},$$

then Eq. (13) equals

$$\text{RE}\left(\widehat{{\beta }_1}\right)=\frac{1+\left(n-1\right)\text{ρ}}{n}\frac{1}{m}\sum _{i=1}^m\frac{n_i}{1+\left(n_i-1\right)\text{ρ}}.$$ (14)

Let τ = (1 – ρ)/ρ, Eq. (14) becomes

$$\text{RE}\left(\widehat{{\beta }_1}\right)=\frac{n+\tau }{n}\frac{1}{m}\sum _{i=1}^m\frac{n_i}{n_i+\tau }.$$ (15)

It is identical to Eq. (9) developed by Breukelen et al.(Van Breukelen, Candel, et al., 2007). For binary outcomes, the same finding is shown in equation (A8) (Candel and Van Breukelen, 2010) and section 7 (Van Breukelen et al., 2015). They show that

1. There is a trade-off between mean cluster size n and τ. Multiplying all n_i’s and τ by a factor >0 does not change the RE$\left(\widehat{{\beta }_1}\right)$.

2. As ρ → 0 or as ρ → 1, the RE$\left(\widehat{{\beta }_1}\right)\to 1$, for any cluster size distribution. For 0<ρ<1, the RE$\left(\widehat{{\beta }_1}\right)$ is smaller than 1 and so the equal cluster size design is optimal.

4. Simulation studies

In order to compute RE$\left(\widehat{{\beta }_1}\right)$ in Eq. (14), Breukelen et al (Van Breukelen, Candel, et al., 2007) proposed a uniform, positively skewed, negatively skewed, bimodal and unimodal distribution of cluster size, respectively. Let X_i be random variable of cluster size n_i. Here, n_i, i = 1, …, m are dependent since they must sum to N = mn. We assume that cluster sizes (X₁, ⋯, X_m) follow a multinomial distribution with N and probabilities (p₁, ⋯, p_m), where ${\sum }_{i=1}^m{p}_i=1$. Under this distribution, the mean of X_i is Np_i, the variance is Np_i(1 − p_i) and covariance of (X_i, X_j) is −Np_ip_j for i ≠ j. Given the number of clusters m and equal cluster size n, the total sample size is $N=mn={\sum }_{i=1}^m{n}_i$ under the designs of Ω_equal and Ω_unequal. For convenience of the following discussion, we sort the i-th cluster with the distribution probability p_i’s, i = 1, ⋯, m, by a non-decreasing order such that p₁ ≤ p₂ ≤ ⋯ ≤ p_m. Obviously, the cluster sizes n_i’s, i = 1, ⋯, m may be different even if p_i’s are equal. Therefore, six patterns of (p₁, ⋯, p_m) are discussed under the design Ω_unequal.

1. Constant: p₁ = p₂ ⋯= p_m

2. Monotonically increasing: p₁ < p₂ < ⋯ < p_m;

3. Constant followed by monotonically increasing: p₁ = p₂⋯ = p_k < ⋯ <p_m;

4. Monotonically increasing followed by constant:p₁ < p₂ ⋯ < p_k = ⋯ = p_m;

5. Constant, monotonically increasing followed by constant: $p_1=p_2\cdots =p_{k_1}<p_{k_1+1}<\cdots <p_{k_2}=\cdots =p_m$;

6. Monotonically increasing, constant followed by monotonically increasing: $p_1<p_2\cdots <p_{k_1}=p_{k_1+1}=\cdots =p_{k_2}<\cdots <p_m$;

These six patterns are shown in Figure 1, which demonstrates the probabilities p_i, i = 1, ⋯, 100. We sort these 100 probabilities by a non-decreasing order, such that the 1^st cluster has the minimum of probabilities and the 100^th cluster has the maximum of the probabilities. For example, pattern 2, the probabilities of 1^st cluster and 100^th cluster are 0.006 and 0.013, respectively; they are 0.004 and 0.0275 for pattern 3. There could be more complicated patterns but they should be combinations of the above six patterns. Therefore, RE$\left(\widehat{{\beta }_1}\right)$ in Eq. (14) will be computed through simulation studies only for these six patterns.

Figure 1.

Six basic patterns of probabilities (p₁, ⋯, p_m).

4.1. Designs

As shown in Eq. (14), RE$\left(\widehat{{\beta }_1}\right)$ is independent of cluster allocations π and the parameters β₀ and β₁. To investigate the RE based on GEE models, the following factors are considered: (1) number of clusters m and equal cluster sizes n (or N); (2) the values of p₁, ⋯, p_m, equivalently, the pattern of p₁, ⋯, p_m; (3) association parameter (ρ) in the exchangeable working correlation structure.

Number of clusters m and equal cluster sizes n results in total sample size of N = nm. We will consider three scenarios: small, medium, and large CRTs; eg. N = 200, 600, 2000, respectively. Within each scenario we investigate three situations: small, medium, and large number of clusters. For example, m = 20 and n = 100; m =50 and n = 40; m =100 and n = 20 in a large CRT with a total sample size of N = 2000. All six patterns in Figure 1 are used for each study design.

Table 1 shows the values of (p₁, ⋯, p_m) for the simulation design with m = 100 and n = 20. During the interval of monotonically increasing within each pattern, we make p_i′s as an arithmetic sequence for code efficiency. All the values of p_i, i = 1, ⋯, m are shown in the 3^rd column for each pattern. For example, pattern 2, p_i = p₁ + (i − 1)d, i = 2, ⋯, m. Once p₁ is given, d will be calculated from p₁ and m and the formula is shown in the last column. Consequentially, p_i, i = 2, ⋯, m will be computed from the formula in the 3^rd column. Pattern 3 has the probabilities of p_i with constant followed by monotonically increasing:p₁ = ⋯ = p_k < p_k+1 < ⋯ < p_m. In this pattern, k and P_k are pre-determined. Here k = 50 and P_k = 0.2 are used, respectively. Then p_i, i = 1, ⋯, k and d will be computed from the formulas in the 3^rd column and the last column, respectively. Finally, p_i, i = k + 1, ⋯, m will be calculated from the formula in the 3^rd column. The logic for pattern 4 is the same as the one for pattern 2. For pattern 5, the pre-determined values are k₁, k₂, and $P_{k_1}$. Then p_i, i = 1, ⋯, k₁ and d will be computed from the formula in the 3^rd column and the last column, respectively. Finally, p_i, i = k₁ + 1, ⋯, k₂ will be calculated from the formula in the 3^rd column. Please note $p_{k_2}=\cdots =p_m$, thus, all the values of p_i, i = 1, ⋯, m are obtained. Pattern 6 is the most complicated case. Four parameters including k₁, k₂, $P_{k_1}$, and $P_{k_2}$ must be pre-determined. Then p_i, i = k₁, ⋯, k₂ and d_i, i = 1, 2 will be computed from the formula in the 3^rd column and the last column, respectively. Finally, p_i, i = 1, ⋯, k₁ – 1, k₂ + 1, ⋯, m will be calculated from the formula in the 3^rd column. The required parameters are marked with # in table 1. Provided the values of (p₁, ⋯, p_m) are available for each pattern, we will simulate the cluster sizes n_i, i = 1, ⋯, m from a multinomial distribution with N and probabilities (p₁, ⋯, p_m).

Table 1.

Multinomial distribution with probabilities (p₁, ⋯, p_m) with m = 100 and n = 20

Pattern	Description	(p1, ⋯, pm)[1]	Other Parameters
1	Constant P1 = p2 ⋯ = pm	Pi = 1/m, i = 1, ⋯, m
2	Monotonically increasing P1 < p2 < ⋯ < pm	P1# = 0.006, pi = p1 + (i − 1)d, i = 2, ⋯, m	$d=2\frac{1/m-p_1}{m-1}$
3	Constant followed by monotonically increasing P1 = ⋯ = pk < pk+1 < ⋯ < pm	k# = 50, $p_i=\frac{P_k}{k}, i=1, \cdots , k$ pi = pk + (i − k)d, i = k + 1, ⋯, m	$P_k{}^{\#}={\sum }_{i=1}^k{p}_i=0.2$, $d=\frac{2}{m-k}\left(\frac{1-P_k+\frac{P_k}{k}}{m-k+1}-\frac{P_k}{k}\right)$
4	Monotonically increasing followed by constant P1 < ⋯ < pk = pk+1 = ⋯ = pm	k# = 50 $P_k{}^{\#}=\sum _{i=k+1}^m{p}_i=0.65$ pi = pk − (k − i)d, i = 1, ⋯, k − 1	$P_k{}^{\#}=\sum _{i=k+1}^m{p}_i=0.65$ $d=\frac{2}{k-1}\left(\frac{P_k}{m-k}-\frac{1-P_k}{k}\right)$
5	Constant, monotonically increasing followed by constant $p_1=p_2\cdots =p_{k_1}<p_{k_1+1}<\cdots <p_{k_2}=\cdots =p_m$	k1# = 20, k2# = 70, $p_i=\frac{P_{k_1}}{k_1}, i=1, \cdots , k_1$ $p_i=p_{k_1}+\left(i-k_1\right)d,i=k_1+1, \cdots , k_2$	$P_{k_1}{}^{\#}=\sum _{i=1}^{k_1}{p}_i=0.05$, $d=\frac{2\left(1-m{P}_{k_1}/k_1\right)}{\left( k_2-k_1\right)\left[2*\left(m- k_2\right)+\left( k_2-k_1+1\right)\right]}$
6	Monotonically increasing, constant followed by monotonically increasing $p_1<p_2\cdots <p_{k_1}=p_{k_1+1}=\cdots =p_{k_2}<\cdots <p_m$	k1# = 25, k2# = 75 $p_i=\frac{P_{k_2}}{k_2-k_1}, i=k_1, \cdots , k_2$ $p_i=p_{k_1}-\left(k_1-i\right)d_1,i=1, \cdots , k_1-1$ $p_i=p_{k_2}+\left(i-k_2\right)d_2,i=k_2+1, \cdots , m$ $\text{Note:}\frac{k_1}{2}\frac{P_{k_2}}{k_2-k_1}<P_{k_1}<k_1\frac{P_{k_2}}{k_2-k_1}$	$P_{k_1}{}^{\#}=\sum _{i=1}^{k_1}{p}_i=0.15, P_{k_2}{}^{\#}=\sum _{i=k_1+1}^{k_2}{p}_i=0.5$ $d_1=\frac{2}{k_1-1}\left(\frac{P_{k_2}}{ k_2-k_1}-\frac{P_{k_1}}{k_1}\right)$, $d_2=\frac{2}{ m-k_2}\left(\frac{1-P_{k_1}-P_{k_2}+\frac{P_{k_1}}{k_1}}{ m-k_2+1}-\frac{P_{k_2}}{ k_2-k_1}\right)$

^[1](p₁, ⋯, p_m) are probabilities and ${\sum }_{i=1}^m{p}_i=1$.

^#:required parameters.

For all the scenarios, the required parameters for (p₁, ⋯, p_m) calculation are shown in Table 2. Even if the ICCs are pretty small for most CRTs (Murray, Varnell, et al., 2004; Turner, Prevost, et al., 2004), the association parameter (ρ) ranged from 0 to 0.95 with steps of 0.01 considered for the illustration purposes. 1000 simulation samples are generated for each design. REs are calculated using Eq. (14) for all the samples, and mean, standard deviation, minimum and maximum of REs are obtained at each ρ correspondingly. Source code to reproduce the results is available as Supporting Information on the journal’s web page (http://onlinelibrary.wiley.com/doi/xxx/suppinfo).

Table 2.

Required parameters in simulation designs

Pattern	Total Sample Size (N = 200)			Total Sample Size (N = 600)			Total Sample Size (N = 2000)
	m = 5	m = 20	m = 40	m = 6	m = 30	m = 60	m = 20	m = 50	m = 100
	n = 40	n = 10	n = 5	n = 100	n = 20	n = 10	n = 100	n = 40	n = 20
2	p1 = 0.1	p1 = 0.006	p1 = 0.006	p1 = 0.006	p1 = 0.006	p1 = 0.006	p1 = 0.006	p1 = 0.006	p1 = 0.006
3	k = 3	k = 10	k = 20	k = 3	k = 15	k = 30	k = 10	k = 25	k = 50
	pk = 0.5	pk = 0.4	pk = 0.4	pk = 0.2	pk = 0.2	pk = 0.2	pk = 0.2	pk = 0.2	pk = 0.2
4	k = 2	k = 10	k = 20	k = 3	k = 15	k = 30	k = 10	k = 25	k = 50
	pk = 0.65	pk = 0.65	pk = 0.65	pk = 0.65	pk = 0.65	pk = 0.65	pk = 0.65	pk = 0.65	pk = 0.65
5	k1 = 2	k1 = 4	k1 = 8	k1 = 2	k1 = 6	k1 = 12	k1 = 4	k1 = 10	k1 = 20
	k2 = 3	k2 = 14	k2 = 28	k2 = 4	k2 = 21	k2 = 42	k2 = 14	k2 = 35	k2 = 70
	$P_{k_1}=0.05$	$P_{k_1}=0.05$	$P_{k_1}=0.05$	$P_{k_1}=0.05$	$P_{k_1}=0.05$	$P_{k_1}=0.05$	$P_{k_1}=0.05$	$P_{k_1}=0.05$	$P_{k_1}=0.05$
6	k1 = 2	k1 = 5	k1 = 10	k1 = 2	k1 = 7	k1 = 15	k1 = 5	k1 = 12	k1 = 25
	k2 = 3	k2 = 15	k2 = 30	k2 = 5	k2 = 22	k2 = 45	k2 = 15	k2 = 37	k2 = 75
	$P_{k_1}=0.3$	$P_{k_1}=0.15$	$P_{k_1}=0.15$	$P_{k_1}=0.2$	$P_{k_1}=0.15$	$P_{k_1}=0.15$	$P_{k_1}=0.15$	$P_{k_1}=0.15$	$P_{k_1}=0.15$
	$P_{k_2}=0.2$	$P_{k_2}=0.5$	$P_{k_2}=0.5$	$P_{k_2}=0.5$	$P_{k_2}=0.5$	$P_{k_2}=0.5$	$P_{k_2}=0.5$	$P_{k_2}=0.5$	$P_{k_2}=0.5$

4.2. Results

Figure 2 shows the plots of the mean, minimum and maximum of RE as a function of association parameter (ρ) based on the simulation data with m = 100 and n = 20 for patterns 1–6, respectively. The parameters used in this simulation study are shown in Table 1. As can be seen, the mean RE starts from 1, reaches the minimum, and then increases to 1 at ρ=1. The minimum of mean REs for these six patterns are 0.9871 at ρ=0.05, 0.9734 at ρ=0.05, 0.8589 at ρ=0.05, 0.9343 at ρ=0.08, 0.8962 at ρ=0.07, and 0.9570 at ρ=0.06, respectively. The standard deviations are so small (<0.008) for all the ρ’s and all six patterns, thus we do not include 95% CIs in the plots. When comparing these six patterns, we find that the first pattern shows the highest RE. This is expected since the setting of p₁ = p₂= ⋯ = p_m has the highest chance of equal cluster sizes. The second pattern has the next highest RE while the third pattern shows the lowest one. Pattern 6 has a better RE than patterns 4 and 5. Simultaneously, after the minimum is reached, the first pattern is the least steep, while the third one is very steep, and pattern 5 is a little steeper than pattern 6. The closer to ρ that the minimum of mean REs is reached, the larger range RE has. The same findings are observed across all six patterns. The plots of the mean and range of RE based on the simulation data with m = 20, n = 100 and m = 50, n = 40 for six patterns are shown in Appendix Figure 1 and 2, respectively. Among three situations including small, medium, and large number of clusters, we notice that the range looks narrower when the cluster size increases at any value of ρ for any pattern. For a small cluster size (n = 20), the range of mean RE stays large when ρ increases after the minimum of mean REs is reached for patterns 2–6. There are not many differences otherwise.

Figure 2.

Relative efficiency of the treatment effect for six patterns, with m = 100 and n = 20.

Table 3 presents mean (min, max) of CV, the minimum and median of mean RE and corresponding ρ’s among 1000 repeated samples for all three total sample sizes (200, 600, and 2000), the three situations of cluster sizes and number of clusters, and all six patterns. The first pattern indeed shows the highest RE for any scenario. Below discussions focus on the remaining five patterns.

Table 3.

Minimum and median of mean RE for the different simulation study designs

Total Sample Size (N)	Number of Clusters (m)	Cluster Sizes (n)	Pattern	CV Mean (min, max)	Mean RE[1]
					ρ, Minimum[2]	ρ, Median[2]
200	5	40	1	0.15 (0.02, 0.34)	0.03, 0.9948	0.48–0.49, 0.9994
			2	0.42 (0.17, 0.69)	0.03, 0.9617	0.48–0.49, 0.9952
			3	0.29 (0.06, 0.59)	0.02, 0.9833	0.48–0.49, 0.9983
			4	0.24 (0.06, 0.45)	0.03, 0.9868	0.48–0.49, 0.9984
			5	0.81 (0.69, 0.92)	0.06, 0.7974	0.48–0.49, 0.9311
			6	0.35 (0.13, 0.59)	0.03, 0.9717	0.48–0.49, 0.9962
	20	10	1	0.31 (0.18, 0.47)	0.10, 0.9754	0.49–0.50, 0.9903
			2	0.63 (0.43, 0.84)	0.14, 0.8918	0.51–0.52, 0.9306
			3	0.40 (0.19, 0.61)	0.10, 0.9612	0.49–0.50, 0.9843
			4	0.52 (0.37, 0.68)	0.16, 0.9170	0.52–0.53, 0.9427
			5	0.62 (0.47, 0.81)	0.14, 0.8908	0.50–0.51, 0.9324
			6	0.46 (0.26, 0.66)	0.13, 0.9425	0.50–0.51, 0.9669
	40	5	1	0.44 (0.29, 0.62)	0.21, 0.9482	0.53–0.54, 0.9649
			2	0.63 (0.43, 0.81)	0.24, 0.8924	0.56–0.57, 0.9146
			3	0.51 (0.34, 0.71)	0.21, 0.9338	0.54–0.55, 0.9544
			4	0.60 (0.43, 0.75)	0.28, 0.8945	0.59–0.60, 0.9112
			5	0.70 (0.50, 0.88)	0.28, 0.8600	0.59–0.60, 0.8816
			6	0.55 (0.36, 0.76)	0.24, 0.9175	0.56–0.57, 0.9365
600	6	100	1	0.09 (0.01, 0.19)	0.01, 0.9979	0.48–0.49, 0.9999
			2	0.73 (0.62, 0.83)	0.03, 0.8485	0.48–0.49, 0.9531
			3	0.76 (0.60, 0.91)	0.01, 0.8848	0.48–0.49, 0.9936
			4	0.51 (0.44, 0.60)	0.03, 0.9071	0.48–0.49, 0.9848
			5	0.72 (0.62, 0.84)	0.02, 0.8450	0.48–0.49, 0.9817
			6	0.51 (0.40, 0.64)	0.02, 0.9315	0.48–0.49, 0.9936
	30	20	1	0.22 (0.13, 0.31)	0.05, 0.9876	0.48–0.49, 0.9974
			2	0.54 (0.44, 0.65)	0.07, 0.9195	0.48–0.49, 0.9725
			3	0.80 (0.64, 0.92)	0.05, 0.8620	0.48–0.49, 0.9625
			4	0.46 (0.37, 0.55)	0.08, 0.9314	0.49–0.50, 0.9693
			5	0.59 (0.48, 0.69)	0.07, 0.9002	0.48–0.49, 0.9636
			6	0.36 (0.26, 0.46)	0.06, 0.9654	0.48–0.49, 0.9899
	60	10	1	0.31 (0.22, 0.44)	0.10, 0.9742	0.49–0.50, 0.9892
			2	0.49 (0.35, 0.61)	0.11, 0.9366	0.49–0.50, 0.9690
			3	0.83 (0.71, 0.96)	0.11, 0.8478	0.49–0.50, 0.9217
			4	0.51 (0.41, 0.58)	0.15, 0.9206	0.50–0.51, 0.9481
			5	0.63 (0.54, 0.72)	0.14, 0.8839	0.51–0.52, 0.9282
			6	0.46 (0.36, 0.56)	0.12, 0.9436	0.50–0.51, 0.9699
2000	20	100	1	0.09 (0.06, 0.15)	0.01, 0.9976	0.48–0.49, 0.9999
			2	0.56 (0.49, 0.61)	0.02, 0.9146	0.48–0.49, 0.9922
			3	0.77 (0.69, 0.84)	0.01, 0.8738	0.48–0.49, 0.9932
			4	0.43 (0.38, 0.48)	0.02, 0.9380	0.48–0.49, 0.9929
			5	0.55 (0.47, 0.60)	0.02, 0.9140	0.48–0.49, 0.9933
			6	0.35 (0.29, 0.41)	0.02, 0.9651	0.48–0.49, 0.9972
	50	40	1	0.16 (0.11, 0.22)	0.03, 0.9938	0.48–0.49, 0.9993
			2	0.44 (0.38, 0.51)	0.03, 0.9475	0.48–0.49, 0.9925
			3	0.79 (0.72, 0.86)	0.03, 0.8667	0.48–0.49, 0.9819
			4	0.43 (0.37, 0.48)	0.04, 0.9393	0.48–0.49, 0.9855
			5	0.57 (0.51, 0.62)	0.04, 0.9044	0.48–0.49, 0.9820
			6	0.34 (0.28, 0.39)	0.03, 0.9686	0.48–0.49, 0.9952
	100	20	1	0.22 (0.18, 0.29)	0.05, 0.9871	0.48–0.49, 0.9972
			2	0.32 (0.27, 0.38)	0.05, 0.9734	0.48–0.49, 0.9939
			3	0.80 (0.74, 0.88)	0.05, 0.8589	0.48–0.49, 0.9612
			4	0.45 (0.39, 0.49)	0.08, 0.9343	0.48–0.49, 0.9723
			5	0.59 (0.53, 0.64)	0.07, 0.8962	0.48–0.49, 0.9619
			6	0.40 (0.34, 0.45)	0.06, 0.9570	0.48–0.49, 0.9867

^[1]The mean RE among 1000 simulations is calculated for each ρ.

^[2]The minimum and median of mean RE including the corresponding ρ’s are identified across all the values of ρ.

1. Total sample size is small (N=200); the minimums of mean REs are less than 90% for one-third of cases. When the number of clusters is small (m = 5), the minimums of mean RE for pattern 4 and pattern 5 are 0.9868 and 0.7974, respectively, the largest and smallest estimates among the five patterns. Additionally, the minimums are reached at ρ ∈ [0.02, 0.06]. The remaining two situations (m = 20, 40). obtain the largest estimate of minimum of mean RE for pattern 3 and the smallest estimate for pattern 5. We also find that the minimum is reached at the larger ρ when cluster size decreases. For example, ρ is around 0.05 for n = 40, around 0.15 for n = 20, and around 0.25 for n = 5, respectively. The median of mean REs are at least 93% for m = 5 and n = 40, 93% for m = 20 and n =10, 88% for m = 40 and n = 5. The corresponding values of ρ are between 0.48 and 0.49, between 0.49 and 0.53, and between 0.53 and 0.60, respectively.

2. Total sample size is medium (N=600); All the minimums of mean REs are at least 84% and more than half are above 90%. When the number of clusters is small (m = 6), the minimums of mean RE for pattern 5 and pattern 6 are 0.8450 and 0.9315, respectively. They are the largest and smallest estimates among patterns 2–6. The remaining two situations (m = 30, 60) obtain the smallest estimate of minimum of mean RE for pattern 3. We have the same findings about the trend of ρ, where the minimum is reached, for different cluster sizes: the smaller the cluster size, the larger the ρ. Additionally, all the medians of mean REs are at least 95% for m = 6 and n = 100, 96% for m = 30 and n = 20, and 92% for m = 60 and n = 10, respectively. The corresponding values of ρ are between 0.48 and 0.49, between 0.48 and 0.50, and between 0.49 and 0.52, respectively.

3. Total sample size is large (N=2000); Such large studies will definitely yield sufficient power. They have the best estimate of minimal REs with at least 85% for all the different settings (m, n, and pattern) and approximately 75% cases are above 90%. The situations (m = 20, 50) obtain the largest estimate of minimum of RE for pattern 6. Pattern 3 demonstrates the smallest estimate of minimum mean RE for all three situations (87.38%, 86.67%, 85.89%). ρ, where the minimum is reached, ranged from 0.01 to 0.08. All the medians of mean REs are larger than 99% for m = 20 and n = 100, above 98% for m = 50 and n = 40, and at least 96% m = 100 and n = 20.

We find that minimum of mean RE decreases when CV increases for all simulation studies. Pattern 1 indicates the lowest CV as expected while pattern 5 has the largest CV in most cases. In summary, the worst scenarios for small CRTs give 21%, 11%, and 14% efficiency loss when the number of cluster is small, medium and large, respectively. They are 16%, 14%, and 16% for medium CRTs. For large CRTs all three are about the same, 14%.

Additionally, Table 3 shows that ρ at which the RE is minimum decreases with increasing average cluster size n given a pattern. Breukelen et al (Van Breukelen, Candel, et al., 2007) shows the RE minimum is reached at ρ = 1/(n + 1) based on their Taylor approximation (Eq. (10)). We have the similar results for large CRTs with mn = 2000, especially for n = 40 and 60. Furthermore, Breukelen et al (Van Breukelen, Candel, et al., 2007) shows that the number of clusters does not affect RE, Eq. (15), given the cluster distribution. We make m twice and keep n same in the simulation designs. The parameters for (p₁, ⋯, p_m) and simulation results are shown in Appendix Tables 1 and 2. When we compare Table 3 and Appendix Table 2, e.g. m = 5, n = 40 and m = 10, n = 40, the different findings within a same pattern are noticed. For example, pattern 2, the minimums of mean REs are 0.9617 and 0.8802, respectively. On the other hand, they have different CVs, 0.42 (0.17, 0.69) and 0.65 (0.52, 0.78). That is, the same patterns in our simulation setting which strongly depends on (p₁, ⋯, p_m) may have the different cluster distributions when the number of cluster doubles.

5. Sample size calculations

In sections 5.1 and 5.2, we show the sample size formulas for the different types of outcome based on GEE models with the assumption of equal cluster sizes with no cost constraints and under a given cost, respectively. Section 5.3 demonstrates how the sample sizes are adjusted due to efficiency loss from unequal cluster sizes.

5.1. No cost constraints

For continuous and binary count outcomes, a two-sided test based on β with type I error of α and nominal power η requires the sample size

$$m={\left(z_{1-\alpha /2}+z_{\eta }\right)}^2\frac{V_{\beta }}{{\beta }^2},$$ (16)

Where β is the treatment effect. Substituting Equations (6) and (9) for a two-group study with the assumption of exchangeable correlation matrix, we have the sample size formulas

$$m={\left(z_{1-\alpha /2}+z_{\eta }\right)}^2\frac{{\sigma }^2\left[1+\left(n-1\right)\text{ρ}\right]}{\pi \left(1-\pi \right)n{\beta }^2},$$ (17)

and

$$m={\left(z_{1-\alpha /2}+z_{\eta }\right)}^2\frac{\left(\frac{1}{\pi p_1\left(1-p_1\right)}+\frac{1}{\left(1-\pi \right)p_0\left(1-p_0\right)}\right)\left[1+\left(n-1\right)\text{ρ}\right]}{n{\left[\text{log}\left(\frac{p_1\left(1-p_0\right)}{p_0\left(1-p_1\right)}\right)\right]}^2},$$

respectively, where z_α is the 100 × α percentile of a standard normal distribution. Please note Eq. (17) reduces to Eq. (3) (Van Breukelen and Candel, 2012) when the equal allocation occurs.

For count outcomes, we use formula (3) (Amatya, Bhaumik, et al., 2013) to obtain

$$m={\left(z_{1-\alpha /2}\sqrt{\frac{1}{\pi }+\frac{1}{1-\pi }}+z_{\eta }\sqrt{\frac{1}{\pi e^{\beta }}+\frac{1}{1-\pi }}\right)}^2\frac{1+\left(n-1\right)\rho }{n{e}^{{\beta }_0}{\beta }^2}$$

through Eq. (11).

5.2. Optimal design

Van Breukelen et al (Van Breukelen and Candel, 2015) define the total budget B in CRTs. Assume the study cost per cluster in the enrollment stage is c currency units (e.g. USD), and each subject costs s currency units. Total budget B includes m clusters and n subjects within a cluster,

$$B=m\left(c+sn\right).$$ (18)

They define “optimal” as that the variance of the treatment effect estimator in the linear regression model is minimized under the cost constraints. In this paper the term “optimal” using GEE models refers to maximum power for a given sampling budget. Actually, they are equivalent given an unbiased point estimator of the treatment effect β. Here, the optimal design is to find the design which maximizes the power given the constraint in Eq. (18).

Define

$${\text{Δ}}_1=\frac{{\sigma }^2}{\pi \left(1-\pi \right){\beta }^2}$$

for continuous outcome, and

$${\text{Δ}}_1= \frac{\frac{1}{\pi p_1\left(1-p_1\right)}+\frac{1}{\left(1-\pi \right)p_0\left(1-p_0\right)}}{{\left[\text{log}\left(\frac{p_1\left(1-p_0\right)}{p_0\left(1-p_1\right)}\right)\right]}^2}$$

for binary outcome, the sample sizes based on GEE models (Shih, 1997; Pan, 2001) can be re-written as

$$m={\left(z_{1-\alpha /2}+z_{\eta }\right)}^2\frac{1+\left(n-1\right)\rho }{n} {\text{Δ}}_1.$$ (19)

Alternatively,

$$z_{\eta }=\sqrt{\frac{n}{1+\left(n-1\right)\rho }\frac{m}{{\text{Δ}}_1}}-z_{1-\alpha /2}.$$

For count data,

$$z_{\eta }=\left(\sqrt{\frac{n}{1+\left(n-1\right)\rho }m{e}^{{\beta }_0}{\beta }^2}-z_{1-\alpha /2}\sqrt{\frac{1}{\pi }+\frac{1}{1-\pi }}\right)/\sqrt{\frac{1}{\pi e^{\beta }}+\frac{1}{1-\pi }}.$$ (20)

Consequently, the power is calculated by ϕ⁻¹ (z_η), where ϕ is cumulative density function of standard normal distribution.

As noted that π, α, Δ₁ in Eq. (19) and (β₀, β) in Eq. (20) are pre-determined values under null and alternative hypothesis for three types of outcomes, maximizing the power means maximizing

$$K=\frac{nm}{1+\left(n-1\right)\rho }.$$

For a known value ρ, the locally optimal design (LOD) is reached at

$$n_{\text{LOD}}=\sqrt{\frac{\vartheta c}{s}}, m_{\text{LOD}}=\frac{B}{\sqrt{\vartheta sc}+c},$$ (21)

Where $\vartheta =\frac{1-\rho }{\rho }$. It is identical to the optimal designs proposed by other researchers (Raudenbush, 1997; Moerbeek et al., 2000; Van Breukelen and Candel, 2015), even though they use the different definition of “optimal.”

In the scenario of an unknown parameter value of ρ, Van Breukelen and Candel (Van Breukelen and Candel, 2015) proposed the Maximin designs (MMDs) using the linear regression model and the variance of the maximum likelihood estimator. However, there is no closed form for binary outcomes and the variance from binary outcomes needs to be transformed to that of continuous outcomes. When using GEE models, Liu et al (Liu et al., 2017) proposed an algorithm, shown as below, to find the optimal design (m_OD, n_OD) based on parameter space (ρ_min, ρ_max) and design space (m_min, m_max).

• Step 1, Define the parameter and design space, (ρ_min, ρ_max) and (m_min, m_max), respectively.

• Step 2, Consider ρ = ρ_max, calculate m_LOD using equation (21).

1. If it is within the range (m_min, m_max), then set m_OD = m_LOD.

2. If it is outside of (m_min, m_max), then set m_OD = m_max. The cluster size n_OD is calculated by $\left(\frac{B}{m_{\text{OD}}}-c\right)/s$.

Liu et al (Liu and Colditz, 2017) shows proof details of LODs and ODs when GEE models are considered in the sample size calculation. When comparing the algorithm in MMDs based on RE and efficiency (Van Breukelen and Candel, 2015), the revised algorithm is same as one of their maximin designs which maximizes the minimum efficiency. However, it works for any type of outcome including continuous, binary, and count.

Given a fixed budget, we can find the optimal design which maximizes the power for the proposed study but please note that the maximized power may be much different from nominal power η, eg. 80%. When a known value ρ is the same, or the parameter and design space are the same for three different types of outcomes, the optimal sample sizes including number of clusters and cluster size are same for these three as well. However, the maximized powers are different and depends on the outcome type.

5.3. Application

Equal cluster sizes are assumed in sections 5.1 and 5.2. However, equal cluster sizes are not guaranteed in practice. When designing a CRT to examine the treatment effect through GEE models, first we assume equal cluster sizes and n is given. The required number of clusters can be obtained from these two sections. We will then increase number of clusters in order to compensate for the efficiency loss due to unequal cluster sizes. Below are examples to demonstrate the calculation of number of clusters for unequal cluster sizes.

If we consider GEE models to design a CRT without financial constraints, the sample size formulas in section 5.1 are used for calculation. For instance, we want to compare two physical therapy treatments designed to increase muscle flexibility in a CRT. We must determine how many clusters are required to achieve a power of at least 80% to detect a group mean difference with n = 40 at the type I error of 5%. The mean difference of muscle flexibility between the standard treatment and the new treatment is 1, the common standard deviation of muscle flexibility is 3.1, and equal allocation (π = 0.5) and ρ = 0.01 are considered. Using equation (17), the number of clusters m = 20 are needed. The total sample size is mn = 800. We consider this study as a medium CRT with medium number of clusters. As shown in section 4.2, REs are dependent of the pattern of p₁, ⋯, p_m and the worst scenarios for medium CRTs give 16%, 14%, and 16% efficiency loss for medium CRTs when the number of cluster is small, medium and large, respectively. The adjusted number of clusters will be 20/0.86=24.

If a study is designed under a given budget, then the sample size formulas in section 5.2 will be used. For example, total budget of the proposed study is $55,000, the study cost per cluster in the enrollment stage is $1,000, and each subject costs $100. Using equation (21), the locally optimal design with n_LOD ≈ 8.00 and m_LOD ≈ 30.55 are obtained given ρ=0.135. It is considered a small CRT with large number of clusters. The worst scenario for small CRTs gives 14% efficiency loss when the cluster size is small. Therefore, the adjusted number of clusters will be 30.55/0.86 ≈ 35.5. We need to enroll 36 clusters with cluster size of 8. The original proposed budget is 30(1000+100*8)=$54,000 and is within the budget while the budget after the adjustment is 36*1000+100*36*8=$64,800 and is beyond the budget. Obviously, the cost could be increased because the number of clusters rise. Therefore, there is a trade-off between the increase in number of clusters due to efficiency loss and the actual cost within the budget in the optimal design.

6. Discussion

The sample size formulas based on GEE methods have been derived in recent years (Liu and Liang, 1997; Shih, 1997; Pan, 2001; Amatya, Bhaumik, et al., 2013). As we notice, the sample size formulas assume the exchangeable working correlation structure and equal cluster sizes. However, the assumption of equal cluster size is not realistic. Therefore, researchers investigate the RE of unequal versus equal cluster sizes when testing the treatment effect. In the paper (Van Breukelen, Candel, et al., 2007), Breukelen et al consider linear regression models to address the RE of unequal versus equal cluster sizes with continuous outcome for CRTs. For a binary outcome in CRTs, Candel and Breukelen (Candel and Van Breukelen, 2010) utilize mixed effect logistic regression model and present the adjusted sample size formula for varying cluster sizes. In this paper, we investigate the REs based on GEE models to test the treatment effect in a two-group comparison in CRTs. The three outcomes of continuous, binary, or count data are discussed simultaneously. The variances of the estimator of the treatment effect were derived for three different types of outcome given the exchangeable working correlation structure. We define RE as the ratio of variance of the estimator of the treatment effect for equal to unequal cluster sizes. The simpler formulas of RE with continuous, binary, and count outcomes are given.

First, we find the formulas of REs are the same for continuous, binary, and count data using GEE models. That is, RE is not dependent on the type of outcome at the stage of study design. Second, RE$\left(\widehat{{\beta }_1}\right)$ is independent of cluster allocations π and the parameters β₀ and β₁. Third, the performance of RE for the exchangeable working correlation structure is investigated through simulations since no closed form is provided. We make a conclusion about efficiency loss for the different sizes of CRTs-small, medium, and large. Fourth, the adjusted number of clusters accounting for efficiency loss are illustrated for CRTs without financial constraints and under a budget. Fifth, the expressions (Eq. (14) and (15)) are same as those (Eq. (8) and (9)) in Breukelen et al (Van Breukelen, Candel, et al., 2007). They are an extension of published results on optimal sample sizes and on the relative efficiency of unequal versus equal cluster sizes in cluster randomized trials using GEE models. Last, Breukelen et al (Van Breukelen, Candel, et al., 2007) approximate the RE through Taylor series by assuming that n_i, i = 1, ⋯, m are independent realizations of a random variable while we conduct simulation studies to investigate RE performance given that unequal cluster sizes follow a multinomial distribution under $mn={\sum }_{i=1}^m{n}_i$. The rationale here is that the number of clusters under the assumption of equal cluster sizes in the sample size calculation are obtained, resulting in a fixed sample size (N = mn). Next we investigate the performance of RE for unequal cluster size n_i. Breukelen et al (Van Breukelen, Candel, et al., 2007) find that the loss of efficiency could be larger than 10% when CV exceeds 0.63. However, such a large CV is not expected to occur in real cluster randomized trials. As such, they conclude that the loss of efficiency due to variation of cluster sizes rarely exceeds 10 per cent and can be compensated by sampling 11 per cent more clusters. Candel and Breukelen (Candel and Van Breukelen, 2010) find that 14% more clusters are needed to cover the efficiency loss in many cases with a binary outcome in CRTs. Our simulation results show that small CRTs give 21%, 11%, and 14% efficiency loss when the number of cluster is small, medium and large, respectively. They are 16%, 14%, and 16% for medium CRTs. The efficiency loss could be approximately 14%, even in large CRTs under the fact of $mn={\sum }_{i=1}^m{n}_i$. Please note that the corresponding CVs are larger than 0.63, thus, we make the similar conclusion as Breukelen et al (Van Breukelen, Candel, et al., 2007).

There are several limitations to this approach. The first limitation is that the covariates are not included in the consideration. If we include the covariates in GEE model, the sample size formulas are not shown to be simple as sections 5.1 and 5.2. Moreover, the performance of the sample size formula is sensitive to the distribution of the covariates (Liu and Liang, 1997). However, it is reasonable to calculate the sample size without the consideration of covariates in the design stage in general. The next limitation is that our proposed RE is investigated on the exchangeable working correlation matrix only. The assumption of the exchangeable working correlation matrix may not hold in a real world application. However, the sample size formulas always use this assumption and it is acceptable in practice. The third limitation is that the performance of RE is determined by the pattern of p₁, ⋯, p_m as well as the number of cluster, cluster sizes, and association parameter (ρ). Obviously, we have no prior knowledge of p₁, ⋯, p_m. This paper shows the different scenarios of efficiency loss from six basic patterns. Sample size adjustments are made based on the worst scenario in order to be conservative. The last limitation is that GEE models may have poor performance when the number of clusters is not large enough. The performance of RE defined in section 3 and the sample size formula in section 5 must be used cautiously.

In conclusion, this paper discusses the efficiency loss based on GEE models when unequal cluster sizes occur in a real world. We believe that the proposed method is very useful and practical, especially for designing CRTs with any type of outcomes.

Supplementary Material

Supp Fig 1

Figure 1 Mean cluster size for six patterns, with m = 100 and n = 20.

Supp Fig 2

Figure 2 Mean cluster size for six patterns, with m = 20 and n = 100.

Supp Fig 3

Figure 3 Mean cluster size for six patterns, with m = 50 and n = 40.

Supp Tables 1-2

Table 1 Multinomial distribution with probabilities (p₁, ⋯, p_m) with m = 100 and n = 20

Table 2 Required parameters in simulation designs

Figure 3.

Relative efficiency of the treatment effect for six patterns, with m = 20 and n = 100.

Figure 4.

Relative efficiency of the treatment effect for six patterns, with m = 50 and n = 40.