Augmented GEE for improving efficiency and validity of estimation in cluster randomized trials by leveraging cluster- and individual-level covariates

Abstract

Recent methodological advances in covariate adjustment in RCTs have used semiparametric theory to improve efficiency of inferences by incorporating baseline covariates; these methods have focused on independent outcomes. We modify one of these approaches, augmentation of standard estimators, for use within cluster randomized trials in which treatments are assigned to groups of individuals, thereby inducing correlation. We demonstrate the potential for imbalance correction and efficiency improvement through consideration of both cluster- and individual-level covariates. To improve small-sample estimation, we consider several variance adjustments. We evaluate this approach for continuous and binary outcomes through simulation, and apply it to data from a cluster randomized trial of a community behavioral intervention related to HIV prevention in Tanzania.

1. INTRODUCTION

1.1. Traditional and cluster randomized trials

Randomized clinical trials (RCTs) are recognized as the gold standard in medical research for evaluating new treatments. Cluster or group randomized trials (GRTs), which assign treatment to groups of individuals, are advantageous when interaction among subjects within a group may impact their respective outcomes. GRTs are therefore especially relevant for assessing prevention and treatment methods for infectious diseases, where subjects within a geographical unit such as a neighborhood, school, or workplace may infect each other. For example, in vaccine studies, a subject’s vaccination status may impact health outcomes not only for that subject but for others as well. Clustered designs also have the advantage of reducing the potential for contamination of effects caused by sharing of information or medication between treated and control subjects. Similarly, group treatment assignment can enhance compliance as subjects within a group are given the same regimen to follow. In some cases, the intervention may be administered at the cluster-level, such as in studies involving schools or medical practices. Klar and Donner provide several examples of intervention trials in which groups were randomized for medical, political, or logistical reasons [1].

Although intervention is assigned at the group level, interest often lies in performing inference on the individual. Generally, subjects within a group are expected to be more similar than subjects in different groups, inducing dependence across study subjects. Cluster randomized designs thus present the additional challenge of accounting for correlation among group members. Standard approaches for estimating treatment effects when responses are correlated include maximum likelihood for generalized linear mixed models (GLMM) and generalized estimating equations (GEE) for restricted mean models [2, 3]. To estimate the marginal effect in a binary treatment setting, one typically fits a model including an intercept and treatment term. The relevant GLMM is defined by the model E(Y_ij|A_i,b_i) = g(β₀ + β₁cA_i + b_i), where Y_ij denotes the outcome for the j_th individual in the i_th cluster, A_i is an indicator for treatment, b_i is a random effect inducing correlation among subjects within a cluster, and g(·) is a monotone link function. The outcome Y_ij and random effect b_i are assumed to follow a particular distribution. We note here that β₁c is interpreted as a cluster-specific treatment effect, but marginalizes over all other covariates. In the analogous GEE approach, estimating equations are constructed following the mean model

$$E(Y_{\mathit{ij}}\mid A_i)=g(A_i;\beta )=g({\beta }_0+{\beta }_1{A}_i),$$ (1)

where correlation is accounted for by incorporating a working covariance matrix V_w. For cluster randomized designs, independence or exchangeable structure is generally assumed. An advantage of the GEE approach is that consistency of $\widehat{\beta }$, the estimate of β, only requires that the mean g(A_i; β) is correctly specified, in which case, $\widehat{\beta }$ is asymptotically normal for all V_w and efficient when V_w takes the true form of V, the variance of response vector Y_i. The exact form of the GEE is reviewed in the following section. GEE differ from maximum likelihood estimation in mixed models by treating correlation as a nuisance parameter. Additionally, GLMM require full specification of the distribution of Y_ij, while GEE follow from semiparametric theory and only specify the first moment of Y_ij while requiring the second moment to be finite. Unlike GLMM, GEE do not make any assumptions about cluster effects, and thus provide a population-averaged effect estimate in contrast to the GLMM cluster-specific estimate. In either approach, treatment is evaluated through inference on β₁.

A second challenge presented by cluster randomized designs is that the number of available experimental units may be fairly small. Inference for model-based methods relies on asymptotic theory, which may not be applicable in trials with relatively few clusters. For GEE, several studies have shown that the sandwich variance estimator typically underestimates the variability of parameter estimates and consequently results in inference that is too liberal [4]. A number of adjustment methods for small sample analysis have been proposed [5, 6, 7, 8, 9]. These adjustments generally take one of two strategies; they account for the variability in the sandwich estimator or correct for its small-sample bias. None of these methods have been uniformly adopted.

The number of available experimental units also affects the degree to which randomization successfully balances baseline characteristics across treatment groups. RCTs with large sample sizes assure a reasonable degree of balance in covariate profiles with high probability, but GRTs often have smaller numbers of experimental units, and therefore provide less assurance of balance [10]. GRTs are also likely to contain subject heterogeneity in cluster-level and individual-level characteristics that can influence estimated treatment effects. Clustered designs therefore require methods that permit controlling for imbalances at the cluster and subject levels.

1.2. Methods for covariate adjustment in randomized trials

Traditionally, adjustment for residual imbalance has been achieved by adding covariates Z_i, X_ij to a model for the effect of treatment on some outcome. The adjusted model for Y_ij is defined by E(Y_ij|A_i, Z_i, X_ij) = g(β₀ + β_1* A_i + β_ZZ_i + β_XX_ij), where Z_i is a vector of covariates shared by all subjects within the i_th cluster, and X_ij is a subject-specific vector of measurements. Standard approaches such as mixed models and GEE can incorporate adjustment at both levels. With the exception of linear and log-linear models, the conditional model differs from the marginal model (1) in the interpretation of β_1*. Inference on β_1* is also affected by the presence of baseline covariates. For uncorrelated continuous outcomes and an identity link function relating covariates to the mean, it has been shown that when X and Y are correlated, β_1* is more precise than the unadjusted estimator [11, 12]. No direct relationship between the efficiency of β₁ (1) and β_1* has been established for non-linear models [13] or correlated outcomes. To provide an alternative that makes fewer parametric assumptions, Gail et al. [14] proposed a permutation approach to covariate adjustment in GRTs. Parametric models are used for adjustment, and permutation inference is conducted on the cluster-averaged model-based residuals. Permutation tests are guaranteed to be valid even for small samples, unlike modeling approaches. A similar model-based permutation approach using an optimally weighted combination of residuals was developed by Braun and Feng [15].

Recent methodological developments in covariate adjustment for RCTs include van der Laan’s Targeted Maximum Likelihood [16] and Tsiatis’ augmentation approach [12, 17]. These methods adapt semiparametric theory developed by Robins [18] and Robins, Rotnitzky, and Zhao [19] for observational studies with time-varying exposures and missing data problems, respectively. RCTs may be conceptualized theoretically in either framework, with counterfactual outcomes under the treatment not received considered missing, or as observational studies with a known probability of point exposure. Robins et al. [19] and Robins [18] characterize the efficient influence function in these settings [19, 18]. van der Laan and Tsiatis solve the set of estimating functions determined by the efficient score using two different approaches, which are equivalent in the absence of model misspecification.

Targeted Maximum Likelihood Estimation (tMLE) is an iterative procedure that involves adding a cleverly defined covariate to standard regression models. Upon convergence, the tMLE estimator solves the efficient influence function for the parameter of interest, resulting in bias reduction and efficiency improvement relative to maximum likelihood. tMLE is currently available for independent binary, continuous, time-to-event outcomes [20, 21], and most recently, clustered or longitudinal outcomes. Tsiatis’ approach involves directly solving a set of augmented estimating equations determined by the efficient influence function for the marginal treatment effect [17]. This method has been explored for continuous, binary, and discrete survival outcomes [22, 12, 17, 23]. Current applications of the augmentation method have focused on independent outcomes, with the exception of a simulation study based on the linear mixed model [17].

While tMLE simultaneously uses baseline covariates from treatment and control groups to target treatment effect estimation, Tsiatis’ method separates covariate adjustment and treatment evaluation. It also has the added advantage of allowing separate adjustment for baseline covariates within treatment arms. If done by separate statistical groups that do not share data, this approach reduces the risk that adjustment models are chosen to yield the most significant result. Even without decoupling of adjustment and treatment effect estimation, all covariate adjustment methods can be made objective by prespecifying the adjustment strategy.

2. THE SIMPLE AUGMENTED GEE

This section demonstrates the use of augmented estimating equations in analyses of cluster randomized trials. In such a trial, m clusters of size n_i, i = 1,…,m, are randomized to either treatment (A_i = 1) or control (A_i = 0) with probability P(A_i = 1) = π. To motivate the augmented GEE, we first review standard GEE. Let Y_ij denote the response for the j_th individual in the i_th cluster. Y_i = (Y_i1, Y_i2,…,Y_ini)^T, where n_i is the number of subjects within the i_th cluster. GEE for the marginal treatment effect are defined by the mean model (1), where β is a p-dimensional parameter. An estimator for β is obtained by solving the estimating equations

$$\sum _{i=1}^m{\psi }_i(\mathrm{Y},A;\beta )=\sum _{i=1}^m{{\mathrm{D}}_{\mathrm{i}}}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}^{-1}\{{\mathrm{Y}}_{\mathrm{i}}-\mathsf{g}(A_i;\beta \left)\right\}=0,$$ (2)

where ${\mathrm{D}}_{\mathrm{i}}=\frac{{\partial }_{\mathsf{g}}(A_i;\beta )}{\partial {\beta }^{\mathrm{T}}}$, ${\mathrm{V}}_{\mathrm{i}}={\mathrm{V}}_{\varphi }^{1∕2}\mathrm{R}\left\{\alpha \right(A_i\left)\right\}{\mathrm{V}}_{\varphi }^{1∕2}$, and bold g(A_i; β) denotes the n_i-dimensional link function for the outcome vector Y_i. Covariance matrix V_i is determined by the n_i × n_i matrix function v(A_i). The variance function v(A_i) is a product of the diagonal matrix V_ϕ, where V_ϕi,i is the variance of Y_ij, and correlation matrix R{α(A_i)}, where we allow α to be treatment specific. This differs from the usual presentation of GEE, in which V_i is constant and does not depend on A_i. Because our model does not place any restrictions on V_i, we generalize the usual approach to allow V_i to be more flexible. Variance parameters ϕ and α_k, where k indexes treatment, are estimated by the method of moments using ${\widehat{\beta }}_{\mathit{init}}$, an initial estimator of β. To recover the GEE fit in standard software, the above expressions simplify such that v(A_i) = v(1) = v(0) = V, and a single correlation parameter α is estimated across all clusters. In a slight abuse of notation, we take V_i to be the matrix function v(A_i) and V the constant variance matrix.

For continuous outcomes with the identity link $\mathsf{g}(A_i;\beta )={\mathrm{A}}_i^{\ast }\beta$, where ${\mathrm{A}}_i^{\ast }$ is the n_i × 2 design matrix composed of rows (1, A_i), the solution to (2), $\widehat{\beta }$, exists in closed form, with

$$\widehat{\beta }={\left(\sum _{i=1}^m{{\mathrm{A}}_i^{\ast }}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}^{-1}{\mathrm{A}}_i^{\ast }\right)}^{-1}\left(\sum _{i=1}^m{{\mathrm{A}}_i^{\ast }}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}^{-1}{\mathrm{Y}}_{\mathrm{i}}\right).$$

For simple designs, a closed form solution for $\widehat{\beta }$ can also be derived for non-identity link functions g(A_i; β) using the discreteness of A. The solution to (2) for the logit link is given in Appendix A of the supplementary material. Generally, for more complex models, GEE coefficient estimates are found using an iterative procedure such as the Newton-Raphson method or Iteratively Reweighted Least Squares (IRWLS).

Robins et al. [19] and Robins [18] established that in a model for data O = (Y, A, X) in which π_k = P(A = k|X) is known, any regular and asymptotically linear estimator for β can be found as the solution to ${\Sigma }_i{\psi }_{i_{\mathit{aug}}}(Y,A,X;{\stackrel{‒}{\gamma }}_K)=0$ for a specific choice of ${\stackrel{‒}{\gamma }}_K\left(X\right)$. Zhang et al. [17] demonstrated use of this theory in RCTs with univariate outcomes. Applying these results to multivariate settings, $\widehat{\beta }$ may be improved by augmenting the standard GEE with a function of baseline covariates X. The general form of the augmented GEE for a K-level treatment is

$$\sum _{i=1}^m{\psi }_{i_{\mathit{aug}}}(\mathrm{Y},A,\mathrm{X};\beta ,{\stackrel{‒}{\gamma }}_K)=\sum _{i=1}^m\left[{{\mathrm{D}}_{\mathrm{i}}}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}^{-1}\{{\mathrm{Y}}_{\mathrm{i}}-\mathsf{g}(A_i;\beta \left)\right\}-\sum _{k=1}^K\left\{I\right(A_i=k)-{\pi }_k\}{\gamma }_k\left({\mathrm{X}}_{\mathrm{i}}\right)\right]=0,$$ (3)

where γ_k(X_i) is a p-dimensional function of X_i.

It was further shown that for the class of estimating functions $\left\{{\psi }_{\mathit{aug}}\right({\stackrel{‒}{\gamma }}_K):{\stackrel{‒}{\gamma }}_K\in {\Gamma }_K\}$, where Γ_K is the set of all functions of X such that $E\left[{\psi }_{\mathit{aug}}{\left({\stackrel{‒}{\gamma }}_K\right)}^{\mathrm{T}}{\psi }_{\mathit{aug}}\right({\stackrel{‒}{\gamma }}_K\left)\right]<\infty$, the optimal estimator within this class for a fixed ψ (Y, A; β) is obtained by setting γ_kopt(X_i) = E{Ϙ_i(Y, A; β)|A_i = k, X_i} [19, 18, 17]. When only two treatment arms are considered, the augmentation term ${\sum }_{k=1}^K\left\{I\right(A_i=k)-{\pi }_k\}{\gamma }_k\left({\mathrm{X}}_{\mathrm{i}}\right)$ can be written as $(A_i-{\pi }_1)\left[{\mathrm{D}}_{\mathrm{i}}{\left(1\right)}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}{\left(1\right)}^{-1}\right\{E({\mathrm{Y}}_{\mathrm{i}}\mid A_i=1,{\mathrm{X}}_{\mathrm{i}})-\mathsf{g}(1;\beta )-{\mathrm{D}}_{\mathrm{i}}{\left(0\right)}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}{\left(0\right)}^{-1}\left\{E\right({\mathrm{Y}}_{\mathrm{i}}\mid A_i=0,{\mathrm{X}}_{\mathrm{i}})-\mathsf{g}(0;\beta \left)\right\}]$ The simple augmented GEE is thus

$$\sum _{i=1}^m{\psi }_{i_{\mathit{opt}}}(\mathrm{Y},A,\mathrm{X};\beta )=\sum _{i=1}^m{{\mathrm{D}}_{\mathrm{i}}}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}^{-1}\{{\mathrm{Y}}_{\mathrm{i}}-\mathsf{g}(A_i;\beta \left)\right\}-(A_i-\pi )\gamma \left({\mathrm{X}}_{\mathrm{i}}\right)=0,$$ (4)

where $\gamma \left({\mathrm{X}}_{\mathrm{i}}\right)=\left[{\mathrm{D}}_{\mathrm{i}}{\left(1\right)}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}{\left(1\right)}^{-1}\right\{E({\mathrm{Y}}_{\mathrm{i}}\mid A_i=1,{\mathrm{X}}_{\mathrm{i}})-\mathsf{g}(1;\beta )\}-{\mathrm{D}}_{\mathrm{i}}{\left(0\right)}^{-1}\{E({\mathrm{Y}}_{\mathrm{i}}\mid A_i=0,{\mathrm{X}}_{\mathrm{i}})-\mathsf{g}(0;\beta )\left\}\right]$

Solving for ${\widehat{\beta }}_{\mathit{aug}}$ therefore requires knowledge of ϕ, α_k, π, and E(Y_i|X_i, A_i = k) for k = 0,1. Following standard practice, we estimate ϕ and α_k using the residuals from a GLM fit under independence. Specifically, ϕ is estimated by the Pearson Chi-Square statistic, and α_k is obtained by solving the treatment-specific moment equations ${\sum }_{i=1}^{m}I(A_i=k)\{{\widehat{∊}}_{\mathit{ij}}{\widehat{∊}}_{\mathit{ij}}-h({\alpha }_k\left)\right\}=0$, where ${\widehat{∊}}_{\mathit{ij}}=Y_{\mathit{ij}}-g(A_i;{\widehat{\beta }}_{\mathit{init}})$, and h(α_k) is determined by the correlation structure assumed. For fixed ψ_i(Y, A; β), the optimality of the augmentation depends on correct estimation of E(Y_i|X_i, A_i = k)=f_k(X_i; η_k). When E(Y_i|X_i, A_i = k) is misspecified, asymptotic normality and consistency hold, but the resulting estimator does not achieve maximum asymptotic efficiency. Several options are available for estimating the conditional mean E(Y_i|X_i, A_i). We propose a strategy below which in large samples is guaranteed to improve on standard GEE. Following Tsiatis’ approach of estimating E(Y_i|X_i, A_i) separately within each arm, estimation proceeds via ordinary least squares (OLS), or maximum likelihood (ML) on an appropriately defined generalized linear model. Although the observations within a cluster are not independent, the predicted values from OLS and ML fits remain consistent. For treatment-specific estimation, the argument in [22] may be generalized to GEE, guaranteeing that when E(Y_i|X_i, A_i) is estimated with OLS, the augmented estimator is at least as efficient as the unaugmented estimator for continuous and discrete outcomes. This property holds even if models are misspecified. To more correctly specify the mean function, one may opt to fit an appropriate GLM, such as logistic regression for a binary outcome. We explore both approaches through simulation. It is also worthwhile to note that if the probability of treatment depends on baseline covariates X_i such that π_k = P(A_i = k|X_i), the simple augmented GEE does provide a valid estimate of treatment effects, but OLS is no longer sufficient to guarantee efficiency improvement over unaugmented methods. For continuous Y_ij and identity link g(A_i; β), the improved estimator is

$$\begin{array}{cc}\hfill & {\widehat{\beta }}_{\mathit{aug}}={\left[\sum _{i=1}^m{{\mathrm{A}}_i^{\ast }}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}^{-1}{A}_i^{\ast }-(A_i-\pi )\left\{A_i^{\ast }{\left(1\right)}^{\mathrm{T}}{\mathrm{V}}_i{\left(1\right)}^{-1}{\mathrm{A}}^{\ast }\right(1)-{\mathrm{A}}_i^{\ast }{\left(0\right)}^{\mathrm{T}}{\mathrm{V}}_i{\left(0\right)}^{-1}{\mathrm{A}}^{\ast }(0\left)\right\}\right]}^{-1}\times \hfill \\ \hfill & \left[\sum _{i=1}^m{{\mathrm{A}}_i^{\ast }}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}^{-1}{\mathrm{Y}}_{\mathrm{i}}-(A_i-\pi )\left\{{\mathrm{A}}_i^{\ast }{\left(1\right)}^{\mathrm{T}}{\mathrm{V}}_i{\left(1\right)}^{-1}\widehat{E}\right({\mathrm{Y}}_{\mathrm{i}}\mid A_i=1,{\mathrm{X}}_i)-{\mathrm{A}}_i^{\ast }{\left(0\right)}^{\mathrm{T}}{\mathrm{V}}_i{\left(0\right)}^{-1}\widehat{E}({\mathrm{Y}}_{\mathrm{i}}\mid A_i=0,X_i\left)\right\}\right]\hfill \end{array}$$

As in the unaugmented case, a closed form solution can be derived for non-identity links under a simple design. Solutions for the logit link may be found in Appendix A.

Implementation of the simple augmented GEE for inclusion of baselines covariates in analysis of a cluster randomized trial is summarized in the following steps:

1. Determine $\widehat{E}({\mathrm{Y}}_{\mathrm{i}}\mid {\mathrm{X}}_{\mathrm{i}},A_i=k)=f_k({\mathrm{X}}_{\mathrm{i}};{\widehat{\eta }}_k)$ from OLS or ML regression of Y onto baseline covariates X within each treatment arm.

2. Fit a GLM under independence to obtain ${\widehat{\beta }}_{\mathit{init}}$.

3. Estimate ϕ and α_k from ${\widehat{∊}}_{\mathit{ij}}$ of the initial fit.

4. Construct the augmented estimating equations ψ_aug(Y, A, X; β).

5. Solve for ${\widehat{\beta }}_{\mathit{aug}}$

The GEE was initially proposed as an iterative procedure, in which fitting involved repeatedly estimating correlation parameters α and mean parameters β until convergence. Since its inception, however, theoretical development and simulation studies have shown that the one-step procedure, as we have proposed for the augmented estimator, provides asymptotically equivalent estimates to the fully iterated approach, with similar finite sample properties [24].

3. VARIANCE ESTIMATION

The asymptotic variance of ${\widehat{\beta }}_{\mathit{init}}$, under m → ∞, is derived through the usual M-estimator Taylor expansion, accounting for the nuisance parameters ${\widehat{\eta }}_k$ involved in estimating E(Y_i|X_i, A_i = k)=f_k(X_i; η_k). We let ${\widehat{\psi }}_{i_{\mathit{opt}}}(\mathrm{Y},A,\mathrm{X};\beta )$ be an estimate of (4) evaluated at $\widehat{\eta }$. The familiar sandwich variance estimator $\mathit{var}\left({\widehat{\beta }}_{\mathit{aug}}\right)={\Gamma }^{-1}\Delta {\Gamma }^{-1^{\mathrm{T}}}$ is obtained, where $\Gamma =E\left[\frac{\partial {\psi }_{i_{\mathit{opt}}}(\mathrm{Y},A,\mathrm{X};\beta )}{\partial {\beta }^{\mathrm{T}}}\right]$, and $\Delta =E\left[{\psi }_{i_{\mathit{opt}}}{(\mathrm{Y},A,\mathrm{X};\beta )}^{\otimes 2}\right]$, where $U^{\otimes 2}=U{U}^{\mathrm{T}}$. By randomization, the augmentation term has mean zero and does not contribute to Γ. We therefore estimate Γ by $\widehat{\Gamma }=m^{-1}{\Sigma }_i{{\mathrm{D}}_{\mathrm{i}}}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}^{-1}{\mathrm{D}}_{\mathrm{i}}$. Estimation of η_k results in additional terms in our expansion of ${\widehat{\psi }}_{i_{\mathit{opt}}}(\mathrm{Y},A,\mathrm{X};\beta )$ shown below.

$$\begin{array}{cc}\hfill & \sum _{i=1}^m{\widehat{\psi }}_{i_{\mathit{opt}}}(\mathrm{Y},A,\mathrm{X};\beta )=\hfill \\ \hfill & \sum _{i=1}^m\{{{\mathrm{D}}_{\mathrm{i}}}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}^{-1}\{{\mathrm{Y}}_{\mathrm{i}}-\mathsf{g}(A_i;{\beta }_0\left)\right\}\phantom{\}}\hfill \\ \hfill & \phantom{\{}-(A_i-\pi )\left[{\mathrm{D}}_{\mathrm{i}}{\left(1\right)}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}{\left(1\right)}^{-1}\right\{f_1({\mathrm{X}}_{\mathrm{i}};{\widehat{\eta }}_1)-\mathsf{g}(1;{\beta }_0)\}-{\mathrm{D}}_{\mathrm{i}}{\left(0\right)}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}{\left(0\right)}^{-1}\{f_0({\mathrm{X}}_{\mathrm{i}};{\widehat{\eta }}_0)-\mathsf{g}(0;{\beta }_0)\left\}\right]\}\hfill \\ \hfill & =\sum _{i=1}^m{\tilde{\psi }}_{i_{\mathit{opt}}}=\sum _{i=1}^m\{{{\mathrm{D}}_{\mathrm{i}}}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}^{-1}\{{\mathrm{Y}}_{\mathrm{i}}-\mathsf{g}(A_i;{\beta }_0\left)\right\}\phantom{\}}\hfill \\ \hfill & -(A_i-\pi )\left[{\mathrm{D}}_{\mathrm{i}}{\left(1\right)}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}{\left(1\right)}^{-1}\left\{f_1\right({\mathrm{X}}_{\mathrm{i}};{\eta }_1^{\ast })-\mathsf{g}(1;{\beta }_0\left)\right\}-{\mathrm{D}}_{\mathrm{i}}{\left(0\right)}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}{\left(0\right)}^{-1}\left\{f_0\right({\mathrm{X}}_{\mathrm{i}};{\eta }_0^{\ast })-\mathsf{g}(0;{\beta }_0\left)\right\}\right]\hfill \end{array}$$ (a)

$$\phantom{\{}-(A_i-\pi )\left[{\mathrm{D}}_{\mathrm{i}}{\left(1\right)}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}{\left(1\right)}^{-1}\left\{f_1^{\prime }\right({\mathrm{X}}_{\mathrm{i}};{\eta }_1^{\ast }\left)\right\}({\widehat{\eta }}_1-{\eta }_1^{\ast })-{\mathrm{D}}_{\mathrm{i}}{\left(0\right)}^{\mathrm{T}}{\mathrm{V}}_{\mathrm{i}}{\left(0\right)}^{-1}\left\{f_0^{\prime }\right({\mathrm{X}}_{\mathrm{i}};{\eta }_0^{\ast }\left)\right\}({\widehat{\eta }}_0-{\eta }_0^{\ast })\right]\}+o_p\left(1\right)$$ (b)

where ${\widehat{\eta }}_k\stackrel{P}{\to }{\eta }_k^{\ast }$. By randomization (b) $\stackrel{P}{\to }0$ as m → ∞, showing that asymptotically there is no additional variability associated with estimating η_k, even when f_k(X_i; η_k) is misspecified. For cluster randomized designs, however, the asymptotics may not hold, as the number of experimental units may be fairly small. In small sample settings, it is likely that $\mathit{var}\left({\widehat{\beta }}_{\mathit{aug}}\right)$ is affected by estimation of η_k. We therefore estimate Δ by $\widehat{\Delta }=m^{-1}{\Sigma }_i{\tilde{\psi }}_{i_{\mathit{opt}}}^{\otimes 2}$, with and without term (b) and evaluate our variance estimator through simulation. Specifically, we estimate $({\widehat{\eta }}_k-{\eta }_k^{\ast })$ by its first order approximation and substitute estimated parameter values for the truth. Inclusion of (b) is not guaranteed to increase the estimated variance but does provide a more unbiased estimate by accounting for estimation of η_k.

The sandwich variance estimator of standard GEE is known to often be biased downward for inference involving relatively few independent units. We examine Fay’s bias-correction approach to recover loss. We choose this approach because unlike other methods which were derived for standard GEE, Fay’s method is generalizable to any M-estimator, including our augmented estimating equations. We apply Fay’s first correction, in which Δ is estimated by ${\widehat{\Delta }}^{\ast }=m^{-1}{\Sigma }_i{\left({\mathrm{H}}_{\mathrm{i}}{\widehat{\psi }}_i\right)}^{\otimes 2}$, where H_i is a diagonal matrix with $H_{i_{\mathit{jj}}}={\left[1-\mathit{min}\{q,{(\frac{\partial {\psi }_i(\mathrm{Y},A,\mathrm{X};\beta )}{\partial {\beta }^{\mathrm{T}}}\times \widehat{\Gamma })}_{\mathit{jj}}\}\right]}^{1∕2}$. Lower bound q is typically set to 0.75 to prevent gross inflation [5]. In total, we consider four standard error estimators for the simple augmented GEE: 1) unadjusted sandwich (SE₁), 2) nuisance $\left({\widehat{\eta }}_k\right)$-adjusted sandwich [term (b)] (SE₂), 3) sandwich with Fay’s small-sample bias correction (SE₃), and 4) sandwich with Fay’s small-sample bias correction and nuisance adjustment (SE₄), and evaluate each through simulation. Formulas for each estimator are provided in Appendix B of the supplementary material.

An alternative estimate $v\widehat{a}r\left({\widehat{\beta }}_{\mathit{aug}}\right)$ can be computed through a resampling technique such as the nonparametric bootstrap. To preserve the number of treated and control clusters within any bootstrap sample, we resample clusters within treatment arm. We use strategy 1 described by Davidson and Hinkley [25], in which the composition of resampled clusters is maintained, and demonstrate this approach through data analysis.

4. APPLICATION: YOUNG CITIZENS STUDY

We applied the simple augmented GEE to data from the Young Citizens study [26]. This trial involved a behavioral intervention designed to train children ages 10-14 to educate their communities about HIV. To facilitate randomization, 30 communities, were grouped into 15 pairs using a clustering algorithm involving several demographic characteristics. One community per pair was randomly assigned to treatment and the other to control. Residents within each community were surveyed post-intervention regarding their beliefs about the ability of children to effectively teach their peers and families about HIV. The primary outcome was a composite score reflecting the strength of this belief (Y₁). A secondary outcome measured residents’ beliefs regarding whether or not the AIDS problem was getting worse in their communities. Residents responded on a 4-point scale with values ‘strongly disagree’, ‘disagree’, ‘agree’, and ‘strongly agree’. Responses were dichotomized by collapsing ‘strongly agree’ and ‘agree’ into one category labeled ‘agree’; ‘strongly disagree’ and ‘disagree’ were collapsed similarly (Y₂). The number of residents surveyed per community ranged from 16-80 by multiples of 16.

We implemented standard and augmented GEE using the customary single correlation parameter as well as the less restrictive approach of allowing treatment-specific correlation. For augmented estimators, we also included estimation under independence to further examine the impact of covariance selection on the efficiency of augmented inference. Adjustment models were determined separately for treatment and control groups by various model selection procedures. The final models used in analysis were selected via cross validation. For child efficacy (Y₁), the adjustment model in the treatment arm included the baseline covariates employment status, residential or urban community, the number of relatives living in the community, age, religion, population density, and whether or not the household had a flushing toilet, which was an indicator of household wealth. Among control communities, employment, age, and flushing toilet were included. The baseline covariates that entered the adjustment models for beliefs about the state of the HIV problem (Y₂) were mean community wealth, ethnic group, and household wealth for the intervention arm, and only mean community wealth for the control arm.

In analyzing our continuous outcome Y₁, we evaluated the marginal treatment effect by considering model (1), where g(A_i; β) was the identity link function. We computed the standard error of ${\widehat{\beta }}_1$ by the sandwich estimator and the nonparametric bootstrap for each estimation procedure. The standard error modifications in section 3 were applied, namely: 1) unadjusted sandwich (SE₁), 2) nuisance $\left(\widehat{\eta }\right)$-adjusted sandwich (term (b) above) (SE₂), 3) sandwich with small-sample bias correction (SE₃), and 4) sandwich with bias correction and nuisance adjustment (SE₄). In our second application, we evaluated the marginal treatment effect on the binary secondary outcome Y₂ and fit model (1) with the inverse logit link. We compared estimates obtained from standard GEE, the simple augmented GEE, adjusted logistic GEE with standardization i.e. the G-formula [27], and inverse probability of treatment weighted (IPTW) methods. In the IPTW approach, we ignore that the treatment probability is known and estimate P(A = 1|X) using a logistic regression model in which covariates were entered linearly.

In standard and augmented analyses, the intervention had a highly significant impact on the perceived ability of children to be peer educators {95% CI Standard (0.182, 0.526), 95% CI Augmented (0.245, 0.482)}. The adjusted sandwich variance estimator suggested over a 70% increase in efficiency resulting from covariate adjustment (ARE, Table 1). Bootstrap estimates showed a similar efficiency gain under common correlation (58%) and a much more modest gain using treatment specific correlation (5%) (‘ARE boot’, Table 1). Comparing within unaugmented and augmented estimators, little difference in standard error was observed between estimators allowing for treatment-specific correlation versus estimators assuming common correlation. Estimates of β₁ were similar across standard and augmented estimators with either correlation structure.

Table 1. Marginal Treatment Effect Analysis: Parameter estimates, sandwich standard errors, and bootstrap standard error.

Std: unaugmented GEE, Aug: augmented GEE. Ind: independence, Exch: exchangeable correlation with single parameter, Exch-TS: exchangeable with treatment-specific correlation parameters. SE₁: unadjusted sandwich SE, SE₂: sandwich with nuisance parameter adjustment, SE₃: sandwich with small-sample adjustment, SE₄: sandwich with small-sample and nuisance adjustments. Exch: Exchangeable correlation with single parameter, Exch-TS: Exchangeable with treatment-specific correlation parameters. ARE: square of the sandwich SE of the Std(Exch) estimator divided by the square of the sandwich SE of the indicated estimator. ARE boot: square of the bootstrap SE of the Std(Exch) estimator divided by the square of the bootstrap SE of the indicated estimator. ARE and Confidence intervals(CIs) are based on SE₃ and SE₄ for unaugmented and augmented estimators, respectively.

	Estimate	Sandwich Standard Error with Adjustments						Bootstrap SE
	${\widehat{\beta }}_1$	SE1	SE2	SE3	SE4	95% CI	ARE	SE boot	ARE boot
Std (Exch)	0.354	0.082	-	0.088	-	(0.182,0.526)	1.000	0.081	1.000
Std (Exch-TS)	0.355	0.082	-	0.086	-	(0.186,0.525)	1.034	0.085	0.910
Aug (Ind)	0.360	0.066	0.060	0.071	0.064	(0.236,0.485)	1.528	0.062	1.681
Aug (Exch)	0.364	0.063	0.057	0.067	0.060	(0.245,0.482)	1.730	0.064	1.582
Aug (Exch-TS)	0.360	0.063	0.061	0.066	0.064	(0.234,0.485)	1.769	0.082	0.966

Examining our binary outcome, Y₂, there was a marked difference in the estimated parameters when comparing standard and augmented GEE (Table 2). The estimate ${\widehat{\beta }}_1$ was −0.238 {95% CI (−0.777, 0.300)} using standard methods, compared to values the range (−0.079,−0.023) for all augmented GEE estimates. In either approach, the effect of treatment on the perception of the AIDS epidemic was not significant at the p=0.05 level {95% CI Augmented GEE=(-0.491,0.332)}. Estimates from the standardized adjusted logistic GEE were also closer to 0. Although effects were not significant at the p=0.05 level for any of the approaches, confidence intervals for the augmented GEE were somewhat tighter, as were those using standard methods of covariate adjustment.

Table 2. Marginal Treatment Effect Analysis with Binary Outcome: Parameter sandwich standard error estimates.

Std: unaugmented GEE, Aug: augmented GEE. Ind: independence, Exch: exchangeable correlation with single parameter, Exch-TS: exchangeable with treatment-specific correlation parameters. GLM, OLS: generalized linear model or ordinary least squares augmentation. ARE: square of the sandwich SE of the Std(Exch) estimator divided by the square of the sandwich SE of the indicated estimator. Confidence intervals (CIs) and ARE based on adjusted sandwich standard errors. Adjusted Logistic GEE-Model 1: logit(P (Y_ij = 1)) = η₀ + η₁Mean_wealth_i, Model 2: logit(P (Y_ij = 1)) = η₀ + η₁Mean_wealth_i + η₂I(Ethnic_ij = 1) + η₃I(Wealth_ij = 0). IPTW: logit(P (A_i = 1)) = η₀ + η₁Know_leader_i + η₂Good_floor_i.

Estimator	${\widehat{\beta }}_1$	SE	95%CI	ARE
Standard GEE
Std Exch	−0.238	0.275	(−0.777,0.300)	1.000
Std Exch TS	−0.219	0.266	(−0.74,0.301)	1.069
Augmented GEE
Aug Ind -GLM	−0.062	0.215	(−0.484,0.361)	1.627
Aug Exch - GLM	−0.079	0.21	(−0.491,0.332)	1.716
Aug Exch -TS - GLM	−0.065	0.204	(−0.465,0.335)	1.811
AugInd-OLS	−0.023	0.22	(−0.454,0.408)	1.561
AugExch-OLS	−0.062	0.214	(−0.481,0.358)	1.648
AugExch-TS-OLS	−0.047	0.206	(−0.451,0.358)	1.773
Adjusted Logistic GEE
Model 1	−0.093	0.167	(−0.420,0.234)	2.712
Model 2	−0.044	0.179	(−0.396,0.308)	2.349
IPTW Logistic GEE
Model 3	−0.293	0.236	( −0.756,0.170)	1.354

Considering efficiency, for both outcomes the estimated variability was lower for the augmented estimator compared to standard GEE. Although there is some uncertainty regarding the behavior of the sandwich estimator in small samples, these results suggest that when the asymptotics hold, augmented GEE is a valid approach that may be substantially more efficient than standard GEE. Randomized trials involving longitudinal data with many subjects or clustered designs with many smaller units, such as households or offspring, are therefore ideal candidates for this method. We evaluate our method and the behavior of the sandwich variance estimator through simulation in the following section.

5. SIMULATIONS

We assessed the performance of the simple augmented GEE in two sets of simulations. The first investigates continuous outcomes with an identity link, and the second set explores binary outcomes using an inverse logit link. We considered the impact of misspecification of the augmentation term and working covariance structure on the performance of our estimator. Results are based on 1000 simulated datasets.

5.1. SIMULATIONS 1

Cluster level covariates were treatment, density, wealth, and community type (eg., urban/rural, etc). Treatment assignment was completed by first fixing the number of treated and control clusters to m/2, where m is the total number of clusters. Clusters were then randomly assigned to treatment or control. Community type was generated from a multinomial distribution. Density and wealth were generated from the exponential and normal distributions, respectively. Individual-level covariates age, employment, security1, and security2 were simulated from normal and multinomial distributions with age treated as continuous and other covariates categorical. Data were generated following the means and variances of covariates in the Young Citizens data. Intracluster correlation was induced by cluster-specific random effects and community-level covariates. We considered varying levels of correlation for treated versus control clusters. To assess small-sample performance, we compared scenarios of m = 30 and m = 100 clusters.

Outcomes were generated from the following models: (Y_ij|X_ij, A_i = 1) = 7.23 + 0.599employed_ij + 0.44mean wealth_i – 0.22I(security1_ij = 3) – 0.06age_ij + 49.702density_i + b_1i + ∈_ij, and (Y_ij|X_ij, A_i = 0) = 2.56 + 0.245employed_ij + 0.691I(community type_i = 4) + 0.921I(security2_ij = 4) + 0.055age_ij + b_0i + ∈_ij, where $b_{\mathit{ki}}~N(0,{\sigma }_k^2)$, and ${∊}_{\mathit{ij}}~N(0,{\sigma }^2)$. Community-level and individual-level covariates therefore contributed to heterogeneity in subject responses. Values of ${\sigma }_1^2$ and ${\sigma }_0^2$ were selected to yield the desired within-cluster marginal correlation (ρ_k). For treatment and control clusters alike, ${\sigma }^2=1$.

We evaluated the effect of working covariance and augmentation misspecification by estimating β₁ and its variance under different covariance structures and augmentation models. Two variations of standard GEE were considered: standard GEE with common exchangeable correlation {Std(Exch)}, and standard GEE with treatment-specific exchangeable correlation {Std(Exch-TS)}. For the class of augmented GEE, we estimated ${\widehat{\beta }}_{\mathit{aug}}$ with independence, exchangeable, and treatment-specific exchangeable correlation structures. Each estimator was evaluated under several augmentation models. The estimator resulting from fitting the true form of E(Y_ij|X_ij, A_i = k) is denoted by ‘C’ for ‘Correct’. Alternative augmentation models were defined by forward (F) and backward (B) selection, and a wrong (W) model. The wrong models were given by E(Y_ij|X_ij, A_i = 1) = η₀ + η₁mean_wealth_i + η₂I(community_type_i = 2) + η₃I(security1_ij = 2) and E(Y_ij|X_ij, A_i = 0) = η₀ + η₁density_i + η₂age_ij + η₃I(community_type_i = 1) + η₄I(security2_ij = 4). Augmentation under the ‘Correct’ model illustrates the largest possible efficiency gain. Alternative model fitting techniques were chosen to be representative of methods commonly used when performing covariate adjustment in analyzing clinical trial data. Forward or backward stepwise selection may be used by analysts favoring more parsimonious or larger models, respectively. The ‘Wrong’ model was included for comparison using models that contain some relevant covariates but omit others. To correct for small-sample variance underestimation, we applied several modifications to the sandwich estimator as detailed in Section 3. For the unaugmented estimators, we calculated the sandwich variance (SE₁), and the sandwich variance with bias correction (SE₃). Standard errors for augmented estimators were calculated using SE₂ and SE₄ as well, which account for η_k-estimation.

Table 3 shows results for m = 30 and 100, ${\sigma }_1^2=0.03$, and ${\sigma }_0^2=0.025$, which correspond to approximately 10% and 5% within-cluster correlation in treated and control clusters, respectively. For Table 4, we raise the level of unexplained similarity among cluster members by setting ${\sigma }_1^2=0.23$ and ${\sigma }_0^2=0.20$ for the sample sizes previously considered.

Table 3. Standard vs. Augmented GEE, Continuous Outcome: 30 & 100 clusters, ρ₀ = 0.05, ρ₁ = 0.10, β₁ = 1.3239.

Std: unaugmented. C,F,B,W: augmented with ‘Correct’, ‘Forward’ selected, ‘Backward’ selected, or ‘Wrong’ models. Ind: independence, Exch: exchangeable with single correlation parameter. Exch-TS: exchangeable with treatment-specific parameter. SE₁: average unadjusted sandwich SE, SE₂: average sandwich SE with nuisance parameter adjustment, SE₃: average sandwich SE with small-sample adjustment, SE₄: average sandwich SE with small-sample and nuisance adjustments. MC SE: Monte Carlo standard deviation. MC RE: square of the Monte Carlo SE of the Std(Exch) estimator divided by the Monte Carlo SE for the indicated estimator. Cov. U: SE₁ coverage, Cov. A: SE₃ and SE₄ coverage for unaugmented and augmented GEE, respectively.

m=30	${\widehat{\beta }}_1$	Bias	SE1	SE2	SE3	SE4	MC SE	MC RE	Cov. U	Cov. A
Std(Exch)	1.324	0.000	0.156	-	0.165	-	0.165	1.000	0.933	0.949
Std(Exch-TS)	1.325	−0.001	0.158	-	0.166	-	0.164	1.004	0.934	0.949
C(Ind)	1.324	−0.001	0.135	0.138	0.140	0.144	0.151	1.194	0.912	0.929
F(Ind)	1.325	−0.001	0.132	0.136	0.137	0.142	0.153	1.162	0.895	0.922
B(Ind)	1.325	−0.001	0.132	0.137	0.137	0.142	0.154	1.149	0.893	0.918
W(Ind)	1.321	0.003	0.143	0.151	0.150	0.158	0.160	1.057	0.911	0.937
C(Exch)	1.328	−0.004	0.130	0.132	0.134	0.137	0.147	1.248	0.909	0.927
F(Exch)	1.328	−0.004	0.128	0.131	0.132	0.135	0.149	1.219	0.898	0.917
B(Exch)	1.328	−0.004	0.128	0.131	0.132	0.135	0.150	1.205	0.893	0.915
W(Exch)	1.326	−0.002	0.139	0.145	0.144	0.150	0.156	1.110	0.904	0.928
C(Exch-TS)	1.329	−0.005	0.134	0.138	0.137	0.143	0.149	1.223	0.914	0.925
F(Exch-TS)	1.329	−0.005	0.133	0.137	0.135	0.142	0.151	1.198	0.895	0.915
B(Exch-TS)	1.329	−0.005	0.132	0.137	0.135	0.142	0.151	1.186	0.890	0.910
W(Exch-TS)	1.328	−0.004	0.142	0.148	0.146	0.154	0.159	1.078	0.900	0.927
m=100	${\widehat{\beta }}_1$	Bias	SE1	SE2	SE3	SE4	MC SE	MC RE	Cov. U	Cov. A
Std(Exch)	1.3261	−0.002	0.088	-	0.090	-	0.090	1.000	0.943	0.946
Std(Exch-TS)	1.3265	−0.003	0.088	-	0.089	-	0.090	1.001	0.942	0.945
C(Ind)	1.3276	−0.004	0.077	0.077	0.078	0.078	0.078	1.333	0.942	0.947
F(Ind)	1.3280	−0.004	0.076	0.077	0.077	0.078	0.079	1.296	0.937	0.943
B(Ind)	1.3279	−0.004	0.076	0.077	0.077	0.078	0.079	1.297	0.937	0.944
W(Ind)	1.3266	−0.003	0.083	0.084	0.084	0.085	0.088	1.048	0.935	0.945
C(Exch)	1.3281	−0.004	0.074	0.074	0.075	0.075	0.075	1.449	0.944	0.945
F(Exch)	1.3286	−0.005	0.073	0.074	0.074	0.075	0.076	1.409	0.933	0.941
B(Exch)	1.3286	−0.005	0.073	0.074	0.074	0.075	0.076	1.410	0.932	0.94
W(Exch)	1.3269	−0.003	0.080	0.081	0.080	0.082	0.083	1.166	0.936	0.943
C(Exch-TS)	1.3282	−0.004	0.074	0.074	0.074	0.075	0.074	1.452	0.946	0.95
F(Exch-TS)	1.3288	−0.005	0.073	0.074	0.074	0.075	0.075	1.412	0.937	0.943
B(Exch-TS)	1.3287	−0.005	0.073	0.074	0.074	0.075	0.075	1.413	0.936	0.943
W(Exch-TS)	1.3273	−0.003	0.079	0.080	0.080	0.081	0.083	1.168	0.934	0.943

Table 4. Standard vs. Augmented GEE, Continuous Outcome: 30 & 100 clusters, ρ₀ = 0.13, ρ₁ = 0.17, β₁ = 1.3239.

Std: unaugmented. C,F,B,W: augmented with ‘Correct’, ‘Forward’ selected, ‘Backward’ selected, or ‘Wrong’ models. Ind: independence, Exch: exchangeable with single correlation parameter. Exch-TS: exchangeable with treatment-specific parameter. SE₁: average unadjusted sandwich SE, SE₂: average sandwich SE with nuisance parameter adjustment, SE₃: average sandwich SE with small-sample adjustment, SE₄: average sandwich SE with small-sample and nuisance adjustments. MC SE: Monte Carlo standard deviation. MC RE: square of the Monte Carlo SE of the Std(Exch) estimator divided by the Monte Carlo SE for the indicated estimator. Cov. U: SE₁ coverage, Cov. A: SE₃ and SE₄ coverage for unaugmented and augmented GEE, respectively.

m=30	${\widehat{\beta }}_1$	Bias	SE1	SE2	SE3	SE4	MC SE	MC RE	Cov. U	Cov. A
Std(Exch)	1.312	0.012	0.217	-	0.229	-	0.233	1.000	0.933	0.937
Std(Exch-TS)	1.313	0.011	0.217	-	0.227	-	0.233	1.000	0.929	0.934
C(Ind)	1.317	0.007	0.209	0.219	0.221	0.232	0.234	0.989	0.907	0.934
F(Ind)	1.317	0.007	0.201	0.216	0.211	0.226	0.255	0.834	0.863	0.908
B(Ind)	1.317	0.007	0.202	0.216	0.211	0.227	0.256	0.828	0.861	0.911
W(Ind)	1.310	0.014	0.213	0.226	0.226	0.240	0.250	0.864	0.897	0.922
C(Exch)	1.319	0.005	0.199	0.206	0.207	0.214	0.215	1.170	0.912	0.934
F(Exch)	1.319	0.005	0.194	0.205	0.201	0.213	0.237	0.963	0.881	0.916
B(Exch)	1.319	0.005	0.194	0.206	0.201	0.213	0.238	0.954	0.879	0.915
W(Exch)	1.314	0.010	0.204	0.214	0.213	0.224	0.232	1.008	0.910	0.94
C(Exch-TS)	1.320	0.004	0.198	0.205	0.206	0.213	0.215	1.172	0.911	0.935
F(Exch-TS)	1.319	0.004	0.193	0.205	0.200	0.212	0.237	0.964	0.884	0.914
B(Exch-TS)	1.320	0.004	0.193	0.205	0.200	0.212	0.238	0.955	0.882	0.914
W(Exch-TS)	1.315	0.009	0.203	0.214	0.212	0.223	0.232	1.008	0.909	0.938
m=100	${\widehat{\beta }}_1$	Bias	SE1	SE2	SE3	SE4	MC SE	MC RE	Cov. U	Cov. A
Std(Exch)	1.318	0.006	0.123	-	0.125	-	0.124	1.000	0.947	0.948
Std(Exch-TS)	1.318	0.006	0.123	-	0.125	-	0.123	1.004	0.947	0.949
C(Ind)	1.322	0.002	0.121	0.123	0.123	0.125	0.124	0.997	0.943	0.949
F(Ind)	1.321	0.003	0.118	0.121	0.120	0.123	0.125	0.979	0.935	0.941
B(Ind)	1.321	0.003	0.118	0.121	0.120	0.123	0.125	0.978	0.935	0.941
W(Ind)	1.320	0.004	0.124	0.127	0.127	0.129	0.128	0.931	0.935	0.944
C(Exch)	1.319	0.005	0.113	0.114	0.114	0.116	0.116	1.128	0.939	0.943
F(Exch)	1.318	0.006	0.112	0.114	0.113	0.115	0.118	1.102	0.929	0.937
B(Exch)	1.318	0.006	0.112	0.114	0.113	0.115	0.118	1.101	0.929	0.937
W(Exch)	1.317	0.007	0.117	0.118	0.118	0.120	0.120	1.056	0.934	0.944
C(Exch-TS)	1.319	0.005	0.113	0.114	0.114	0.115	0.116	1.133	0.941	0.945
F(Exch-TS)	1.318	0.006	0.111	0.114	0.113	0.115	0.118	1.106	0.931	0.939
B(Exch-TS)	1.318	0.006	0.111	0.114	0.113	0.115	0.118	1.105	0.931	0.939
W(Exch-TS)	1.317	0.007	0.116	0.118	0.118	0.120	0.120	1.061	0.935	0.942

For small-sample and large-sample inference, bias was similar across all estimators. Working covariance specification affected the variance of the augmented estimator, with exchangeable (true) correlation structures resulting in smaller average standard errors than independence. Comparing estimators calculated with an exchangeable correlation structure, augmented estimators were often more efficient than the standard approach. Monte Carlo relative efficiency estimates suggest that in the small-sample setting with low levels of unexplained intracommunity correlation, considerable improvement (5-19%) is observed even when misspecifying the augmentation model (Table 3). When intracluster correlation was larger, additional variability associated with automated model selection resulted in loss of efficiency associated with augmentation (Table 4). Average sandwich standard errors were overly optimistic in comparing augmented GEE to standard GEE in small samples, consistently estimating lower variability with augmentation. For large samples, efficiency gains were not hindered by higher levels of unexplained cluster similarity, with Monte Carlo efficiency improving by 5-40% (Table 4).

Coverage results show that for small samples, the uncorrected sandwich variance underestimates the variability of the augmented estimator (Tables 3 & 4). Bias correction fully recovered small-sample loss of variance for standard GEE. For augmented estimators, correction was less effective. Coverage was slightly increased by accounting for augmentation in the sandwich variance but did not quite reach nominal levels. For large-sample inference, neither adjustment substantially increased coverage, which was already close to nominal levels for the uncorrected sandwich variance without the nuisance term.

5.2. SIMULATIONS 2

To explore the performance of the augmented GEE for clustered binary outcomes, we again generated datasets of m clusters with probability of treatment P(A = 1) = 1/2. Cluster-level variables X₁ and X₂ were simulated from exponential and multinomial distributions with mean 0.002 and probabilities p=(0.46, 0.27, 0.07, 0.17, 0.03), respectively. Individual-level covariates X₃, X₄, X₅, and X₆ were generated such that (X₃, X₄) ~ Normal $\left(\left(00\right),\left(\begin{array}{cc}\hfill 4\hfill & \hfill 6\hfill \\ \hfill 6\hfill & \hfill 25\hfill \end{array}\right)\right)$ X₅ ~ Bernoulli(p = 0.28), and X₆ ~ Multinomial{1, p = (0.45, 0.15, 0.30, 0.10)}. We used the random logistic model to simulate correlated binary outcomes Y. Random intercepts b_i were drawn from the bridge distribution for the logit link [28], B_l(0, 1 – ρ), where 0 is the mean and ρ is the desired correlation. The bridge distribution was selected to preserve the logistic shape after marginalizing over random effects and provide a simple scaling relationship between parameters of the models for E(Y|X, A, b) and E(Y|X, A). Outcome generating models were: logit{E(Y_ij|X_ij,A_i = 1, b)} = η₁₀ + η₁₁X_3ij + η₁₂X_4ij + b_i, and $\log \mathit{it}\left\{E\right(Y_{\mathit{ij}}\mid X_{\mathit{ij}},A_i=0,b\left)\right\}={\eta }_{0_0}+{\eta }_{0_1}{X}_{4_{\mathit{ij}}}+{\eta }_{0_2}{X}_{4\mathit{ij}}^2+{\eta }_{0_3}{X}_{5_{\mathit{ij}}}+b_i$.

For low association between Y and X, we set η₀ = (3.4, –0.6, 0.03, 0.5)^T and η₁ = (2.5, −0.62, 0.86)^T. Coefficients η₀ = c(2.0, −0.9, 0.03, 0.5)^T and η₁ = (1.5, −0.62, 0.86)^T were used for a high association. We again compared small-sample versus large-sample performance by implementing standard and augmented GEE under m = 30, m = 100, and m = 250 clusters. We also considered two levels of intracluster correlation (ρ=0.05, 0.20). Results for m = 250 are included in Appendix C of the supplementary material.

We applied the augmented GEE under independent and exchangeable correlation structures and evaluated different methods of fitting the augmentation term. To guarantee improved efficiency relative to standard GEE, we fit augmentation models E(Y|X, A = k) using OLS. We contrast this approach with logistic regression, which correctly specifies the form of the relationship between Y and X, but is not guaranteed to improve efficiency under model misspecification. For each model fitting technique, we fit the correct augmentation model (C), a forward selection model (F), and two wrong models (O & W). Wrong models denoted by ‘O’ contained one baseline covariate. Specifically, the models fit were E(Y_ij|X_ij, A = 1) = g(α₁₀ + α₁₁X_4ij) and E(Y_ij|X_ij, A = 0) = g(α₀₀ + α₀₁X_3ij). Wrong models ‘W’ are given by E(Y_ij|X_ij, A = 1) = g(α₁₀ + α₁₁X_5ij + α₁₁X_2i) and E(Y_ij|X_ij, A = 0) = g(α₀₀ + α₀₁X_4ij + α₀₂X_1ij + α₀₃X_5ij).

Results were similar to those obtained for the continuous outcomes in the first set of simulations (Tables 5 & 6). Bias was similar across all methods of estimation for small- and large-sample inference, and correct specification of the working covariance resulted in more efficient estimation for augmented estimators. Small-sample results suggested that for low association of baseline covariates and outcome, small gains are possible for reasonably specified models (2%-10%), but for automated model selection and poorly specified models, efficiency loss occurs (−17%- −3%) because of additional variability introduced by model selection and estimation of the augmentation terms. Efficiency increased by 8%-35% when baseline covariates were more strongly related to the outcome, and unexplained intracluster correlation was low. For higher intracluster correlation, efficiency gains were lower (−5% - 12%, Table 6), with loss of efficiency for automated model selection. Similar to the continuous outcome, standard error adjustments were partially effective in recovering nominal coverage. When 100 clusters were sampled, augmentation increased efficiency by 1%-35% for high association or low intracluster correlation. With low association between X and Y and high intracluster correlation, augmentation decreased efficiency for poorly specified and automated models. Considering 250 clusters, augmented estimators were more efficient than unaugmented estimators across the levels of intracluster correlation, degree of X,Y association, and methods of model fitting that were considered (see supplementary material).

Table 5. Standard vs. Augmented GEE, Binary Outcome: 30 & 100 clusters, low and high association, ρ = 0.05, β₁ = −0.2959 low association, β₁ = 1.1362 high association.

Std: unaugmented. Correlation exchangeable unless denoted by ‘Ind’ for independence. C,F,O,W: augmentation with ‘Correct’, ‘Forward’ selected, ‘One-variable’, or ‘Wrong’ model. ML, OLS: augmentation fit with maximum likelihood or ordinary least squares. SE₁: average unadjusted sandwich SE, SE₂: average sandwich SE with nuisance parameter adjustment, SE₃: average sandwich SE with small-sample adjustment, SE₄: average sandwich SE with small-sample and nuisance adjustments. MC RE: square of the Monte Carlo SE of the Std(Exch) estimator divided by the Monte Carlo SE for the indicated estimator. Cov. U: SE₁ coverage, Cov. A: SE₃ and SE₄ coverage for unaugmented and augmented GEE, respectively.

Low	Estimator	${\widehat{\beta }}_1$	Bias	SE1	SE2	SE3	SE4	MC SE	MC RE	Cov. U	Cov. A
m=30	Std	−0.299	0.003	0.196	-	0.209	-	0.220	1.000	0.923	0.945
	C - ML (Ind)	−0.300	0.004	0.193	0.196	0.207	0.210	0.220	1.002	0.924	0.942
	C - ML	−0.299	0.003	0.187	0.189	0.198	0.200	0.210	1.101	0.920	0.937
	C - OLS	−0.300	0.004	0.188	0.191	0.200	0.202	0.213	1.075	0.919	0.936
	F - ML	−0.298	0.003	0.178	0.180	0.187	0.189	0.225	0.956	0.880	0.901
	F - OLS	−0.302	0.006	0.179	0.183	0.188	0.192	0.226	0.948	0.878	0.906
	O - ML	−0.299	0.003	0.190	0.192	0.202	0.204	0.215	1.051	0.918	0.935
	O - OLS	−0.299	0.003	0.190	0.192	0.202	0.205	0.215	1.054	0.914	0.937
	W - ML	−0.295	−0.001	0.191	0.195	0.202	0.206	0.224	0.971	0.904	0.929
	W - OLS	−0.298	0.002	0.191	0.195	0.202	0.207	0.224	0.970	0.902	0.933
m=100	Std	−0.293	−0.003	0.115	-	0.117	-	0.116	1.000	0.944	0.947
	C - ML (Ind)	−0.292	−0.004	0.113	0.113	0.115	0.116	0.117	0.987	0.941	0.946
	C - ML	−0.293	−0.003	0.109	0.109	0.111	0.111	0.112	1.089	0.938	0.940
	C - OLS	−0.293	−0.003	0.110	0.110	0.112	0.112	0.112	1.077	0.943	0.945
	F - ML	−0.293	−0.003	0.107	0.108	0.109	0.110	0.113	1.057	0.934	0.944
	F - OLS	−0.293	−0.003	0.108	0.109	0.109	0.110	0.114	1.052	0.937	0.943
	O - ML	−0.293	−0.003	0.111	0.111	0.113	0.113	0.113	1.070	0.944	0.950
	O - OLS	−0.293	−0.003	0.111	0.111	0.113	0.113	0.113	1.070	0.943	0.948
	W - ML	−0.293	−0.003	0.113	0.114	0.115	0.115	0.116	1.014	0.935	0.945
	W - OLS	−0.293	−0.003	0.113	0.114	0.115	0.116	0.116	1.015	0.937	0.943
High	Estimator	${\widehat{\beta }}_1$	Bias	SE1	SE2	SE3	SE4	MC SE	MC RE	Cov. U	Cov. A
m=30	Std	1.137	−0.001	0.153	-	0.163	-	0.169	1.000	0.925	0.946
	C - ML (Ind)	1.132	0.004	0.136	0.138	0.146	0.149	0.151	1.254	0.935	0.948
	C - ML	1.135	0.001	0.132	0.134	0.140	0.142	0.144	1.382	0.932	0.952
	C - OLS	1.134	0.002	0.134	0.136	0.143	0.145	0.146	1.348	0.938	0.951
	F - ML	1.135	0.002	0.125	0.128	0.132	0.135	0.155	1.195	0.901	0.921
	F - OLS	1.137	−0.001	0.127	0.131	0.134	0.139	0.156	1.172	0.896	0.924
	O - ML	1.135	0.001	0.135	0.137	0.144	0.146	0.148	1.306	0.928	0.955
	O - OLS	1.135	0.001	0.136	0.139	0.146	0.148	0.150	1.278	0.930	0.950
	W - ML	1.137	−0.001	0.141	0.145	0.150	0.154	0.161	1.101	0.924	0.947
	W - OLS	1.137	−0.001	0.141	0.146	0.150	0.155	0.163	1.085	0.921	0.945
m=100	Std	1.138	−0.002	0.089	-	0.090	-	0.090	1.000	0.946	0.949
	C - ML (Ind)	1.139	−0.003	0.079	0.079	0.080	0.081	0.083	1.162	0.934	0.935
	C - ML	1.139	−0.003	0.076	0.076	0.078	0.078	0.080	1.257	0.936	0.941
	C - OLS	1.140	−0.004	0.077	0.078	0.079	0.079	0.081	1.234	0.943	0.945
	F - ML	1.138	−0.002	0.075	0.075	0.076	0.076	0.082	1.210	0.931	0.935
	F - OLS	1.140	−0.003	0.076	0.076	0.077	0.078	0.082	1.197	0.934	0.943
	O - ML	1.139	−0.003	0.078	0.078	0.080	0.080	0.081	1.222	0.943	0.954
	O - OLS	1.140	−0.004	0.079	0.079	0.080	0.081	0.082	1.203	0.946	0.952
	W - ML	1.139	−0.003	0.083	0.083	0.084	0.085	0.086	1.098	0.948	0.954
	W - OLS	1.140	−0.003	0.083	0.084	0.084	0.085	0.086	1.095	0.945	0.952

Table 6. Standard vs. Augmented GEE, Binary Outcome: 30 & 100 clusters, low and high association, ρ = 0.20 β₁ = −0.2164 low association, β₁ = 1.0501 high association.

Std: unaugmented. Correlation exchangeable unless denoted by ‘Ind’ for independence. C,F,O,W: augmentation with ‘Correct’, ‘Forward’ selected, ‘One-variable’, or ‘Wrong’ model. ML, OLS: augmentation fit with maximum likelihood or ordinary least squares. SE₁: average unadjusted sandwich SE, SE₂: average sandwich SE with nuisance parameter adjustment, SE₃: average sandwich SE with small-sample adjustment, SE₄: average sandwich SE with small-sample and nuisance adjustments. MC RE: square of the Monte Carlo SE of the Std(Exch) estimator divided by the Monte Carlo SE for the indicated estimator. Cov. U: SE₁ coverage, Cov. A: SE₃ and SE₄ coverage for unaugmented and augmented GEE, respectively.

Low	Estimator	${\widehat{\beta }}_1$	Bias	SE1	SE2	SE3	SE4	MC SE	MC RE	Cov. U	Cov.A
m=30	Std	−0.228	0.012	0.317	-	0.335	-	0.344	1.000	0.925	0.935
	C - ML (Ind)	−0.235	0.019	0.328	0.332	0.354	0.358	0.373	0.855	0.908	0.934
	C - ML	−0.229	0.013	0.312	0.313	0.329	0.330	0.338	1.041	0.927	0.939
	C - OLS	−0.230	0.013	0.313	0.314	0.330	0.331	0.339	1.031	0.931	0.937
	F - ML	−0.218	0.001	0.294	0.296	0.308	0.310	0.365	0.891	0.892	0.908
	F - OLS	−0.224	0.008	0.297	0.303	0.310	0.317	0.376	0.838	0.887	0.905
	O - ML	−0.232	0.015	0.314	0.315	0.331	0.332	0.338	1.036	0.924	0.935
	O - OLS	−0.232	0.016	0.314	0.315	0.331	0.332	0.339	1.034	0.926	0.935
	W - ML	−0.221	0.004	0.311	0.314	0.328	0.330	0.350	0.968	0.915	0.925
	W - OLS	−0.225	0.009	0.312	0.315	0.329	0.332	0.352	0.958	0.912	0.926
m=100	Std	−0.213	−0.003	0.183	-	0.186	-	0.188	1.000	0.937	0.943
	C - ML (Ind)	−0.213	−0.003	0.192	0.193	0.196	0.197	0.201	0.868	0.938	0.943
	C - ML	−0.212	−0.004	0.180	0.180	0.182	0.182	0.185	1.031	0.939	0.941
	C - OLS	−0.212	−0.004	0.180	0.180	0.183	0.183	0.185	1.023	0.930	0.935
	F - ML	−0.213	−0.003	0.176	0.177	0.179	0.179	0.190	0.975	0.925	0.930
	F - OLS	−0.213	−0.003	0.176	0.178	0.179	0.180	0.191	0.968	0.930	0.933
	O - ML	−0.213	−0.003	0.181	0.181	0.184	0.184	0.186	1.021	0.930	0.936
	O - OLS	−0.213	−0.004	0.181	0.181	0.184	0.184	0.186	1.018	0.930	0.936
	W - ML	−0.213	−0.004	0.181	0.182	0.184	0.185	0.188	0.996	0.937	0.943
	W - OLS	−0.213	−0.003	0.181	0.182	0.184	0.185	0.188	0.995	0.935	0.943
High	Estimator	${\widehat{\beta }}_1$	Bias	SE1	SE2	SE3	SE4	MC SE	MC RE	Cov. U	Cov. A
m=30	Std	1.067	−0.017	0.244	-	0.258	-	0.251	1.000	0.934	0.948
	C - ML (Ind)	1.065	−0.015	0.244	0.247	0.264	0.267	0.258	0.941	0.923	0.949
	C - ML	1.069	−0.019	0.230	0.231	0.243	0.244	0.235	1.132	0.935	0.950
	C - OLS	1.069	−0.019	0.231	0.232	0.245	0.246	0.237	1.120	0.934	0.947
	F - ML	1.065	−0.015	0.217	0.219	0.228	0.230	0.257	0.952	0.889	0.909
	F - OLS	1.069	−0.019	0.219	0.224	0.230	0.235	0.262	0.914	0.895	0.913
	O - ML	1.074	−0.024	0.233	0.234	0.247	0.247	0.243	1.061	0.931	0.941
	O - OLS	1.075	−0.025	0.234	0.235	0.247	0.248	0.244	1.052	0.928	0.943
	W - ML	1.066	−0.016	0.234	0.237	0.248	0.250	0.248	1.021	0.931	0.942
	W - OLS	1.067	−0.017	0.235	0.237	0.248	0.250	0.248	1.018	0.932	0.944
m=100	Std	1.062	−0.012	0.138	-	0.140	-	0.137	1.000	0.946	0.949
	C - ML (Ind)	1.059	−0.009	0.140	0.140	0.143	0.144	0.143	0.928	0.941	0.950
	C - ML	1.059	−0.009	0.130	0.130	0.133	0.133	0.129	1.124	0.952	0.958
	C - OLS	1.059	−0.009	0.131	0.131	0.133	0.133	0.130	1.117	0.953	0.956
	F - ML	1.057	−0.007	0.128	0.129	0.130	0.131	0.132	1.076	0.944	0.948
	F - OLS	1.058	−0.007	0.128	0.130	0.130	0.132	0.133	1.070	0.946	0.953
	O - ML	1.060	−0.010	0.132	0.132	0.134	0.134	0.131	1.096	0.947	0.952
	O - OLS	1.060	−0.010	0.132	0.132	0.134	0.134	0.131	1.095	0.950	0.953
	W - ML	1.061	−0.011	0.134	0.135	0.136	0.137	0.135	1.041	0.943	0.949
	W - OLS	1.062	−0.011	0.134	0.135	0.136	0.137	0.134	1.044	0.945	0.951

In summary, large-sample results suggest improvement with augmentation, whereas results for small-sample estimation are less consistent. Across the number of clusters evaluated, augmentation was less beneficial as the degree of intracluster correlation increased. Regarding augmentation fit, the variability of ${\widehat{\beta }}_1$ was similar when comparing augmented estimators resulting from predictions from OLS and ML.

6. DISCUSSION

This paper demonstrates the use of methodology based on semiparametric theory to improve efficiency of inferences in randomized studies with correlated outcomes through augmenting the standard GEE. This method extends the work of [17] by focusing on multivariate outcomes, and is the first application of this approach to a cluster randomized trial.

The binary outcome analysis illustrates an additional advantage of augmented GEE - double robustness. Results from standard GEE may result in misleading estimates in settings where randomization has led to imbalance in important predictors. Augmentation involves specifying a conditional model E(Y|X, A) that corrects for imbalances and therefore recovers unbiased estimates of treatment effects, even when randomization does not result in independence of X and A in the observed data. Alternative methods for correction, such as IPTW using a predictive model for the probability of treatment given baseline covariates, may not perform well given the cluster-level assignment. Predictive models for treatment only make use of cluster-level information; individual-level covariates may be averaged by cluster to create cluster-level covariates, but this data-coarsened approach can lead to poorly specified models. Generally, inference on the probability of treatment will be poor given the small number of randomized units. Alternatively, augmentation exploits relationships among individual-level covariates and outcomes. Since there are multiple individuals per cluster, there is more information available for estimating E(Y|X, A) compared to P(A = 1|X). Estimation of E(Y|X, A) may consequently result in a better estimator of β₁.

Simulation studies explored the possibility of efficiency gains using the augmented GEE in small- and large-sample settings. For large samples, the augmented GEE improved efficiency compared to the standard GEE for marginal treatment effects, which ignores baseline covariates. In the small-sample setting, efficiency gain was less consistent; low levels of between-community heterogeneity and high degrees of association between baseline covariates and outcomes were required to benefit from augmentation. Gail et al. [14] found a similar trend in their studies of permutation inference, noting that covariate adjustment did not improve efficiency when between-community variability was high. These results highlight the importance of measuring all covariates that contribute to within-community similarities in response. Interpreting the results from the Young Citizens study using the insight obtained through simulations, the low intracluster correlation (0.02) suggests improvement in efficiency when adjusting for baseline covariates. The degree of improvement, however, may be overstated by sandwich standard errors. Small-sample estimation also resulted in coverage slightly below nominal levels, even after standard error adjustment. The standard error modifications used only consider first-order approximations to the sandwich variance and nuisance parameter distributions. The simulation results suggest second order effects of nuisance parameter estimation may impact variance underestimation. The shortcomings of this approach in small samples motivate investigation into the use of augmented estimators with permutation-based inference.

We implemented augmentation using separate models for treatment and control, with ML and OLS for binary outcomes, and OLS for continuous outcomes. Asymptotically, treatment-specific OLS including an intercept term is guaranteed to be at least as efficient as the unadjusted estimator [22, 12]. As discussed by Zhang and Gilbert [23], data splitting can be inefficient in finite samples compared to fitting a common model for E(Y|X, A). For studies involving relatively few randomized units, fitting a common conditional model may better utilize covariate information. The effect of data splitting in finite sample inference has not yet been examined in practice. To guarantee efficiency gain over unadjusted methods when fitting a common model, van der Laan’s empirical efficiency maximization approach [29] may be used. This method estimates nuisance parameters by empirically minimizing the asymptotic variance of a scalar targeted parameter. It results in fitting adjustment models with a weighted least squares procedure, in which weights depend on treatment probabilities.

Although the simple augmented GEE improves estimation in large samples, it is not the semiparametric efficient estimator for our restricted mean model for multivariate outcome data, even under correct specification of E(Y|A, X). Nonetheless, the simple augmented GEE builds upon standard GEE in an intuitive way and provides insight into how augmentation may be used with multivariate data to improve efficiency. Development of a locally semiparametric efficient estimator for restricted mean models for multivariate data and an understanding of its behavior remain important research questions. A locally efficient estimator is an estimator that remains consistent and asymptotically normal under the restricted mean model, and that achieves the semiparametric efficiency bound for the model at the submodel where nuisance parameters are correctly specified. When model misspecification of nuisance parameters is present, it is not clear whether the locally efficient estimator will still improve efficiency compared to standard techniques. Additional modification of the locally efficient estimator is needed to ensure improvement relative to standard GEE. Further research is warranted in this area.